Monitor Your Elk-M1, Log to MySQL, and Daemonize it!

I received a request a couple of days ago from someone who saw my posts on the Elk–Control perl library that I took over responsibility for. I had mentioned that I logged every Elk event into a MySQL database.

I am happy to report that I have this script running as a daemon, logging every ElkM1 event into my database. And it runs for months without issue.

I will keep these instructions brief. If you have detailed questions, please ask in the comments and I’ll help the best I can.

Download

You can download all the scirpts, just click here.

Notables

  1. If you don’t know a least a little perl and some MySQL basics, you may want to look elsewhere.
  2. The script and accompanying files are designed to work in concert as a daemonized process.
  3. Everything is logged into ‘elk_monitor.log’ in the same directory (or directory of your choosing) as the script. If you have a lot of passive infra-red motion detectors, you will want to monitor the log size.

Basic File Structure

file structure

elk_monitor.pl

This is the main script, it runs in a loop checking for new messages every second. You will need edit the %zonelist, %outputs, %areas near the top. Also, you will need to set some paths, look near the top of the file.

Required: ElkM1::Control, Log4perl

elk_monitor.sh

This is the shell script that controls elk_monitor.pl as a daemon. Setting up daemons can be complex as each system is different, so google around for some help.

elkm1.sql

A MySQL script to setup the tables and fields that the elk_monitor.pl script uses. You will need to add a database named ‘elkm1’. After that you can execute this script to create everything else for you.

lib/elkm1_db.pl

The database setting and basic sql commands are abstracted into this library. This is where you will need to set MySQL’s host, name, password, etc.

lib/file_tickler.pl

I use a custom ruby script to monitor the daemon. This library just touches a file every minute to let us know the elk_monitor.pl script is still running. Why? Well, I have had the script ‘freeze’ without exiting and most monitoring applications do not detect this. Obviously, this part is optional.

Conclusion

If you get all the settings right, she should run all day/week/month for you.

I and not a perl expert. Far from it. So I’m sure everything could use some tidying. If you do, please let me know and I’ll incorporate your changes.

Posted in Home Automation, perl. Tagged with , , .

Sinatra-Tailer: a small app for viewing server log files

UPDATE (Jan 9, 2015): I have not updated this project in year. While it may work, I doubt it. If you would like to take over the project, leave me a comment and we’ll get in touch.

I was reading Jason Seifer’s: Offline Gem Server Rdocs, which is an apache/passenger served Sinatra app that allows you to view the Rdocs of installed gems without using gem server. Nice. So I installed it on our sandbox server for all to enjoy.

But it got me thinking, there is another think I like to keep an eye on on our servers… log files. Oh, and I was looking for good excuse to play around with Sinatra. So, “with a one, and-a-two, and-a-three…” we have Sinatra-Tailer.

You can read all about it on the github page, but in short it simply performs a tail and displays the last X lines of the log file.

[sourcecode language=’bash’]
tail -n /path/to/my/log/file.log
[/sourcecode]

Features

  1. refreshes the list every X seconds, set by the user
  2. only one config.yml file to edit
  3. supports file globs, so you can grab a whole list of file with one line
  4. specify the number of lines to show
  5. completely unobtrusive js, because I’m cool?

Requirements

Sinatra framework

      (tested on 0.9.1.1, earlier may work)

I whipped this up pretty quickly so I’m sure there are a few bugs. There is some testing for a few unit tests, but nothing functional.

One word of warning, if you want to put this on a production server my recommendation is to put it on a separate port (like 9876) and for heaven’s sake, at a minimum use http basic authentication. From the sinatra readme:

Enjoy!

Posted in Ruby. Tagged with , , , .

Apache Virtual Hosts, Rails, Rack, Passenger on Local Net

This one has been bugging me for a couple of years and I just didn’t want to put in the time and testing to nail it down. But tonight was my night.

The Problem

Here in my development palace I have a couple of laptops that I use for development and an Ubuntu (hardy) server as a Rails/Rack sandbox that I run with Passenger. While I have always been able to serve up either a Rails or Sinatra project, I typically have 3 or 4 going at the same time. But I had to access them with urls like: http://192.168.0.2/project1 and http://192.168.0.2/project2.

So, what’s the problem, eh?

Well, for starters, you can’t do anything concerning subdomains. That’s a show stopper for many apps.

Second, it really messes with your environment variables like [“PATH_INFO”]. So if you do any routing and you expect the base url to be ‘/’, it won’t be. The root url will be ‘/projectx’. Bummer.

The Solution

To remedy the issues with non-tld virtual hosts (also called sub URIs?) we need to add the domains to the laptops and server, then add the virtualhosts on the server for apache. This means that I can now goto http://project1.ubuntu.remote url to get to my rails/rack app.

For reference, I am using the following:

On Each Laptop

These instructions are for Mac OS X, but should be identical for your favorite flavor of *nix. We are going to add some friendly names to our ubuntu server using it’s ip address (192.168.0.2). In this example we will be able to access the root web documents using http://ubuntu.remote, and one of our Sinatra projects using http://project1.ubuntu.remote.

Important! Don’t use a name ending in ‘.local’. While it did work for me, I read about a few instances where some systems will not resolve domain names ending in .local outside the local machine. Also, don’t change anything else in your /etc/hosts file except to add the lines at the end.

[source language=’python’]
mate /etc/hosts

# add the following line to the END of your /etc/hosts file and save
# note: there is a character between the ip address and each host name
192.168.0.2 ubuntu.remote project1.ubuntu.remote
[/source]

On The Server: Hosts File

Nothing will work until we tell the server machine that it can accept incoming traffic from our new friendly names. Similar to on the mac, we need to edit our /etc/hosts file but we will add a slightly different line. In this case we are telling the server that incoming traffic from ‘ubuntu.remote’ and ‘gems.ubuntu.remote’ are just names for the localhost (127.0.0.1).

[source lang=’shell’]sudo pico /etc/hosts

# add the following line to the end of our /etc/hosts file and save
# you can added after your existing 127.0.0.1 line if you wish
127.0.0.1 ubuntu.remote project1.ubuntu.remote
[/source]

On The Server: VirtualHosts

As a final step we will need to add or change the virtual host definitions for your rails/rack project. Depending on your Apache setup or Linux distro, the default enabled site may be slightly different and you may have to adapt your virtual host file to your default setup.

First, in my /etc/apache2/apache.conf file, I have the following line near the end of the file. This was to eliminate a warning every time apache was started/restarted. It may affect your virtual host default, but shouldn’t, so add it if you get errors.

[source lang=’shell’]
#added near end of /etc/apache2/apache.conf
ServerName localhost
[/source]

Next we need to check our default virtual host setup to see how the NameVirtualHost is defined. We will duplicate that in each new virtual host that we add. The parts we are looking is the NameVirtualHost and VirtualHost near the top. They could be ‘NameVirtualHost *’ and ‘<VirtualHost *>’ or ‘NameVirtualHost *:80’ and ‘<VirtualHost *:80>’. See the pattern? We need to use the same definition in our new virtual files.

Let’s check our default virtual host setup:

[source lang=’shell’]
cd /etc/apache2/sites-enabled
ls # your are looking for the name of your default site file

total 0

lrwxrwxrwx 1 root root 36 2009-02-27 18:49 000-default -> /etc/apache2/sites-available/default

cat /etc/apache2/sites-enabled/000-default

NameVirtualHost *

  ServerAdmin webmaster@localhost
# snipped several lines #

[/source]

On my system the NameVirtualHost is ‘*’, so that is what we need to use for the new virtual host. In this case, I am going to add a virtual host file for a rack based project. It’s a Sinatra app, but that doesn’t matter as this will work for any rack based project, even a rails/rack project (since rails 2.2.2?).

Also, I like using the Apache2 system to enable and disable sites. It’s clean, easy to use, and if any apache admin comes along they will know exactly what is going on.

It’s also work mentioning that you should disable or delete any existing virtual host files that you were using previously.

[source lang=’shell’]cd /etc/apache2/sites-available
sudo pico gems

# enter this as your virtual host setup

ServerName project1.ubuntu.remote

DocumentRoot “/path/to/your/application/project/public”

RackEnv production

Order allow,deny
Allow from all

# save the file and reload apache

sudo /etc/init.d/apache2 reload
[/source]

The Fun Part

Assuming you saw no errors when you reloaded apache, you should be able to go to your new urls see both the root files and your rack application.

http://ubuntu.remote
http://project1.ubuntu.remote

Dyn-O-Mite!

Bonus Points!

For extra credit, let’s say you have admin access to your router, and you are one of those gotta-have-control-of-everything dudes (or dudettes) so you installed a 3rd party firmware. You’re in luck! You can probably add some custom entries in your DNS and you don’t have to edit the /etc/hosts files on your laptop.

I installed Tomato firmware on my Linksys wireless router. Tomato uses Dnsmasq with allows you to set internal host names.

Maybe I’ll cover that in another post.

Posted in apache, Rails, Ruby. Tagged with , , , , .

Elk M1 Security System Control with Perl Script

I have an Elk M1 security system that I like to monitor and record all the activity. It’s useful to know what doors are open and how long they have been open. For several years in the past I have use Homeseer to do the monitoring where I wrote VBScript files to push the activity and changes to my MySQL server. Honestly, I have never likes Homseer, it’s flaky and I prefer that I don’t depend on WinXP either.

About a year ago, James Russo released a script on Cocoontech to monitor and control your Elk M1 from a perl script. You can read the forum post for more details. Thanks to James and other Cocoontech members, it works fairly well.

But there have been no updates or bug fixes in over a year. Not wanting to relive my perl days (sorry, I never really liked perl), I decided to revive the project and give it a new home. I asked James for permission to take over the project and he gracefully agreed.

ElkM1::Control New Home on Github

Go check it out! ElkM1::Control on github.

Why github? First, I have stopped using SVN. And sourceforge, ElkM1::Control’s original home, doesn’t support git. Second, github makes it too easy to fork a project and make changes yourself, then push them back up so that I can integrate those changes. You really should try git if you haven’t before. And you don’t need git to download ElkM1::Control and install it.

Updates and Bug Fixes!

You read that right, I have made a few bug fixes and added a couple of new features. I have been running ElkM1::Control on my Ubuntu Hardy server for about a month with no problems. I have also run and do some debugging on OS X (10.5.x).

I also changed the version numbering system to the more common major.minor.release notation. We are not at version 0.1.0.

Quick Installation Instructions

  1. Goto the project home page and click the download button. Select the ZIP version (for some reason the TAR doesn’t work sometimes).
  2. Follow the README file, don’t forget the PREREQUISITES section. For those who don’t rtfm…
  3. From your favorite shell, you may need to add sudo in front of each: ‘perl Makefile.PL’
  4. ‘make’
  5. ‘make test’
  6. Then to install: ‘make install’

Next, you’ll need a little script to get going. There are several scripts posted on Cocoontech, but this one should get you started:

[source lang=’php’]
# Example 2: Display all M1 messages
use ElkM1::Control;
    my $elk = ElkM1::Control->new(host => ‘192.168.0.251’, port => 2101);
    while (1) {  # Loop forever
        while (my $msg = $elk->readMessage) { # Read the M1’s messages
                print $msg->toString;  # Print the messages
        }
    }
[/source]

Posted in Home Automation, perl. Tagged with , , .

Authlogic – Rails Authentication Done Right

Sorry to borrow the title line directly from Ben’s site, but Authlogic is the authentication system I have been looking for. Bye, bye restful authentication. Hello easy, simple, get out of my way, easily upgradeable, smartly written Authlogic.

I shouldn’t disrespect restful authentication much as she has been with me for over a year now. But every time I had to install, setup, and get the basics working in RA I couldn’t help gnashing my teeth. I jumped on the RA bandwagon like so many other Rails developers looking for an authentication system that just and move on to more important things.

Enter Authlogic by Ben Johnson of BinaryLogic. By luck, I was starting a new rails app this week, so I decided to take Authlogic out for a test drive. Following Ben’s Tutorial: Authlogic Basic Setup I had a basic login/logout/session management system up and running in a ridiculously little amount of code. The best part of Authlogic is that it truly get’s out of my way and provides me with what I need… a robust and secure methodology of authentication and session management.

The benefits of Authlogic are:

  • It’s a plugin and a gem. When Ben pushes an update, getting the latest release is super simple.
  • It’s a plugin and a gem. This keeps the authentication code separate from your codes, the way it should be.
  • Session are treated like ActiveRecord objects. This is just as cools as it sounds and is very Rails-like.
  • Better security. Authlogic uses a non-reversible token that is changed on every session initiation and password change, thus virtually eliminating session persistence and brute force session attacks.
  • Ben Johnson. Ben knows what he is doing and has been quickly releasing updates.
  • Ben Johnson. Sorry for the repetition, but Ben also has a nice series of tutorials with supporting project code you can download.
Read more on on BinaryLogic or get it at Github.
Posted in Rails. Tagged with , .

iPhoto AppleScript to Remove Duplicates

Update May 6, 2015: This script has not been tested with Mac OS X Yosemite 10.10.3 new Photos application.

Short Story:

I had several years of photos that I needed to identify and remove the duplicate. Instead of manually combing through 12,000 (read Long Story below) and before carpal tunnel set in, I needed a script to help me out. My situation may or may not be unique, so this script may not work 100% out-of-the-box for you, but it should get you started.

This script will identify duplicate photos in your iPhoto library and mark them with a comment (keyword) of “duplicate”. It will not delete anything.

To use:

  1. Download and unzip the script
  2. Double-click the script to open in Script Editor
  3. Go into iPhoto and select a group of photos you want to compare
  4. Switch back to Script Editor and run the script
  5. Don’t Touch Anything! Just let the script finish, it could take a while if you are comparing a lot of photos
  6. After the script is done, go back into iphoto and search for “duplicate”
  7. You can highlight all the duplicates and delete them or move them some place safe

Photos are considered a duplicate if:

  1. both heights match
  2. both widths match
  3. the photo date in iPhoto match, this is typically the EXIF creation date
There are no error checks in this script and it presents no interface except an alert when it’s done. If you need help, just post a comment below and I’ll do my best.

Long Story:

Continue reading

Posted in Mac. Tagged with , .

Batter Blaster – Wrong for America

Ok, the title is a tad hyperbolic, but when I saw this in Costco a few months ago a tiny rage welled inside. We were shopping on a Saturday when we came across Batter Blaster. The spiel was along the lines of how much easier breakfast would be with pancakes from a can.

Breakfast from a Can? WTF!

Continue reading

Posted in Peeves.

Butt Biter: Rails ‘truncate’ Method Broken in Ruby 1.8.7

This one threw me for far too many minutes. In ActionView::Helpers::TextHelper the truncate method is broken for Ruby 1.8.7. I’m not exactly sure the version of ruby that it stopped working for, but I was previously on 1.8.6, and I don’t recall it not working. But after I upgraded to the latest patchlevel of 1.8.7, truncate stopped working.

Update 12/2/08: Fixed in Rails 2.2! (read more)

The error that I received was thus:

ActionView::TemplateError (undefined method `length' for #) on line #83 of dashboard/index.html.erb:

Fortunately, someone has submitted a ticket to Rails core. I hope it’s applied in the Rails 2.2.0 release.

Continue reading

Posted in Rails. Tagged with , .

Ruby on Rails Ubuntu Server VM with VMPlayer

I had to setup a Ubuntu (8.04) Server virtual machine and get Rails up-to-speed on it this morning, so I thought I would document my steps for those who find the need to repeat. Here is what I needed:

What you need to start:

Continue reading

Posted in Rails, Ruby. Tagged with , , .

Ruby: Send Email Reminders from iCal Calendar

This is going to be long, so hunker down for a good read.

Problem: Our Cub Scout den leader sends email reminders to all the parent about upcoming events. I decided to setup a gCal (google calendar) will all the events so that other parents could just check the calendar as they wish, or subscribe to the ical file to integrate with their own calendar. But several of the parents don’t have calendars (shame, shame) and everyone still liked the idea of receiving email reminders. So I decided to write a ruby script to email reminders automatically two days before the event.

Solution: Ruby is a great language to handle this. Especially since there are gems to handle many of the functions we need. Some of the gems we will be using are as follows:

Continue reading

Posted in Ruby. Tagged with , , , , .