Monitor your passenger applications via SNMP

Basic Idea:

Monitor the output of the command passenger-status, "Active Sites, Total Running, and Waiting Sites"

On your production box:
ubuntu@prodweb01:~$ cat /home/ubuntu/scripts/monit-passenger.sh
#!/bin/bash
# Monitor Passenger-status by outputting to a file
# then grepping the variables and outputing them to seperate files
# you must add your own ruby path variables. 

export PATH=""
cmd="/usr/local/rvm/gems/ruby-1.8.7-p371@rails2.3.15/bin/passenger-status"
outputdir="/home/ubuntu/scripts/monitor/passenger"
mainfile="passenger-status.txt"
greping=( max count active inactive )
wait=waiting
$cmd > $outputdir/$mainfile

for item in "${greping[@]}"
do
    egrep ^$item $outputdir$mainfile | awk -F"=" '{print $2}' | sed 's/ //g' > $outputdir$item.txt
done

egrep ^$wait $outputdir$mainfile | awk -F":" '{print $2}' | sed 's/ //g' > $outputdir$wait.txt

Add those files to snmpd for zenoss to read.

root@prodweb01:~$ cat /etc/snmp/snmpd.conf
extend QUEUE /bin/cat /home/ubuntu/scripts/monitor/passenger/waiting.txt
extend ACTIVE /bin/cat /home/ubuntu/scripts/monitor/passenger/active.txt
extend COUNT /bin/cat /home/ubuntu/scripts/monitor/passenger/count.txt
extend INACTIVE /bin/cat /home/ubuntu/scripts/monitor/passenger/inactive.txt
extend MAX /bin/cat /home/ubuntu/scripts/monitor/passenger/max.txt

Set the script to run every 5 minutes so zenoss can graph something

root@prodweb01:~$ crontab -u root -l
*/5 * * * * /home/ubuntu/scripts/monit-passenger.sh

Now restart snmpd and do a quick cat on the files to show that they are outputting data.

root@prodweb01:~$ /etc/init.d/snmpd restart
root@prodweb01:~$ cat /home/ubuntu/scripts/monitor/passenger/* | head
On your Zenoss box

Run the command:

[root@zenoss ~]# snmpwalk -v 2c -cYOURGROUP 111.222.333.444 NET-SNMP-EXTEND-MIB::nsExtendOutput1Line
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."MAX" = STRING: 200
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."COUNT" = STRING: 147
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."QUEUE" = STRING: 0
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."ACTIVE" = STRING: 0
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."INACTIVE" = STRING: 147
[root@zenoss ~]# snmpwalk -v 2c -On -cYOURGROUP 111.222.333.444 NET-SNMP-EXTEND-MIB::nsExtendOutput1Line
.1.3.6.1.4.1.8072.1.3.2.3.1.1.3.XX.XX.XX = STRING: 200
.1.3.6.1.4.1.8072.1.3.2.3.1.1.5.XX.XX.XX.XX.XX = STRING: 147
.1.3.6.1.4.1.8072.1.3.2.3.1.1.5.XX.XX.XX.XX.XX = STRING: 0
.1.3.6.1.4.1.8072.1.3.2.3.1.1.6.XX.XX.XX.XX.XX.XX = STRING: 0
.1.3.6.1.4.1.8072.1.3.2.3.1.1.8.XX.XX.XX.XX.XX.XX.XX.XX = STRING: 147

Thats an O for Orange, not a Zero.
Side Note: The redacted XX's are the ASCII HEX values. MAX == 77.65.88. (I am so glad I did this blog post :P)

Now You have your MIB ID's for your COUNT QUEUE ACTIVE and INACTIVE and MAX, you can easily add it to your Zenoss/Nagios. Make sure also that you "Override Template Here" and set it as /Device, and then it will turn it from saying (/Devices) to (Locally Defined). Otherwise you will add all of your MIB points to every server you operate.