Setting up cricket and rrdtool

*** TODO: Check CPU load. Fix memory usage. Catalyst switches. New cricket-config.tgz ***

rrdtool is a utility for storing and viewing information in "round robin database" files; cricket is a frontend which collects data via SNMP and displays it using a CGI. Together they form an SNMP graphing tool which is much more efficient than MRTG, since it only generates graphs when someone is looking at them.

This document is a quick guide to installation under FreeBSD, but does not attempt to explain fully how Cricket works; for that you need to see the documentation at http://cricket.sourceforge.net/ or online in /usr/local/cricket/doc/ after installation. The "Gentle Introduction to Cricket" is a good starting point. There is also a very useful set of contributed device support at http://www.certaintysolutions.com/tech-advice/cricket-contrib/index.shtml

1. Install the software

We will use the FreeBSD 'ports' mechanism as it automatically downloads and installs all the dependent modules. Since cricket installs a lot of dependent packages, make sure you are pointing to the local FTP server by putting the following line in /etc/make.conf:

MASTER_SITE_OVERRIDE=ftp://noc.ws.afnog.org/pub/FreeBSD/ports/distfiles/

Then build as follows:

# cd /usr/ports/net/cricket
# make
# make install
# make clean
Fix broken permissions:
# cd /usr/local/cricket
# chmod +x compile collector collect-subtrees *.cgi util/listInterfaces

Note: installation will automatically download the following source files into
/usr/ports/distfiles/ or you can fetch them by hand:
Digest-MD5-2.12.tar.gz
HTML-Parser-3.13.tar.gz
HTML-Tagset-3.03.tar.gz
MIME-Base64-2.11.tar.gz
SNMP_Session-0.75.tar.gz
Time-HiRes-01.20.tar.gz
TimeDate-1.10.tar.gz
URI-1.09.tar.gz
cricket-1.0.2.tar.gz
gettext-0.10.35.tar.gz
libnet-1.0703.tar.gz
libtool-1.3.4.tar.gz
libwww-perl-5.48.tar.gz
make-3.79.1.tar.gz
rrdtool-1.0.28.tar.gz

2. Create a 'cricket' user

This is so that the cricket data collection mechanism does not have to run as 'root'

# pw useradd cricket -d /usr/local/cricket -c "Cricket user" -u 77
Check using:
# grep cricket /etc/passwd

(-u 77 sets the uid/gid to 77, or you can omit this to let the system choose a 'normal' uid over 1000)

3. Create data directories

Note that the 'cricket-data' and 'cricket-logs' directories must be owned by 'cricket' so that it can create files there.

# cd /usr/local/cricket
# mkdir cricket-data
# mkdir cricket-logs
# chown cricket:cricket cricket-data
# chown cricket:cricket cricket-logs

4. Set up a skeleton configuration

The sample-config tree which comes with Cricket is actually not very well structured (as agreed by most people on the cricket-users mailing list), so we will use a different one.

Cricket's configuration is based upon a tree structure. In each directory there can be a file called "Defaults" which is read first, other files, and other directories. The other files and other directories 'inherit' any values which are set in the Defaults file, unless they are explicitly overridden. This makes creation of configurations for thousands of devices very straightforward: any information which is shared between those devices, say a common snmp-community string, can be put in the Defaults file and used by them all.

# cd /usr/local/cricket
# tar -xvzf /path/to/cricket-config.tgz

Now we will create a configuration for a Cisco router; if you have multiple Cisco routers you put them under the same directory. Note that there are actually two things we are going to monitor: (1) the router itself (CPU load, temperature etc) and (2) the individual interfaces.

# cd /usr/local/cricket/cricket-config
# cd routers/cisco

Have a look in the Defaults file. Note how it defines parameters which can be collected for some common types of Cisco router. Look for Cisco-3600-Router, which we will be using.

# mkdir afnog-border
# cd afnog-border
# vi Defaults

# parameters for the router which are also inherited by the interfaces
target	--default--
	router = 213.172.132.66
	snmp-community = "community"

# the router itself
target	afnog-border
	target-type = Cisco-3600-Router
	short-desc = "Border router AFNOG to NCS"


# mkdir interfaces
# cd interfaces
# vi Defaults

target --default--
	target-type = standard-interface
	inst = map(interface-name)

# /usr/local/cricket/util/listInterfaces 213.172.132.66 'community' >Targets

The 'map(interface-name)' function allows you to give Cricket an interface name which is converted into the interface index at run-time. Hence if the router reboots, and the interfaces are renumbered, Cricket will continue to correct information from the correct interface. The mapping function is actually defined in the top-level Defaults file:

map interface-name
    base-oid    =    ifDescr
    match       =    %interface-name%

The utility 'listInterfaces' is similar to MRTG's cfgmaker, in that it automatically reads the available interfaces and descriptions. However, Cricket's config files contain much less information, since they inherit most of it from higher directories, and therefore are much easier to maintain.

5. Compile the configuration

This step must be done after every configuration change

# cd /usr/local/cricket
# ./compile -base cricket-config

The result should look something like this:

[30-Apr-2001 16:10:19 ] Starting compile: Cricket version 1.0.2 (2000-05-19)
[30-Apr-2001 16:10:20 ] Processed 12 nodes (in 6 files) in 2 seconds.

(If you see "Processed 0 nodes" then something is wrong :-)

This process creates cricket-config/config.db which is read by the data collector and graphing programs.

6. Test data collection for one part of subtree

We 'su' to user 'cricket' so that the newly-created data files are owned by 'cricket'. If you forget this, then rememeber to chown everything in the cricket-data directory to 'cricket'

# su - cricket
$ ./collector /routers
[30-Apr-2001 16:37:46 ] Starting collector: Cricket version 1.0.2 (2000-05-19)
[30-Apr-2001 16:37:46 ] Retrieved data for afnog-border (): 0,0,1,2707164,46100764
[30-Apr-2001 16:37:46 ] Retrieved data for serial0_0 (1): 6747137,1217602,46,0,13168,13043
[30-Apr-2001 16:37:46 ] Retrieved data for serial0_1 (2): 10333709,12106021,5,0,17589,26725
[30-Apr-2001 16:37:46 ] Retrieved data for ethernet1_0 (5): 13192669,17116879,0,0,36912,31776
[30-Apr-2001 16:37:46 ] Retrieved data for null0 (6): 0,0,0,0,0,0
[30-Apr-2001 16:37:46 ] Processed 5 targets in 1 seconds.
$ ctrl-D

If you have other subtrees (e.g. 'switches') teset them in the same way. Then set up a set of subtrees for collection:

# cd /usr/local/cricket
# vi subtree-sets

set normal:
	/routers
	/switches


# su - cricket
$ ./collect-subtrees normal
$

Notice that 'collect-subtrees' does not generate any output on stderr. Instead, it logs its output in files: this makes it more suitable for running regularly from 'cron'

# ls cricket-logs
normal.0
# cat normal.0

7. Set up regular collection

# crontab -u cricket -e

Set up the following line:

*/5 * * * /usr/local/cricket/collect-subtrees normal

OK, Cricket is now collecting data and putting it into .rrd files (in the cricket-data subdirectory). Now we just need to be able to display them.

8. Install apache (if not already installed)

# cd /usr/ports/www/apache13
# make
# make install
# make clean

This will fetch and compile source files: apache_1.3.14.tar.gz

# cd /usr/local/www
# rm cgi-bin data
# mkdir cgi-bin data

# cd /usr/local/etc/apache
# vi httpd.conf
Set ServerAdmin to your E-mail address

9. Install the cricket cgi-bin and images

# cd /usr/local/www/data
# mkdir cricket
# cd cricket
# cp -pr /usr/local/cricket/images .
# cp -p /usr/local/cricket/*.cgi .

We need to configure Apache to allow execution of .cgi files within this directory; at the same time, because our graphs are confidential, we can restrict access. The example below allows access from certain IP addresses without authentication, or from any other IP address if a username and password is given

Uncomment the line:
AddHandler cgi-script .cgi

And add the following:
<Directory /usr/local/www/data/cricket>
	Options FollowSymLinks ExecCGI
	DirectoryIndex grapher.cgi
	AllowOverride None
	Satisfy any
	AuthName "Cricket stats page"
	AuthType Basic
	AuthUserFile /usr/local/www/htpasswd
	Require valid-user
	Order deny,allow
	Allow from 213.172.132.0/23 127.0.0.1
	Deny from all
</Directory>

10. Fix up minor problems

Make sure that mail to user 'cricket' is forwarded to a real user, otherwise any errors from the crontab entry will not be seen

/etc/aliases

cricket:    root

then run 'newaliases'.

The grapher will always display a red message saying that the config.db is out of date, even if it isn't. To fix this, edit grapher.cgi, and make the following change to function fixHome:

...
sub fixHome {

	# brute force:
	$ENV{'HOME'} = '/usr/local/cricket';
	chdir('/usr/local/cricket');
	return;

	$my($sname) = $gQ->script_name();
	...