What is it?
Downloading
Requirements
What's New
Upgrading
Installation
Overview
Unix
Windows
Securing SNMPTT
Configuration Options - snmptt.ini
Logging
Standard
Syslog
EventLog
Database
MySQL
PostgreSQL
ODBC
Windows ODBC
Unknown Traps
Executing an external program
Modes of Operation
Command line arguments
SNMPTT.CONF Configuration file format
EVENT
FORMAT
Variable-substitutions
EXEC
NODES
MATCH
REGEX
SDESC
EDESC
SNMPTT.CONF Configuration file Notes
Name resolution / DNS
Sample1 SNMPTT.CONF file
Sample2 SNMPTT.CONF file
Notes
Limitations
Feedback & Bugs
Integration with other software
Nagios / Netsaint
Copyright 2002, 2003 Alex Burger
alex_b@users.sourceforge.net
4/3/2002
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
02111-1307 USA
Many network devices including but not limited to network switches, routers, remote access servers, UPSs, printers and operating systems such as Unix and Windows NT have the ability to send notifications to an SNMP manager running on a network management station. The notifications can be either SNMP Traps, or SNMP Inform messages.
The notification can contain a wide array of information such as port failures, link failures, access violations, power outages, paper jams, hard drive failures etc. The MIB (Management Information Base) available from the vendor determines the notifications supported by each device.
The MIB file contains TRAP-TYPE (SMIv1) or NOTIFICATION-TYPE (SMIv2) definitions, which define the variables that are passed to the management station when a particular event occurs.
The Net-SNMP program snmptrapd is an application that receives and logs SNMP trap and inform messages via TCP/IP. Following is a sample syslog entry for a Compaq cpqDa3LogDrvStatusChange trap that notifies that the drive array is rebuilding:
Feb 12 13:37:10 server11 snmptrapd[25409]: 192.168.110.192: Enterprise Specific Trap (3008) Uptime: 306 days, 23:13:24.29, .1.3.6.1.2.1.1.5.0 = SERVER08, .1.3.6.1.4.1.232.11.2.11.1.0 = 0, .1.3.6.1.4.1.232.3.2.3.1.1.4.8.1 = rebuilding(7)
or
Feb 12 13:37:10 server11 snmptrapd[25409]: 192.168.110.192: Enterprise Specific Trap (3008) Uptime: 306 days, 23:13:24.29, sysName.0 = SERVER08, cpqHoTrapFlags.0 = 0, cpqDaLogDrvStatus.8.1 = rebuilding(7)
The output from snmptrapd can be changed via the -O option to display numeric or symbolic OIDs and other display options, but it generally follows the format of variable name = value, variable name = value etc.
A more descriptive / friendly trap message can be created using SNMPTT's variable substitution. Following is the same trap, logged with SNMPTT:
Feb 12 13:37:13 server11 TRAPD: .1.3.6.1.4.1.232.0.3008 Normal "XLOGONLY" server08 - Logical Drive Status Change: Status is now rebuilding
The definition for the cpqDa3LogDrvStatusChange trap in the SNMPTT configuration file would be defined as follows:
FORMAT Logical Drive Status Change: Status is now $3.
The $3 represents the third variable as defined in the MIB file, which for this particular trap, is the cpqDaLogDrvStatus variable.
Another example of an SNMPTT configuration entry is:
FORMAT Compaq Drive Array Spare Drive on controller $4, bus $5, bay $6 status is $3.
Which could result in the following output:
"Compaq Drive Array Spare Drive on controller 3, bus 0, bay 3 status is Failed."
Snmptt can log to any of the following destinations: text log, syslog, NT Event log or a SQL database. External programs can also be run to pass the translated trap to an email client, paging software etc.
In addition to variable substitution, SNMPTT allows complex configurations allowing:
SNMPTT can be downloaded from http://www.sourceforge.net/projects/snmptt
UCD-SNMP v4.2.3 appears to have a bug that prevents it from working correctly with SNMPTT. UCD-SNMP v4.2.1, v4.2.4 and v4.2.5 appear to work fine. If you need to use UCD-SNMP v4.2.3 (for example you are using the Compaq supplied UCD-SNMP package) then you should at the very least use the snmptrapd program from v4.2.5. Net-SNMP does not have this problem.
Note:
SNMPTT only requires the Net-SNMP Perl module if you want to have variable names translated into symbolic form, want to be able to have snmptrapd pass traps using symbolic form, or you enable the options translate_integers, translate_trap_oid or translate_oids.
To upgrade from v0.8 to v0.9, you should:
To upgrade from v0.7 to v0.8, you should:
To upgrade from v0.6 to v0.7, you should:
To upgrade from v0.5 to v0.6, you should:
To upgrade from v0.1, v0.2 to v0.3 to v0.4, you should:
To upgrade from v0.1, v0.2 to v0.3 to v0.4, you should:
1. Read this entire file to understand how snmptt works
2. Copy snmptt to /usr/sbin/ and ensure it is executable (chmod +x snmptt)
3. Copy snmptthandler to /usr/sbin/ and ensure it is executable (chmod +x snmptthandler)
4. Copy snmptt.ini to /etc/snmp/ or /etc/ and edit the options inside the file.
5a. For standlone mode: Modify the Net-SNMP snmptrapd.conf file by adding the following line:
traphandle default /usr/sbin/snmptt5b. For daemon mode: Modify the Net-SNMP snmptrapd.conf file by adding the following line:Note: It is possible to configure snmptrapd to execute snmptt based on the specific trap received, but using the default option is preferred
traphandle default /usr/sbin/snmptthandlerLaunch snmptt using:
snmptt --daemon
A startup script is included which can be used to start and stop snmptt on Mandrake, RedHat and other systems. Copy the script to the init.d directory (renaming the file during the copy):6. Start snmptrapd using the command line: snmptrapd -On.cp snmptt.init.d to /etc/rc.d/init.d/snmptt
Add the service using chkconfig:
chkconfig --add snmptt
Configure the service to start at runlevel 2345:
chkconfig --level 2345 snmptt on
Snmptt will be started at the next reboot, or can be started immediately with:
service snmptt start
or
/etc/rc.d/init.d/snmptt start
As an alternative, you can edit your snmp.conf file to include the line: printNumericOids 1. This setting will take effect no matter what is used on the command line.
7. See the Securing SNMPTT section.
1. Read this entire file to understand how snmptt works
2. Create the directory c:\snmp and copy snmptt and snmptthandler to it.
3. Copy snmptt.ini-nt to %SystemRoot%\snmptt.ini (c:\winnt\snmptt.ini) and edit the options inside the file.
4a. For standlone mode: Modify the Net-SNMP snmptrapd.conf file by adding the following line:
traphandle default perl c:\snmp\snmptt
Note: It is possible to configure snmptrapd to execute snmptt based on the specific trap received, but using the default option is preferred4b. For daemon mode: Modify the Net-SNMP snmptrapd.conf file by adding the following line:
traphandle default perl c:\snmp\snmptthandler5. Start SNMPTRAPD using the command line: SNMPTRAPD -On.Launch snmptt using:
snmptt --daemon
Windows NT EventLog:
If you have enabled Windows Event Log support, then you must install an Event Message File to prevent "Event Message Not Found" messages from appearing in the Event Log. Microsoft Knowledge Base article KB166902 contains information on this error.
The Event Message File is a binary DLL file. Included with SNMPTT is a pre-compiled binary. To compile the DLL yourself, see 'Compiling' below.
To install the DLL:
Windows NT Service:
To configure SNMPTT as a service under Windows NT, follow these steps. More information can be obtained from the Windows NT Resource Kit.
1. Install the Windows NT resource kit
2. Copy the srvany.exe program to c:\winnt\system32 from c:\Program Files\Resource Kit *
3. Install the SNMPTT service using:
instsrv SNMPTT c:\winnt\system32\srvany.exe
4. Configure the service:
Launch REGEDIT
Go to HKLM\SYSTEM\CurrentControlSet\SNMPTT
Create a key called: Parameters
Inside of Parameters, create a Sting Value (REG_SZ) called Application with the value of: c:\perl\bin\perl.exe
Inside of Parameters, create a Sting Value (REG_SZ) called AppParameters with the value of: c:\snmp\snmptt --daemon
5. Start the service from the control panel, or from a command prompt, type: net start snmptt
To remove the service, type: instsrv SNMPTT removeAs mentioned throughout this document, configuration options are set by editing the snmptt.ini file.
For Linux / Unix, the snmptt.ini file should be located in either /etc/snmp/ or /etc/. SNMPTT will search for the the file in both directories in that order.
For Windows, the file should be in %SystemRoot%\. For example, c:\winnt.
The location of the ini file can be set on the command line using the -ini= parameter. See Command Line Arguments.
A sample snmptt.ini is provided in this package. For Windows NT, be sure to copy the snmptt.ini-nt file to %SystemRoot%\snmptt.ini. For Windows NT, be sure to remove the -nt from the end of the filename.
This
readme file does not document all configuration options available in
the snmptt.ini as the snmptt.ini file
contains detailed descriptions for each option.
Translated traps can be sent to standard output and to a log file. The output format is:
date trap-oid severity category hostname translated-trapTo configure standard output or regular logging, edit the snmptt.ini file and modify the following variables:
enable_stdout
enable_log
log_file
Translated traps can also be sent to syslog. The format of the entries will be similar to above without the date (as syslogd logs the date):
trap-oid severity category hostname translated-trapSyslog entries normally start with: date hostname snmptt[pid]:
syslog_enableSNMPTT system errors can be sent to syslog by editing the snmptt.ini file and modifying the following variables:
syslog_facility
syslog_level
syslog_system_enableSyslog system entries normally start with: date hostname snmptt-sys[pid]:
syslog_system_facility
syslog_system_level
SNMPTT (version) started (*)
Unable to enter spool dir x (*)
Unable to open spool dir x (*)
Unable to read spool dir x (*)
Could not open trap file x (*)
Unable to delete trap file x from spool dir (*)
Unable to delete !reload file spool dir (*)
Reloading configuration file(s) (*)
SNMPTT (version) shutdown (*)
Loading snmpttconfigfile (*)
Could not open configuration file: snmpttconfigfile(*)
Finished loading x lines from snmpttconfigfile (*)
MySQL error: Unable to connect to database
Can not open log file logfile
MySQL error: Unable to perform INSERT INTO
DBI DBD::ODBC error: Unable to perform INSERT INTO
PostgreSQL error: Unable to connect to database
PostgreSQL error: Unable to perform INSERT INTO
* (daemon mode only)
Translated traps can also be sent to the NT EventLog. All traps are logged under EventID 2 under the source SNMPTT. The format of the entries will be similar to above without the date (as the Event Log logs the date):
trap-oid severity category hostname translated-trapTo configure eventlog support, edit the snmptt ini file and modify the following variables:
eventlog_enableSNMPTT system errors can be sent to the Event Log by editing the snmptt.ini file and modifying the following variables:
eventlog_type
eventlog_system_enableThe following errors are logged. Note that each error contains a unique EventID:
EventID 0: SNMPTT (version) started (*)Note:
EventID 3: Unable to enter spool dir x (*)
EventID 4: Unable to open spool dir x (*)
EventID 5: Unable to read spool dir x (*)
EventID 6: Could not open trap file x (*)
EventID 7: Unable to delete trap file x from spool dir (*)
EventID 20: Unable to delete !reload file spool dir (*)
EventID 8: Reloading configuration file(s) (*)
EventID 1: SNMPTT (version) shutdown (*)
EventID 9: Loading snmpttconfigfile (*)
EventID 10: Could not open configuration file: snmpttconfigfile(*)
EventID 11: Finished loading x lines from snmpttconfigfile(*)
EventID 12: MySQL error: Unable to connect to database
EventID 13: SQL error: Unable to connect to DSN dsn
EventID 14: Can not open log file logfile
EventID 15: MySQL error: Unable to perform INSERT INTO
EventID 16: DBI DBD::ODBC error: Unable to perform INSERT INTO
EventID 17: Win32::ODBC error: Unable to perform INSERT INTO
EventID 18: PostgreSQL error: Unable to connect to database
EventID 19: PostgreSQL error: Unable to perform INSERT INTO
* (daemon mode only)
To prevent "Event Message Not Found" messages in the Event Viewer, an Event Message File must be used. For information on installing the message file, see the Installation section for Windows.
Translated traps can also be sent to a database. MySQL (tested under Linux), PostgreSQL (tested under Linux) and ODBC (tested under Windows NT) can be used.
To configure SNMPTT for MySQL, modify the following variables in the snmptt.ini file.
mysql_dbi_enableNote: Sample values are defined in the default ini file.
mysql_dbi_host
mysql_dbi_port
mysql_dbi_database
mysql_dbi_table
mysql_dbi_username
mysql_dbi_password
The following MySQL script can create the database and table. Permissions etc should also be defined.
CREATE DATABASE snmptt;All variables are INSERTted into the database as standard text including the date and time. The variable lengths I have chosen above should be sufficient, but they may need to be increased depending on your environment.
USE snmptt;DROP TABLE snmptt;;
CREATE TABLE snmptt (
eventname VARCHAR(50),
eventid VARCHAR(50),
trapoid VARCHAR(100),
enterprise VARCHAR(100),
community VARCHAR(20),
hostname VARCHAR(100),
agentip VARCHAR(16),
category VARCHAR(20),
severity VARCHAR(20),
uptime VARCHAR(20),
traptime VARCHAR(30),
formatline VARCHAR(255));
To configure SNMPTT for PostgreSQL, modify the following variables in the snmptt.ini file.
postgresql_dbi_enableNote: Sample values are defined in the default ini file.
postgresql_dbi_module
postgresql_dbi_hostport_enable
postgresql_dbi_host
postgresql_dbi_port
postgresql_dbi_database
postgresql_dbi_table
postgresql_dbi_username
postgresql_dbi_password
The following PostgreSQL script can create the database and table (not tested!). Permissions etc should also be defined.
su - postgresAll variables are INSERTted into the database as standard text including the date and time. The variable lengths I have chosen above should be sufficient, but they may need to be increased depending on your environment.
createdb snmptt
psql snmptt
DROP TABLE snmptt;
CREATE TABLE snmptt (
eventname VARCHAR(50),
eventid VARCHAR(50),
trapoid VARCHAR(100),
enterprise VARCHAR(100),
community VARCHAR(20),
hostname VARCHAR(100),
agentip VARCHAR(16),
category VARCHAR(20),
severity VARCHAR(20),
uptime VARCHAR(20),
traptime VARCHAR(30),
formatline VARCHAR(255));
\q
SNMPTT can access ODBC data sources using either the DBD::ODBC module on Linux and Windows, or the WIN32::ODBC module on Windows.
To configure SNMPTT for ODBC access using the module DBD::ODBC, modify the following variables in the snmptt script.
dbd_odbc_enable = 1;Note:
dbd_odbc_dsn = 'snmptt';
dbd_odbc_table = 'snmptt';
dbd_odbc_username = 'snmptt';
dbd_odbc_password = 'password';
The following MS SQL Server script can create the table inside an existing database. Permissions etc should also be defined.
CREATE TABLE snmptt (All variables are inserted into the database using 'INSERT INTO' as text including the date and time. The variable lengths I have chosen above should be sufficient, but they may need to be increased depending on your environment.
eventname character(50) NULL,
eventid character(50) NULL,
trapoid character(100) NULL,
enterprise character(100) NULL,
community character(20) NULL,
hostname character(100) NULL,
agentip character(16) NULL,
category character(20) NULL,
severity character(20) NULL,
uptime character(20) NULL,
traptime character(30) NULL,
formatline character(255) NULL)
SNMPTT can access ODBC data sources using either the DBD::ODBC module on Linux and Windows, or the WIN32::ODBC module on Windows.
To configure SNMPTT for MS SQL via ODBC on Windows, modify the following variables in the snmptt script.
sql_win32_odbc_enable = 1;Note:
sql_win32_odbc_dsn = 'snmptt';
sql_win32_odbc_table = 'snmptt';
sql_win32_odbc_username = 'snmptt';
sql_win32_odbc_password = 'password';
The following MS SQL Server script can create the table inside an existing database. Permissions etc should also be defined.
CREATE TABLE snmptt (All variables are inserted into the database using 'INSERT INTO' as text including the date and time. The variable lengths I have chosen above should be sufficient, but they may need to be increased depending on your environment.
eventname character(50) NULL,
eventid character(50) NULL,
trapoid character(50) NULL,
enterprise character(50) NULL,
community character(20) NULL,
hostname character(100) NULL,
agentip character(16) NULL,
category character(20) NULL,
severity character(20) NULL,
uptime character(20) NULL,
traptime character(30) NULL,
formatline character(255) NULL)
Logging of non recognized traps is also possible. This would be used mainly for troubleshooting purposes.
To configure unknown trap logging, edit the snmptt.ini file and modify the following variables:
enable_unknown_trap_log
unknown_trap_log_file
An external program can be launched when a trap is received. The command line is defined in the configuration file. For example, to send a page using QPAGE (http://www.qpage.org), the following command line could be used:
qpage -f TRAP notifygroup1 "$r $x $X Compaq Drive Array Spare Drive on controller $4, bus $5, bay $6 status is $3."$r is translated to the hostname, and $x is the current date, and $X is the current time (described in detail below)
To enable or disable the execution of EXEC definitions, edit the snmptt.ini file and modify the following variable:
exec_enable
SNMPTT can be run in two modes: standalone mode and daemon mode.
To use SNMPTT in standalone mode, the snmptrapd.conf file would contain a traphandle statement such as:
traphandle default /usr/sbin/snmpttWhen a trap is received by SNMPTRAPD, the trap is passed to the /usr/sbin/snmptt script. SNMPTT performs the following tasks:
Running SNMPTT without the --daemon command line option will result standalone mode unless the mode variable in the snmptt.ini file is set to daemon. For standalone mode, the mode variable in the snmptt.ini file should be set standalone.
Note: Enabling the UCD-SNMP / Net-SNMP Perl module will greatly increase the startup time of SNMPTT. Daemon mode is recommended.
When SNMPTT is run in daemon mode, the snmptrapd.conf file would contain a traphandle statement such as:
traphandle default /usr/sbin/snmptthandlerWhen a trap is received by SNMPTRAPD, the trap is passed to the /usr/sbin/snmptthandler script. SNMPTTHANDLER performs the following tasks:
Running SNMPTT with the --daemon command line option or setting the mode variable in the snmptt.ini file to daemon will cause SNMPTT to run in daemon mode.
By setting the .ini variable use_trap_time to 1 (default), the date and time used for logging etc will be the date and time passed inside the trap spool file. If use_trap_time is set to 0, the date and time that the trap was processed by SNMPTT is used. Setting use_trap_time to 0 can result in inaccurate time stamps in log files due to the length of time SNMPTT sleeps between spool directory polling.
Note: When running on a non Windows platform, SNMPTT will fork to the background and create a pid file in /var/run/snmptt.pid if daemon_fork is set to 1. If the user is not able to create the /var/run/snmptt.pid file, it will attempt to create one in the current working directory.
Sending the HUP signal to SNMPTT when running as a daemon will cause it to reload the configuration file including the .ini file, any snmptt.conf files listed in the .ini file and any NODES files if dynamic_nodes is disabled. A reload can also be forced by adding a file to the spool directory called !reload. The filename is not case sensitive. If this file is detected, it will flag a reload to occur and will delete the file. This would be the only way to cause a reload when using Windows.
The following command line arguments are supported:
Usage:
snmptt [<options>]
Options:
--daemon
Start in daemon mode
--debug=n
Set debug level (1 or 2)
--debugfile=filename Set debug output
file
--dump
Dump (display) defined traps
--help
Display this message
--ini=filename
Specify path to snmptt.ini file
--version
Display author and version information
--time
Use to see how long it takes to load and
process trap file (eg: time snmptt --time)
The configuration file (usually /etc/snmp/snmptt.conf or c:\snmp\snmptt.conf) contains a list of all the defined traps.
If your snmptt.conf file is getting rather large and you would like to divide it up into many smaller files, then do the following:
-create additional snmptt.conf files
-add the file names to the snmptt_conf_files section in the snmptt.ini
file.
For example:
snmptt_conf_files = <<ENDThe syntax of the snmptt.conf file is:
/etc/snmp/snmptt.conf.generic
/etc/snmp/snmptt.conf.compaq
/etc/snmp/snmptt.conf.cisco
/etc/snmp/snmptt.conf.hp
/etc/snmp/snmptt.conf.3com
END
EVENT event_name event_OID "category" severityFORMAT format_string
[EXEC command_string]
[NODES sources_list]
[MATCH [MODE=[or | and]] | [$n:[!][( ) | n | n-n | > n | < n | x.x.x.x | x.x.x.x-x.x.x.x | x.x.x.x/x]]
[REGEX ( )( )[i][g][e]]
[SDESC]
[EDESC]Note: Lines starting with a # will be ignored.
Note: The EVENT and FORMAT line are REQUIRED. Commands in [] are optional. Do NOT include the []s in the configuration file!
EVENT:
EVENT event_name event_OID "category" severityevent_name:
Unique text label (alias) containing no spaces. This would match the name on the TRAP-TYPE or NOTIFICATION-TYPE line in the MIB file when converted using snmpttconvertmib.event_OID:Object identifier string in dotted format or symbolic notation containing no spaces.category:For example, a Compaq (enterprise 1.3.6.1.4.1.232) cpqHoGenericTrap trap (trap 11001) would be written as:
1.3.6.1.4.1.232.0.11001.
Symbolic names can also be used if the UCD-SNMP / Net-SNMP Perl module is installed and enabled by setting net_snmp_perl_enable in the snmptt.ini file. For example:
linkDown
IF-MIB::linkDown
Note:
The current version of Net-SNMP (5.0.9 at the time of this writing) and everything before it does not support including the module name (eg: IF-MIB::) when translating an OID. A patch is available for 5.0.8+ that will appear in later releases of Net-SNMP (5.1). The patch is available from the contrib folder, or it can be downloaded from the Net-SNMP patch page. If the version of Net-SNMP you are using does not support this feature and the event OID is specified with the module name, the event definition will be ignored. Also note that UCD-SNMP may not properly convert symbolic names to numeric OIDs which could result in traps not being matched.SNMP V1 traps are in the format of enterprise ID (1.3.6.1.4.1.232) followed by a 0, and then followed by the trap number (11001).Wildcards in dotted format notation can also be used. For example:There can be multiple entries for the same trap OID in the configuration file. If multiple_event is enabled in the snmptt.ini, then it will process all matching traps. If multiple_event is disabled, only the first matching entry will be used.
1.3.6.1.4.1.232.1.2.*
Note:
Specific trap matches are performed before wildcards so if you have an entry for 1.3.6.1.4.1.232.1.2.5 AND 1.3.6.1.4.1.232.1.2.*, it will process the .5 trap when received even if the wildcard is defined first.Wildcard matches are only matched if there are NO exact matches. This takes into consideration the NODES list. Therefore, if there is a matching trap, but the NODES list prevents it from being considered a match, the wildcard entry will only be used if there are no other exact matches.
Character string enclosed in double "s. Used when logging output (see above).severity:If the category is "IGNORE", no action will take place even if the snmptt.conf contains FORMAT and / or EXEC statements.
If the category is "LOGONLY", the trap will be logged as usual, but the EXEC statement will be ignored.
Note: If you plan on using an external program such as Netsaint for logging, paging etc, you probably do not want any traps defined with LOGONLY as the EXEC line would never be used to submit the passive service check.
Character string of the severity of the event. Used in the output when logging. Example: Minor, Major, Normal, Critical, Warning. The snmptt.ini contains options to match the syslog level or NT Event Log type to the severity level.FORMAT:
FORMAT format_stringThere can be only one FORMAT line per EVENT.
The format string is used to generate the text that will be logged to any of the supported logging methods.
Variable substitution is performed on this string using the following variables:
$A - Trap agent host name (see Note 1)Example:
$aA - Trap agent IP address
$c - Category
$C - Trap community string
$E - Enterprise trap OID in symbolic format
$e - Enterprise trap OID in number format
$G - Generic trap number (0 if enterprise trap)
$S - Specific trap number (0 if generic trap)
$N - Event name defined in .conf file of matched entry
$i - Event OID defined in .conf file of matched entry (could be a wildcard OID)
$O - Trap OID in symbolic format (see Note 4)
$o - Trap OID in numerical format (see Note 4)
$R, $r - Hostname (see Note 1)
$aR, $ar - IP address
$s - Severity
$T - Uptime: Time since network entity was initialized
$X - Time trap was spooled (daemon mode) or current time (standalone mode)
$x - Date trap was spooled (daemon mode) or current date (standalone mode)
$# - Number of (how many) variable-bindings in the trap
$$ - Print a $
$@ - Number of seconds since the epoch
$n - Expand variable-binding n (1-n) (see Note 2,5)
$+n - Expand variable-binding n (1-n) in the format of variable name:value (see Note 2,3,5)
$-n - Expand variable-binding n (1-n) in the format of variable name (variable type):value (see Note 2,3,5)
$vn - Expand variable name of the variable-binding n (1-n)(see Note 3)
$* - Expand all variable-bindings (see Note 5)
$+* - Expand all variable-bindings in the format of variable name:value (see Note 2,3,5)
$-* - Expand all variable-bindings in the format of variable name (variable type):value (see Note 2,3,5)FORMAT NIC switchover to slot $3, port $4 from slot $5, port $6
Note:
For the text log file, the output will be formatted as:Note (1):date time trap-OID severity category hostname - formatFor all other log files except MySQL, DBD::ODBC and Win32::ODBC, the output will be formatted as:trap-OID severity category hostname - formatFor MySQL, DBD::ODBC and Win32::ODBC, the formatline column will contain only the format text.See the section 'Name Resolution / DNS' for important DNS information.Note (2):If translate_integers is enabled in the snmptt.ini file, SNMPTT will attempt to convert integer values received in traps into text by performing a lookup in the MIB file.Note (3):
You must have the UCD-SNMP / Net-SNMP Perl module installed for this to work and you must enable support for it by enabling net_snmp_perl_enable in the snmptt.ini file.
For this feature to work, you must ensure UCD-SNMP / Net-SNMP is configured correctly with all the required MIBS. If the option is enabled, but the value can not be found, the integer value will be used. If the MIB files are present, but translations do not occur, ensure UCD-SNMP / Net-SNMP is correctly configured to process all the required mibs. This is configured in the UCD-SNMP / Net-SNMP snmp.conf file. Alternatively, you can try setting the mibs_enviroment variable in snmptt.ini to ALL (no quotes) to force all MIBS to be initialized at SNMPTT startup.
If translate_integers is enabled while using stand-alone mode, it may take longer to process each trap due to the initialization of the MIB files.
$vn, $+n and $-n variable names and variable type are translated into the text name by performing a lookup in the MIB file. You must have the UCD-SNMP / Net-SNMP Perl module installed for this to work and you must enable support for it by enabling net_snmp_perl_enable in the snmptt.ini file. If net_snmp_perl_enable is not enabled, the $vn variable will be replaced with the text 'variablen' where n is the variable number (1+).For the name translation to work, you must ensure UCD-SNMP / Net-SNMP is configured correctly with all the required MIBS. If the option is enabled and the correct name is not returned, ensure UCD-SNMP / Net-SNMP is correctly configured to process all the required mibs. This is configured in the UCD-SNMP / Net-SNMP snmp.conf file. Alternatively, you can try setting the mibs_enviroment variable in snmptt.ini to ALL (no quotes) to force all MIBS to be initialized at SNMPTT startup.
Note (4):
If translate_trap_oid is enabled in the snmptt.ini file, SNMPTT will attempt to convert the numeric OID of the received trap into symbolic form such as IF-MIB::linkDown. You must have the UCD-SNMP / Net-SNMP Perl module installed for this to work and you must enable support for it by enabling net_snmp_perl_enable in the snmptt.ini file. If net_snmp_perl_enable is not enabled, it will default to using the numeric OID.
The current version of Net-SNMP (5.0.8 at the time of this writing) and everything before it does not support including the module name (eg: IF-MIB::) when translating an OID and most of the 5.0.x versions do not properly tranlsate numeric OIDs to long symbolic names. A patch is available for 5.0.8+ that will appear in later releases of Net-SNMP (5.1). The patch is available from the contrib folder, or it can be downloaded from the Net-SNMP patch page.
Note (5):
If translate_oids is enabled in the snmptt.ini file, SNMPTT will attempt to convert any numeric OIDs found inside the variables passed inside the trap to symbolic form. You must have the UCD-SNMP / Net-SNMP Perl module installed for this to work and you must enable support for it by enabling net_snmp_perl_enable in the snmptt.ini file. If net_snmp_perl_enable is not enabled, it will default to using the numeric OID.
The current version of Net-SNMP (5.0.9 at the time of this writing) and everything before it does not support including the module name (eg: IF-MIB::) when translating an OID and most of the 5.0.x versions do not properly tranlsate numeric OIDs to long symbolic names. A patch is available for 5.0.8+ that will appear in later releases of Net-SNMP (5.1). The patch is available from the contrib folder, or it can be downloaded from the Net-SNMP patch page.EXEC:
[EXEC command_string]There can be multiple EXEC lines per EVENT.
Optional string containing command to execute when trap is received and arguments to pass to the program. The EXEC lines are executed in the order that they appear.
EXEC uses the same variable substitution as the FORMAT line.
Example:
EXEC /usr/bin/qpage -f TRAP alex "$r: $x $X - NIC switchover to slot $3, port $4 from slot $5, port $6"orEXEC c:\snmp\pager netops "$r: $x $X - NIC switchover to slot $3, port $4 from slot $5, port $6"Note: Unlike the FORMAT line, nothing is prepended to the message. If you would like to include the hostname and date in the page above, you must use the variables such as $r, $x and $X.NODES:
[NODES sources_list]Used to limit which devices can be mapped to this EVENT definition.
There can be multiple NODES lines per EVENT.Optional string containing any combination of host names, IP addresses, CIDR network address, network IP address ranges, or a filename. If this keyword omitted then ALL sources will be accepted. Each entry is checked for a match. As soon as one match occurs, searching stops.
For example, if you only wanted devices on the subnet 192.168.1.0/24 to trigger this EVENT, you could use a NODES entry of:
NODES 192.168.1.0/24
If a filename is specified, it must be specified with a full path.
Example 1:
This example will match any hosts called fred, barney, betty or wilma:Example 2:
NODES fred barney betty wilma
This example will load the file /etc/snmptt-nodes (see below), and match any hosts called fred, barney, betty, network ip addresses 192.168.1.1, 192.168.1.2, 192.168.1.3, 192.168.2.1, network range 192.168.50.0/22 or network range 192.168.60.0-192.168.61.255:Example 3:
NODES /etc/snmptt-nodes
This example will load both files /etc/snmptt-nodes and /etc/snmptt-nodes2 (see above example):Example 4:
NODES /etc/snmptt-nodes /etc/snmptt-nodes2
NODES 192.168.4.0/22 192.168.60.0-192.168.61.255 /etc/snmptt-nodes2Example 5:
NODES fred /etc/snmptt-nodes pebbles /etc/snmptt-nodes2 barneywhere snmptt-nodes contains:fredNotes:
barney betty
# comment lines
192.168.1.1 192.168.1.2 192.168.1.3
192.168.2.1
192.168.50.0/22
192.168.60.0-192.168.61.255
wilmaThe names are NOT case sensitive and comment lines are permitted by starting the line with a #.
CIDR network addresses must be specified using 4 octets followed by a / followed by the number of bits. For example: 172.16.0.0/24. Using 172.16/24 will NOT work.
Do not use any spaces between network ranges as they will be interpreted as two different values. For example, 192.168.1.1 - 192.168.1.20 will not work. Use 192.168.1.1-192.168.1.20 instead.
By default, NODES files are loaded when the snmptt.conf files are loaded (during startup of SNMPTT). The snmptt.ini option dynamic_nodes can be set to 1 to have the nodes files loaded each time an EVENT is processed.
See the section 'Name Resolution / DNS' for important DNS information.
MATCH:
[MATCH [MODE=[or | and]] | [$n:[!][( )[i] | n | n-n | > n | < n | x.x.x.x | x.x.x.x-x.x.x.x | x.x.x.x/x]]Optional match expression that must be evaluated to true for the trap to be considered a match to this EVENT definition.
If a MATCH statement exists, and no matches evaluate to true, then the default will be to NOT match this EVENT definition.
The following Perl regular expression modifiers are supported:
i - ignore case when trying to matchThe following command formats are available:
MATCH MODE=[or | and]
MATCH $n: [!] (reg) [i]
MATCH $n: [!] n
MATCH $n: [!] n-n
MATCH $n: [!] < n
MATCH $n: [!] > n
MATCH $n: [!] x.x.x.x
MATCH $n: [!] x.x.x.x-x.x.x.x
MATCH $n: [!] x.x.x.x/x
where:
or or and set the default evaluation mode for ALL matches
$n is the enterprise variable starting with $1
reg is a regular expression
! is used to negate the result (not)
n is a number
x.x.x.x is an IP address
x.x.x.x-x.x.x.x is an IP network address range
x.x.x.x/x is an IP CIDR network addresss
REGEX:
[REGEX( )( )[i][g][e]]Optional regular expression to perform a search and replace on the translated FORMAT / EXEC line. Multiple REGEX ( )( ) lines are permitted.
First ( ) contains the search expression.
Second ( ) contains the replacement textThe following Perl regular expression modifiers are supported:
i - ignore case when trying to match left side
g - replace all occurances instead of only the first
e - execute the right side (eval) as codeTo use substitution with captures (memory parenthesis) or the e modifier, you must first enable support in the snmptt.ini file by setting allow_unsafe_regex to 1. Note: This is considered unsafe because the contents of the right expression is executed (eval) by Perl which could contain unsafe code. If this option is enabled, BE SURE THAT THE SNMPTT CONFIGURATION FILES ARE SECURE!
Each REGEX line is processed in order from top to bottom and are accumulative. The second REGEX operates on the results of the first REGEX etc.
Example:
FORMAT line before: UPS has detected a building alarm. Cause: UPS1 Alarm #14: Building alarm 3.
REGEX (Building alarm 3)(Computer room high temperature)
REGEX (Building alarm 4)(Moisture detection alarm)
REGEX (roOm)(ROOM)ig
REGEX (UPS)(The big UPS)
REGEX (\s+)( )g
FORMAT line after: The big UPS has detected a building alarm. Cause: UPS1 Alarm #14: Computer ROOM high temperatureTo use parentheses ( or ) in the search expression, they must be backslashed (\) otherwise it is interpreted as a capture (see below). The replacement text does not need to be backslashed.
Example:
FORMAT line before: Alarm (1) and (2) has been triggered
REGEX (\(1\))(One)
REGEX (\(2\))((Two))
FORMAT line after: Alarm One and (Two) has been triggered
Example:
Example 1:
Example 2:
Example 3
Example 4This example is a sample of using Perl subroutines inside of a REXEC statement.
FORMAT line before: Extremely severe error has occured
REGEX (Extremely severe error has occured)(("Better get a lotto ticket!! Here is a lotto number to try:".sprintf ("%s", lottonumber());sub lottonumber { for(my $i=0;$i<6;$i++) { $temp = $temp . " " . (int(rand 49) +1); } return $temp; } )ie
FORMAT line after: Better get a lotto ticket!! Here is a lotto number to try: 36 27 38 32 29 6
Note: The REGEX expression is executed on the final translated FORMAT / EXEC line, after all variable substitutions have been completed.
SDESC
[SDESC]Optional start of a description. All text between this line and the line EDESC will be ignored by snmptt. This section can be used to enter comments about the trap for your own use. If you use a SDESC, you MUST follow with a EDESC.
EDESC
[EDESC]Used to end the description section.
Example:
SDESC
Trap used when power supply fails in
a server.
EDESC
When there are multiple definitions of the same trap in the configuration file, the following rules apply:
A match occurs when:
Snmptrapd passes the IP address of the device sending the trap (host), the
host name of the device sending the trap (host) (if configured to resolve host
names) and the IP address of the actual SNMP agent (agent).
If the configuration setting dns_enable is set to 0 (dns disabled), then the
host name of the AGENT will not be available for the $A variable, NODES
matches, and the hostname column in SQL databases. The only exception to
this is if the (host) IP address matches the (agent) IP address and snmptrapd
is configured to resolve host names. In that case, the host name of the
(host) will be used for the (agent) host name as they are obviously the same
host.
If the configuration setting dns_enable is set to 1 (dns enabled), then the
host name of both the host and the AGENT will be resolved via DNS. NODES
entries will also be resolved to IP addresses before performing matches.
The host name may resolve to the Fully Qualified Domain Name (FQDN). For
example: barney.bedrock.com. Adding an entry for the host in your
/etc/hosts file or %systemroot%\system32\drivers\etc\hosts may result in the
short name being used instead (barney). You can also enable the
strip_domain / strip_domain_list options to have SNMPTT strip
the domain of any FQDN host. See the snmptt.ini file for details.
Note: The examples folder also contains a sample snmptt.conf file.
Following is a sample of two defined traps in snmptt.conf:
#
EVENT COMPAQ_11003 .1.3.6.1.4.1.232.0.11003 "LOGONLY" Normal
FORMAT Compaq Generic Trap: $*
EXEC qpage -f TRAP notifygroup1 "Compaq Generic Trap: $*"
NODES /etc/snmp/cpqnodes
SDESC
Generic test trap
EDESC
#
#
EVENT cpqDa3AccelBatteryFailed .1.3.6.1.4.1.232.0.3014 "Error Events" Critical
FORMAT Battery status is $3.
EXEC qpage -f TRAP notifygroup1 "$s $r $x $X: Battery status is $3"
NODES ntserver1 ntserver2 ntserver3
#
#
Following is a sample of a list of files to load in snmptt.ini:
snmptt_conf_files = <<ENDFollowing is a sample of one defined traps in /etc/snmp/snmptt-compaq.conf:
/etc/snmp/snmp-compaq.conf
/etc/snmp/snmp-compaq-hsv.conf
END
#Following is a sample of one defined traps in /etc/snmp/snmptt-compaq-hsv.conf:
EVENT COMPAQ_11003 .1.3.6.1.4.1.232.0.11003 "LOGONLY" Normal
FORMAT Compaq Generic Trap: $*
EXEC qpage -f TRAP notifygroup1 "Compaq Generic Trap: $*"
NODES /etc/snmp/cpqnodes
SDESC
Generic test trap
EDESC
#
#
EVENT mngmtAgentTrap-16025 .1.3.6.1.4.1.232.0.136016025 "Status Events" Normal
FORMAT Host $1 : SCellName-TimeDate $2 : EventCode $3 : Description $4
EXEC qpage -f TRAP notifygroup1 "Host $1 : SCellName-TimeDate $2 : EventCode $3 : Description $4"
SDESC
"Ema EMU Internal State Machine Error [status:10]"
EDESC
#
An existing HP Openview trapd.conf can be used in most cases but the file must be a VERSION 3 file. SNMPTT does not support all the variables implemented in HPOV, but most are available. The following variables may or may not match exactly to HPOV: $O, $o, $r, $ar, $R, $aR.
Some vendors (such as Compaq and Cisco ) provide a file that can be imported in to HP Openview using an HP Openview utility. Snmpttconvert can be used to convert the file to snmptt.conf format.
Some vendors provide a MIB file that contains TRAP or NOTIFICATION
definitions. Snmpttconvertmib
can be used to convert the file to snmptt.conf format.
With a 450 Mhz PIII and a 9000 line snmptt.conf containing 566 unique traps (EVENTs), it takes under a second to process the trap including logging and executing the qpage program. The larger the snmptt.conf file is, the longer it will take to process. If there are a large number of traps being received, daemon mode should be used. If it takes 1 second to process one trap, then obviously you shouldn't try to process more than one trap per second.
Note: Enabling the UCD-SNMP / Net-SNMP Perl module will greatly increase the startup time of SNMPTT. Daemon mode is recommended.
The SNMPTRAPD program blocks when executing traphandle commands. This means that if the program called never quits, SNMPTRAPD will wait forever. If a trap is received while the traphandler is running, it is buffered and will be processed when the traphandler finishes. I do not know how large this buffer is.
The program called by SNMPTT (EXEC) blocks SNMPTT. If you call a
program that does not return, SNMPTT will be left waiting. In standalone
mode, this would cause snmptrapd to wait forever also.
Please send me any comments - good or bad - to alex_b@users.sourceforge.net. If you have any problems including converting trap files, please send me an email and include the file you are trying to convert and I will try to take a look at it.
Please also send any bug reports, patches or improvements so I can fix /
add them and add it to the next release. You can also use Sourceforge
for bugs and
feature requests.
Nagios / Netsaint is a system monitoring application. It monitors hosts and services and alerts you when things go wrong. The name changed from Netsaint to Nagios in July of 2002.
This section will outline the basic steps to integrate SNMPTT with Nagios. It will not attempt to explain how Nagios works. There is very good documentation available on the Nagios web page. You should be able to install and configuration Nagios before attempting to integrate it with SNMPTT. You should also have a functioning SNMPTT system that can at least log translated traps to a log file.
Passive service checks allow Nagios to process service check results that
are submitted by external applications. Using SNMPTT's EXEC statement,
the received trap can be passed to Nagios using the submit_check_result script included with
Nagios. Once received by Nagios, the trap can be logged, a page sent,
email sent etc.
One service is defined for each Nagios host that is to receive traps from SNMPTT. The benefits of using only one service entry is that it is makes it easier to set up Nagios. Trying to define every possible trap for every host you have is not recommended. For example, after converting the MIBS from Compaq, there are over 340 traps defined. Trying to define this for every Compaq server would not be a good idea as 40 servers * 340 traps = 13,600 service definitions.
The downside of using only one service entry is that you will only see the
last trap that was recieved on the Nagios console. Each received trap
will be logged, emailed, paged etc but the console will only show the last one
as being in the warning or critical state. The service will remain in this state until you
manually force a service check. See Clearing received traps in Nagios
below.
When defining the service for receiving the SNMPTT translated trap, the service must be defined as volatile. When a service is changed from an OK state to a non-OK state, contacts are notified etc. Normally, a service is Nagios is NOT defined volatile which means if another service check is performed and the state is STILL non-OK then NO contacts are notified. Because there is only one service entry for the SNMP traps, we need to make sure we are contacted every time a trap comes in.
Following is a sample service entry for Nagios. The Description column should not be included in the definition.
define service{ | Description |
||
host_name | server01 | Name of host |
|
service_description | TRAP | Name of service. What you use here must match the same value for the submit_check_result script | |
is_volatile | 1 | Enables volatile services | |
check_command | check-host-alive | Used to reset the status
to OK when 'Schedule an immediate check of
this service' is selected. |
|
max_check_attempts | 1 |
Leave as 1. | |
normal_check_interval | 1 |
Leave as 1. | |
retry_check_interval | 1 |
Leave as 1. | |
passive_checks_enabled | 1 | Enables passive
checks |
|
check_period | none | When this servcie can be checked. Because it is a passive service, it never needs to be automatically checked | |
notification_interval | 31536000 | Notification
interval. Set to a very high number to prevent you from getting pages of
previously received traps (1 year - restart Nagios at least once a year! - do
not set to 0!). |
|
notification_period | 24x7 | When you can be notified. Can be changed | |
notification_options | w,u,c,r | Notify on warning,
unknown, critical and recovery |
|
notifications_enabled | 1 |
Enable
notifications |
|
contact_groups | cg_core | Name of contact group to notify | |
} |
Following is a sample service entry for Netsaint.
service[server01]=TRAP;1;none;1;1;1;cg_core;31536000;24x7;0;1;1;;check-host-alive
See the Netsaint documentation and the Nagios table above for an explanation of the various fields.