Mon -- King of Simplicity among Unix Monitoring Packages

Mon ( mon download SourceForge.netand kernel.org page) is the king of simplicity among monitoring products that we reviewed here. It was written by Jim Trocki and released under GPL.

Active development slowed considerably after 2001. Still there have been several new features added and bugs fixed in mon 1.2.0.

The author wisely resisted the overcomplexity trap and mon remains simple and capable product with the transparent code base, the codebase that can be easily understood by wide audience of system administrators. In one of his posts David Nolan aptly stated:

My best summary of Mon is that its monitoring for sysadmins. Its not pretty, its not designed for management, its designed to allow a sysadmin to automate the performance monitoring that might otherwise be done ad-hoc or with cron jobs.

It doesn't trivially provide the typical statistics gathering that many bean-counters are looking for, but its extensible and scalable in amazing ways. (See recent posts on this list about one company deploying a network of 2400 mon servers and 1200 locations, and my mon site which runs 500K monitoring tests a day, some of those on hostgroups with hundreds of hosts.)

This is really a quintessential monitoring package "by sysadmin for sysadmins" -- simple, very flexible (each probe can be heavily modified to suit your needs without much digging into documentation) but functional enough to provide great value. And in his reply to the David Nolan post the author of the mon stated:

... the design of mon is extremely flexible, and was purposefully built the way it was in order to leverage other tools which already exist. it follows the traditional Unix design philosophy, which i think is the most elegant system design in existence to this very day. it is all about having a mechanism to connect together lots of smaller tools which do one job very well in order to solve larger problems, rather than writing a large tool for each new problem.

you can also think of this design in terms of using natural language, words and grammar to phrase something you want to say. perl itself also follows this model. larry wall is a linguist (a cunning one at that, sorry couldn't resist the pun), and he applied that to perl.

for example, mon leverages fping, the net-snmp tools, traceroute, rrdtool, etc. another example of mon's flexibility is how an on-call notification system with escalation was added without changing anything in mon at all, it was just a matter of writing a custom alert and plugging it in to your mon configuration file with the correct grammar.

in order to get a good idea of how mon works, i would recommend reading the slides from this presentation:

ftp://ftp.kernel.org/pub/software/admin/mon/mon-talk-0.4.tar.gz

And that fact that is written in "simple Perl" without OO perversions only increases its value. One of the first versions (0.38j) is small enough to get basic ideas without getting headache and I recommend to get it first. You should use the latest version, but to get the ideas and architecture you still need to look at code of 0.38j.

Due to its design philosophy mon remains a rather simple package and by-and-large escaped Christmas Tree effect that hunts other monitoring packages like Nagios. It's still completely written in Perl and you can study what it is doing in debugger. It is also relatively easy to understand how particular checks are programmed and that fact alone constitutes tremendous advantage as it permit easy customarization.

Mon fundamentally is a scheduler which executes the monitors (each tests a specific condition or report specific data), and calls the appropriate alerts if the monitor fails.

It views resource monitoring as two separate tasks: the testing of a condition, and triggering an action upon failure. It is important to understand that mon implements the testing and action-taking tasks as separate, stand-alone programs: invocation of an alert is governed by logic which offers various "squelch" features and dependencies, configurable by the user.

The current version is available from ftp://ftp.kernel.org/pub/software/admin/mon/devel

mon was developed under Linux, but it is known to work under Solaris. As both the clients and server are written completely in Perl portability should not be an issue. I would be very surprised if there are serious portability issues with mon. It also is included as standard vendor provided package on many Linux distributions such as Suse and Ubuntu.

Configuration

On startup, mon reads a configuration file and then performs tests as scheduled in the configuration file.

Each mon monitor (called "service monitor" in documentation) is a Perl script located in /etc/mon/mon.d  or any other directory specified in /etc/mon/mon.cf configuration file. For example Suse packages uses iether /usr/lib/mon/mod.d or /usr/lib64/mon/mod.d depending on whether you use 64-bit kernel or not.

Probes

Mon was designed to be open and extensible in the sense that Mon's monitor programs can be any executable. The relevant quote from the documentations is:

Monitor processes are invoked with the arguments specified in the configuration file, appended by the hosts from the applicable host group.

.... should return an exit status of 0 if it completed successfully (found no problems), or nonzero if a problem was detected. The first line of output from the monitor script has a special meaning: it is used as a brief summary of the exact failure which was detected, and is passed to the alert program. All remaining output is also passed to the alert program, but it has no required interpretation.

There are more then a dozen prepackaged monitors that come with mon and most can be easily customized to your own needs:

• Disk space Disk space monitoring via NFS or local mounts. Disk space monitoring is probably the first and the most important task that sysadmin need to solve.
• ICMP echo (ping) Efficient host reachability via ICMP echo, using fping, which multiplexes pings using a single process. If a group of hosts is pinged, and if more than one fails, then the collective timeout before the monitor script returns is what it would normally be if you're pinging only one host.
• SMTP SMTP connectivity is tested by connecting to the SMTP port on one or more servers, listening for a "220" response, sending a "QUIT", and waiting for a "221" response and then a shutdown.
• Telnet Waits for an actual login prompt to determine if the service is working. This uses Wietse Venema's tcp_scan utility.
• FTP FTP connectivity is tested by connecting to the FTP server, sending a "quit", and waiting for the appropriate response.
• NNTP Similar to SMTP testing, speaks the NNTP protocol.
• HTTP Speaks the HTTP protocol to fetch a URL from an HTTP server. The http_t monitor can measure page delivery latency and report failures if a minimum transfer speed is not met. There are several variants of the HTTP monitor for use in different situations.
• POP-3 Similar to SMTP, speaks POP-3 to connect and then log out.
• IMAP Similar to POP-3, speaks IMAP to connect and then log out.
• TCP-based services Any simple TCP-based service can be tested. mon comes with a "tcp.monitor" utility (which is a wrapper for Wietse Venema's tcp_monitor from the Satan distribution). This will send output to a server, and for a particular response, with a timeout.
• Uptime via Asynchronous SNMP Calculate the uptime and downtime of any SNMP-capable host. Hundreds of hosts can be polled in a few seconds using the asynchronous method.
• HP Printers via SNMP Using SNMP, hpnp.monitor polls HP printers with JetDirect interfaces and can detect paper outages, paper jams, toner cartridge problems, and more. When a failure is detected, it can report the message which is displayed on the printer's LCD, e.g. "TRAY 2 EMPTY".
• Processes via SNMP process.monitor polls the UCD SNMP agents on some number of hosts and reports problems with particular processes. This monitor is useful to detect if a particular process has died off.
• Network Appliance free space via SNMP netappfree.monitor polls Network Appliance filers for free space on each exported filesystem. It does not require that you mount the filesystems on the mon host. Per-filesystem low watermarks are configured in a separate configuration file for easy changing without resetting the mon server.
• Network Appliance quota trees via SNMP na_quote.monitor can monitor free space on multiple quota trees. Currently a work-in-progress.
• LDAP ldap.monitor searches an LDAP server for objects which match a particular filter.
• DNS dns.monitor queries a zone master for base information, and then queries one or more secondaries, comparing the serial number found in the SOA record to that of the master.
• Dial-in terminal servers dialin.monitor can periodically dial in to a terminal server or some other modem host and report whether or not a connection was successfully made to the remote host, and if a login prompt was obtained.
• mSQL and MySQL databases The msql-mysql monitor connects to the database in question and list the tables to ensure that the host is responding to queries.
• Network latency up_rtt.monitor uses the UDP or TCP echo service to measure the round-trip time between the monitoring server and a set of hosts. If the measured latency exceeds a minimum, then a failure is reported.
• Sun RPC Services The RPC monitor can call the RPC NULL function to determine if a server is responding to queries. This method can be used to poll NIS, NFS, mountd, and other popular services which use Sun RPC.
• Foundry chassis Report power supply, fan, temperature, module failures in Foundry BigIron and FastIron routers and bridges.
• Compaq Presario chassis Report power supply, fan, and temperature problems with Presario systems via the "Insight Manager" SNMP agent.
• Brocade SilkWorm FCAL switches Report PSU, fan, and temperature, PHY, and link problems.

If a test fails, Mon calls the alert as specified in the configuration file. For example (taken from version 038j):

use File::Df;

foreach (@ARGV) {
    ($path, $minavail) = split (/:/, $_, 2);

    ($fs_type, $fs_desc, $used, $avail, $fused, $favail) = df ($path);
    if (!defined ($used)) {
	push (@failures, "statfs error for $path: $!");
	next;
    }

    if ($avail < $minavail) {
    	push (@failures, sprintf ("%1.1fGB free on %s", $avail / 1024 / 1024,
	    $path));
    }
}

if (@failures) {
    print join (", ", @failures), "\n";
    exit 1;
}

exit 0;

Most are simple scripts that notify users about a particular problem via particular communication channel like example SMTP mail or instant messaging. For example mail.alert that channel alerts into SMTP mails is structured as following:

open (MAIL, "| /usr/lib/sendmail -oi -t") ||
    die "could not open pipe to mail: $!\n";
print MAIL ... 	

Alerts

Mon alerts are also Perl scripts. They are located in /etc/mon/alert.d. Several prepackaged alerts that can be easily adapted to the environment that come with mon:

• Email notification. This is the most common type of modification those days and with the popularity of Blackberries the most portable.
• Alphanumeric paging notification via modem, using QuickPage. Pagers are still used although not that much like 10 years ago.
• Alphanumeric messaging to portable devices using SNPP.
• Alphanumeric paging via email.
• Traps can be delivered to remote mon servers.

Web interface

There is also several WEB interfaces for mon.

• mon.cgi. This is the most powerful interface available. Implemented as a pretty big monolythic Perl script (152K or 3822 lines) originally written by Arthur K. Chan and for some time (Feb 2000-May 2001) maintained by Ryan Clark. The last version is mon.cgi-1.52.tar.gz dated 2001.
• bbmon.cgi (BBmon, The BB-like Mon Web interface) Simplified Web status interface for the Mon resource monitoring system. bbmon.cgi was written to offer a BigBrother-like interface to Mon. It only returns a visual state of the monitored systems. As of today it doesn't offer any interaction, use mon.cgi instead.
• monshow, which ships with mon in the clients/ subdirectory of the mon distribution. monshow is a simple report-only tool which supports configurable "views" of the mon configuration. monshow also operates as a textmode report generator.
• Minotaure by Gilles Lamiral's, which can be found at ftp://ftp.kernel.org/pub/software/admin/mon/contrib/ .

NEWS CONTENTS

• 20100115 : INSTALL - Mon Wiki ( INSTALL - Mon Wiki, Jan 15, 2010 )
• 20100114 : Monitoring with MON by Muhammad Kamran Azeem ( March 17, 2009 , WBITT! )
• 20070904 : Letter to the editor from Jim Trocki ( Sept 4, 2007 )

Old News ;-)

[Jan 15, 2010] INSTALL - Mon Wiki

The "mon" daemon uses Perl 5.n, where n >= 005_01.

Mon requires that *.ph be created from the system header files. If you try to
run mon and Perl complains with the "did you run h2ph?" message, then chances
are this step wasn't done, either by your package manager or manually after
Perl installation. You can fix it by doing the following, as root:

cd /usr/include
h2ph -r -l .

You'll need the following modules for the server to function, all of
which are available from your nearest CPAN archive. The listed
CPAN paths relative to /cpan/modules/by-authors/id/ -- versions of
modules on CPAN change quickly, so there may be newer versions available,
but the following are known to work:

Time::Period PRYAN/Period-1.20.tar.gz
Time::HiRes J/JH/JHI/Time-HiRes-1.59.tar.gz

2. INSTALLING THE PERL CLIENT MODULE
------------------------------------

The Perl client module is distributed as a separate package. It is named
"mon-client-*.tar.gz". Refer to that for installation instructions.
It is available on kernel.org mirrors in the /pub/software/admin/mon directory,
and in CVS on sourceforge.net. Be sure to match the version of mon-client with
the version of mon you are using. At this time, branch "mon-1-2-branch" of the
mon CVS module matches the "mon-client-1-2-branch" branch of the mon-client CVS
module. See http://sourceforge.net/projects/mon/ for information on CVS access.

3. COMPILING THE C CODE (optional)
----------------------------------

Some of the monitors included with mon are written in C and need to
be compiled for your system. If you want to use the RPC monitor or the
dialin.monitor wrapper,

cd mon.d
(edit Makefile)
make
make install
cd ..

Keep in mind that although this is known to work on Linux, Solaris, and AIX,
it may not compile on your system. It is not required for the operation of mon
itself.

4. MONITORS
-----------

All of the monitor and alert scripts that are packaged with mon are
actually *optional*. However, this is what you'll need for each special
monitor, with CPAN paths relative to /cpan/modules/by-author/id/

freespace.monitor - requires Filesys::Diskspace from CPAN,
in FTASSIN/Filesys-DiskSpace-0.05.tar.gz

fping.monitor - requires the 'fping' binary, from http://www.fping.com
RPM packages available at http://dag.wieers.com/packages/fping/

telnet.monitor - requires the Net::Telnet from CPAN,
in J/JR/JROGERS/Net-Telnet-3.03.tar.gz

reboot.monitor
asyncreboot.monitor
netappfree.monitor
process.monitor
snmpdiskspace.monitor
cpqhealth.monitor
foundry-chassis.monitor
na_quota.monitor
silkworm.monitor
snmpvar.monitor
xedia-ipsec-tunnel.monitor
hpnp.monitor
Use the 'net-snmp' package (formerly UCD SNMP), from
http://sourceforge.net/projects/net-snmp
with G.S. Marzot's Perl module G/GS/GSM/SNMP-4.2.0.tar.gz

ldap.monitor - requires Net::LDAPapi from CPAN,
CDONLEY/Net-LDAPapi-1.42.tar.gz
dialin.monitor - requires the Perl Expect module from CPAN,
R/RG/RGIERSIG/Expect-1.15.tar.gz

dns.monitor - requires Net::DNS from CPAN,
C/CR/CREIN/Net-DNS-0.47.tar.gz

msql-mysql.monitor
requires the MSQL/MySQL DBD module and DBI front-end.

There are a good number of useful monitors included which are not listed
above, so please take the time to read doc/README.monitors to get the
details.

5. MON.CF CUSTOMIZATION AND STARTUP
-----------------------------------

-Read the man page for "mon" and "moncmd" in the doc/ directory to get
an overview of the directories involved, i.e. the configuration,
alert, monitors, state, and run directories.

cd doc
nroff -man mon.8 | more

-read the "READMEs" in the doc/ directory for some useful
insight on system configuration.

-Be sure you have the required Perl modules, which are listed in this
file.

-Make your own mon.cf file, using the suppled "example.cf" (located
in the etc/ directory) as a template, or the m4-based "example.m4":

cp etc/example.cf mon.cf

or

cp etc/example.m4 mon.m4

-Edit the "auth.cf" file. This file controls which users can perform
what command. The default is pretty restrictive (read-only), but that's
only for safety. Currently, "moncmd", "monshow", and "mon.cgi" are the
only clients which are able to authenticate themselves to the server;
the 2-way pager interface does not yet perform authentication. However,
these programs work fine in read-only mode.

-Add the following lines to /etc/services:

mon 2583/tcp # MON
mon 2583/udp # MON traps

-You may want to make a DNS CNAME entry called "monhost" for your
host that will run "mon". You can then set the environment variable
MONHOST to be this host. "moncmd" uses this variable.

-The Perl scripts look for perl in /usr/bin. You might want to change
this. I'd advise keeping a locally-installed copy of Perl if you're
going to monitor network resources and you expect this stuff to work
when some component of the network is down.

-Test it by starting "mon" from the distribution directory. Use these
arguments if you chose the non-m4 config:

./mon -f -c mon.cf -b `pwd`

and these arguments for the m4-based config:

./mon -f -M -c mon.m4 -b `pwd`

To see if it's running on your machine:

./clients/moncmd -s localhost list pids

If you get some output, then things are probably OK. Check the
syslog for further diagnostics.

Mon doesn't really need to be installed in any special location. Just
keep it on the local disk of the machine which will be running the server.

WEB INTERFACE
-------------

This distribution contains two web interfaces: monshow and mon.cgi. monshow is
a simple report-only tool which supports configurable "views" of the mon
configuration. monshow also operates as a textmode report generator.

mon.cgi, however, supports the full functionality of mon, including the ability
to disable/enable groups and hosts and services, acknowledge failed services,
show alert and downtime history, authenticate users, among many other things.

To install monshow, simply copy clients/monshow into your web server's cgi-bin
path and name it "monshow.cgi". You may want to read the man page in the doc/
directory so that you can understand how to configure a "view" to your liking.

To install mon.cgi, follow the instructions found in doc/README.mon.cgi.

[Jan 14, 2010] Monitoring with MON by Muhammad Kamran Azeem

March 17, 2009 | WBITT!

This is an extract from an implementation I did on a heartbeat based system. Mon is an excellent monitoring and reporting tool.

It is assumed that there are two servers connected to each other over multple heartbeat links.

MON for heartbeat monitoring:
----------------------------

both servers:
--------------
wget ftp://ftp.kernel.org/pub/software/admin/mon/mon-1.2.0.tar.gz
wget ftp://ftp.kernel.org/pub/software/admin/mon/mon-client-1.2.0.tar.gz

tar xzf mon-1.2.0.tar.gz
tar xzf mon-client-1.2.0.tar.gz

cd mon-client-1.2.0
perl Makefile.PL
make && make install

[Sept 4, 2007] Letter to the editor from Jim Trocki

Error corrected. Thanks for feedback...

This states "Active development stopped approximately in 2001", which is incorrect. Things slowed down for a bit, but there have been many new features added and bugs fixed in mon 1.2.0 which was released at the end of June this year.

Thank you for the nice write-up, though.

Recommended Links

Softpanorama Recommended

Top articles

Sites

mon- Service Monitoring Daemon Jim Trocki at kernel.org

mon - Freshmeat page

FSF page - FSF page

• The current versions (mon-1.1.0pre1.tar.gz and mon-client-1.0.0pre2.tar.gz) is available from ftp://ftp.kernel.org/pub/software/admin/mon/devel

mon maillist

ftp://ftp.kernel.org/pub/software/admin/mon/mon-talk-0.4.tar.gz

http://www.ryanclark.org. Home of mon CGI interface by Ryan Clark. He also started working on a newer version of the mon client. See also http://moncgi.sourceforge.net/

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: January 09, 2020