Linux – How to monitor CPU and Memory usage with Nagios

After you have installed it, I would recommend the following to get a quick head start:

• take a look at the checks that are available to you (usually found in the /usr/local/libexec directory, depending on the distribution) so you can see the wide range of options out of the box
• apply those checks to your servers based on recent issues you have had

This will let you get going right away with results. Once you've got the problem areas covered, begin applying the checks to all known services (http, https, ssl certificate checking, pop3, etc).

For long term trending, give serious consideration to a tool like Cacti. This is great for gathering SNMP info across Unix and Windows boxes (if using windows, make sure that you install the free SNMP Informant) and allowing to see how it changes over a period of time.

Ok I finally got it working.

The problem was that both nagios servers are performing service checks and reporting results to master node, and all those check were performed perfectly. Master node had service freshness checking so if the monitoring servers could not complete checks master server would scheduled those checks from itself.

Anyways, new servers were on new ip range and by default nrpe port was closed on master server.

Opening the port solved the problem. Although it's still odd that it returned "Service check timed out" instead of "Socket timeout error".