Nagios Check that Simulates Web Full Rendering Time

Ok I finally got it working.

The problem was that both nagios servers are performing service checks and reporting results to master node, and all those check were performed perfectly. Master node had service freshness checking so if the monitoring servers could not complete checks master server would scheduled those checks from itself.

Anyways, new servers were on new ip range and by default nrpe port was closed on master server.

Opening the port solved the problem. Although it's still odd that it returned "Service check timed out" instead of "Socket timeout error".

I'd say your best bet is OpenNMS with RT or OTRS integration. Unlike Nagios, it's a complete SNMP management solution with an FCAPS (fault/configuration/accounting/performance/security management) focus. How well it tackles each one of those categories is sort of up to the implementer. It's a great solution for people who are looking to "upgrade" from Nagios and have a Cacti server sitting around doing similar things. The integration of the performance and fault data is absolutely indispensable. The documentation is sort of behind the current state of the product, but I've been personally working on this as of late.

If you want to give it a try, go ahead and follow the quick start instructions on the opennms.org wiki, but stop at "discovery", and take a look at the new provisiond feature whitepaper. It's a great migration tool as well.

The event based system it provides triggers alarms for an alarm panel and notifications for... notifications. These can be phone calls via asterisk, pages, email, twitter, etc. When you or on-call staff are notified, you can reply to the email with the work "ack" and have the notification acknowledged and your ticket updated with start times, etc.

The separation of notifications and alarms is a great feature for your de-duplication request. Depending on what's going on, you can reduce these alarms by a reduction key and only be notified on the threshold (but still have all the alarm triggered so you have the data). There's some advanced correlation features, but I haven't really dug into it.