How to search Nagios by service

I recently divided files by business area, then into staging/production, but that required too much prior knowledge from other colleagues to just find something and change it. Also we moved some servers around and I realised it would be useful to have config-per-host files that could be moved as well. Also, having config-per-host allows you to neatly encapsulate all the monitoring for a particular host (or hostgroup) in one file, and see easily what you are doing for that host. On top of that we have a central commands.cfg file, divided up by each business area which can be easily searched.

However none of this is truly satisfactory - using flat-file storage for configuration commits you to favouring one particular ordering method over another. This sort of configuration really benefits from the relational model. Most new monitoring products have implemented configuration from a database for this reason, I guess.

Currently NDOUtils (warning: pdf) gets your config and event data into a database, but as far as I know there's no way to tell Nagios to use that database as config source. It does give you the opportunity to create tables aside from those required by NDOUtils, perhaps creating a separate nagios_config database where you can map things out and then script up a solution that pulls this information out of the database and creates your config.

I'm going to try and use Puppet to generate my Nagios configs in the near future, hopefully it will let me resolve these sorts of issues.

You can nest hostgroups but not in the way you're describing. See this writeup:

Nagios 4 and hostgroup inheritance