This stuff is definitely confusing when you first start looking under the hood.
You've actually figured out a lot of the key pieces, let's see if I can help you understand how they fit together.
Here's what I've learned through my own experimentation:
1) PNP4Nagios
See their main site: http://docs.pnp4nagios.org/pnp-0.4/start
To verify that you're using PNP4Nagios, look at:
$ more /usr/local/nagios/etc/pnp/pnp4nagios_release
PKG_REL_DATE="05-02-2009"
PKG_VERSION="0.4.14"
PKG_NAME="pnp"
PNP configuration is done in /usr/local/nagios/etc/nagios.cfg
Here's a nice overview that I recently found, it includes a description of the various PNP-related settings in nagios.cfg:
http://bitflip.net/files/pnp4nagios-presentation-20090409.pdf
2) RRD Data Sources and Graph Templates
See: http://docs.pnp4nagios.org/pnp-0.4/tpl
The php template scripts are found in /usr/local/nagios/share/pnp/.
A php graph template is invoked within the context of the RRD data file that's associated with the host or service command whose data you're trying to graph. If there is no template for the command, default.php is used.
As you found, you can't simply add a fourth graph because there's no corresponding datasource:
the 4th graph displays, but with a broken picture icon.
And, as you also found, you can customize the graph that is displayed for a particular datasource:
Using only the fourth combined graph as $def[1] works.
3) default.php
Remember, that default.php is used whenever a host or service command doesn't have an associated graph template.
This means that any change you make will affect the default graph for your entire system.
If that's what you want, fine. Otherwise, simply add a check for the host and/or service whose graph you're trying to customize:
if ($hostname == 'MyHost') {
// generate a customized graph
}
else {
// generate the default graph
}
4) Finally, how to create a combined graph AND display the original three data sources.
I would simply have your perl script generate a fourth datasource and populate it with zeroes:
val1=1;; val2=5;; val3=10;; val4=0;;
Then, in default.php, you can create a custom graph for that datasource:
foreach ($DS as $i) {
if ($servicedesc == 'My_Test_Service') {
if ($i == 4) { # the "val4" placeholder datasource
// your combined graph logic goes here
}
else {
// datasource 1 to 3 graph logic goes here
}
}
}
This just begins to cover this topic, I hope it helps!
Best Answer
Rrdtool graph has some "smarts" about the Y axis to prevent the labels all turning into weird numbers with odd decimals. So it prefers to go up to 2, 5, 10, 20, 50, etc. The docs describe this as "normally the scale is selected from a predefined set of ranges". The upper graph has a max around 12k so that gets rounded up to 20k. There is an option to rrdgraph called --alt-autoscale (and also --alt-autoscale-min and --alt-autoscale-max) which tend to produce tighter graphs, which I generally prefer. See http://oss.oetiker.ch/rrdtool/doc/rrdgraph.en.html for more details.