Using Puppet to incrementally add lines to a file, from multiple classes

puppetsplunk

I am trying to use Puppet to automatically configure Splunk monitoring. This involves adding a list of file paths to a Splunk configuration file (inputs.conf).

Each role (webserver, db, etc.) in our application has its own Puppet module, which includes some base modules common to all roles.

The monitored files will vary between roles – for webservers we want /var/log/nginx/error.log to be monitored. For db servers, /var/log/postgresql/postgresql.log. There are some files (/var/log/syslog) that should be monitored for every role.

I populate the inputs.conf file using an EBR template, e.g.:

# $files is actually a parameter to the logging class
$files = ['/var/log/syslog', '/var/log/nginx/error.log']
file {"/opt/splunkforwarder/etc/apps/search/local/inputs.conf":
    ensure  => "present",
    content => template("logging/splunk_inputs.conf.erb"),
    require => Package["splunkforwarder"],
}

# splunk_inputs.conf.erb:
<% files.each do |file| %>
    [monitor://<%= file %>]
    disabled = false  
<% end -%>

Ideally, I want to include a "logging" module in each of these classes and incrementally build up a list of files that should be monitored. This list should be combined, and the $files variable used in the template.

Something like this:

class base {
    class {'logging':
        files => ['fileA', 'fileB']
    }
}

class webserver {
    require base

    class {'logging':
        files => ['file1', 'file2']
    }
}

… would result in the following files being monitored: fileA, fileB, file1, file2.

Splunk will need to be restarted once the final line has been added. I have an exec that does this, but am not sure of the 'best' time to run it. I would preferably like to avoid restarting Splunk after every additional line.

I am having difficulty designing a clean module layout that supports this. I've tried a few different layouts, but unfortunately without success so far. Virtual resources might be a good solution, but I don't think it is possible to have a "virtual variable" that is realized just before it is used in the File's template. Augeas also does not seem like the right solution, although perhaps I am missing something here.

Does anyone have any suggestions as to how I could achieve this?

Best Answer

This should do the trick for you

https://github.com/ripienaar/puppet-concat

Related Topic