FreeBSD Periodic(8) Too Noisy – How to Control Noise Level

cronfreebsdmac-osxmonitoring

FreeBSD & MacOSX ship with the the periodic(8) utility, which is a neatly organized set of utilities to periodically run system functions like ZFS filesystem checks, security checks, checking for out of date ports, etc.

The problem is that periodic sends too many emails which contain too much superfluous information. This causes people to ignore the emails, and we miss many problems picked up by periodic(8). The Daily emails are sent once per day, the security emails are sent once per day, and the weekly & monthly emails are also sent periodically. These emails have a subject line like one of these:

Subject: $HOSTNAME daily run output
Subject: $HOSTNAME security run output
Subject: $HOSTNAME weekly run output
Subject: $HOSTNAME monthly run output

How can I reduce the amount of email sent from periodic(8)?

I'll post my own answer below, but I would like to see what others have done.

Note: I have a similar question regarding Linux, at Linux: logwatch(8) & cron.daily is too noisy. How can I control the noise level?

Best Answer

Place something like the following in /etc/periodic.conf .

The following configuration will reduce the noise in the emails. If these messages are blank, then periodic(8) will not send an email. In addition, the security emails will be included with the daily emails, which also reduces noise.

# /etc/periodic.conf overrides the defaults in /etc/defaults/periodic.conf
# This file can be overriden by /etc/periodic.conf.local

# *_show_success, *_show_info & *_show_badconfig are disabled
# per recomendation of periodic(8) and "Absolute FreeBSD" p. 310-311
# and "Essential system administration, 3rd Ed." p. 98

# *_show_badconfig="NO" will suppress messages for tools which are not installed on this system (e.g. ZFS on a system without ZFS). 

daily_show_success="NO"
daily_show_info="NO"
daily_show_badconfig="NO"

weekly_show_success="NO"
weekly_show_info="NO"
weekly_show_badconfig="NO"

monthly_show_success="NO"
monthly_show_info="NO"
monthly_show_badconfig="NO"

# Include security jobs with daily email. No need to send second email.
daily_status_security_inline="YES"
security_show_success="NO"

# Don't need to know about denied packets every day
daily_status_security_ipfdenied_enable="NO"

### Now, enable services which you DO want to be aware of    
# Check host for old ports
daily_status_security_portaudit_enable="YES"

# Perform ZFS filesystem checks
daily_status_zfs_enable="YES"

The above example will get you 90% there. However, there is one additional problem which cannot be resolved using the standard FreeBSD config (as of 20110601). The script at /etc/periodic/daily/450.status-security will still print the following pointless message:

Security check:

-- End of daily output --

The fix is to apply the patch found in http://www.freebsd.org/cgi/query-pr.cgi?pr=138692 . This patch will modify the return codes for 450.status-security, so that extra messages won't be printed.

Related Solutions

Linux – How to Control Noise Level in logwatch(8)

Overall, the available documentation for Logwatch lacks adequate explanation and is often far too vague. I pieced together some useful examples, and have reduced the Logwatch noise by over 95%.

Here's what I have found.

Keep in mind that you can find some Logwatch documentation at /usr/share/doc/logwatch-*/HOWTO-Customize-LogWatch, and it contains a few useful examples.

On RHEL/CentOS/SL, the default logwatch configuration is under /usr/share/logwatch/default.conf/logwatch.conf

These settings can be overriden by placing your local configuration under /etc/logwatch/conf/logwatch.conf. Place the following in that file to tell logwatch to completely ignore services like 'httpd' and the daily disk usage checks:
```
# Don't spam about the following Services
Service = "-http"
Service = "-zz-disk_space"
```
Sometimes I don't want to completely disable logwatch for a specific service, I just want to fine tune the results to make them less noisy. /usr/share/logwatch/default.conf/services/*.conf contains the default configuration for the services. These parameters can be overridden by placing your local configuration under /etc/logwatch/conf/services/$SERVICE.conf. Unfortunately, logwatch's ability here is limited, and many of the logwatch executables are full of undocumented Perl. Your choice is to replace the executable with something else, or try to override some settings using /etc/logwatch/conf/services.

For example, I have a security scanner which runs scans across the network. As the tests run, the security scanner generates many error messages in the application logs. I would like logwatch to ignore errors from my security scanners, but still notify me of attacks from other hosts. This is covered in more detail at Logwatch: Ignore certain IPs for SSH & PAM checks?. To do this, I place the following under /etc/logwatch/conf/services/sshd.conf:
```
# Ignore these hosts
*Remove = 192.168.100.1
*Remove = X.Y.123.123
# Ignore these usernames
*Remove = testuser
# Ignore other noise. Note that we need to escape the ()
*Remove = "pam_succeed_if$sshd:auth$: error retrieving information about user netscan.*
```
"
logwatch also allows you to strip out output from the logwatch emails by placing regular expressions in /etc/logwatch/conf/ignore.conf. HOWTO-Customize-LogWatch says:

ignore.conf: This file specifies regular expressions that, when matched by the output of logwatch, will suppress the matching line, regardless of which service is being executed.

However, I haven't had much luck with this. My requirements need a conditional statement, which is something like 'If there are security warnings due to my security scanner, then don't print the output. But if there are security warnings from my security scanner and from some bad guys, then print the useful parts-- The header which says "Failed logins from:", the IPs of the bad hosts, but not the IPs of scanners.'
Nip it at the source (As suggested by @user48838). These messages are being generated by some application, and then Logwatch is happily spewing the results to you. In these cases, you can modify the application to log less.

This isn't always desirable, because sometimes you want the full logs to be sent somewhere (to a Central syslog server, central IDS server, Splunk, Nagios, etc.), but you don't want logwatch to email you about this from every server, every day.

Freebsd – How to put FreeBSD configuration under Revision Control

If you just want to track the changes of your own configurations on the same local host, you can happily use RCS.

A wonderful and easy howto is available on the internet.

Best Answer

Related Solutions

Linux – How to Control Noise Level in logwatch(8)

Freebsd – How to put FreeBSD configuration under Revision Control

Related Topic