Go to Views>Graphs>Curves
, there you shoudl find your Curve 'd--Used-Space'. Edit this curve and check the box 'Print Max value'.
In your line 3.65Gb;15.64;17.59;0.00;19.55
you have in fact (value)(metric);(warn level);(crit level);(min level);(max level)
. So you need to graph as well the max value on this metrics.
We keep a "mean time till full" or "mean time to failure" metric for this purpose, using the statistical trend and its standard deviation to add the smarter (less dumb) logic over a simple static threshold.
Simplest Alert: Just an arbitrary threshold. Doesn't consider anything to do with the actual diskspace usage.
Simple TTF: A little smarter. Calculate the unused percentage minus a buffer and divide by the zero protected rate. Not very statistically robust, but has saved my butt a few times when my users upload their cat video corpus (true story).
- Example: (100% - 5% - current%) / MAX(rate(current%), 0.001%)
Better TTF: But I wanted to avoid alerting for static read-only volumes at 99% (unless they ever have any changes), and I wanted more proactive notice for noisy volumes, and to detect applications with un-managed diskspace footprints. Oh, and the occasional negative values in the Simple TTF just bothered me.
- Example: MAX(100% - 1% - stdev(current%) - current%, 0) / MAX(rate(current%), 0.001%)
I still keep a static buffer of 1%. Both the standard deviation and the consumption rate increase on abnormal usage patterns, which sometimes over compensates. In graphana or alertmanager speak you'll end up with some rather expensive sub-queries. But I did get the smoother timeseries, and less noisy alert I was seeking.
- Example:
clamp_min((100 - 1 - stddev_over_time(usedPct{}[12h:]) - max_over_time(usedPct{}[6h:])) / clamp_min(deriv(usedPct{}[12:]),0.00001), 0)
Quieter drives make for very smooth alerts.
Longer ranges tame even the noisiest public volumes.
Best Answer
This question looks a lot like that question
To receive an email with cfengine 3, you need to use the cf-execd rather than the cf-agent, and define somewhere where to send the email :
Reference : cfengine documentation
So the daemon will send any new output from the agent to the email adress you specified.
Hope this will help