How to delete a counter in graphite whisper

graphitestatsd

I have a counter at stats.message.foo and want to move it to stats.messages.foo.

I've updated my code to populate the new counter, however the old one still exists.

I've read all I need to do to remove a stat from graphite is to delete the appropriate whisper file on disk, however it seems that within a few seconds of removing the wsp it gets regenerated (without data).

This is an annoyance if I want to rename the key that the data is stored under as I need to remember which key is the correct one.

Does anyone know how to permanently remove the old counter?

Best Answer

The reason so many people are struggling with this issue (me included) is because statsd (depending on its settings) continues to write data to old stats (whisper files) even after you delete the whisper file (.wsp).

Check your statsd settings at /etc/statsd/config.js (on Ubuntu). Example settings at https://github.com/etsy/statsd/blob/master/exampleConfig.js. I would recommend setting deleteIdleStats to true so that statsd does not write stats when they are not received.

The downfall of deleteIdleStats is that Graphite may now have null values for certain stats. This can be handled on the visualization layer (I recommend Grafana).

You have two options to stop regeneration of deleted whisper files:

  • simply restart statsd with service statsd restart or
  • update statsd settings to not send stats to backend if the stats are not received AND then service statsd restart

To delete wsp files that you don't need anymore:

  1. List the files to be deleted withfind pathToUnwantedStatistic -print
  2. if that is printing only the files you want deleted, run the following line to delete the unwanted files find pathToUnwantedStatistic -delete
Related Topic