Das Werkstatt

[PROBLEM]
I'm just trying to setup monitoring using "Munin" on a Raspberry Pi.
The installation was super easy, and graphs are updated every 5 minutes out of the box.

Unfortunately, some graphs were only generated when munin-node was started, but don't get updated.
In my case, the plugins which seem to have problems are:

• diskstats
• munin_stats

I can see that there's something wrong, because the timestamp on these graphs stays frozen at one time in the past, and they don't display any readings

Following the "Plugin Debugging" article in the Munin docs, I tried executing the "diskstats" plugin manually (as root): I get the following error:

diskstats: Could not open statefile '/var/lib/munin-node/plugin-state/nobody/diskstats-' for writing: No such file or directory

It's weird that it mentions user "nobody" and no IP at the end of "diskstats-", but:
It's telling the truth: "/var/lib/munin-node" exists, but is completely empty and belongs to "root:root" (see Debian Bug #679897).

[MID-SOLUTION]
I've changed ownership to "munin:munin" and created the "plugin-state" subfolder: Now, "munin-run diskstats" runs without error and also created the file: /var/lib/munin-node/plugin-state/nobody/diskstats-
Still, no graphs for diskstats

So I tried connecting to munin-node directly and fetch diskstats: Returns:

Bad exit

So I checked the "/var/log/munin-node.log" (again).

Now I get:

2014/03/26-22:52:28 [31601] Error output from diskstats:
2014/03/26-22:52:28 [31601] diskstats: Could not open statefile '/var/lib/munin-node/plugin-state/nobody/diskstats-127.0.0.1' for writing: Permission denied
2014/03/26-22:52:28 [31601] Service 'diskstats' exited with status 13/0.

[SOLUTION]
Okay. I've now copied the "diskstats-" file in "/var/lib/munin-node/plugin-state/nobody" to both IPs of my server:

• /var/lib/munin-node/plugin-state/nobody/diskstats-127.0.0.1
• /var/lib/munin-node/plugin-state/nobody/diskstats-192.168.1.x

and changed their access rights to "666".

Now, running the "fetch diskstats" per "nc" again, returns values! YAY!
The timestamp in the diskstats graphs is now updated like the others

Victory is mine!


It currently works, but I'm not 100% sure which of the "nobody/diskstats-*" files are now actually used/valid. Furthermore, I'd like to figure out the correct chown/chmod settings for it.


Links:

• Munin: Plugin Debugging
• Debian Bugreport #679897: "/var/lib/munin/plugin-state/ should be writable by munin group"
• Redhat Bugreport #652936: "Munin plugins unable to save state"

[PROBLEM]
In the munin-node.log, I found another error which is unrelated to the above problem, but still:

2014/03/26-23:02:41 [851] Error output from irqstats:
2014/03/26-23:02:41 [851] Argument "usb_fiq" isn't numeric in addition (+) at /etc/munin/plugins/irqstats line 95, <$in> line 11.

[SOLUTION]
Seem Raspberry Pi related. There's a pull-request with a fix by user "cgzones" on github:
https://github.com/munin-monitoring/munin/pull/144
I also took the liberty of copy/pasting cgzones' changes into a patch file for Munin v2.0.6. It's also been committed upstream in Munin's tracker as ticket #1462.