on one of servers running 64bit debian lenny i can see from time to time corrupted kernel messages in syslog file.
they look ok in dmesg:
usb-storage: device found at 21
usb-storage: waiting for device to settle before scanning
but in /var/log/syslog [ produced sysklog daemon ] i can see:
Aug 19 10:01:09 hostname kernel: usb-storage: device found at 21
Aug 19 10:01:09 hostname kernel: usb-storage: waitin fordvc ostl eoesann
this affects only kernel logs. entries from CRON / dhcpd and other services come cleanly. all happens on local host – it's not syslog-over-the-network.
have you ever seen similar problems? any hints? i'm quite sure it's not related to io subsytem or corrupted memory… machine is used heavily, does not have stability issues and consistency of data stored on it is actually checked daily [ witch checksums being calculated on other machines and verified locally ].
thanks!
ps – i think actual content of messages is irrelevant in this case, corruption occurs to different complains from kernel including entering/leaving promiscuous mode.
Best Answer
I have seen this before from remote syslog messages. Since it uses UDP (and UNIX sockets for local events), if the content is modified in a way that the basic UDP checksum doesn't notice the change, it will go through..
The funny think is that it went away after I started using rsyslog...