Reliable export of journalctl logs

journalctljournaldsyslog

I search a way for a reliable export of journalctl logs.

I could use the --since=... option, but this is a bit fuzzy.

In my case a script would call journalctl --output=json every ten minutes.

I don't want to miss a single line and (if possible) I would like to avoid duplicate lines.

Some days after asking this question I came across RELP: https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protocol

Best Answer

You can install a syslog daemon such as rsyslog (the default on Red Hat derived systems). This will log all journal entries in a more backward compatible manner, and of course you can specify a custom log for whatever you wish.

If you don't need logs exported in real time, you can use journalctl --since as some people have mentioned. You can run it daily at midnight with the time specifier yesterday to get exactly 24 hours of logs.

If you really need to get logs at short intervals and don't want to miss a single entry, then you need to learn about the cursor. For each log entry journalctl will provide a cursor which can be used to skip to exactly that log entry with --cursor, or the immediately following log entry with --after-cursor. Consider the following sample JSON:

{
    "__CURSOR" : "s=6ad7dcf190f3409c8bf8086fec22888c;i=286c44;b=6b134acc25e94d69b4713422b7c773be;m=46f7a97d25;t=55f5e93131a32;x=aecce3d8b96df5dc",
    "__REALTIME_TIMESTAMP" : "1512234682620466",
    "__MONOTONIC_TIMESTAMP" : "304802790693",
    "_BOOT_ID" : "6b134acc25e94d69b4713422b7c773be",
    "PRIORITY" : "6",
    "_MACHINE_ID" : "770056613d554df2abcb7757ba2e6270",
    "_HOSTNAME" : "dalaran.example.us",
    "_PID" : "1",
    "_UID" : "0",
    "_SELINUX_CONTEXT" : "system_u:system_r:init_t:s0",
    "_GID" : "0",
    "_CAP_EFFECTIVE" : "3fffffffff",
    "SYSLOG_FACILITY" : "3",
    "SYSLOG_IDENTIFIER" : "systemd",
    "_TRANSPORT" : "journal",
    "_COMM" : "systemd",
    "_EXE" : "/usr/lib/systemd/systemd",
    "_CMDLINE" : "/usr/lib/systemd/systemd --switched-root --system --deserialize 25",
    "_SYSTEMD_CGROUP" : "/init.scope",
    "_SYSTEMD_UNIT" : "init.scope",
    "_SYSTEMD_SLICE" : "-.slice",
    "CODE_FILE" : "../src/core/unit.c",
    "CODE_LINE" : "1505",
    "CODE_FUNC" : "unit_status_log_starting_stopping_reloading",
    "MESSAGE_ID" : "de5b426a63be47a7b6ac3eaac82e2f6f",
    "MESSAGE" : "Stopping OpenSSH server daemon...",
    "UNIT" : "sshd.service",
    "_SOURCE_REALTIME_TIMESTAMP" : "1512234682615526"
}

For your purposes, the __CURSOR is an opaque blob. Just capture the value from the last log entry you receive on one call to journalctl and feed it to the next call:

journalctl --output=json --after-cursor="s=6ad7dcf190f3409c8bf8086fec22888c;i=286c44;b=6b134acc25e94d69b4713422b7c773be;m=46f7a97d25;t=55f5e93131a32;x=aecce3d8b96df5dc" ...
Related Topic