Seeing that there is not even a single comment, I decided to continue my research and finally pieced the configuration together.
OS: Ubuntu 16.04
systemd: 229-1ubuntu2
systemd-journal-remote: 229-1ubuntu2
Upload server configuration
This one is actually simple, online example are correct and only need to touch one configuration file.
Use following command to install systemd-journal-remote
sudo apt-get install systemd-journal-remote
Edit /etc/systemd/journal-upload.conf.
/etc/systemd/journal-upload.conf
[Upload]
URL=http://10.0.0.1:19532
# ServerKeyFile=/etc/ssl/private/journal-upload.pem
# ServerCertificateFile=/etc/ssl/certs/journal-upload.pem
# TrustedCertificateFile=/etc/ssl/ca/trusted.pem
To make sure journal-upload auto start on boot
sudo systemctl enable systemd-journal-upload.service
Restart journal-upload after configuration.
sudo systemctl restart systemd-journal-upload.service
If you are using http, you can do as above and leave the bottom 3 lines commented. For active mode https, uncomment them and create those cert files.
The URL actually dictate the transfer protocol(http/https) and the destination port to use.
Additionally if you want to prevent accidental overwrite by future package update, you can create a /etc/systemd/journal-upload.conf.d directory and put your config file inside, as long as the file end with a .conf extension.
As a side notes, I am doing this within a LXC container and seems the service will not use /etc/hosts for dns resolution, I end up using IP address here. So if you use hostname and see error message that journal-upload cannot reach the target, try with IP address.
Receiving server configuration
The receiving server give me most of the trouble when looking for configuration information. And unlike the uploading server, configuration is scattered on this side.
Use following command to install systemd-journal-remote and enable the listening port
sudo apt-get install systemd-journal-remote
sudo systemctl enable systemd-journal-remote.socket
There are two ways, active and passive, to configure journal-remote. I am using passive mode here.
Port number
The configuration file for journal listening port is /etc/systemd/system/sockets.target.wants/systemd-journal-remote.socket as follow. ListenStream is the port number.
Unlike the uploading side, this setting has nothing to do with which protocol(http/https) to use. It only specify the listening port number.
[Unit]
Description=Journal Remote Sink Socket
[Socket]
ListenStream=19532
[Install]
WantedBy=sockets.target
Protocol(http/https) and journal/log location
To change the protocol of the journal transfer and the save location, copy /lib/systemd/system/systemd-journal-remote.service into /etc/systemd/system/, then edit /etc/systemd/system/systemd-journal-remote.service.
[Unit]
Description=Journal Remote Sink Service
Documentation=man:systemd-journal-remote(8) man:journal-remote.conf(5)
Requires=systemd-journal-remote.socket
[Service]
ExecStart=/etc/systemd/systemd-journal-remote \
--listen-http=-3 \
--output=/var/log/journal/remote/
User=systemd-journal-remote
Group=systemd-journal-remote
PrivateTmp=yes
PrivateDevices=yes
PrivateNetwork=yes
WatchdogSec=3min
[Install]
Also=systemd-journal-remote.socket
The --listen-http=-3 specify the incoming journal is using http. If you want to use https, change it to --listen-https=-3.
--output=/var/log/journal/remote/ specify the sink (saving directory) of incoming journal. If it does not exist, create it and change its owner to systemd-journal-remote.
sudo mkdir /var/log/journal/remote
sudo chown systemd-journal-remote /var/log/journal/remote
Restart journal-remote.socket after configuration.
sudo systemctl daemon-reload
What about the most obvious /etc/systemd/journal-remote.conf?
[Remote]
# Seal=false
# SplitMode=host
# ServerKeyFile=/etc/ssl/private/journal-remote.pem
# ServerCertificateFile=/etc/ssl/certs/journal-remote.pem
# TrustedCertificateFile=/etc/ssl/ca/trusted.pem
Since I am not using https, don't need to change anything.
Best Answer
Firstly, it doesn't seems much of high cpu-usage.
journald, has a more complex workflow thanrsyslogd, since it process and keep the data in a more structured form. If you have very high log output to thejournaldby your services, it is normal thatjournaldis overloaded.In order to optimize journald; Journald has following settings as defaults within following file:
If your service logs are non-critical, you can lower the rate limits to suspend the logs of the service which overloads the
journald. It also comes with disadvantage that you may lose your logs. Which you will be notifed byjournaldstating the suspended log counts withinRateLimitInterval, as an enrty instead of logs which are suspended.Setting does to "0" will disable suspending, and you will have much higher cpu-usage.
If logs are important for you seperate service logs from
journaldand usersyslogd. But again this is not high cpu-usage.