Kafka logfile location

kafka

Just to make some things clear: I'm talking about the process logfile that contains the stdout and stderr messages.

This is my systemd unit file:

[Unit]
Description=Apache Kafka server
Documentation=http://kafka.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
PIDFile=/var/run/kafka.pid
User=kafka
Group=kafka
Environment="JAVA_HOME=/usr/lib/jvm/jre"
Environment="KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:/var/log/kafka"

ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-failure
SyslogIdentifier=kafka

[Install]
WantedBy=multi-user.target

Note that I added the KAFKA_LOG4J_OPTS environment variable.

However, that doesn't seem to do anything. This is the output, when I try to start my service:

Feb 11 00:55:30 kafka01 kafka[4047]: mkdir: cannot create directory ‘/opt/kafka/bin/../logs’: Permission denied
Feb 11 00:55:30 kafka01 kafka[4047]: OpenJDK 64-Bit Server VM warning: Cannot open file /opt/kafka/bin/../logs/kafkaServer-gc.log due to No such file or directory
Feb 11 00:55:30 kafka01 kafka[4047]: log4j:WARN No appenders could be found for logger (kafka.Kafka$).
Feb 11 00:55:30 kafka01 kafka[4047]: log4j:WARN Please initialize the log4j system properly.

So, can please someone tell me the correct way to change the location of the kafka log file? KAFKA_LOG4J_OPTS doesn't work, KAFKA_LOG_DIR doesn't work either (=I was hoping kafka implemented this similar to zookeeper). The kafka documentation doesn't tell me either.

Best Answer

For other unfortunate lads like my, you need to modify LOG_DIR environment variable (tested for Kafka v0.11).

If you open script kafka-server-start or /usr/bin/zookeeper-server-start, you will see at the bottom that it calls kafka-run-class script. And you will see there that it uses LOG_DIR as the folder for the logs of the service (not to be confused with kafka topics data).

Related Topic