UTF-8 Characters in Apache Access Log ✔

apache-2.4charsetencodingloggingutf-8

The issue

I'm using PHP's apache_note() to log variables from web requests to a CustomLog format. However, try as I might, Apache doesn't want to log UTF-8 characters the way I'd like.

In PHP, I have apache_note('some_value', '✔'); which corresponds to the VHost config which looks like this:

LogFormat "%{some_value}n" custom_format
CustomLog ${APACHE_LOG_DIR}/access.log custom_format

However, Apache ends up logging the literal version like this:

\xe2\x9c\x94

What I've tried

  • Checked the values of both LANG and LC_ALL and they are both set to en_US.UTF-8
  • Updated /etc/apache2/envvars to use /etc/default/locale by default
  • Using mod_charset_lite I have set CharsetSourceEnc UTF8 and
    CharsetDefault UTF8 in the Apache config for the site (I know this is for content in/out)
  • Checked that /etc/apache2/conf.d/charset has set AddDefaultCharset UTF-8
  • Tried sending the logging output through a piped log to another program – its \xe2\x9c\x94 by the time it gets there, so it certainly seems like its something to do with the Apache process itself.
  • Read through the Apache logs docs

Ultimately, I want that access log to show something like:

✔

but I'm pulling my hair out trying to get there.

Other information

  • Apache version 2.4.10
  • Debian 8.4

Update

Per Esa's suggestion, I modified the LogFormat directive:

LogFormat "%{some_value}n ✔" custom_format

And I get the following:

\xe2\x9c\x94 ✔

Which is interesting, because it suggests Apache's willingness to log UTF-8. However, I'm still not convinced that the issue has anything to do with PHP passing non UTF-8 values. 

  apache_note('some_value', '✔');
  $value = apache_note('some_value');
  print_r($value);

in PHP still prints out

✔

I will try re-compiling Apache next to see fi it helps, but I do need this in production which may be dicey.

Best Answer

Escaped logging is a feature

Starting from 2.0.49, the Apache logging API escapes everything that goes to error_log, therefore if you're annoyed by this feature during the development phase (as your error messages will be all messed up) you can disable the escaping during the Apache build time:

% CFLAGS="-DAP_UNSAFE_ERROR_LOG_UNESCAPED" ./configure ...

Do not use that CFLAGS in production unless you know what you are doing.

Related Topic