Tomcat – Logging X-Forwarded-For IP in Shibboleth’s Audit Logs

shibbolethtomcatx-forwarded-for

This is a follow up from my earlier question on capturing the X-Forwarded-For IP address in across multiple proxies. Now, I'm looking to capture the Client's IP in the application's (Shibboleth's IdP) logs.

The follow is:

Client > Load Balancer > Apache httpd server > Tomcat server (running Shibboleth's IdP)

The variable with the Client's IP at the Load Balancer is ClientIP which I can capture in the web server's logs using LogFormat "%{ClientIP}i ... " and similarly in Tomcat's access logs using these inserver.xml`:

<Valve className="org.apache.catalina.valves.RemoteIpValve" 
remoteIpHeader="ClientIP" 
protocolHeaderHttpsValue="https" />

<Valve className="org.apache.catalina.valves.AccessLogValve" 
directory="logs" 
prefix="localhost_access_log." 
suffix=".txt"
pattern="%{ClientIP}i %h %l %u %t %r %>s %b %{Referer}i %{User-Agent}i" resolveHosts="false"/>

Now, I'm looking to capture this IP in Shibboleth's IdP audit logs. This discussion talks about using mod_rpaf for Apache but doesn't get into the details. I'm hoping to do without installing additional modules.

Looks like I'm missing a couple of (small?) pieces here. Any ideas? Many thanks!

Best Answer

After digging at LogBack's documentation on logging client IPs and the corresponding code for the MDCInsertingServletFilter servlet, turns out you must use the X-Forwarder-For header.

We replaced ClientIP with X-Forwarder-For in the load balancer, updated httpd.conf's LogFormat to log X-Forwarder-For, then updated Shibboleth's logging.xml file as follows for the audit logs:

<appender name="IDP_AUDIT" class="ch.qos.logback.core.rolling.RollingFileAppender">
  <File>/app/shibboleth-idp/logs/idp-audit.log</File>

  <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
    <FileNamePattern>/app/shibboleth-idp/logs/idp-audit-%d{yyyy-MM-dd}.log</FileNamePattern>
  </rollingPolicy>
  <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
    <charset>UTF-8</charset>
  <Pattern>%msg|%X{req.xForwardedFor}|%n</Pattern>
  </encoder>
</appender>

We made similar changes to the idp-process as well, and did not have to update Tomcat's logging.

Hope this helps others.

Edit 1: Tomcat Logging

Turns out, after a while, the X-Forwarded-For IP address stopped appearing in Shib's IdP logs. We ended up adding the RemoteIpValve in tomcat and restarting. Looks like it is needed after all: 

  <Valve className="org.apache.catalina.valves.RemoteIpValve"
   remoteIpHeader="X-Forwarded-For"
   trustedProxies="--IPs here--"
   protocolHeaderHttpsValue="https" />
Related Topic