Nginx – Log Complete Request/Response with All Headers

nginx

We have an application server that sometimes hangs. We suspect it is due to a bad request from a client.

Can nginx log the complete request/response (like fiddler captures) to files, so we can see the requests that were sent before the hang?

(We probably need to avoid pcap and that approach and do it all in nginx)

If nginx is not the right tool for this, what (other than a network analyzer) might be?

Best Answer

To get the request body sent by visitors, use client_body_in_file_only on; and log the "temporary" file it's written to in the logs by appending var $request_body_file to the log format. "Temporary" files will be located in client_temp directory by default.

You can log request headers $http_<header> too and sent headers with $sent_http_<header>.

If you have request body and headers you should be able to replay it and get the response your visitor had.

Also something like gor should highly be considered so you could replay the traffic on an other environment where you could let nginx write these temporary files without causing IO issues in production (nginx won't purge them with on value that's why It's not that "temporary" in this case).

Correct way in new versions of nginx

Turn out my first answer to this question was correct at certain time, but it turned into another pitfall - to stay up to date please check Taxing rewrite pitfalls

I have been corrected by many SE users, so the credit goes to them, but more importantly, here is the correct code:

server {
       listen         80;
       server_name    my.domain.com;
       return         301 https://$server_name$request_uri;
}

server {
       listen         443 ssl;
       server_name    my.domain.com;
       # add Strict-Transport-Security to prevent man in the middle attacks
       add_header Strict-Transport-Security "max-age=31536000" always; 

       [....]
}

Nginx – Does nginx allow an upstream server to respond to and close a request before it has finished

You don't mention what your clients are, however, this sounds like something that you would achieve with an expect header. In essence, the client sets an "Expect" header with a "100-continue" expectation. The client then waits for a 100 Continue response from the server, before sending its request body.

If the server does not want to receive the body, it can respond with a final status, and the client does not send the body.

This process is defined in RFC2616, section 8.2.3

Best Answer

Related Solutions

NGINX Redirect – How to Rewrite All HTTP Requests to HTTPS While Maintaining Sub-Domain

Correct way in new versions of nginx

Nginx – Does nginx allow an upstream server to respond to and close a request before it has finished

Related Topic