Java – What are the possible causes for Layer 4 and Layer 7 timeouts in HAProxy

haproxyjavaload balancingreverse-proxytimeout

I am copying my Stackoverflow question : https://stackoverflow.com/q/63683206/6344947

I saw that I was getting 503 for my application when I was hitting the API's. When I checked the HAProxy LB stats for my instances, I found that one of them was showing Layer 4 timeout and the other one was showing Layer 7 timeout.

HAProxy LB stats screenshot : https://i.stack.imgur.com/6Vtnc.png

The instances had a Java process running in them which was acting as the backend service. On restarting the process, I found that things were back to normal.

So is there any way to find out what the cause could be? Could it be because of out of memory issues too? I had seen the logs, but there were nothing related to OOM; also, we had OOM issues in that instance in the past – but at that time I didn't check HAProxy stats.

Any help would be appreciated.

Best Answer

Layer 4 means it got timeout waiting for TCP handshakes, maybe server is down? Layer 7 means it got timeout waiting for HTTP answer, so application is running (starting maybe?), listening on the specified port and accepting TCP connections, but didn't provide HTTP response in time. Maybe it is still launching? Some java applications take minutes to actually start. Both cases rather indicate problem with backend application and not a problem with haproxy itself. I'd say it's time to check logs of this java application.

Related Topic