I am trying to load balance between two Apache http webservers. I created a load balancer using mod_proxy_balance from Apache.
When I run my client (which generates ~100s of threads every second and tries to browse through the website), after a few seconds I start getting 502 and 503 error response codes for few threads. There are few other threads that function properly.
I feel the loadbalancer might be overloaded but im not exactly sure. Can someone please let me know how to proceed in solving this issue? Thanks!
My proxy balancer setup:
<IfModule mod_proxy_balancer.c>
<Proxy balancer://hgcc13.org>
BalancerMember http://hgccvm90.org/ loadfactor=1
BalancerMember http://hgccvm92.org/ loadfactor=1
</Proxy>
<Location />
ProxyPass balancer://hgcc13.org/
</Location>
</IfModule>
When I saw the error_log in the proxy , First I find
" connection time out to the backend servers"
, then i see" ap_proxy_connect_backend disabling worker"
, and then" All workers are in error state",
The above 3 messages are repeated for a short amount of time. Then I find only
"do_ypcall: clnt_call: RPC: Unable to send; errno = Operation not permitted ".
I ran the experiment only for 5 minutes
Best Answer
I suspect an issue with the backend servers, not the proxy itself.
The 502 errors are indicative of a lack of response from the balancer members.
The 503 errors indicate that one of the Apache services has hit its
MaxClients
setting - this could be the proxy, but if it is, then I suspect that's only because it's waiting on other requests to come back from the unresponsive backends.Verify which system is hitting its limit by checking for "
server reached MaxClients
" in the Apache error log on each system.You may need to adjust the timeout on the balancer members (the
ProxyTimeout
directive or thetimeout=x
setting on theBalancerMember
lines), and/or theMaxClients
(and related MPM settings) of the servers.