I'm working on analyzing an intermittent load balancing failure in our application.
Previously we had been using the AWS Elastic Load Balancer in the following configuration:
HTTP 80 -> HTTP 80
HTTPS 443 -> HTTPS 443, presenting the same certificate as the backend IIS servers are
Now we realize this is a bit of a hairbrained setup (both the LB & the IIS server doing the same encryption … wasted work), and it has now started to cause some problems for us. Specifically, we will intermittently see a request to the ELB spike in latency, take a full 60s (the default timeout), and then report an error to the client. We spent a good deal of time confirming the latency spike is not related to a processing delay in the application.
As mentioned, we now realize this setup is weird. For example, a more natural config of ELB works just fine:
HTTP 80 -> HTTP 80
TCP 443 -> TCP 443, straight passthrough, all encryption happening on the IIS backend
Zooming out for a moment, we became curious if we could reproduce the intermittent failure in the bad configuration on HAProxy. That is have HAProxy do SSL termination, and then initiate another full SSL connection to the backend server. Again, we realize this is silly, but are investigating for purposes of comparing the black-ish box that is ELB and HAProxy.
Here is the simple config I tried:
frontend https_frontend
bind *:443 ssl crt /etc/ssl/certs/ourpublicandprivatecert.pem
mode http
default_backend web_server
backend web_server
mode http
server s1 10.0.1.4:443 check
Then when browsing to the HAProxy host, we get:
504 Gateway Time-out
The server didn't respond in time.
I'm guessing HAProxy is freaking out about the mismatch on the cert, but I can't get logs to confirm that. The other possibility is that because that is such a weird setup (mixing SSL termination and passthrough) HAProxy just doesn't support it, instead forcing you down the much saner paths of passthrough OR termination.
Anyone have any insight?
Best Answer
Naturally you have a recent version of HAProxy with OpenSSL support built in.
It seems you need some extra parameters to use HTTPS backend. HAProxy tries normal HTTP connection by default, regardless of the port number.
The
ssl
parameter ensures SSL connection:The server certificate is not verified by default. If you wish to verify it, you need to add
verify required
andca-file <cafile>
to the server line.See more about the options here: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.2