I know it's a frequently asked question which often means there's a problem with certificate validation. It doesn't seem to be the case, because I do not verify the certificate.
This is how my server specification looked in the beginning:
server 1.base.maps.ls.hereapi.com 1.base.maps.ls.hereapi.com:443 ssl verify none check resolvers mydns
Later it evolved to
server 1.base.maps.ls.hereapi.com ipv4@1.base.maps.ls.hereapi.com:443 ssl verify none force-tlsv12 check resolvers mydns resolve-prefer ipv4
But it always returns the same error:
Server freehere_maps_redirect/1.base.maps.ls.hereapi.com is DOWN, reason: Layer6 invalid response, info: "SSL handshake failure"
URL is the real server name. There are 3 more servers in this backend (2.base.maps, 3.base.maps, 4.base.maps…), and all of them return the same error. These servers are not under my control.
I also tried to enable certificate verification, but, of course, it didn't fix the problem. No need to say that curl and other attempts to connect to the backend server work flawlessly.
HA-Proxy version 1.8.4-1deb90d 2018/02/08 running on CentOS7.
Your ideas will be greatly appreciated 🙂
Update:
Here is the output of a successful request made with curl:
$ curl -v --tls-max 1.2 'https://1.base.maps.ls.hereapi.com/maptile/2.1/maptile/newest/normal.day/11/525/761/256/png8?apiKey=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
* Trying 13.33.243.105:443...
* Trying 2600:9000:2118:be00:2:b190:a500:93a1:443...
* Connected to 1.base.maps.ls.hereapi.com (13.33.243.105) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=NL; ST=Noord-Brabant; L=Eindhoven; O=HERE Global BV; CN=*.base.maps.ls.hereapi.com
* start date: May 10 17:54:34 2020 GMT
* expire date: May 11 17:54:34 2021 GMT
* subjectAltName: host "1.base.maps.ls.hereapi.com" matched cert's "*.base.maps.ls.hereapi.com"
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign RSA OV SSL CA 2018
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55b593a60020)
> GET /maptile/2.1/maptile/newest/normal.day/11/525/761/256/png8?apiKey=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx HTTP/2
> Host: 1.base.maps.ls.hereapi.com
> user-agent: curl/7.72.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< content-type: image/png
< content-length: 30653
< access-control-allow-origin: *
< cache-control: public,max-age=86400
< date: Thu, 03 Dec 2020 12:51:37 GMT
< etag: 4742ccdf6d
< expires: Fri, 04 Dec 2020 12:51:37 GMT
< last-modified: Thu, 19 Nov 2020 20:58:37 GMT
< server: nginx
< x-nlp-irt: D=68945
< x-ols-tid: hjBM-ZZ2dSrOCS0R3IovQk_icd0LanCeb81VKszvSQzzTLygjksosg==
< x-served-by: i-094a2ddaf9791ccf9.eu-west-1b
< x-cache: Hit from cloudfront
< via: 1.1 228e9f9ffd3a938a52da99b2c67d587f.cloudfront.net (CloudFront)
< x-amz-cf-pop: HEL50-C1
< x-amz-cf-id: gALOyDPx1QY7dsP0NlaEBivgFHjgP8kLWPw5l2U2DfS1ZCEXOg7FDQ==
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* stopped the pause stream!
* Connection #0 to host 1.base.maps.ls.hereapi.com left intact
Update2:
I tried to replace HTTP checks with TCP checks by modifying the backend to:
backend freehere_maps_redirect
http-send-name-header Host
http-request set-uri http://%[req.hdr(Host)]%[path]?apiKey=xxxxxxxxxxx&%[query]
option tcp-check
tcp-check connect port 443
server 1.base.maps.ls.hereapi.com ipv4@1.base.maps.ls.hereapi.com:443 check
tcp-check passes, but I still can't get any results from the uplink server. On the other hand, it may point to a deeper problem, because I get HTTP 400 (Bad request) from CloudFront:
<H1>400 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Bad request.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
Update 3:
Some more details. I tried to use ssldump to view the SSL handshake process. Here's how it looks with haproxy:
1 1 0.1478 (0.1478) C>S Handshake
ClientHello
Version 3.3
cipher suites
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_DH_DSS_WITH_AES_256_GCM_SHA384
TLS_DHE_DSS_WITH_AES_256_GCM_SHA384
TLS_DH_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
TLS_DH_RSA_WITH_AES_256_CBC_SHA256
TLS_DH_DSS_WITH_AES_256_CBC_SHA256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_DSS_WITH_AES_256_CBC_SHA
TLS_DH_RSA_WITH_AES_256_CBC_SHA
TLS_DH_DSS_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA
TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA
TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA
TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_256_GCM_SHA384
TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_CAMELLIA_256_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_DH_DSS_WITH_AES_128_GCM_SHA256
TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
TLS_DH_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
TLS_DH_RSA_WITH_AES_128_CBC_SHA256
TLS_DH_DSS_WITH_AES_128_CBC_SHA256
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_DSS_WITH_AES_128_CBC_SHA
TLS_DH_RSA_WITH_AES_128_CBC_SHA
TLS_DH_DSS_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_SEED_CBC_SHA
TLS_DHE_DSS_WITH_SEED_CBC_SHA
TLS_DH_RSA_WITH_SEED_CBC_SHA
TLS_DH_DSS_WITH_SEED_CBC_SHA
TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA
TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA
TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA
TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_SEED_CBC_SHA
TLS_RSA_WITH_CAMELLIA_128_CBC_SHA
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_DH_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA
TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_IDEA_CBC_SHA
TLS_ECDHE_RSA_WITH_RC4_128_SHA
TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
TLS_ECDH_RSA_WITH_RC4_128_SHA
TLS_ECDH_ECDSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_MD5
TLS_EMPTY_RENEGOTIATION_INFO_SCSV
compression methods
NULL
1 2 0.3019 (0.1541) S>C Alert
level fatal
value handshake_failure
1 3 0.3019 (0.0000) S>C Alert
level warning
value close_notify
1 0.3024 (0.0004) S>C TCP FIN
1 0.3112 (0.0087) C>S TCP FIN
And here's the successful exchange using curl:
New TCP connection #1: localhost.localdomain(60630) <-> server-13-33-243-73.hel50.r.cloudfront.net(443)
1 1 0.2425 (0.2425) C>S Handshake
ClientHello
Version 3.3
cipher suites
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
Unknown value 0xcca9
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Unknown value 0xcca8
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_DSS_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
Unknown value 0xccaa
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_DSS_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_AES_256_GCM_SHA384
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_MD5
compression methods
NULL
1 2 0.3793 (0.1367) S>C Handshake
ServerHello
Version 3.3
session_id[32]=
f8 f2 8b 5b 48 eb bb 7f d8 5c 70 e0 9c 86 30 0d
f7 3d 6c 52 2f 66 b7 33 84 09 1f bb 25 14 d9 f6
cipherSuite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
compressionMethod NULL
1 3 0.3994 (0.0201) S>C Handshake
Certificate
1 4 0.3994 (0.0000) S>C Handshake
ServerKeyExchange
1 5 0.3994 (0.0000) S>C Handshake
ServerHelloDone
etc.
Does it mean it's the ciphers mismatch problem? But the TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 selected by the remote server during curl exchange is also present in the ciphers list offered by haproxy.
Tried also to add sni str(d2t4vhngii5504.cloudfront.net) or sni str(1.base.maps.ls.hereapi.com), but it didn't help either.
Best Answer
The server in question requires SNI. Thus
check-sni nameinstead ofcheckneed to be used. From the documentation:Therefore it should be: