Haproxy not delivering certificate chain

haproxyssl-certificate

We're setting up haproxy 1.5.14 to do SSL termination for the upcoming release of our massively tenanted application. Hundreds of domain names are used with the app; most of the certs are for wildcards.

I have put all the *.pem files in a directory, and pointed haproxy at that directory with this bind line:

bind *:443 ssl crt /media/windowsshare/aleyant-fs/ssl/pem/

It works great — the browser uses SNI to send the domain name, haproxy locates the correct certificate, TLS is negotiated and then haproxy passes it to the backend. It's wonderful…

… except that intermediate certificates are not being used.

I have followed the directions here. Each pem file contains the private key, the domain certificate, and the intermediate certificate, concatenated. For example here's one of the pem files (private key elided):

Bag Attributes
    Microsoft Local Key set: <No Values>
    localKeyID: 01 00 00 00
    friendlyName: le-6636f293-52ab-4054-9479-d004fe39ca60
    Microsoft CSP Name: Microsoft RSA SChannel Cryptographic Provider
Key Attributes
    X509v3 Key Usage: 10
-----BEGIN PRIVATE KEY-----
---------------OMITTED FOR SECURITY ------------
-----END PRIVATE KEY-----
Bag Attributes
    localKeyID: 01 00 00 00
    friendlyName: v6.pressero.com (Wildcard)
subject=/OU=Domain Control Validated/CN=*.v6.pressero.com
issuer=/C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2
-----BEGIN CERTIFICATE-----
MIIE3TCCA8WgAwIBAgISESHTmCuIWLtnW8IH/eJb79kQMA0GCSqGSIb3DQEBCwUA
....
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIETTCCAzWgAwIBAgILBAAAAAABRE7wNjEwDQYJKoZIhvcNAQELBQAwVzELMAkG
...
-----END CERTIFICATE-----

But checking with openssl, I see that haproxy is not delivering the full chain:

CONNECTED(00000003)
--- Certificate chain  0 s:/OU=Domain Control Validated/CN=*.abp-inc.com
i:/C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2
-----BEGIN CERTIFICATE----- 
MIIE0TCCA7mgAwIBAgISESHIIsDizkD1ipb3UIUxxmbbMA0GCSqGSIb3DQEBCwUA
....
-----END CERTIFICATE-----
--- 
Server certificate 
subject=/OU=Domain Control Validated/CN=*.abp-inc.com 
issuer=/C=BE/O=GlobalSign nv-sa/CN=AlphaSSL CA - SHA256 - G2
--- 
No client certificate CA names sent 
Peer signing digest: SHA512 
Server Temp Key: ECDH, P-256, 256 bits
--- 
SSL handshake has read 1936 bytes and written 482 bytes
--- 
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA256 
Server public key is 2048 bit 
Secure Renegotiation IS supported 
Compression: NONE 
Expansion: NONE 
No ALPN negotiated 
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-SHA256
    Session-ID: 6112D93EF9ABB9FC9286BA4D5761012208599691BFCF6258ECA6E03B25146F71
    Session-ID-ctx: 
    Master-Key: 400EF52DB453A3D4AB9BC00CDFD713B7298DE03F101F3646D95A6F3D7E78E4D8FD582688109AF3E8A7B957DE8F788D0A
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - f3 a9 9e 90 5d a1 b7 75-b7 ef db 10 92 ef 87 1f   ....]..u........
    0010 - 3d ff f1 17 e4 34 6f 72-c8 fe 2b d5 ca c0 4e 81   =....4or..+...N.
    0020 - f7 85 5b 6f 87 b0 51 47-d9 4a 2c 2a 0d 98 59 62   ..[o..QG.J,*..Yb
    0030 - ca ef dc cd 9e 09 43 dd-37 a9 8a a3 1a c3 f9 b1   ......C.7.......
    0040 - 9e e5 c6 99 b4 96 f0 4b-52 22 f8 db 17 50 9d 0a   .......KR"...P..
    0050 - 75 4d 33 f2 96 32 6d 67-b7 ec fc a6 c9 5e c9 1e   uM3..2mg.....^..
    0060 - 47 6c f0 69 61 5d 12 ed-9c 52 51 2e 5a f5 74 68   Gl.ia]...RQ.Z.th
    0070 - e7 17 9d ca 14 49 3e 84-c6 da 4a 8d 8b 18 f8 a4   .....I>...J.....
    0080 - ca 1b 3c 17 60 0d 42 15-48 55 64 74 79 12 8f 4d   ..<.`.B.HUdty..M
    0090 - 17 2f 28 d1 72 01 11 bc-e5 b3 02 0c da 47 1b 3b   ./(.r........G.;

    Start Time: 1444168700
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
--- 
HTTP/1.0 400 Bad request 
Cache-Control: no-cache 
Connection: close 
Content-Type: text/html

<html><body><h1>400 Bad request</h1> Your browser sent an invalid request. </body></html>

closed

Any suggestions would be greatly appreciated.

Best Answer

I'm a fool. I did it right by following this, but I've been testing wrong. I forgot the -servername argument to openssl s_client -connect .... With no SNI being sent to haproxy, it picked the first PEM file it found in the directory, which was one I had not added the intermediate cert to (yet).

Related Topic