Ssl – curl fails to retrieve HTTPS content: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

I'm trying to access the website https://www.lawsociety.com.au with curl on Windows 10 and Ubuntu 16.04. It works on Ubuntu, but fails on Windows with the message error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure. I'm not sure what is wrong and how to fix it.

Here is the curl output on Windows machine:

> curl -i -v -I https://www.lawsociety.com.au
* Rebuilt URL to: https://www.lawsociety.com.au/
*   Trying 125.7.104.7...
* TCP_NODELAY set
* Connected to www.lawsociety.com.au (125.7.104.7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: D:\dev\curl\bin\curl-ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS alert, Server hello (2):
* error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure
* stopped the pause stream!
* Closing connection 0
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

curl version:

curl 7.57.0 (x86_64-pc-win32) libcurl/7.57.0 OpenSSL/1.1.0g (WinSSL) zlib/1.2.11 WinIDN libssh2/1.8.0 nghttp2/1.28.0
Release-Date: 2017-11-29
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTP2 HTTPS-proxy MultiSSL    

Same command on Ubuntu box:

$ curl -I -v https://www.lawsociety.com.au
* Rebuilt URL to: https://www.lawsociety.com.au/
*   Trying 125.7.104.7...
* Connected to www.lawsociety.com.au (125.7.104.7) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.0 / RSA_3DES_EDE_CBC_SHA1
*        server certificate verification OK
*        server certificate status verification SKIPPED
*        common name: *.lawsociety.com.au (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: C=AU,postalCode=2000,ST=NSW,L=Sydney,street=170 Phillip Street,O=THE LAW SOCIETY OF NEW SOUTH WALES,OU=PremiumSSL Wildcard,CN=*.lawsociety.com.au
*        start date: Fri, 17 Mar 2017 00:00:00 GMT
*        expire date: Mon, 16 Apr 2018 23:59:59 GMT
*        issuer: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA Limited,CN=COMODO RSA Organization Validation Secure Server CA
*        compression: NULL
* ALPN, server did not agree to a protocol
> HEAD / HTTP/1.1
> Host: www.lawsociety.com.au
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Tue, 26 Dec 2017 09:04:02 GMT
Date: Tue, 26 Dec 2017 09:04:02 GMT
< Server: Oracle-Application-Server-11g
Server: Oracle-Application-Server-11g
< Cache-Control: no-cache
Cache-Control: no-cache
< Content-Length: 36272
Content-Length: 36272
< Set-Cookie: JSESSIONID=kGs3hCQCW7FPhQ2Lh0JvKn9JvXhhHCK2GQKXvLps308Ww1D70pMp!1826685759; path=/; HttpOnly
Set-Cookie: JSESSIONID=kGs3hCQCW7FPhQ2Lh0JvKn9JvXhhHCK2GQKXvLps308Ww1D70pMp!1826685759; path=/; HttpOnly
< X-ORACLE-DMS-ECID: 005OJeGQSZb9xWGayxQ_MG0007Z60000EU
X-ORACLE-DMS-ECID: 005OJeGQSZb9xWGayxQ_MG0007Z60000EU
< X-Powered-By: Servlet/2.5 JSP/2.1
X-Powered-By: Servlet/2.5 JSP/2.1
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8

<
* Connection #0 to host www.lawsociety.com.au left intact

I tried openssl s_client command on Windows:

> openssl s_client -connect www.lawsociety.com.au:443
CONNECTED(00000224)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify error:num=19:self signed certificate in certificate chain
---
Certificate chain
 0 s:/C=AU/postalCode=2000/ST=NSW/L=Sydney/street=170 Phillip Street/O=THE LAW SOCIETY OF NEW SOUTH WALES/OU=PremiumSSL Wildcard/CN=*.lawsociety.com.au
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
 1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
 2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
 3 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
... <removed to save space> ...
-----END CERTIFICATE-----
subject=/C=AU/postalCode=2000/ST=NSW/L=Sydney/street=170 Phillip Street/O=THE LAW SOCIETY OF NEW SOUTH WALES/OU=PremiumSSL Wildcard/CN=*.lawsociety.com.au
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
---
No client certificate CA names sent
---
SSL handshake has read 5683 bytes and written 621 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA
    Session-ID: 8198FE887E4FC12D68E2388D4C052ABF
    Session-ID-ctx:
    Master-Key: 93688C15A75E3E9F596AF96DFF72B557AD28A3FEF8764401CBD12D1F432EAF4D216595D74338AF24498AB29FF5ABE759
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1514278771
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---

So connection seems to be fine. Opening the URL in a browser works just fine, too.

What am I missing?

UPDATE

1. One of the possible problems is that the website is using outdated RC4-SHA cipher. I tried to enable it explicitly with curl, but curl rejects it:

> curl -i -v -I --ciphers "RC4-SHA" https://www.lawsociety.com.au
* Rebuilt URL to: https://www.lawsociety.com.au/
*   Trying 125.7.104.7...
* TCP_NODELAY set
* Connected to www.lawsociety.com.au (125.7.104.7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* failed setting cipher list: RC4-SHA
* Closing connection 0
curl: (59) failed setting cipher list: RC4-SHA

2. Another potential problem is that root certificates might not be available. However, curl comes with the latest Mozilla CA bundle (curl-ca-bundle.crt), so I believe it is using correct certificates.

I also copied all public certificates from working Ubuntu box to the Windows machine and specified cert path to curl using --capath param – it doesn't help.

3. Just for completeness sake, I tried with the latest Python 3.6.4:

import urllib.request
with urllib.request.urlopen('https://www.lawsociety.com.au/') as u:
    print(u.read())

Python SSL should use Windows facilities for HTTPS. However, it fails with the same error:

ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:777)
...
During handling of the above exception, another exception occurred:
...
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:777)>

So it looks like the problem is not with missing certificate, but happens somewhere earlier. I'm not an SSL expert, so I may be wrong.

SOLUTION

The root cause of the problem is outdated SSL protocol+ciphers used by the website. In order to make curl work I downgraded it to the version that uses OpenSSL 1.0.2, which still supports RC4 ciphers. After that everything works flawlessly.

For those who need Python solution:

import ssl
import urllib.request

ctx = ssl.SSLContext(protocol=ssl.PROTOCOL_SSLv3)
ctx.set_ciphers('SSLv3')
# alternatively, for this particular website:
#ctx.set_ciphers('RC4-SHA:RC4-MD5')

with urllib.request.urlopen('https://www.lawsociety.com.au/', context=ctx) as u:
    print(u.read())