Nginx ssl handshke failed: peer closed connection in SSL handshake while SSL handshaking

javanginxssl

I'm having issue with some client failed to connect to server through HTTPS.
Our setup is like this:

[client (JVM)] –(https)–> [NGINX] –proxy_pass(http)–> [AppServer (jvm)]

I can connect to AppServer from my java client (runtime:jre1.8.0_65), browser and openssl without any issue.
Here is my openssl log with successful connection:

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

Here is my print out from JAVA with successful connection:

*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1493949464 bytes = { xxx, xxx, ... }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
***
main, WRITE: TLSv1.2 Handshake, length = 195
main, READ: TLSv1.2 Handshake, length = 89
*** ServerHello, TLSv1.2
RandomCookie:  GMT: 1493949462 bytes = { XXX, XXX, ... }
Session ID:  {XXX, XXX, ...}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
Extension ec_point_formats, formats: [uncompressed, ansiX962_compressed_prime, ansiX962_compressed_char2]
***

However, one of our client failed at handshake and get

java.net.SocketException: Connection reset by peer: socket write error

ClientHello print out are the same but cannot receive ServerHello…

Here is the error print out:

Manual Logon Thread, WRITE: TLSv1.2 Handshake, length = 195
Manual Logon Thread, waiting for close_notify or alert: state 1
Manual Logon Thread, Exception while waiting for close java.net.SocketException: Connection reset
Manual Logon Thread, handling exception: java.net.SocketException: Connection reset
Manual Logon Thread, SEND TLSv1.2 ALERT:  fatal, description = unexpected_message
Manual Logon Thread, WRITE: TLSv1.2 Alert, length = 2
Manual Logon Thread, Exception sending alert: java.net.SocketException: Connection reset by peer: socket write error
Manual Logon Thread, called closeSocket()
Manual Logon Thread, called close()
Manual Logon Thread, called closeInternal(true)
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at sun.security.ssl.InputRecord.readFully(Unknown Source)
    at sun.security.ssl.InputRecord.read(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.waitForClose(Unknown Source)
    at sun.security.ssl.HandshakeOutStream.flush(Unknown Source)
    at sun.security.ssl.Handshaker.kickstart(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.kickstartHandshake(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)

I looked into the Nginx server log and find the error message kind of random…

2017/05/05 12:02:06 [info] 21996#21996: *126588 SSL_do_handshake() failed (SSL: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol) while SSL handshaking, client: client.ip.address, server: 0.0.0.0:443
2017/05/05 12:02:07 [info] 21991#21991: *126591 peer closed connection in SSL handshake while SSL handshaking, client: client.ip.address, server: 0.0.0.0:443
2017/05/05 12:02:08 [info] 21997#21997: *126592 SSL_do_handshake() failed (SSL: error:1408A10B:SSL routines:SSL3_GET_CLIENT_HELLO:wrong version number) while SSL handshaking, client: client.ip.address, server: 0.0.0.0:443
2017/05/05 12:02:08 [info] 21998#21998: *126593 peer closed connection in SSL handshake while SSL handshaking, client: client.ip.address, server: 0.0.0.0:443
... many peer closed connection print out ...
2017/05/05 12:02:13 [info] 21995#21995: *126603 SSL_do_handshake() failed (SSL: error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher) while SSL handshaking, client: client.ip.address, server: 0.0.0.0:443

Anybody has suggestion on these error?

Best Answer

It turns out the client side has some program will scan and block HTTPS request if not permitted, however, don't scan the http connection...

After asking them to permit our host, the issue has been resolved.

Related Topic