Java ssl/https client using a self-signed certificate

httpsjavaself-signedssltruststore

I am attempting to write a Java https client using jdk version 1.6.0_32. I have a self-signed public certificate that I have imported into a new truststore. The issue is that I keep getting "Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake" The SSL debug output is as follows:

C:\Users\csheets\eclispe_workspace\sdpweb\InstallSSLCert>java TestCert
keyStore is :
keyStore type is : jks
keyStore provider is :
init keystore
init keymanager of type SunX509
trustStore is: c:\users\csheets\4startrust.ts
trustStore type is : jks
trustStore provider is :
init truststore
adding as trusted cert:
Subject: CN=4starserver.servehttp.com
Issuer:  CN=4STAR
Algorithm: RSA; Serial number: 0x200000001
Valid from Mon May 14 11:25:15 MDT 2012 until Tue May 14 11:25:15 MDT 2013

trigger seeding of SecureRandom
done seeding SecureRandom
Allow unsafe renegotiation: true
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1320442869 bytes = { 175, 184, 30, 195, 10, 55, 219, 232, 23
, 237, 63, 239, 83, 49, 125, 80, 10, 174, 112, 210, 61, 53, 232, 66, 179, 22, 16
1, 80 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH
_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC
_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_
DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SH
A, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_
WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WI
TH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: SSLv3 Handshake, length = 527
*** ServerHello, SSLv3
RandomCookie:  GMT: 1320442987 bytes = { 158, 143, 79, 29, 193, 160, 122, 201, 8
1, 67, 17, 26, 159, 243, 54, 202, 255, 156, 125, 121, 132, 174, 17, 202, 222, 65
, 252, 77 }
Session ID:  {131, 30, 0, 0, 6, 235, 145, 226, 5, 214, 118, 217, 18, 123, 46, 20
4, 51, 182, 211, 225, 48, 172, 95, 70, 144, 4, 178, 150, 166, 75, 166, 29}
Cipher Suite: SSL_RSA_WITH_RC4_128_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
%% Created:  [Session-1, SSL_RSA_WITH_RC4_128_SHA]
** SSL_RSA_WITH_RC4_128_SHA
*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=4starserver.servehttp.com
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 129409122589634486230897608496688768265641316152022572826296995250983
  80968933262586507340653723460037384941316405007365056646964455523390263136350462
  59738068084572819329229707448458528878467480278641098016863640927986379246142644
  62745346179244207665720440347282685862962453661441013596685879879277368109494267

public exponent: 65537
Validity: [From: Mon May 14 11:25:15 MDT 2012,
           To: Tue May 14 11:25:15 MDT 2013]
Issuer: CN=4STAR
SerialNumber: [    02000000 01]

]
Algorithm: [SHA1withRSA]
Signature:
0000: 00 8E A1 F4 58 22 F2 C2   A9 1D C6 CB 5A 23 F5 A5  ....X"......Z#..
0010: 02 3A C9 FF 83 96 1A 13   3A 0F 59 D5 1E 1F 56 85  .:......:.Y...V.
0020: AB 4A 46 8D F3 43 E8 BA   B3 F9 B7 8C FB 76 AD D5  .JF..C.......v..
0030: 9F 15 47 DC 30 72 F9 BA   B1 FF DA 2C 25 89 FF 30  ..G.0r.....,%..0
0040: C4 4F BA D6 0C B9 30 10   B0 4B 74 EF 8A F4 5D F1  .O....0..Kt...].
0050: AC 2C 47 D9 C2 F5 A0 AF   CE 8B 76 53 36 A3 BE 11  .,G.......vS6...
0060: 7E BA 1F 4A 67 C1 69 EF   C3 E6 32 E2 0D 09 93 66  ...Jg.i...2....f
0070: 92 21 66 88 95 CA BD C8   FF CF 79 9D 7E F3 DC E0  .!f.......y.....

]
***
Found trusted certificate:
[
[
  Version: V3
  Subject: CN=4starserver.servehttp.com
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 129409122589634486230897608496688768265641316152022572826296995250983
80968933262586507340653723460037384941316405007365056646964455523390263136350462
59738068084572819329229707448458528878467480278641098016863640927986379246142644
62745346179244207665720440347282685862962453661441013596685879879277368109494267

 public exponent: 65537
 Validity: [From: Mon May 14 11:25:15 MDT 2012,
           To: Tue May 14 11:25:15 MDT 2013]
 Issuer: CN=4STAR
 SerialNumber: [    02000000 01]

 ]
Algorithm: [SHA1withRSA]
Signature:
0000: 00 8E A1 F4 58 22 F2 C2   A9 1D C6 CB 5A 23 F5 A5  ....X"......Z#..
0010: 02 3A C9 FF 83 96 1A 13   3A 0F 59 D5 1E 1F 56 85  .:......:.Y...V.
0020: AB 4A 46 8D F3 43 E8 BA   B3 F9 B7 8C FB 76 AD D5  .JF..C.......v..
0030: 9F 15 47 DC 30 72 F9 BA   B1 FF DA 2C 25 89 FF 30  ..G.0r.....,%..0
0040: C4 4F BA D6 0C B9 30 10   B0 4B 74 EF 8A F4 5D F1  .O....0..Kt...].
0050: AC 2C 47 D9 C2 F5 A0 AF   CE 8B 76 53 36 A3 BE 11  .,G.......vS6...
0060: 7E BA 1F 4A 67 C1 69 EF   C3 E6 32 E2 0D 09 93 66  ...Jg.i...2....f
0070: 92 21 66 88 95 CA BD C8   FF CF 79 9D 7E F3 DC E0  .!f.......y.....

]
*** ServerHelloDone
*** ClientKeyExchange, RSA PreMasterSecret, SSLv3
main, WRITE: SSLv3 Handshake, length = 132
SESSION KEYGEN:
PreMaster Secret:
0000: 03 00 07 86 97 89 23 A4   73 85 54 59 A4 76 DD 85  ......#.s.TY.v..
0010: 12 1A 28 1B 71 CC 7A B2   EE 0F 65 60 26 30 6C B4  ..(.q.z...e`&0l.
0020: B4 92 2D 15 50 51 E5 10   77 96 8E B0 4F 30 57 73  ..-.PQ..w...O0Ws
CONNECTION KEYGEN:
Client Nonce:
0000: 4F B4 5C F5 AF B8 1E C3   0A 37 DB E8 17 ED 3F EF  O.\......7....?.
0010: 53 31 7D 50 0A AE 70 D2   3D 35 E8 42 B3 16 A1 50  S1.P..p.=5.B...P
Server Nonce:
0000: 4F B4 5C 6B 9E 8F 4F 1D   C1 A0 7A C9 51 43 11 1A  O.\k..O...z.QC..
0010: 9F F3 36 CA FF 9C 7D 79   84 AE 11 CA DE 41 FC 4D  ..6....y.....A.M
Master Secret:
0000: 60 59 16 75 E0 5E 4B 64   D6 6B 56 18 9B F2 C8 7A  `Y.u.^Kd.kV....z
0010: F8 DF 65 C6 C0 12 92 62   15 A1 4E 5F 53 D3 02 EF  ..e....b..N_S...
0020: 9B EF ED FD 1E 01 61 6F   AC 39 E0 5B AD 87 BF 25  ......ao.9.[...%
Client MAC write Secret:
0000: C1 36 79 97 E6 71 22 D0   27 D0 41 88 F9 F5 8D C2  .6y..q".'.A.....
0010: EA A3 97 FB                                        ....
Server MAC write Secret:
0000: C4 00 15 49 31 29 B2 F3   06 90 59 F0 5A 4D 3D 45  ...I1)....Y.ZM=E
0010: 32 B2 B6 83                                        2...
Client write key:
0000: E6 46 87 A2 16 52 04 11   73 15 E8 23 9F E6 02 A3  .F...R..s..#....
Server write key:
0000: 04 63 1D 64 E7 25 FC E4   53 FC 43 04 33 3C ED 6E  .c.d.%..S.C.3<.n
... no IV used for this cipher
main, WRITE: SSLv3 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 61, 221, 238, 253, 97, 36, 152, 79, 254, 95, 226, 136, 55, 16, 2
07, 66, 58, 197, 233, 254, 125, 99, 11, 0, 138, 51, 139, 62, 175, 123, 52, 167,
131, 216, 245, 97 }
***
main, WRITE: SSLv3 Handshake, length = 60
main, received EOFException: error
main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host close
d connection during handshake
main, SEND SSLv3 ALERT:  fatal, description = handshake_failure
main, WRITE: SSLv3 Alert, length = 22
main, called closeSocket()
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host clos
ed connection during handshake
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)

    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Un
known Source)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Sou
rce)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Sou
rce)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect
(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unkn
own Source)
    at TestCert.main(TestCert.java:92)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
    ... 9 more

Not sure how to interpret all of this information but it does seem like my certificate is being found and trusted – what does the ClientKeyExchange mean – could it be that the server is expecting client authentication or is this just the certificate info being exchanged. The whole Java keystore, trustore and self-signed certificates seems to create a lot of complexity and confusion – at least for me.

My test client code is as follows:

import java.io.*;
import java.net.URL;
import java.net.URLConnection;

import javax.net.ssl.*;
import javax.net.ssl.X509TrustManager;
import java.security.cert.X509Certificate;
import java.security.*;

public class TestCert {
public static void main(String[] args) throws Exception {
    System.setProperty("javax.net.debug", "ssl");
    System.setProperty("sun.security.ssl.allowUnsafeRenegotiation", "true");
    System.setProperty("javax.net.ssl.trustStore", "c:\\users\\csheets\\4startrust.ts");
    System.setProperty("javax.net.ssl.trustStorePassword", "mypassword");


    SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();

    URL url = new URL("https://4starserver.servehttp.com:777/?username=0&password=0&command=WEBAUTH&TRAN=2&MERCHANT=9999999999119911&FNAME=TONY&LNAME=PISCOPO&CC=4111111111111111&EXP=0613&AMOUNT=99.98");
    HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
    conn.setSSLSocketFactory(sslsocketfactory);

    InputStream inputstream = conn.getInputStream();
    InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
    BufferedReader bufferedreader = new BufferedReader(inputstreamreader);

    String string = null;
    while ((string = bufferedreader.readLine()) != null) {
        System.out.println("Received " + string);
    }
}
}

I have tried other things like creating my own TrustManager and TrustManager as well but that has not seemed to work either. Any assistance would be greatly appreciated.

UPDATE:
The server port was changed to 443 so could test with Qualys SSL Lab test and I got the following results:

Common Error Messages

Connect timed out - server did not respond to our connection request
No route to host - unable to reach the server
Unable to connect to server - failed to connect to the server
Unrecognized SSL message, plaintext connection? - the server responded with plain-text HTTP on HTTPS port
Received fatal alert: handshake_failure - this is either a faulty SSL server or some other server listening on port 443; if the SSL version of the web site works in your browser, please report this issue to us
Known Issues

Could not generate DH keypair - due to a known problem with the underlying SSL library (Sun's JSSE implementation) we are unable to assess the sites that offer only DHE handshakes stronger than 1024 bits.

The strange thing is that I seem to get a response back from the server if I just put the URL https://server.com/ in the browser – but on the server evidently it seems that there are several requests / connections being made. Seems like there be something not quite right on the server side?

Best Answer

You certificate is trusted (using your custom trust store), this is not a trust manager issue. You don't need to allow unsafe renegotiation (nor to specify the default SSLSocketFactory).

There seems to be a problem with the TLS setup, since it works with System.setProperty("https.protocols", "SSLv3");. You also get a handshake problem if you force -tls1 with openssl s_client.

If you're in control of the server, I'd suggest trying to put it on port 443 and test it via the Qualys SSL labs test (which is more detailed and catches more issues than many other tests available).

Related Solutions

Java HTTPS client certificate authentication

Finally managed to solve all the issues, so I'll answer my own question. These are the settings/files I've used to manage to get my particular problem(s) solved;

The client's keystore is a PKCS#12 format file containing

The client's public certificate (in this instance signed by a self-signed CA)
The client's private key

To generate it I used OpenSSL's pkcs12 command, for example;

openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12 -name "Whatever"

Tip: make sure you get the latest OpenSSL, not version 0.9.8h because that seems to suffer from a bug which doesn't allow you to properly generate PKCS#12 files.

This PKCS#12 file will be used by the Java client to present the client certificate to the server when the server has explicitly requested the client to authenticate. See the Wikipedia article on TLS for an overview of how the protocol for client certificate authentication actually works (also explains why we need the client's private key here).

The client's truststore is a straight forward JKS format file containing the root or intermediate CA certificates. These CA certificates will determine which endpoints you will be allowed to communicate with, in this case it will allow your client to connect to whichever server presents a certificate which was signed by one of the truststore's CA's.

To generate it you can use the standard Java keytool, for example;

keytool -genkey -dname "cn=CLIENT" -alias truststorekey -keyalg RSA -keystore ./client-truststore.jks -keypass whatever -storepass whatever
keytool -import -keystore ./client-truststore.jks -file myca.crt -alias myca

Using this truststore, your client will try to do a complete SSL handshake with all servers who present a certificate signed by the CA identified by myca.crt.

The files above are strictly for the client only. When you want to set-up a server as well, the server needs its own key- and truststore files. A great walk-through for setting up a fully working example for both a Java client and server (using Tomcat) can be found on this website.

Issues/Remarks/Tips

Client certificate authentication can only be enforced by the server.
(Important!) When the server requests a client certificate (as part of the TLS handshake), it will also provide a list of trusted CA's as part of the certificate request. When the client certificate you wish to present for authentication is not signed by one of these CA's, it won't be presented at all (in my opinion, this is weird behaviour, but I'm sure there's a reason for it). This was the main cause of my issues, as the other party had not configured their server properly to accept my self-signed client certificate and we assumed that the problem was at my end for not properly providing the client certificate in the request.
Get Wireshark. It has great SSL/HTTPS packet analysis and will be a tremendous help debugging and finding the problem. It's similar to -Djavax.net.debug=ssl but is more structured and (arguably) easier to interpret if you're uncomfortable with the Java SSL debug output.
It's perfectly possible to use the Apache httpclient library. If you want to use httpclient, just replace the destination URL with the HTTPS equivalent and add the following JVM arguments (which are the same for any other client, regardless of the library you want to use to send/receive data over HTTP/HTTPS):
```
-Djavax.net.debug=ssl
-Djavax.net.ssl.keyStoreType=pkcs12
-Djavax.net.ssl.keyStore=client.p12
-Djavax.net.ssl.keyStorePassword=whatever
-Djavax.net.ssl.trustStoreType=jks
-Djavax.net.ssl.trustStore=client-truststore.jks
-Djavax.net.ssl.trustStorePassword=whatever
```

Java – Accept server’s self-signed ssl certificate in Java client

You have basically two options here: add the self-signed certificate to your JVM truststore or configure your client to

Option 1

Export the certificate from your browser and import it in your JVM truststore (to establish a chain of trust):

<JAVA_HOME>\bin\keytool -import -v -trustcacerts
-alias server-alias -file server.cer
-keystore cacerts.jks -keypass changeit
-storepass changeit

Option 2

Disable Certificate Validation:

// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { 
    new X509TrustManager() {     
        public java.security.cert.X509Certificate[] getAcceptedIssuers() { 
            return new X509Certificate[0];
        } 
        public void checkClientTrusted( 
            java.security.cert.X509Certificate[] certs, String authType) {
            } 
        public void checkServerTrusted( 
            java.security.cert.X509Certificate[] certs, String authType) {
        }
    } 
}; 

// Install the all-trusting trust manager
try {
    SSLContext sc = SSLContext.getInstance("SSL"); 
    sc.init(null, trustAllCerts, new java.security.SecureRandom()); 
    HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (GeneralSecurityException e) {
} 
// Now you can access an https URL without having the certificate in the truststore
try { 
    URL url = new URL("https://hostname/index.html"); 
} catch (MalformedURLException e) {
}

Note that I do not recommend the Option #2 at all. Disabling the trust manager defeats some parts of SSL and makes you vulnerable to man in the middle attacks. Prefer Option #1 or, even better, have the server use a "real" certificate signed by a well known CA.