SSL – How to Enable TLS 1.3 Support on Windows

Reboot. Changes to Schannel settings do not take effect until the system is rebooted.

I wondered about the same thing, looked at the source code (“Use the source, Luke!” :), and it simply is the return value of SSL_connect() from the OpenSSL library. The documentation states that:

• Accepted (1) means:

  The TLS/SSL handshake was successfully completed, a TLS/SSL connection has been established.

• Rejected (0) means:

  The TLS/SSL handshake was not successful but was shut down controlled and by the specifications of the TLS/SSL protocol. Call SSL_get_error() with the return value ret to find out the reason.

• Failed (<0) means:

  The TLS/SSL handshake was not successful, because a fatal error occurred either at the protocol level or a connection failure occurred. The shutdown was not clean. It can also occur [if] action is need to continue the operation for non-blocking BIOs. Call SSL_get_error() with the return value ret to find out the reason.

The latter two can be followed by a N/A, if https is not available.

So I think sslscan should contain a --verbose or -v option that calls SSL_get_error() and outputs the actual reason it failed (or was rejected).
That would be rather useful. Because right now, it isn’t.

For now, all I can recommend is to manually connect with a more real-world client, force the usage of a certain cipher on said client or on the server, and then have it show you the actual reason.
Unless you want to improve sslscan’s code, of course. :)