Using the Same SSL/TLS Certificate for Server and Client – Guide

If you have 5 web servers behind a load balancer (...) do you need SSL certificates for all the servers,

It depends.

If you do your load balancing on the TCP or IP layer (OSI layer 4/3, a.k.a L4, L3), then yes, all HTTP servers will need to have the SSL certificate installed.

If you load balance on the HTTPS layer (L7), then you'd commonly install the certificate on the load balancer alone, and use plain un-encrypted HTTP over the local network between the load balancer and the webservers (for best performance on the web servers).

If you have a large installation, then you may be doing Internet -> L3 load balancing -> layer of L7 SSL concentrators -> load balancers -> layer of L7 HTTP application servers...

Willy Tarreau, the author of HAProxy, has a really nice overview of the canonical ways of load balancing HTTP/HTTPS.

If you install a certificate on each server, then be sure to get a certificate that supports this. Normally certificates can be installed on multiple servers, as long as the servers all serve traffic for one Fully Qualified Domain Name only. But verify what you're buying, certificate issuers can have a confusing product portfolio...

In order to download the certificate, you need to use the client built into openssl like so:

echo -n | openssl s_client -connect $HOST:$PORTNUMBER -servername $SERVERNAME \
    | openssl x509 > /tmp/$SERVERNAME.cert

That will save the certificate to /tmp/$SERVERNAME.cert.

The -servername is used to select the correct certificate when multiple are presented, in the case of SNI.

You can use -showcerts if you want to download all the certificates in the chain. But if you just want to download the server certificate, there is no need to specify -showcerts. The x509 at the end will strip out the intermediate certs, you will need to use sed -n '/-----BEGIN/,/-----END/p' instead of the x509 at the end.

echo -n gives a response to the server, so that the connection is released

openssl x509 removes information about the certificate chain and connection details. This is the preferred format to import the certificate into other keystores.