The first thing you may want to do is check that your server is presenting it's certificates properly. You can do this by trying to connect to your server using OpenSSL. On a client machine with access, run:
openssl s_client –connect target_server_fqdn:636
This should return a nice print out of the server's certificate. The key here is checking the "Verify return code" printed at the end. You may get different codes, but generally speaking, you should get 0 for a valid certificate, and possibly 19 if you're self-signing.
If this fails, go back and check to ensure you have imported your server side certificates properly.
If you've passed this test, move on to testing your TLS connections from the client side.
On a client machine, run
ldapsearch -z -ZZ '(uid=<testusername>)'
This will force an LDAP lookup over an encrypted connection. If that's successful, you should get some user information back, and a check into the DS logs should yield the following:
[23/Sep/2011:07:48:57 -0500] conn=1631 op=0 EXT oid="X.X.X.X.XX.X.XX" name="startTLS"
[23/Sep/2011:07:48:57 -0500] conn=1631 op=0 RESULT err=0 tag=120 nentries=0 etime=0
[23/Sep/2011:07:48:57 -0500] conn=1631 SSL 256-bit AES
If this fails, you'll want to ensure the certificates were properly imported on the client side.
When troubleshooting, some common areas I've found myself looking frequently are:
1.) On the clients, in some cases (which someone here may be able to better explain), you might try to require signing by editing ldap.conf and including the line
TLS_REQCERT allow
2.) If the authentication GUI is giving you problems, you might try just explicitly turning on TLS for LDAP with
authconfig --enableldaptls --update
I've had problems with the GUI before, so I tend to stick to using CLI commands.
3.) And a final thing you might try (again , just for testing), is calling
cacertdir_rehash <dir where certs are stored>
Update
If you're looking for more help in actually creating self sign certificates, try the following:
1.) Create your own, self-signed CA Certificate:
certutil -S -n "<CA Certificate Name Here>" -s "cn=<CN Name Here>, dc=<Your DC's FQDN>" -2 -x -t "CT,," -m 1000 -v 120 -d . -k rsa
2.) Create a server certificate for the directory server
certutil -S -n "Cert-Name" -s "cn=<Server FQDN>" -c "<Name of CA Certificate>" -t "u,u,u" -m 1001 -v 120 -d . -k rsa
3.) Import both of these certificates into the directory server in the "Manage Certificates" section, selected under "Tasks"
4.) Enable TLS encryption
5.) Create an exportable certificate for clients and output it to a .pem file
certutil -d . -L -n "<CA Certificate Name>" -a > cacert.pem
6.) By means of your choosing - download the client certificate onto each client.
7.) Rehash the certificates by using the previously mentioned command
cacertdir_rehash <dir where certs are stored>
I would start by verifying the certificate as follows.
How to troubleshoot LDAP over SSL connection problems
http://support.microsoft.com/kb/938703
Step 1: Verify the Server Authentication certificate
Make sure that the Server Authentication certificate that you use meets the following requirements:
Step 2: Verify the Client Authentication certificate
In some cases, LDAPS uses a Client Authentication certificate if it is available on the client computer. If such a certificate is available, make sure that the certificate meets the following requirements:
Step 3: Check for multiple SSL certificates
Determine whether multiple SSL certificates meet the requirements that are described in step 1. Schannel (the Microsoft SSL provider) selects the first valid certificate that Schannel finds in the Local Computer store. If multiple valid certificates are available in the Local Computer store, Schannel may not select the correct certificate. A conflict with a certification authority (CA) certificate may occur if the CA is installed on a domain controller that you are trying to access through LDAPS.
Step 4: Verify the LDAPS connection on the server
Use the Ldp.exe tool on the domain controller to try to connect to the server by using port 636. If you cannot connect to the server by using port 636, see the errors that Ldp.exe generates. Also, view the Event Viewer logs to find errors. For more information about how to use Ldp.exe to connect to port 636, click the following article number to view the article in the Microsoft Knowledge Base:
321051 How to enable LDAP over SSL with a third-party certification authority
http://support.microsoft.com/kb/321051
Step 5: Enable Schannel logging
Enable Schannel event logging on the server and on the client computer. For more information about how to enable Schannel event logging, click the following article number to view the article in the Microsoft Knowledge Base:
260729 How to enable Schannel event logging in IIS
http://support.microsoft.com/kb/260729
Best Answer
Answering my own question. Turns out you can perform the import of certificate and keys from a p12 using pk12util however there are some issues with it. These are based on the binaries that follow with redhat DS 9.0
Pk12util will not respect the -n flag on import. The imported certificate entry nickname will be the alias in the p12 file. Also, for whatever reason, I had issues when importing 2 different certificates with different nicknames in their p12 keystores. The nickname in the store would end up the same as another one in there. This may have be triggered by the fact that the subject of the certificates were the same. Adding OU attribute with different values fixed the issue. Certutil can then be used to import the CA certificates