Summary

I had to rebuild a server, and run into an interesting issue. Puppet refuses to verify the certificates, even after removing /var/lib/puppet/ssl and cleaning the certificate off of the master.

Servers

Master:
OS: Ubuntu 14.04
Puppet Master version: 3.4.3-1 (using Webrick)

Agent:
OS: Ubuntu 12.04
Puppet version: 2.7.11
(Note that I replaced the hostname with 'agent-server.com' in the below output)

Replication Steps

• Remove SSL dir on agent-server:

  rm -fr /var/lib/puppet/ssl

• Clean the certificate on the Puppet Master:

  puppet cert clean agent-server.com

• Restart Puppet Master:

  /etc/init.d/puppetmaster restart

• Run puppet agent:

  puppet agent -t

Error messages:

Agent:

root@agent-server:~# puppet agent -t
info: Creating a new SSL key for agent-server.com
info: Caching certificate for ca
info: Creating a new SSL certificate request for agent-server.com
info: Certificate Request fingerprint (md5): F2:2A:AD:3C:D5:E8:13:82:1D:C5:80:B4:FD:23:C4:86
info: Caching certificate for agent-server.com
info: Caching certificate_revocation_list for ca
err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
err: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client 

Puppet Master

[2017-08-29 18:08:51] 10.88.0.208 - - [29/Aug/2017:18:08:51 UTC] "GET /production/certificate/ca? HTTP/1.1" 200 1939
[2017-08-29 18:08:51] - -> /production/certificate/ca?
[2017-08-29 18:08:51] 10.88.0.208 - - [29/Aug/2017:18:08:51 UTC] "GET /production/certificate/agent-server.com? HTTP/1.1" 404 78
[2017-08-29 18:08:51] - -> /production/certificate/agent-server.com?
[2017-08-29 18:08:51] 10.88.0.208 - - [29/Aug/2017:18:08:51 UTC] "GET /production/certificate_request/agent-server.com? HTTP/1.1" 404 86
[2017-08-29 18:08:51] - -> /production/certificate_request/agent-server.com?
[2017-08-29 18:08:51] 10.88.0.208 - - [29/Aug/2017:18:08:51 UTC] "PUT /production/certificate_request/agent-server.com HTTP/1.1" 200 1448
[2017-08-29 18:08:51] - -> /production/certificate_request/agent-server.com
[2017-08-29 18:08:51] 10.88.0.208 - - [29/Aug/2017:18:08:51 UTC] "GET /production/certificate/agent-server.com? HTTP/1.1" 200 1448
[2017-08-29 18:08:51] - -> /production/certificate/agent-server.com?
[2017-08-29 18:08:56] 10.88.0.208 - - [29/Aug/2017:18:08:56 UTC] "GET /production/certificate_revocation_list/ca? HTTP/1.1" 200 11220
[2017-08-29 18:08:56] - -> /production/certificate_revocation_list/ca?
[2017-08-29 18:08:56] ERROR OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: sslv3 alert certificate revoked
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick.rb:35:in `accept'
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick.rb:35:in `block (2 levels) in listen'
/usr/lib/ruby/1.9.1/webrick/server.rb:191:in `call'
/usr/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
[2017-08-29 18:08:56] ERROR OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: sslv3 alert certificate revoked
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick.rb:35:in `accept'
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick.rb:35:in `block (2 levels) in listen'
/usr/lib/ruby/1.9.1/webrick/server.rb:191:in `call'
/usr/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread

So judging by the logs it looks like the certificate is being revoked, even though it is a brand new one. In addition, it can't be a time issue because the two servers are very close, only apart by 2-3 seconds.

I'm rather stumped, unfortunately. Any help is appreciated.