Server Not Found in Kerberos Database – Locating the Database

active-directorykerberosmitkerberosredhat

Testing setup:

Weblogic 12.2.1.4 running on a Windows 10 machine joined to an active directory
- JVM 1.8.0_281
- The java web application is using Java GSSAPI to access the fileshare over Samba essentially using the code from https://github.com/hierynomus/smbj/issues/304#issuecomment-375603115
- This is a developer machine – the java system property "user.name" shows the developer's windows username
Linux fileserver running Red Hat Enterprise Linux 7, configured with sssd to connect to Active Directory
Linux fileserver running Red Hat Enterprise Linux 6, not using sssd, not using winbind (unclear how precisely it is configured for Active Directory)

From what I've been able to gather third-hand on conversations with sysadmins, MIT Kerberos is somehow involved in connecting the linux servers to Active Directory – but I have no more information on that.

(note: serverfault is having markdown table rendering issue – table displays fine in preview, but not in actual posted question, so surrounded in code block for now so that it doesn't all run together)

Test Results

| Source (all on same Windows 10 machine)  | Target FileServer     | Result  |
|------------------------------------------|-----------------------|---------|
| Weblogic application                     | RHEL 6                | Success |
| Weblogic application                     | RHEL 7                | **Fail:** Server not found in Kerberos database |
| Windows Explorer                         | RHEL 6                | Success |
| Windows Explorer                         | RHEL 7                | Success |

All tests done using the same user credentials.

The trace results from Weblogic (after setting the java system property sun.security.krb5.debug to true) are:

KrbAsReqBuild: PREAUTH FAILED/REQ, re-send AS-REQ
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsReq creating message
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=***************. UDP:88, timeout=30000, number of retries =3, #bytes=233
>>> KDCCommunication: kdc=***************. UDP: 88, timeout=30000,Attempt =1, #bytes=233
>>> KrbKdcReq send: #bytes read=100
>>> KrbKdcReq send: kdc=*****************. TCP:88, timeout=30000, number of retires =3, #bytes=233
>>> KDCCommunication: kdc=****************. TCP:88, timeout=30000,Attempt =1, #bytes=233
>>>DEBUG: TCPClient reading 2695 bytes
>>>KrbKdcReq send: #bytes read=2695
>>>KdcAccessibility: remove **********************.:88
>>>Etype: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>>KrbAsRep cons in KrbAsReq.getReply ******
Found ticket for ******@******** to go to krbtgt/******@****** expiring on ******
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for ******@******** to go to krbtgt/******@****** expiring on ******
Service ticket not found in the subject
>>> Credentials serviceCredsSingle: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes256CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=************. TCP:88, timeout=30000, number of retries =3, #bytes=2633
>>> KDCCommunication: kdc=************. TCP:88, timeout=30000,Attempt =1, #bytes=2633
>>>DEBUG: TCPClient reading 104 bytes
>>> KrbKdcReq send: #bytes read=104
>>> KdcAccessibility: remove *************.:88
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
        STime is **********
        suSec is **********
        error code is 7
        error Message is Server not found in Kerberos database
        sname is cifs/***********@***********
        msgType is 30

Searching for "Server not found in Kerberos database" yields a number of possibilities (DNS seems to be most common suggestion, other answers have suggested SPN registrations, TLS certs, not using FQDN, invalid host to realm mapping, host not part of domain, IPV4 vs IPV6)

The network admins say DNS is correct, which would appear to be substantiated by the fact that Windows explorer can connect to the RHEL 7 server just fine. But neither am I prepared to just blame the java code, since it does successfully connect to the RHEL 6 server.

I'm having difficulty finding a clear explanation of what Kerberos entries need to be configured where.

Is this "Kerberos database" on the Windows 10 machine, the fileserver, or the Active Directory KDC? Or are there multiple copies of this Kerberos database that each need entries?

Edit – Additional Details
I've learned a few new things and can provide some additional details.

The following are all from a Window's command prompt on the Developer machine.

Realm is determined by:

C:\>echo %userdnsdomain%
DC1.DC2.DC3

For the successful connection (RHEL 6 server)

C:\>powershell Get-ADComputer server1

DistinguishedName : CN=SERVER1,OU=ou1,OU=ou2,OU=ou3,DC=dc1,DC=dc2,DC=dc3
DNSHostName       : server1.dc2.dc3
Enabled           : True
Name              : SERVER1
ObjectClass       : Computer
ObjectGUID        : aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
SamAccountName    : server1$
SID               : ************************************
UserPrincipalName :

C:\>setspn -L server1
Registered ServicePrincipalNames for CN=SERVER1,OU=ou1,OU=ou2,OU=ou3,DC=dc1,DC=dc2, DC=dc3:
        HOST/server1.dc2.dc3
        HOST/SERVER1

C:\>nslookup server1
Server:  aa1.dc2.dc3
Address:  123.456.789.01

Name:    server1.dc2.dc3
Address:  123.456.7.890

C:\>nslookup 123.456.7.890
Server:  aa1.dc2.dc3
Address:  123.456.789.01

Name:    server1.dc2.dc3
Address:  123.456.7.890

For the failing connection (RHEL 7 server)

C:\>powershell Get-ADComputer server2

DistinguishedName : CN=SERVER2,OU=ou4,DC=dc1,DC=dc2,DC=dc3
DNSHostName       : server2.dc1.dc2.dc3
Enabled           : True
Name              : SERVER2
ObjectClass       : Computer
ObjectGUID        : bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb
SamAccountName    : SERVER2$
SID               : ************************************
UserPrincipalName :

C:\>setspn -L server2
Registered ServicePrincipalNames for CN=SERVER2,OU=ou4,DC=dc1,DC=dc2,DC=dc3:
        RestrictedKrbHost/SERVER2
        HOST/SERVER2
        RestrictedKrbHost/SERVER2.dc1.dc2.dc3
        HOST/SERVER2.dc1.dc2.dc3

C:\>nslookup server2
Server:  aa1.dc2.dc3
Address:  123.456.789.01

Name:    server2.dc1.dc2.dc3
Address:  12.345.6.78

C:\>nslookup 12.345.6.78
Server:  aa1.dc2.dc3
Address:  123.456.789.01

Name:    server2.dc2.dc3
Address:  12.345.6.78

Comparing the Weblogic trace results for the success vs failure:

success:

>>> DEBUG: ----Credentials----
        client: username@DC1.DC2.DC3
        server: cifs/server1.dc2.dc3@DC1.DC2.DC3
        ticket: sname: cifs/server1.dc2.dc3@DC1.DC2.DC3

failure:

>>>KRBError:
  ...
         error code is 7
         error Message is is Server not found in Kerberos database
         sname is cifs/server2.dc2.dc3@DC1.DC2.DC3
         msgType is 30

So I note a few things:

The successful (RHEL 6) server omits "dc1" in a number of places (but not all)
The successful and failing sname differ only in the server portion – they otherwise line up on the "dc" values.
The two servers are in different Organizational Units (though I don't think this is the issue)
There are a few difference in capitalization in the command results
There appears to be a difference in the nslookup results for server2 depending on whether it is looked up by server name (includes dc1) or ip address (excludes dc1)

Best Answer

This seems to be the culprit:

    error Message is Server not found in Kerberos database
    sname is cifs/***********@***********

The first question to ask is if the failing server has appropriate computer account in AD (and the fact it has sssd configured doesn't necessarily imply that depending on HOW sssd is configured).

Assuming this is present you may use setspn Windows command or OpenLDAP ldapsearch command to check if that computer object has appropriate service principals present. If I remeber correctly HOST/* SPN should suffice but if not (please check Microsoft documentation on that topic) cifs/ SPN may be necessary. setspn command can be used to create them (given appropriate privileges of the user trying to do that).

The next step is to make sure samba configuration on the Linux server has keytab created and accessible and making sure the above-mentioned cifs/* principal is present there.

I am not AD expert (just attaching some Linux systems there) so this may not be complete list but hopefully - it gives you a start.

Related Solutions

NFS (with Kerberos) mount failing due to “Server not found in Kerberos database” error

It seems that the reverse name resolution for the IP is not matching the name you are expecting. Make sure that mercury.domain.com and sol.domain.com are the first name that you added to /etc/hosts after the relevant IP address. To be safe, just add a couple of lines on the top with the IP address of the machine and the hostname that kerberos expects.

10.x.y.z sol.domain.com sol ip-blah-blah
10.a.b.c mercury.domain.com mercury ip-other-other

Make sure that both of these lines are present on both the client and the server.

It is also a good idea verify the setup by running the following command on both the client and the server. Make sure that the first hostname printed for each IP address is the one you are expecting.

getent hosts 10.x.y.z 10.a.b.c

Client not found in kerberos database while getting initial

This command:

c:/> ktpass -princ HTTP/vmproxy.mydomain.com@MYDOMAIN.COM -mapuser squid@MYDOMAIN.COM -crypto rc4-hmac-nt -pass P@ssw0rd -ptype KRB5_NT_PRINCIPAL -out krb5.keytab

I believe sets HTTP/vmproxy.mydomain.com@MYDOMAIN.COM to be a service principal associated with the squid@MYDOMAIN.COM user in AD. Active Directory does not typically allow you to authenticate as a service principal (specifically, does not let it acquire a TGT via an AS_REQ); in theory, service principals are supposed to be for accepting user credentials, not for authenticating to your kerberos realm.

This is different from Unix KDCs, which typically do not distinguish between "service principals" and "user principals" by default, allowing either to authenticate via kinit. In Active Directory, instead the KDC pretends that the principal doesn't exist when you try to kinit as it, which tends to be rather confusing.

If you can successfully authenticate as any other user, I would try testing if the service principal is functional by instead running:

$ kinit some_other_user
$ kvno HTTP/vmproxy.mydomain.com@MYDOMAIN.COM

Which just acquires a service ticket for HTTP/vmproxy.mydomain.com@MYDOMAIN.COM. Or, you may be able to authenticate as squid@MYDOMAIN.COM instead of the service principal you set up.

If you need to be able to kinit as HTTP/vmproxy.mydomain.com@MYDOMAIN.COM directly, I think that is possible, but you need to change the userPrincipalName attribute on the relevant AD account. I can't remember at the moment how to achieve that, but if I recall correctly, you can only have one such UPN; you can't have multiple different principal names you can kinit as for the same account.

Best Answer

Related Solutions

NFS (with Kerberos) mount failing due to “Server not found in Kerberos database” error

Client not found in kerberos database while getting initial

Related Topic