Ubuntu – ssh “cannot resolve host” if host is an A entry, but works with CNAME

domain-name-systemgithubresolvesshUbuntu

I have a very strange behavior (bug?) with my ssh client. When I try to use it with an A record, say ssh -vvvT github.com I get the following error:

ssh -vvvT github.com
OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolve_addr: could not resolve name github.com as address: Name or service not known
ssh: Could not resolve host "github.com"

Which is different from the answer I get when I try with a CNAME that points also to the same A entry (www.github.com IN CNAME github.com) :

ssh -vT www.github.com                                                                                                                                           
OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Re-reading configuration after hostname canonicalisation
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to www.github.com [192.30.253.113] port 22.
debug1: Connection established.
[...]
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
debug1: Remote protocol version 2.0, remote software version libssh_0.7.0
debug1: no match: libssh_0.7.0
debug1: Authenticating to www.github.com:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
[...]
Permission denied (publickey).

We see that here it waits for the canonicalisation of DNS and then it can resolve it.

I am struggling with this since a couple of days. I have tried, I think, almost anything I found, such as:

  • disabling dnsmasq
  • refreshing dns cache
  • changing resolver servers

I have the same problem if I am at home or at work (laptop machine). So I guess the problem is really something with my machine.

The problem is not related only to github, I have the same problem with any A record I have tested…

And all the DNS clients or debug software I have tested can resolve theses records without problems (dig, nslookup, etc).

Does anyone have an idea? I'd like to understand and solve this, and not re-install the machine as I am starting to think about…

system: ubuntu 16.04
ssh version: ssh OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g  1 Mar 2016

EDIT 1:

As said, none of the "standard" techniques seems to work, because all seems to be working fine:

getent hosts github.com
192.30.253.113  github.com
192.30.253.112  github.com

sudo service nscd stop                                                                                                                                         
Failed to stop nscd.service: Unit nscd.service not loaded.

ssh -4 github.com
ssh: Could not resolve host "github.com"

EDIT 2:
this is my /etc/ssh/ssh_config file :

~ grep -v '^#' /etc/ssh/ssh_config
Host *
    SendEnv LANG LC_*
    HashKnownHosts yes
    GSSAPIAuthentication yes
    GSSAPIDelegateCredentials no
    CanonicalizeHostname yes
    CanonicalizeFallbackLocal no

and the dig output for github.com:

~ dig github.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15271
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;github.com.                        IN      A

;; ANSWER SECTION:
github.com.         3       IN      A       192.30.253.113
github.com.         3       IN      A       192.30.253.112

;; Query time: 0 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Mon Aug 28 14:33:48 CEST 2017
;; MSG SIZE  rcvd: 71

and for www.github.com :

~ dig www.github.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> www.github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11119
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.github.com.                    IN      A

;; ANSWER SECTION:
www.github.com.             3600    IN      CNAME   github.com.
github.com.         45      IN      A       192.30.253.112
github.com.         45      IN      A       192.30.253.113

;; Query time: 16 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Mon Aug 28 14:34:14 CEST 2017
;; MSG SIZE  rcvd: 89

EDIT 3:

I have tried to set CanonicalizeHostname=no instead of CanonicalizeHostname=yes and it works.

I do not understand the implication of the Canonicalization, because according to the openssh manual (as I understand it) tells that this is used only with CanonicalizePermittedCNAMEs rules. Which means that it should not avoid the use of A entries, or am I wrong?

Best Answer

Check if you have something in your /etc/hosts file that interfere with lookup. The hosts file is processed before DNS resolution.

Try the following to show what the system actually returns when ssh tries to resolve github.com:

getent hosts github.com

If this returns something else than the expected addresses, and your hosts file looks ok, you could try to disable DNS-caching.

dnsmasq (if installed):

sudo service dnsmasq stop

nscd (if installed):

sudo service nscd stop

It could also be interesting check if forcing ipv4 lookup and connection gives a different result:

ssh -4 github.com