Bind not able to query some servers / domains

binddig

We have been experiencing problems with our company's DNS server when trying to resolve only certain domains, we are running BIND 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 on a CentOS 6.5 server. We are autoritative for some zones and our internal clients and mail system resolve using this server. One of the domains we are having trouble resolving is www.dhl.com, here's what we get when querying using dig:

[root@serverx etc] dig www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> www.dhl.com
;; global options: +cmd
;; connection timed out; no servers could be reached

and

[root@serverx etc]# dig +trace www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +trace www.dhl.com
;; global options: +cmd
.           517419  IN  NS  g.root-servers.net.
.           517419  IN  NS  a.root-servers.net.
.           517419  IN  NS  h.root-servers.net.
.           517419  IN  NS  m.root-servers.net.
.           517419  IN  NS  f.root-servers.net.
.           517419  IN  NS  b.root-servers.net.
.           517419  IN  NS  l.root-servers.net.
.           517419  IN  NS  j.root-servers.net.
.           517419  IN  NS  k.root-servers.net.
.           517419  IN  NS  e.root-servers.net.
.           517419  IN  NS  i.root-servers.net.
.           517419  IN  NS  d.root-servers.net.
.           517419  IN  NS  c.root-servers.net.
;; Received 496 bytes from 192.168.X.X#53(192.168.X.X) in 11 ms

com.            172800  IN  NS  a.gtld-servers.net.
com.            172800  IN  NS  d.gtld-servers.net.
com.            172800  IN  NS  k.gtld-servers.net.
com.            172800  IN  NS  b.gtld-servers.net.
com.            172800  IN  NS  j.gtld-servers.net.
com.            172800  IN  NS  c.gtld-servers.net.
com.            172800  IN  NS  h.gtld-servers.net.
com.            172800  IN  NS  l.gtld-servers.net.
com.            172800  IN  NS  e.gtld-servers.net.
com.            172800  IN  NS  g.gtld-servers.net.
com.            172800  IN  NS  m.gtld-servers.net.
com.            172800  IN  NS  f.gtld-servers.net.
com.            172800  IN  NS  i.gtld-servers.net.
;; Received 489 bytes from 202.12.27.33#53(202.12.27.33) in 6128 ms

dhl.com.        172800  IN  NS  ns4.dhl.com.
dhl.com.        172800  IN  NS  ns6.dhl.com.
dig: couldn't get address for 'ns4.dhl.com': no more

And when I do dig using google's dns server:

dig @8.8.4.4 www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> @8.8.4.4 www.dhl.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11325
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.dhl.com.           IN  A

;; ANSWER SECTION:
www.dhl.com.        1619    IN  CNAME   ngw.dhl.com.edgesuite.net.
ngw.dhl.com.edgesuite.net. 8520 IN  CNAME   a1085.g.akamai.net.
a1085.g.akamai.net. 19  IN  A   23.74.2.113
a1085.g.akamai.net. 19  IN  A   23.74.2.120

;; Query time: 229 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Thu Dec  4 14:47:56 2014
;; MSG SIZE  rcvd: 129

No problem!, and from the same server…

When you look to "/var/log/messages" nothing gets logged!!!. Again, this only happens with certain domains and this server was working ok a couple of days ago, we also have disabled selinux for testing purposes.

This is our named.conf file (named is runnig in a chroot environment):

options {
//      listen-on port 53 { 127.0.0.1;192.168.xx.x; };
//      listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
//      allow-query     { localhost; };
        allow-query     { any; };
        recursion yes;
        allow-recursion { recursive-clients; };
//      query-source address * port 53;
//      dnssec-enable yes;
        dnssec-enable no;
//      dnssec-validation yes;
        dnssec-validation no;
        dnssec-lookaside auto;

        /* Path to ISC DLV key */
        bindkeys-file "/etc/named.iscdlv.key";

        managed-keys-directory "/var/named/dynamic";
        allow-transfer { xxx.x.x.x; xxx.x.x.x; xxx.x.x.x; 127.0.0.1; };
        allow-update { 192.168.xx.xx; };
//        forwarders { 8.8.4.4; };
};

acl recursive-clients { xxx.x.x.x/24; 127.0.0.1; xxx.xxx.xx.x/24; xx.xxx.xxx.xxx/29; xxx.xxx.xx.x;};

logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };

};

zone "." IN {
        type hint;
        file "named.ca";
};


zone "domain.com.xx" IN {
        type master;
        file "db.domain.com.xx";
        allow-transfer { xxx.xxx.xx.xx; xxx.xxx.xxx.x; };
};

zone "xx.xxx.xxx.in-addr.arpa" IN {
        type master;
        file "db.xxx.xxx.xx";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

Any idea guys???… I've been doing tests and researching since yesterday and I cannot figure what is happening!!!

Thanks in advance for any help or idea.

Here it the result using dig +trace +additional www.dhl.com:

dig +trace +additional www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +trace +additional www.dhl.com
;; global options: +cmd
.           518340  IN  NS  h.root-servers.net.
.           518340  IN  NS  l.root-servers.net.
.           518340  IN  NS  e.root-servers.net.
.           518340  IN  NS  k.root-servers.net.
.           518340  IN  NS  i.root-servers.net.
.           518340  IN  NS  m.root-servers.net.
.           518340  IN  NS  b.root-servers.net.
.           518340  IN  NS  c.root-servers.net.
.           518340  IN  NS  g.root-servers.net.
.           518340  IN  NS  f.root-servers.net.
.           518340  IN  NS  d.root-servers.net.
.           518340  IN  NS  a.root-servers.net.
.           518340  IN  NS  j.root-servers.net.
k.root-servers.net. 518345  IN  A   193.0.14.129
k.root-servers.net. 518345  IN  AAAA    2001:7fd::1
b.root-servers.net. 518345  IN  A   192.228.79.201
b.root-servers.net. 518345  IN  AAAA    2001:500:84::b
c.root-servers.net. 518345  IN  A   192.33.4.12
c.root-servers.net. 518345  IN  AAAA    2001:500:2::c
i.root-servers.net. 518345  IN  A   192.36.148.17
i.root-servers.net. 518345  IN  AAAA    2001:7fe::53
f.root-servers.net. 518345  IN  A   192.5.5.241
f.root-servers.net. 518345  IN  AAAA    2001:500:2f::f
h.root-servers.net. 518345  IN  A   128.63.2.53
h.root-servers.net. 518345  IN  AAAA    2001:500:1::803f:235
a.root-servers.net. 518345  IN  A   198.41.0.4
;; Received 508 bytes from 192.168.x.x#53(192.168.x.x) in 11363 ms

com.            172800  IN  NS  d.gtld-servers.net.
com.            172800  IN  NS  l.gtld-servers.net.
com.            172800  IN  NS  b.gtld-servers.net.
com.            172800  IN  NS  m.gtld-servers.net.
com.            172800  IN  NS  c.gtld-servers.net.
com.            172800  IN  NS  k.gtld-servers.net.
com.            172800  IN  NS  i.gtld-servers.net.
com.            172800  IN  NS  a.gtld-servers.net.
com.            172800  IN  NS  f.gtld-servers.net.
com.            172800  IN  NS  g.gtld-servers.net.
com.            172800  IN  NS  j.gtld-servers.net.
com.            172800  IN  NS  h.gtld-servers.net.
com.            172800  IN  NS  e.gtld-servers.net.
a.gtld-servers.net. 172800  IN  A   192.5.6.30
b.gtld-servers.net. 172800  IN  A   192.33.14.30
c.gtld-servers.net. 172800  IN  A   192.26.92.30
d.gtld-servers.net. 172800  IN  A   192.31.80.30
e.gtld-servers.net. 172800  IN  A   192.12.94.30
f.gtld-servers.net. 172800  IN  A   192.35.51.30
g.gtld-servers.net. 172800  IN  A   192.42.93.30
h.gtld-servers.net. 172800  IN  A   192.54.112.30
i.gtld-servers.net. 172800  IN  A   192.43.172.30
j.gtld-servers.net. 172800  IN  A   192.48.79.30
k.gtld-servers.net. 172800  IN  A   192.52.178.30
l.gtld-servers.net. 172800  IN  A   192.41.162.30
m.gtld-servers.net. 172800  IN  A   192.55.83.30
a.gtld-servers.net. 172800  IN  AAAA    2001:503:a83e::2:30
;; Received 489 bytes from 202.12.27.33#53(202.12.27.33) in 4335 ms

dhl.com.        172800  IN  NS  ns4.dhl.com.
dhl.com.        172800  IN  NS  ns6.dhl.com.
ns4.dhl.com.        172800  IN  A   165.72.192.16
ns6.dhl.com.        172800  IN  A   199.40.254.166
dig: couldn't get address for 'ns4.dhl.com': no more

Output from dig +tcp www.redhat.com

dig +tcp www.redhat.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +tcp www.redhat.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62110
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 8, ADDITIONAL: 8

;; QUESTION SECTION:
;www.redhat.com.            IN  A

;; ANSWER SECTION:
www.redhat.com.     11  IN  CNAME   wildcard.redhat.com.edgekey.net.
wildcard.redhat.com.edgekey.net. 20484 IN CNAME wildcard.redhat.com.edgekey.net.globalredir.akadns.net.
wildcard.redhat.com.edgekey.net.globalredir.akadns.net. 2485 IN CNAME e1890.b.akamaiedge.net.
e1890.b.akamaiedge.net. 20  IN  A   172.229.164.152

;; AUTHORITY SECTION:
b.akamaiedge.net.   2874    IN  NS  n2b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n3b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n4b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n1b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n7b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n5b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n6b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n0b.akamaiedge.net.

;; ADDITIONAL SECTION:
n5b.akamaiedge.net. 6917    IN  A   201.144.215.107
n1b.akamaiedge.net. 4917    IN  A   23.61.206.74
n6b.akamaiedge.net. 2917    IN  A   201.144.215.108
n2b.akamaiedge.net. 6917    IN  A   192.204.11.244
n4b.akamaiedge.net. 4917    IN  A   201.144.215.110
n7b.akamaiedge.net. 4917    IN  A   201.144.215.113
n0b.akamaiedge.net. 2917    IN  A   23.61.206.68
n3b.akamaiedge.net. 2917    IN  A   201.144.215.114

;; Query time: 132 msec
;; SERVER: 192.168.x.x#53(192.168.x.x)
;; WHEN: Thu Dec  4 16:03:03 2014
;; MSG SIZE  rcvd: 463

Other tests:

traceroute -U -p 53 165.72.192.16
traceroute to 165.72.192.16 (165.72.192.16), 30 hops max, 60 byte packets
 1  192.168.17.10 (192.168.17.10)  0.724 ms  0.718 ms  0.681 ms
 2  168.243.205.74 (168.243.205.74)  4.188 ms  4.677 ms  3.945 ms
 3  172.26.64.21 (172.26.64.21)  179.831 ms  180.282 ms  181.633 ms
 4  172.24.0.13 (172.24.0.13)  182.433 ms  182.585 ms  179.870 ms
 5  172.24.0.9 (172.24.0.9)  180.654 ms  183.023 ms  180.876 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *

And:

ping 165.72.192.16
PING 165.72.192.16 (165.72.192.16) 56(84) bytes of data.
64 bytes from 165.72.192.16: icmp_seq=1 ttl=240 time=356 ms
64 bytes from 165.72.192.16: icmp_seq=2 ttl=240 time=325 ms
64 bytes from 165.72.192.16: icmp_seq=3 ttl=240 time=291 ms
64 bytes from 165.72.192.16: icmp_seq=4 ttl=240 time=260 ms

traceroute -U -p 53 199.40.254.166
traceroute to 199.40.254.166 (199.40.254.166), 30 hops max, 60 byte packets
 1  192.168.17.10 (192.168.17.10)  0.710 ms  0.683 ms  0.764 ms
 2  168.243.205.74 (168.243.205.74)  3.136 ms  3.875 ms  4.191 ms
 3  172.26.64.21 (172.26.64.21)  19.367 ms  18.988 ms  19.698 ms
 4  172.24.0.177 (172.24.0.177)  4.657 ms  6.608 ms  7.088 ms
 5  172.24.0.9 (172.24.0.9)  5.126 ms  7.412 ms  5.518 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *

ping 199.40.254.166
PING 199.40.254.166 (199.40.254.166) 56(84) bytes of data.
64 bytes from 199.40.254.166: icmp_seq=1 ttl=237 time=287 ms
64 bytes from 199.40.254.166: icmp_seq=2 ttl=237 time=280 ms
64 bytes from 199.40.254.166: icmp_seq=3 ttl=237 time=286 ms

UPDATE – 5 Dic 2014

Well, I have installed bind in another server on the same subnet, same OS version and same bind release, and it works just fine!!!, the only thing that changes is the IP address and I can't change that to test because the server with the problem is in production… so I think the IDS theory that Andrew sugests is true… I'll talk to our ISP and investigate our external IP if its blacklisted and post how it goes…

UPDATE – 6 Dic 2014

The server's IP address is not listed in any black list that I checked on the Internet…

Best Answer

Rerun your dig +trace with the +additional flag. I expect it to still fail, but I'll explain what's happening.

+trace with +additional will display the nameserver glue (the ADDITIONAL section), which will at least tell you that the TLD nameservers are properly returning the IP address of ns4.dhl.com.
The couldn't get address for 'ns4.dhl.com': no more failure is coming from your local nameserver. This is an obscure detail, but dig +trace does not use data from the ADDITIONAL section to determine the IP address of the "next hop" nameserver. It is actually querying your nameserver in /etc/resolv.conf to obtain the IP address of ns4.dhl.com, and this query is failing.

Based on the result of the dig commands I had you run in the comments, there appears to be a problem communicating with DHL's nameservers on port 53. A UDP based traceroute on port 53 (traceroute -U -p 53) will give you a better idea of where the packet is being lost. It's either a device on your network or DHL's losing the query/reply. In the latter case, your nameserver's IP address may have somehow gotten itself into a reputation database for an IDS device.

Related Solutions

Overriding some DNS entries in BIND for internal networks

The best method is via the response policy zone in Bind 9.8.1 or newer. It allows you to override single records in arbitrary zones (and there's no need to create a whole subdomain for that, only the single record you want to change), it allows you to override CNAMEs, etc. Other solutions such as Unbound cannot override CNAMEs.

https://www.redpill-linpro.com/sysadvent/2015/12/08/dns-rpz.html

EDIT: Let's do this properly then. I will document what I've done based on the tutorial linked above.

My OS is Raspbian 4.4 for Raspberry Pi, but the technique should work without any changes on Debian and Ubuntu, or with minimal changes on other platforms.

Go to where your Bind config files are kept on your system - here it's in /etc/bind. Create in there a file called db.rpz with the following contents:

$TTL 60
@            IN    SOA  localhost. root.localhost.  (
                          2015112501   ; serial
                          1h           ; refresh
                          30m          ; retry
                          1w           ; expiry
                          30m)         ; minimum
                   IN     NS    localhost.

localhost       A   127.0.0.1

www.some-website.com    A        127.0.0.1

www.other-website.com   CNAME    fake-hostname.com.

What does it do?

it overrides the IP address for www.some-website.com with the fake address 127.0.0.1, effectively sending all traffic for that site to the loopback address
it sends traffic for www.other-website.com to another site called fake-hostname.com

Anything that could go in a Bind zone file you can use here.

To activate these changes there are a few more steps:

Edit named.conf.local and add this section:

zone "rpz" {
  type master;
  file "/etc/bind/db.rpz";
};

The tutorial linked above tells you to add more stuff to zone "rpz" { } but that's not necessary in simple setups - what I've shown here is the minimum to make it work on your local resolver.

Edit named.conf.options and somewhere in the options { } section add the response-policy option:

options {
  // bunch
  // of
  // stuff
  // please
  // ignore

  response-policy { zone "rpz"; };
}

Now restart Bind:

service bind9 restart

That's it. The nameserver should begin overriding those records now.

If you need to make changes, just edit db.rpz, then restart Bind again.

Bonus: if you want to log DNS queries to syslog, so you can keep an eye on the proceedings, edit named.conf.local and make sure there's a logging section that includes these statements:

logging {
    // stuff
    // already
    // there

    channel my_syslog {
        syslog daemon;
        severity info;
    };
    category queries { my_syslog; };
};

Restart Bind again and that's it.

Test it on the machine running Bind:

dig @127.0.0.1 www.other-website.com. any

If you run dig on a different machine just use @the-ip-address-of-Bind-server instead of @127.0.0.1

I've used this technique with great success to override the CNAME for a website I was working on, sending it to a new AWS load balancer that I was just testing. A Raspberry Pi was used to run Bind, and the RPi was also configured to function as a WiFi router - so by connecting devices to the SSID running on the RPi I would get the DNS overrides I needed for testing.

Bind is not resolving specific host

In your dig output, the name servers for that domain appear to be ns.agasoft.co.il and ns3.agasoft.co.il.

However, as of right now, their name servers are webserver1.agasoft.co.il and webserver2.agasoft.co.il.

It appears they changed their name servers.

In addition, mailhost.agasoft.co.il does not exist, according to those servers. Instead, their mail server (as identified by their MX record) is mailserver1.agasoft.co.il. This host does exist, and both name servers return the same address for it: 82.80.246.156.

Also note that in your dig output above, you requested a trace for mailhost.agasoft.co.il but the output shows for mailserver1.agasoft.co.il. Was that a typo in your question?

Summary: the agasoft.co.il domain appears to have recently changed their name servers and possibly also their mail server. If you are trying to find out why mail couldn’t be delivered to them, that’s why. The old name servers (according to your dig output) have a TTL of 1 day, so the problem should clear up in a day or so.

Best Answer

Related Solutions

Overriding some DNS entries in BIND for internal networks

Bind is not resolving specific host

Related Topic