Excessive Outbound DNS Traffic

denial-of-servicedomain-name-systempostfix

I have a VPS system which I have had for 3 years on one host without issue. Recently, the host started sending an extreme amount of outbound DNS traffic to 31.193.132.138. Due to the way that Linode responded to this, I have recently left Linode and moved to 6sync. The server was completely rebuilt on 6sync with the exception of postfix mail configurations.

Currently, the daemons run are as follows:

sshd
nginx
postfix
dovecot

php5-fpm (localhost only)
spampd (localhost only)
clamsmtpd (localhost only)

Given that the server was 100% rebuilt, I can't find any serious exploits against the above stated daemons, passwords have changed, ssh keys don't even exist on the rebuild yet, etc… it seems extremely unlikely that this is a compromise which is being used to DoS the address.

The provided IP is noted online as a known SPAM source. My initial assumption was that it was attempting to use my postfix server as a relay, and the bogus addresses it was providing were domains with that IP registered as their nameservers. I would imagine given my postfix configuration that DNS queries for things such as SPF information would come in with equal or greater amount than the number of attempted spam e-mails sent.

Both Linode and 6Sync have said that the outbound traffic is extremely disproportionate. The following is all the information I received from Linode regarding the outbound traffic:

21:28:28.647263 IP 97.107.134.33.32775 > 31.193.132.138.53: 28720 op8+% [b2&3=0x4134] [17267a] [30550q] [28773n] [14673au][|domain]
21:28:28.647264 IP 97.107.134.33 > 31.193.132.138: udp
21:28:28.647264 IP 97.107.134.33.32775 > 31.193.132.138.53: 28720 op8+% [b2&3=0x4134] [17267a] [30550q] [28773n] [14673au][|domain]
21:28:28.647265 IP 97.107.134.33 > 31.193.132.138: udp
21:28:28.647265 IP 97.107.134.33.32775 > 31.193.132.138.53: 28720 op8+% [b2&3=0x4134] [17267a] [30550q] [28773n] [14673au][|domain]
21:28:28.647266 IP 97.107.134.33 > 31.193.132.138: udp

6sync cannot confirm whether or not the recent spike in outbound traffic was to the same IP or over DNS, but I have presumed as such. For now my server is blocking the entire 31.0.0.0/8 subnet to help deter this while I figure it out.

Anyone have any idea what is going on?

Best Answer

not an answer, just some random thoughts:

  • can you see this traffic when you just run tcpdump on your [virtual] network interface? if so - can you try to figure out if there's any daily/hourly pattern? you can create iptables rule to count the traffic and then allow munin plugin to collect the stats..
  • can you try to determine which app is generating this traffic? i see two methods here:
    • brutal method is to wait till the traffic shows and start killing the applications one after another.
    • gentle approach - use iptables on the OUTPUT chain and owner match to log output packets going on the port 53 to your syslog. something like: iptables -I OUTPUT -p udp --dport 53 --match owner --uid-owner 33 -j LOG --log-prefix "uid 33" applied to all your used uids. check syslog to see which process generates the unwanted traffic.
  • do you have local dns server [eg bind] running? if so:
    • sniff also on the loopback to see what app might be sending requests that cause unwanted traffic.
    • can external servers talk to your dns server? if so - maybe it's some sort of back-scatter attack where your server receives packets from spoofed addresses and responds bombarding a victim.
  • are you 110% sure your php code was not altered? can it be that some of your scripts contain few malicious lines?
Related Topic