Dynamic ARP Entries turning into Static ARP entries

arpwindows-server-2003

I recently acquired a client that has a strange ARP caching issue on one of thier servers.

I have a server that will eventually start turning it's dynamic ARP entries into static ARP entries. This causes problems because when the machine that has a static ARP entries on this server receives a new IP via DHCP, then the server is not able to communicate with the clients. Clearing the ARP cache resolves the issue and the server is fine for about a week and then it starts slowly turning ARP entries into static ARP entries. I haven't narrowed it down to when or how many it starts to do, but slowly you start seeing 1 static ARP and then 5 and then 10.

The server in question is a Windows Server 2003 SP2. It is a DC, DHCP, and DNS server. I've checked the DHCP scope options and there's nothing in there that would indicate anything to do with static ARP entries. The only thing different between this DNS server and our other DNS server is that the 'Dynamically Update DNA A and PTR records for DHCP clients that do not request updates' is checked on the problematic server.

I've done a bit of research about this and it seems that this may happen if any PXE type services are running, from what I can tell, there is nothing running a PXE server.

I'm a bit lost as I have never seen dynamic ARP entries start to turn into static ARP entries. Right now my solution is a schedule task that runs every 24 hours to clear the ARP cache (arp -d *). I would like to not rely on this schedule task.

Has anybody seen this before or have any suggestions on how to troubleshoot this?

Best Answer

This could be benign, or malign. Let's hope for benign: there is something running on your machine that thinks it knows better than ARP and is updating the ARP table "by hand". I suspect something like a firewall or other endpoint protection type of program, but if you really can't track it down by reviewing what's installed then your only recourse is to break out heavy-duty audit tools like WPR/WPA or ProcessInternals, let them do their thing, and then tie the events back.

It could be malign: a classic man-in-the-middle attack is to send out an ARP claiming to be Alice when you are really Bob: everyone updates their cache and from then on everyone who sends to Alice thinks they are talking to her when in fact their traffic is going to Bob. Or (another way around) someone breaks into your machine and sets up static ARPs to the "wrong" targets.

An old strategy for defeating the first, btw, is to set up static ARP entries for all the local targets you want to talk to. For the second, well, if the attacker is on your machine, it's too late.