Zabbix agents with the same IP moving between proxies

zabbix

We have a Zabbix environment where we are attempting to monitor systems on customer networks through proxies installed at each location. Many customer sites share the same IP range which seems to be a problem for Zabbix.

We're having an issue where monitored hosts are bouncing between proxies. When it's OK, the hosts look like:

good config

but for some reason, the server3.office.wolpertinger.com host at this site gets assigned to aardvark's proxy:

bad config

(this happens with a few different hosts, but I've chosen to focus on this particular one for the purpose of diagnosis)

The end result of this problem is that when the zabbix server builds the configurations for the proxies, it may not include all the necessary host information so that the proxy can properly monitor the agents.

For example, the server will fail to include information about server3.office.wolpertinger.com when sending to wolpertinger's proxy and then suddenly that server is marked as unreachable for an hour.

I've tried:

  • Changing Device uniqueness criteria to 'IP address' (this was original config)
  • Changing Device uniqueness criteria to 'system.uname'
  • Disabling discovery action rules

all with no effect.

What do I need to do to fix this?


Discovery rule status for aardvark:
aardvark discovery rule
Monitoring page for discovery rule:
aardvark discovery

(you can see how zabbix is getting confused about which host it's seeing, despite the discovery rule being set to distinguish by system.uname)


Discovery rule status for wolpertinger:
wolpertinger discovery rule
Monitoring page for discovery rule:
wolpertinger discovery


Discovery action rules:
discovery action rules


The actual host configurations for the respective hosts are:
wolpertinger server3
note: the proxy here should be wolpertinger

and:
aardvark server1


At one point I realized that Windows doesn't use the FQDN in system.uname so I thought it might be the same across hosts:

server2.office.aardvark.com:     Windows SERVER2 6.1.7601 Microsoft Windows 7 Professional Service Pack 1 x86
server3.office.ostrich.com:      Windows SERVER3 6.1.7600 Microsoft Windows Server 2008 R2 Standard Edition x64
server2.office.ostrich.com:      Windows SERVER2 6.1.7600 Microsoft Windows Server 2008 R2 Standard Edition x64
server3.office.wolpertinger.com: Windows SERVER3 6.1.7601 Microsoft Windows 7 Professional Service Pack 1 x64
server2.office.wolpertinger.com: Windows SERVER2 6.0.6002 Microsoft Windows Server 2008 Standard Edition Service Pack 2 x86

Best Answer

This has been identified as a problem in Zabbix and fixed by the new feature ZBXNEXT-1267.