We're having issues where SNMPv3 ceases to accept SNMP requests on our Juniper EX series switches. In what seems like random occurrences, Juniper nodes refuse SNMP authentication credentials.
This is what we end up getting:
rj@MHN00525SYS01:~$ snmpwalk -v 3 -a SHA -A "PASS" -l authPriv -x AES -X "PASS"
-u "username" mhn00416an01 iso.3.6.1.2.1.1.1.0
snmpwalk: Unknown user name
rj@MHN00525SYS01:~$
Our devices were working, but then just stop. The usernames are correct and loaded into the system.
rj@MHN00416AN01# run show snmp v3 users
Engine ID: 80 00 0a 4c 04 31 33 <...snip...>
User Auth/Priv Storage Status
username sha/aes128 nonvolatile active
After dealing with this for some time, we finally decided to investigate. I'm seeing this entry before the system stops accepting requests:
Nov 6 09:44:32 MHN00416AN01 mgd[7996]: UI_COMMIT: User 'rj' requested 'commit'
operation (comment: none)
Nov 6 09:44:35 MHN00416AN01 snmpd[1260]: LIBJSNMP_NS_LOG_ERR: ERR: snmpd: send_trap: USM
unknown security name (no such user exists) (plaintext scopedPDU header type 00: s/b 30)
Nov 6 09:44:35 MHN00416AN01 snmpd[1260]: SNMPD_ENGINE_ID_CHANGED: Warning: Local
engine ID has changed. Passwords for all the users under [edit snmp v3 usm local-engine]
need to be reconfigured.
I have found that simply re-inputting the username/passwords restore access to the device.
rj@MHN00416AN01# set snmp v3 usm local-engine user username
authentication-sha authentication-password PASS
{master:0}[edit]
rj@MHN00416AN01# set snmp v3 usm local-engine user username
privacy-aes128 privacy-password PASS
After a commit
operation, we can access it again.
rj@MHN00525SYS01:~$ snmpwalk -v 3 -a SHA -A "PASS" -l authPriv -x AES -X "PASS"
-u "username" mhn00416an01 iso.3.6.1.2.1.1.1.0
iso.3.6.1.2.1.1.1.0 = STRING: "Juniper Networks, Inc. ex3300-48p Ethernet Switch, kernel
JUNOS 12.3R3.4, Build date: 2013-06-14 02:21:01 UTC Copyright (c) 1996-2013 Juniper
Networks, Inc."
rj@MHN00525SYS01:~$
This issue is happening across a ton of different devices. Models include EX2200-c, EX3300, EX4200 and EX4550 on version 12.3.
Why is this happening and how can I make it stop?
Best Answer
This issue is a result of the engine-id changing. If you run into this syslog message, that is the exact definition of the problem.
Unfortunately, I have found that this is happening for no apparent reason. The commit performed before this engine-id changed was a simple
syslog-host
addition that resulted in the engine-id changing. Juniper covers theengine-id
option in their TechLibrary, which says the default option is to use the default IP address of the device.After a lot of beating our heads against the desk, we have found that simply defining one with an explicit setting is the best option, as it remains consistent.
Juniper also has a note for changing the engine-id that needs to be done before it will work. It effectively means once you commit a new engine-id, you must also commit another username/password.
You'll need to commit new users into the configuration after the engine change. This is important because SNMPv3 users won't function if you don't do this despite what is actually said under the
show snmp v3 users
. So make sure you define new users.Before the change:
After the change:
This provides a more reliable SNMP setup that isn't prone to engine-id changes and eliminating access. Juniper does recommend that you use the
use-mac-address
option as opposed to thelocal engine-id-suffix
. This option is susceptible to issues in a virtual chassis setup. If a backup routing engine takes over, the SNMP engine-id will change to theme0
interface of the new master.