OpenIndiana (illumos): vmxnet3 interface lost on reboot

illumosopenindianaopensolarissolarisvmware-esxi

I want my VMware vmxnet3 interface to be brought up with DHCP on boot. I can manually configure the NIC with:

# ifconfig vmxnet3s0 plumb
# ipadm create-addr -T dhcp vmxnet3s0/v4dhcp

But after creating /etc/dhcp.vmxnet3s0 and rebooting, the interface is down and the logs show:

Aug 13 09:34:15 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x200000) -> no
Aug 13 09:34:15 neumann vmxnet3s: [ID 715698 kern.notice] vmxnet3s:0: stop()
Aug 13 09:34:17 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x200000) -> no
Aug 13 09:34:17 neumann vmxnet3s: [ID 920500 kern.notice] vmxnet3s:0: start()
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(TxRingSize) -> 256
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(RxRingSize) -> 256
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(RxBufPoolLimit) -> 512
Aug 13 09:34:17 neumann nwamd[491]: [ID 605049 daemon.error] 1: nwamd_set_unset_link_properties: dladm_set_linkprop failed: operation not supported
Aug 13 09:34:17 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x20000) -> no
Aug 13 09:34:17 neumann nwamd[491]: [ID 751932 daemon.error] 1: nwamd_down_interface: ipadm_delete_addr failed on vmxnet3s0: Object not found
Aug 13 09:34:17 neumann nwamd[491]: [ID 819019 daemon.error] 1: nwamd_plumb_unplumb_interface: plumb IPv4 failed for vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 160156 daemon.error] 1: nwamd_plumb_unplumb_interface: plumb IPv6 failed for vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 771489 daemon.error] 1: add_ip_address: ipadm_create_addr failed on vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 405346 daemon.error] 9: start_dhcp: ipadm_create_addr failed for vmxnet3s0: Operation not supported on disabled object

I then tried disabling network/physical:nwam in favour of network/physical:default. This works, the interface is brought up but physical:default fails and my network services (e.g. NFS) refuse to start.

# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
    inet 127.0.0.1 netmask ff000000 
vmxnet3s0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:1: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:2: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:3: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:4: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:5: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:6: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:7: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:8: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
    inet6 ::1/128 
vmxnet3s0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 9000 index 2
    inet6 ::/0

# cat /var/svc/log/network-physical\:default.log
[ Aug 16 09:46:39 Enabled. ]
[ Aug 16 09:46:41 Executing start method ("/lib/svc/method/net-physical"). ]
[ Aug 16 09:46:41 Timeout override by svc.startd.  Using infinite timeout. ]
starting DHCP on primary interface vmxnet3s0
ifconfig: vmxnet3s0: DHCP is already running
[ Aug 16 09:46:43 Method "start" exited with status 96. ]

NFS server not running:

# svcs -xv network/nfs/server
svc:/network/nfs/server:default (NFS server)
State: offline since August 16, 2012 09:46:40 AM UTC
Reason: Service svc:/network/physical:default
        is not running because a method failed.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/milestone/network:default
            svc:/network/physical:default
Reason: Service svc:/network/physical:nwam is disabled.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/milestone/network:default
            svc:/network/physical:nwam
Reason: Service svc:/network/nfs/nlockmgr:default is disabled.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/network/nfs/nlockmgr:default
   See: man -M /usr/share/man -s 1M nfsd
Impact: This service is not running.

I'm new to the world of Solaris, so any help solving would be much appreciated. Thanks!

Best Answer

Don't use ifconfig to plumb the interface use ipadm create-if instead. You've also managed to configure the adaptor 9 times, all of which are configured to use DHCP. You should run ipadm delete-addr on all 9 and perhaps even ipadm delete-if vmxnet3.

Once you've done that, you should also stop using vmxnet3 interfaces in your virtual machine configuration and use e1000 interfaces instead, I've had much better luck with those.

More info: ifconfig vs ipadm comparison and example of network setup using dladm and ipadm in OmniOS (also applies to other Illumos deritives like OpenIndiana).

Related Topic