Vmware ESXi 5 cant delete file from host

vmware-esxi

I was uploading a file to a datastore on ESXi 5 host, when something went wrong. I rebooted the host, but now cannot delete the file. (I renamed it to BADFILE). As you can see from the commands below, I've done all the right things…but just can't delete it! Any ideas?

Note this file is on the ESXi host – not used by a VM guest. lsof shows no processes using the file.

/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # touch BADFILE 
touch: BADFILE: Device or resource busy
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # vmkfstools -D BADFILE 
Lock [type 10c00001 offset 10063872 v 215, hb offset 3579904
gen 133, mode 2, owner 00000000-00000000-0000-000000000000 mtime 480
num 1 gblnum 0 gblgen 0 gblbrk 0]
RO Owner[0] HB Offset 3579904 511c5a3a-8cabaab1-ae4c-001731f8a1de
Addr <4, 3, 170>, gen 206, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 3695179776, nb 441 tbz 0, cow 0, newSinceEpoch 441, zla 3, bs 8388608
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # tail /var/log/vmkernel.log 
len 3695179776, nb 441 tbz 0, cow 0, newSinceEpoch 441 zla 3, bs 8388608
2013-02-14T04:15:13.635Z cpu0:5826)FS3: 173: <END BADFILE>
2013-02-14T04:16:57.856Z cpu0:2105)FS3Misc: 1465: Long VMFS rsv time on 'MIRROR1' (held for 331 msecs). # R: 1, # W: 1 bytesXfer: 2 sectors
2013-02-14T04:23:34.691Z cpu1:6190)FS3: 171: <START BADFILE>
2013-02-14T04:23:34.691Z cpu1:6190)Lock [type 10c00001 offset 10063872 v 215, hb offset 3579904
gen 133, mode 2, owner 00000000-00000000-0000-000000000000 mtime 480
num 1 gblnum 0 gblgen 0 gblbrk 0]
2013-02-14T04:23:34.692Z cpu1:6190)Addr <4, 3, 170>, gen 206, links 1, type reg, flags 0x0, uid 0, gid 0, mode 600
len 3695179776, nb 441 tbz 0, cow 0, newSinceEpoch 441 zla 3, bs 8388608
2013-02-14T04:23:34.692Z cpu1:6190)FS3: 173: <END BADFILE>
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # lsof | grep BADFILE 
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # vmkvsitools lsof | grep BADFILE 
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # ls
BADFILE                      Windows_Server_2012_x64.iso
/vmfs/volumes/4ed306ea-ac7ce83d-0559-001731f8a1de/ISO Images # rm -f BADFILE 
rm: can't remove 'BADFILE': Device or resource busy

Best Answer

The file was likely locked by an individual VM. Being an .ISO file, was it possibly attached to one of the running virtual machines?

You resolved this by entering maintenance mode (thus shutting down all of the running VM's). The same thing could have been accomplished without downtime by opening the virtual machines' settings and removing the .ISO from the CDROM device field.