I have a RHEL system set up as an iSCSI target with 2 luns shared out via one target via the following configuration in /etc/tgt/targets.conf (which I built following the commented out examples in /etc/tgt/targets.conf):
<target iqn.2014-04.com.local.box:ESXi.target1>
backing-store /dev/ESXi_VG/ESXi_LUN_01 #LUN01
backing-store /dev/ESXi_VG/TEST_LUN_01 #LUN02
</target>
These are intended to be shared out to a pair of ESXi hosts as a Datastore.
I originally created a single 4.00 TB LVM logical volume on the vg ESXi_VG for the datastore; however I realized I'd need a second LUN for Iops testing via RDM to a vm stored on local storage on the ESXi host. To that end I created a second 5.00 GB LVM logical volume on the same volume group and added the LUN entry for the second LUN in the target configuration I showed above. I then unmounted the 4.00 Tb LUN on my ESXi host, and restarted tgtd.
I'm seeing some weird stuff now. First, Here is the output from a tgt-admin -s command to show current LUNs:
[root@box ~]# tgt-admin -s
Target 1: iqn.2014-04.com.local.box:ESXi.target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 5369 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/ESXi_VG/TEST_LUN_01
Backing store flags:
Account information:
ACL information:
ALL
As you can see, only 1 LUN is displaying, not 2 – and the LUN that is displaying is only the 5.00 GB LUN that I added at a later date. If I remove or comment out the LUN entry it only shows LUN 0, the controller, and not LUN 1.
However, ESXi can see the 4.00 TB LUN still and can mount it, format it, etc. what it can't see is the 5.00 GB partition, despite it being shared out via the same iSCSI target.
I have a couple questions as a result:
-
Does tgt-admin only show LUNs not currently connected to an initiator? If not why is my 4.00 TB datastore no longer showing?
-
How can I get the 2nd LUN to appear to ESXi so I can mount it to a VM via RDM for iops testing?
Best Answer
Here is what I discovered that the problem was. After being called away from this project due to a corporate merger, it was dumped back on my plate and I decided to restart from scratch. This time I created a LUN, RH_ISCSI_LV_001, and shared it out via the iscsi daemon. I was able to mount and format it in vSphere.
Once again I unmounted the LUN and stopped tgtd to edit
/etc/tgt/targets.conf
to add a second test LUN which I would use for performance testing by mapping it to a guest using RDM. Once again when I started tgtd up, only 1 LUN was displayed.I hypothesized it might be a problem with the LUN not being readable by tgtd after VMware formatted it, so I attempted to destroy it with lvremove. it threw an ioctl error (
device-mapper: remove ioctl on failed: Device or resource busy
) and wouldn't delete.After some digging I ran the command
lsof /dev/ESXi_VG/RH_ISCSI_LV_001
and discovered that pid 2251 was the blocking process. a quickps -ef
and I learned that tgtd had, at some point, not shut down cleanly, and as nothing prevented a second tgtd process from starting up (apparently) it retained "ownership" of the LUN and thus prevented each successive tgtd process I spawned from touching it. a quick kill -9 andservice tgtd start
later and both my LUNs were visible to ESXi.