We have a Windows Server 2012 Failover cluster running a number of virtual machines. Storage for the cluster is provided by a Windows Storage Server 2012 instance and mapped to the nodes via iSCSI. In addition, I use DPM 2012 SP1 to protect the clustered VMs – these backups are stored to a different server from the one running the cluster. The snapshot provider configuration was set up in accordance with the instructions and guidance at
http://blogs.technet.com/b/filecab/archive/2012/10/08/iscsi-target-storage-vds-vss-provider.aspx
We had the same setup with Server 2008 R2, but recently upgraded to take advantage of the hardware snapshot provider and performance improvements claimed by Microsoft.
Since configuring the protection group for the clustered virtual machines, the group shows that all of the members are in an ideal state and there are no errors in the cluster event log.
However, when looking at the iSCSI devices on the storage server, many virtual disks are showing up with paths such as '\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy{95EEC618-9594-11E3-93F9-00259067F05B}\csv.vhd'. Each of these virtual disks corresponds with the backup of one server, and each of these devices also has an entry the device manager. These should not remain behind as artefacts after a backup is run, but for some reason they are.
When running 'vssadmin list shadows', many shadows are listed as well.
Nothing significant is showing up in the system logs, but whenever a new backup is run, another disk is added to the list of iSCSI disks, and another entry is added to the list of vss shadows.
This issue was posted to the technet forums, and they were unable to help. Microsoft support has not helped either.
Does anyone have any idea why snapshots are appearing as iSCSI virtual disks, or how this can be prevented?
Best Answer
First things first. I would run this solution by your Microsoft Support Technician before using the suggestion!
We have a very similar issue with our cluster and brought it to Microsoft's attention. In our situation, the volumes are appearing on each node of our cluster in the registry under the keys:
\HKLM\SYSTEM\CurrentControlSet\Enum\STORAGE\VolumeSnapshot\HarddiskVolumeSnapshot###
We had about 400-500 ghost entries from the vss provider not cleaning up after itself. Their suggestion was to remove the ghost devices with DevNodeClean.x64.exe and that did the trick.
When running DevNodeClean.x64.exe /l (the list extension) on the storage node, I was able to see that the DriverKey (on the storage node) matched the Driver string value within the registry key (on a cluster node).
I would assume you can remove the ghost entries from the DPM server as we did from the cluster nodes.
As for why they are sticking around, we have gotten no answer from Microsoft either...
P.S. - DevNodeClean.x64.exe is only available from a Microsoft Support Technician as far as I know.