Can’t complete interrupted update of ESXi 5 to U2

updatevmware-esxi

I've been trying to do an interactive update of VMware ESXi 5.0.0 update 1 to 5.0.0 update 2 via the installer image burned to a DVD.

The first time I ran the update, all seemed to go fine at first. I picked the "Upgrade ESXi, preserve VMFS datastore" option.

The installer, however, crashed with an "unexpected error", "OSError: [Errno 39] Directory not empty". It referenced a $RECYCLE.BIN directory in the root of a VMFS volume which otherwise contained only imgdb.tgz. Using the maintenance console I also found that the $RECYCLE.BIN directory contained only a DESKTOP.INI whose contents were consistent with Windows shell extensions.

I suspect this file managed to wind up in there accidentally while I was using Windows-based tools (booted into separately) to inspect and manage the RAID volumes the host is installed into. In any event the file didn't look like it belonged and did seem to be disrupting the update, so I simply deleted it via the management console.

I then attempted to do the update again, but this time the installer did not provide the "Upgrade ESXi, preserve VMFS datastore" option. The installer now indicates for the drive:

ESX(i) Found: No

(It had said "Yes" before the installer crash.)

It would have let me reinstall ESXi from scratch while preserving the datastore, but I've had issues in past with things like MAC addresses getting changed around when reimporting VMs so I didn't want to go that route.

I also tried removing the $RECYCLE.BIN directory itself but this changed nothing.

The host still runs fine but I need to install update 2 in order to support Solaris 11.1.

Any info or suggestions would be appreciated!

Thanks,
Kevin

Best Answer

I ran into this issue as well and I was able to figure out a solution.

Explanation

Some explanation to start with, This issue is caused by booting Windows that has access to the datastore drives, in every partition Windows sees it creates a $RECYCLE:BIN folder, but the $ symbol creates problems with linux so we need to remove it. So if you ever have to booted to Windows I recommend cleaning these up before upgrading, or better yet after you boot into Windows.

Fixing the root cause

The first step is to clean up the $RECYCLING.BIN folders.

  1. Enable SSH and connect to the host
  2. Run cd /vmfs/volumes/
  3. Run ls -allh this will list all of your volumes
  4. Run the following for each volume and take note of which ones have the $RECYCLING.BIN folder ls -allh [volume id] (replace [volume id] with each volume listed before)

Now that we've identified which volumes have the folders we need to clean them up.

  1. Navigate to the volume with cd [volume id]
  2. We first will rename the folder to get rid of the $, run mv '$RECYCLING.BIN' REC (including the single quotes) (I don't think this is required but I did it anyway)
  3. Now we delete the folder with rm -rf REC

The folder should be gone now, I ran ls again to ensure the folder was gone but it's optional, I also ran ls REC/ to ensure there wasn't anything else in the directory before deleting.

Finishing the upgrade

Now we will deal with actually finishing the upgrade Since we've essentially "burned" the upgrade from DVD option we need another way, fortunately there are a few ways to upgrade.

  1. First we will need to shut down your VMs and put the host into maintenance mode
  2. Enable SSH if it is not already from before
  3. We need to enable the firewall rule httpclient, you can either do this from the gui or with the command esxcli network firewall ruleset set -e true -r httpClient

Now for the actual upgrade, there are two ways to do this, a. let the host download the required files or b. download the upgrade zip from VMWare's site and upload to the host (this requires the appropriate license on your MyVMWare account).

  1. Run the command

esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.5.0-4564106-standard

In this command I am upgrading to 6.5, if you are upgrading to a newer version then you will need to know the profile name, fortunately you can get this off VMWare site or the iso image, on the root of the disk there is an XML file called PROFILE.XML, if you open this up you will see the <name> tag, this is the version on the disk, replace this in the last part of the command.

If you are upgrading from the ZIP you will use the command esxcli software vib update -d /vmfs/volumes/DATASTORE/VMware-ESXi-6.0.0-2494585-depot.zip

  1. Now reboot and when the host comes back up you should be on the latest version
  2. Finally disable the httpclient firewall rule from the gui or with the command esxcli network firewall ruleset set -e false -r httpClient, exit maintenance mode, and don't forget to disable SSH

I know this is 4 years old but it is still one of the top results on Google and also there is only one other relevant result so this is for anyone that comes along.

Related Topic