VMWare: ESX or 3i? Do I need the COS

Note: I've never done this before.

The shared storage RDM should be possible, although having five nodes accessing it could introduce hilarity.

The technique you want is similar to how two node Microsoft Clustering Services is implemented (with a shared quorum drive); VMware provide a documented method for how to achieve it.

The solution looks well documented, if a little hairy. I'd recommend building and testing it in a lab before considering letting it anywhere near your production cluster.

Good luck.

VMWare ESX or ESXi CAN run inside a virtual machine, provided certain prerequisites are satisfied.
This kind of setup is of course completely useless (and totally unsupported) in a production environment, but can be very useful for two purposes:

• Testing or studying ESX or ESXi if you don't have a physical server available.
• Testing or studying the whole Virtual Infrastructure if you don't have at least two servers and a SAN.

Prerequisites:

• You need some physical resources. In order to run ESX or ESXi in VM, the VM needs at least 1.5 GB of memory, two VCPUs and enough disk space for the server itself and for the VMs you will run inside it.
• You absolutely need a physical CPU with native virtualization support (Intel VT or AMD-V).
• You need to run VMWare Workstation 6.5, VMWare Server 2 or VMWare Fusion 5 on the physical host. Previous versions can't succesfully run ESX or ESXi in a VM.
• A 64-bit OS on the physical host is useful but not required.

Setup:

• Enable native virtualization support for your CPU in the motherboard BIOS (it's often not enabled by default).
• Install your preferred virtualization software. I've tested everything succesfully using VMWare Workstation 6.5.2 on a Windows XP x64 host, but it should work with VMWare Server 2.0 and/or Linux hosts, too.
• Create a custom VM using those setting:
  • Hardware compatibility level: latest
  • Guest operating system: other 64-bit
  • Virtual CPUs: at least 2
  • Memory: at least 1.5 GB
  • Networking: Host-only or NAT
  • SCSI adapter: LSI Logic
  • Virtual disk type: SCSI
  • Virtual disks: as you wish; I suggest using at least two virtual disks, a 10-GB one for installing the system and another one where to create a datastore. The space should be pre-allocated.
  • Remove floppy, sound card, USB controller, etc. Leave only networking and storage.
  • CPU Execution mode: Intel VT-x or AMD-V (very important).

• Manually edit the VMX file of the virtual machine you created, setting the following parameters:

  guestOS = "vmkernel"
monitor_control.vt32 = "TRUE"
monitor_control.restrict_backdoor = "TRUE"

• Start the VM and install ESX or ESXi from the installation ISO image.

• Configure the networking to allow the ESX or ESXi virtual server to talk with the host.

Usage:

• Use your web browser to connect to you virtual server's IP address and download the VI Client.
• Install the VI Client on the host.
• Connect to the virtual ESX/ESXi server.
• Create a VM as you wish.
• Power up the VM.

• If everything is done correctly, the VM will start. If it complains about not being able to power on a VM inside a VM, then there is an error with the `monitor_control.restrict_backdoor' parameter (or you're using an old version of VMWare Workstation/Server).

• Enjoy :-)