How to run VMWare ESX or ESXi in a virtual machine

virtualizationvmware-esxvmware-esxi

Can VMWare ESX or ESXi be installed and used inside a virtual machine?

It can be installed inside VMWare Workstation or Server, but then it doesn't work; the main symptoms are:

  • It runs REALLY slowly.
  • It lets you create VMs, but when powering up them it gives an error stating "You may not power on a virtual machine in a virtual machine".

Best Answer

VMWare ESX or ESXi CAN run inside a virtual machine, provided certain prerequisites are satisfied.
This kind of setup is of course completely useless (and totally unsupported) in a production environment, but can be very useful for two purposes:

  • Testing or studying ESX or ESXi if you don't have a physical server available.
  • Testing or studying the whole Virtual Infrastructure if you don't have at least two servers and a SAN.

Prerequisites:

  • You need some physical resources. In order to run ESX or ESXi in VM, the VM needs at least 1.5 GB of memory, two VCPUs and enough disk space for the server itself and for the VMs you will run inside it.
  • You absolutely need a physical CPU with native virtualization support (Intel VT or AMD-V).
  • You need to run VMWare Workstation 6.5, VMWare Server 2 or VMWare Fusion 5 on the physical host. Previous versions can't succesfully run ESX or ESXi in a VM.
  • A 64-bit OS on the physical host is useful but not required.

Setup:

  • Enable native virtualization support for your CPU in the motherboard BIOS (it's often not enabled by default).
  • Install your preferred virtualization software. I've tested everything succesfully using VMWare Workstation 6.5.2 on a Windows XP x64 host, but it should work with VMWare Server 2.0 and/or Linux hosts, too.
  • Create a custom VM using those setting:
    • Hardware compatibility level: latest
    • Guest operating system: other 64-bit
    • Virtual CPUs: at least 2
    • Memory: at least 1.5 GB
    • Networking: Host-only or NAT
    • SCSI adapter: LSI Logic
    • Virtual disk type: SCSI
    • Virtual disks: as you wish; I suggest using at least two virtual disks, a 10-GB one for installing the system and another one where to create a datastore. The space should be pre-allocated.
    • Remove floppy, sound card, USB controller, etc. Leave only networking and storage.
    • CPU Execution mode: Intel VT-x or AMD-V (very important).
  • Manually edit the VMX file of the virtual machine you created, setting the following parameters:

    guestOS = "vmkernel"
    monitor_control.vt32 = "TRUE"
    monitor_control.restrict_backdoor = "TRUE"

  • Start the VM and install ESX or ESXi from the installation ISO image.

  • Configure the networking to allow the ESX or ESXi virtual server to talk with the host.

Usage:

  • Use your web browser to connect to you virtual server's IP address and download the VI Client.
  • Install the VI Client on the host.
  • Connect to the virtual ESX/ESXi server.
  • Create a VM as you wish.
  • Power up the VM.
  • If everything is done correctly, the VM will start. If it complains about not being able to power on a VM inside a VM, then there is an error with the `monitor_control.restrict_backdoor' parameter (or you're using an old version of VMWare Workstation/Server).

  • Enjoy :-)