Linux – Prepping a virtual appliance

linuxvirtual-appliances

Before converting a running virtual machine to an OVA (redistributable virtual appliance), what does one need to do to ensure it is in a ready state so the instances of the OVA don't bring along unnecessary or potentially disruptive cruft from the build process? This is what I have so far. Am I missing anything? If this has already been answer or there is a Best Common Practices document, I'd appreciate a pointer in the right direction. Thanks. 


#################################
##
## Get all packages up2date and 
## clean out any cruft in the 
## local packages
##
#################################
yum -y update ;
yum clean all ;


#################################
##
## Get rid of the signs I was 
## tinkering with this
##
#################################
[[ -a /etc/issue-original,v ]] && unlink /etc/issue-original,v ;
[[ -a /etc/issue,v ]] && unlink /etc/issue,v ;
ci -u /etc/issue ;



#################################
##
## Remove the host-keys to they
## will be regenerated when the
## new VM is spun-up
##
## Also make sure I remove any 
## personal keys I may have been
## using while setting up
##
#################################
find /etc/ssh/*host* |xargs unlink ;
find /root/.ssh/ -type f |xargs unlink ;
find /home/*/.ssh/ -type f |xargs unlink ;



#################################
##
## Get rid of the use of UUID in 
## FSTAB and any NIC configuration
## so the new VM can find then when
## the UUIDs are regenerated 
##
## Since we use LVM, only the /boot
## slice is a direct slice reference
## the rest are logical volumes
##
#################################
sed -i -e 's/UUID=[0-9a-f-]*\s/\/dev\/sda1\t/' /etc/fstab ;
sed -i -e '/^UUID=[0-9a-f-]*.*/d' /etc/sysconfig/network-scripts/ifcfg-eno* ;
sed -i -e '/^UUID=[0-9A-F-]*.*/d' /etc/sysconfig/network-scripts/ifcfg-eno* ;
find /etc/udev/rules.d/ -iname '70*net*' |xargs unlink ;


#################################
##
## Let the NTP daemon know to 
## expect a big jump in time, so 
## he doesn't freak out. Also let
## him know that if the walls melt,
## it is the acid, speaking and 
## he'll be alright
##
#################################
[[ -a /etc/ntp.conf ]] && \
  [[ "$(head -1 /etc/ntp.conf)" == "tinker panic 0" ]] || \
  sed -i -e '1itinker panic 0\n' /etc/ntp.conf ;



#################################
##
## Trunate the command-histories
## because the learning-process
## can contain some embarrassing
## mistakes, some of which are 
## also bad opsec
##
#################################
>/root/.bash_history ;
>/home/*/.bash_history ;
>/root/anaconda-ks.cfg ;



#################################
##
## Lastly, instruct the OS to redo
## the initial setup and put back
## that new-machine-smell
##
#################################
sys-unconfig ;

Best Answer

I don't have access to our current cleanup script right now, but one of things we take into account is the appliance can maybe be deployed without running the proper customization steps. That means, for instance, the hostname on the original image might live on, and has to be reset before closing. We usually set ours to localhost.

With that in mind, these are extra steps you might need to take care of

  • clean up /etc/hosts, /etc/resolv.conf, /etc/sysconfig/network of any customized values
  • in addition to removing the ifcfg files, don't forget to remove any route files (/etc/sysconfig/network-scripts/route-eno*), if you have them
  • rotate and clean up every single logfile, audit log and the wtmp file (don't remove the wtmp file, just cat /dev/null > /var/log/wtmp), so they start up empty
  • clean up /var/tmp and /tmp
  • clean up the vmware provisioning log, if you have it (iirc, /var/log/vmware-imc/*)
  • if you registered your template into something like RH Satellite, SpaceWalk, SuSE Manager or RHN itself, make sure to restore /etc/sysconfig/rhn or similar back to default values. From experience, rm'ing these is a bad idea. We usually just make a backup when first creating the image, and mv that back when closing it down. Same goes for /etc/sysconfig/osad, if your environment uses it.
  • remove any extra users that might've been created, and EVERY SINGLE password on the shadow file
  • make sure the initialization, first boot, post-deploy-customization scripts are actually set to run on first boot.

Last, but not least, you can zero out every single filesystem plus the extra unused VG space, so you can better compress the image. We have a script that goes into every filesystem, dd's a bunch of zeroes until it fills up, then removes the file. Same for a VG with empty space, create an LV that fills 100%FREE, zero it out and remove it. After finishing these last steps (the last ones before powering down), you can use a trick depending on the kind of image you're creating:

  • For VMWare, clone the image into a new VM, with the destination set as thin provisioning.. When doing this, all the contiguous zeros will be converted into nothing
  • For openstack/kvm images, you might want to convert them using qemu-img to qcow2 (if your provider supports it), and enable the compress flag: qemu-img convert -f raw -O qcow2 -c source.raw destination.qcow2

One thing we've taken from this process is, every time we run thru a new cycle of images, we learn of something new we could've added, usually only after it has shipped. This gets added to the next cycle, and so on.

EDIT: on the spirit of the "every time we learn something new", the following steps were added last time:

  • yum history new
  • yum reinstall basesystem -y
Related Topic