Docker – Relationship between Vagrant, Docker, Chef and OpenStack (or similar products)

chefdockerlxcopenstackvagrant

I am a web developer, but I am also interested in a few administrative tasks. Hence, the new move from pure administration to dev-ops comes handy for me.

Anyway, I have some problems to put a few things into a relationship. Maybe there isn't any, so I wanted to ask for help to clarify.

Basically, what I want to put into relation is four types of software (from my understanding). The exact products don't matter, you can place any similar software as an alternative:

  • Vagrant: From my understanding is to automate creation and management of VMs: Setting them up, starting and stopping them. This can be done using a local VM or remote, e.g. on a cloud platform.
  • Docker: A "lightweight VM", based on a few Linux kernel concepts, which can be used to run processes in isolation, e.g. in a shared web hosting environment.
  • Chef: A tool to setup and configure an operating system, e.g. inside a VM.
  • OpenStack: A tool that allows you to build your own private cloud, hence comparable to something such as AWS.

Question #1: Are my explanations right, or am I wrong with some (or all) of these consumptions?

Question #2: How could I mix all those tools? Would that make any sense?

In my imagination and from my point of understanding, you could go and

  • use OpenStack to build your own cloud,
  • use Vagrant to manage the VMs run in the cloud,
  • use Chef to setup these VMs
  • and finally use Docker to run processes inside the VMs.

Is this correct? And if so, can you give me an advice in how to start using all this (it's quite a lot at the same time, and I don't know yet where to start)?

Best Answer

Let's use their respective web pages to find out what are all these projects about. I'll change the order in which you listed, though:

  • Chef: Chef is an automation platform that transforms infrastructure into code.

    This is a configuration management software. Most of them use the same paradigm: they allow you to define the state you want a machine to be, with regards to configuration files, software installed, users, groups and many other resource types. Most of them also provide functionality to push changes onto specific machines, a process usually called orchestration.

  • Vagrant: Create and configure lightweight, reproducible, and portable development environments.

    It provides a reproducible way to generate fully virtualized machines using either Oracle's VirtualBox or VMWare technology as providers. Vagrant can coordinate with a configuration management software to continue the process of installation where the operating system's installer finishes. This is known as provisioning.

  • Docker: An open source project to pack, ship and run any application as a lightweight container

    The functionality of this software somewhat overlaps with that of Vagrant, in which it provides the means to define operating systems installations, but greatly differs in the technology used for this purpose. Docker uses Linux containers, which are not virtual machines per se, but isolated processes running in isolated filesystems. Docker can also use a configuration management system to provision the containers.

  • OpenStack: Open source software for building private and public clouds.

    While it is true that OpenStack can be deployed on a single machine, such deployment is purely for proof-of-concept, probably not very functional due to resource constraints.

    The primary target for OpenStack installations are bare metal multi-node environments, where the different components can be used in dedicated hardware to achieve better results.

    A key functionality of OpenStack is its support for many virtualization technologies, from fully virtualized (VirtualBox, VMWare), to paravirtualized (KVM/Qemu) and also containers (LXC) and even User Mode Linux (UML).

I've tried to present these products as components of an specific architecture. From my point of view, it makes sense to first be able to define your needs with regards to the environment you need (Chef, Puppet, Ansible, ...), then be able to deploy it in a controlled fashion (Vagrant, Docker, ...) and finally scale it to global size if needs be.

How much of all this functionality you need should be defined in the scope of your project.

Also note I've over-simplified mostly all technical explanations. Please use the referenced links for detailed information.