Let's say I have a Docker container with a web server (like Apache 2). Now I want to update the OS under it. This SF answer says the best way is to rebuild the base image and my Apache image. But deploying the image means downtime because I have to delete the old container before I can create the new one, so there is only one container that is binding to port 80/443.
But how can I deploy this update with zero downtime? Should I use a load balancer and use inter-container communication? And how do I update the load balancer?
Best Answer
The ideal target scenario
Yes, you should use a load balancer and update one instance at a time. I'm not sure where inter-container communication comes in.
As an example, imagine you have a load balancer which serves your site A. Users only connect to it as and only know it as "A". The load balancer knows that there are two or more backends (B, C, etc.), and whether they're VMs or containers doesn't matter.
Then, you want to upgrade the backends, which in this case are Apache instances.
Then, do the same process for C, D, etc.
Note that there's an open request for in-place upgrades of Docker containers, from Nov 2013, but it doesn't appear to have much progress so the above solution is what you should do in the mean time.
What to do for an existing live site
Presumably, you're asking this because you're already running a live site in this model and you would like to upgrade it without downtime. So, we need to get to the ideal target state above, but incrementally.
Let's assume that:
If these assumptions are false, you should first fix it such that this is correct.
Then, follow these steps:
How to update a load balancer
The easy (hosted) way
The easiest option is to not run your own balancer. For example, if you're using a cloud platform which provides load balancing as a service, consider using it and then maintenance and update of the load balancer is not an issue.
The manual way
If you are running your own load balancer, adding another layer of indirection (i.e., DNS) will help. Let's assume the following:
We proceed as follows:
add B's IP address to the DNS resolution along with A
if anything is wrong with B, undo as follows:
and you're done.
Details, diagrams, and tooling
See these write-ups and tools that can help you automate the process, but the general idea is the same:
The Moral
"All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections." — David Wheeler