Nginx – How to use nginx with coreos docker containers and fleet

coreosdockerdomain-name-systemnginx

I have three docker containers configured..

  • ghost – a custom docker container which runs my ghost blog
  • ghost-data – docker container which holds all the persistent data.. content/* config.js, newrelic.js
  • jwilder/nginx-proxy – a docker container with nginx which sits in front of the ghost container

Using a single server I get how to link the nginx to the ghost container and get stuff served up through nginx.

However I want to start learning about coreos and migrate to a fleet of coreos servers. I know that in order to do this i need to write up service files for two of the containers (ghost, ghost-data) and make sure that ghost, and ghost-data are on the same host and nginx isn't.

As far as I understand, the whole point of fleet/clustering in coreos is uptime. containers will move from node to node as needed. The problem is, each of these nodes have their own IPs to the world. How do I get my DNS to know of these IPs and which IP has the nginx server on it?

The way I see it, something running like this..

  • node1 – runs nginx-proxy, public ip of 1.1.1.1
  • node2 – runs ghost, ghost-data, public ip of 1.1.1.2
  • node3 – idle node, public ip of 1.1.1.3

DNS is pointing example.com to 1.1.1.1

What happens when coreos decide to move nginx-proxy to node3 1.1.1.3?

How can i get the DNS to follow this? I'm sure in the service file there is a way to force nginx to stay on the same node, but that completely defeats the purpose doesn't it?

Best Answer

You have three main options:

  1. Use a specific host for routing

This would be similar to your set up described above, but I'd make sure your unit file stays on that host with MachineID=abc123.... It's not super HA however, since you have a single point of failure.

  1. Use a Cloud Load Balancer

If you're on a cloud provider, use a LB and set up a health check on port 80. Only the machine running on 80 will be given traffic. You'll have a bit of downtime (seconds to minutes) if the LB doesn't detect your failure quickly enough.

  1. Two Layered Routing

Something similar to https://github.com/robszumski/varnish_etcd, but you can do it with varnish/nginx/haproxy and confd, or vulcand. Basically, you have a routing layer on each machine, that redirects to the backend's location dynamically.

Related Topic