I have three networks on Rackspace:
- Public network
- Service network
10.181.XXX.XXX
(they call it a private network sometimes but it is not private per-se, it is datacenter-wide private so their tenants may share it) - A real private network of
192.168.3.0/24
created via the networking UI
My plan is to launch a secure CoreOS cluster whose etcd
peers and clients are limited to the real private networking so it is not available for the outside world.
So, I launched a CoreOS server and it had three interfaces:
eth0
has the IP address of the public network – goodeth1
has the IP address of the service network – goodeth2
has the IP address of the private network – great!
All looked well but etcd
is bound to the service network instead of the private one I created manually. Here's a piece of cloud-config
:
#cloud-config
coreos:
etcd2:
discovery: https://discovery.etcd.io/XXX
advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
initial-advertise-peer-urls: http://$private_ipv4:2380
listen-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
listen-peer-urls: http://$private_ipv4:2380
Which means that $private_ipv4
variable gets expanded to eth1
IP address, here's the contents of /etc/environment
:
COREOS_PUBLIC_IPV4=166.78.XXX.XXX
COREOS_PRIVATE_IPV4=10.181.XXX.XXX
Ok, looks like Rackspace injects its own networks, it is explainable. But does this mean that etcd is locked only to the first two networks and there's no way to configure it to use the real private one?
I've tried a bunch of hacks such as this:
listen-peer-urls: http://`/usr/bin/ifconfig eth2 | /usr/bin/grep --word-regexp inet | /usr/bin/awk '{print $2}'`:2380
and others but neither got properly executed nor substituted in the cloud-config
.
My questions are:
- Am I doing something weird in terms on CoreOS cluster setup field, so that I can not achieve the target easily and naturally? If so, how would you lay out a secure cluster whose
etcd
operates within a real private network? - Are there ways to "inject" dynamic values into
cloud-init
file so it gets interpolated at runtime? The issue is that the IP address is not known beforehand so it needs to get gotten and injected somehow I think.
Best Answer
They also provide a way to create your own private network and attach your server to it. As explained KB Article ID: 2163. If you did that and had the servicenet connected interface removed, it would no longer be an issue. However, the caveat to this would be that you would no longer have a way to utilize servicenet related services. More on intricacies about that is their KB Article ID: 2250.
Or, you could configure your firewall so that only the services that you use/need servicenet for are able to reach your server. You could even venture into the world of orchestrating a true private network through a cloud vyatta appliance and still retain access to Rackspace services as explained in KB Article ID: 3454.
All of those articles can be found at: http://www.rackspace.com/knowledge_center/
Ultimately, you'd have to weigh what option works best for you.
HTH