Iptables – How to have multiple IP addresses on a kubernetes node and then source NAT a specific service / pod to a specific IP address


I have a kubernetes node that is set up with multiple IP addresses (it happens to be a bare-metal node, and the hardware has multiple NICs, one of the NICs has multiple IP addresses assigned – but this could be a VM node with multiple virtual adapters, or a VM node with multiple IP addresses assigned to a single adapter – the important thing is that the node has multiple IP addresses).

When kubernetes performs source NAT on IP communications coming from a specific service running on this node, it always SNATs from the first IP address on the first adapter on the system (i.e. all communications from any service appear to come from the first IP address attached to the first adapter on the system).

However, I would like to be able to have a specific service (or pod, if necessary) communicate with the outside world from one of the other IP addresses. i.e. I would like to specify on a service-by-service basis which of the IP addresses assigned to the node the service is SNAT-ed from / communicates from.

Just in case the answer I get is 'you shouldn't need to do that'… The specific reason for this problem is that I am accessing a service that has a rate restriction per IP address, and I need to access from multiple IP addresses to bypass the rate restriction. But this problem would apply equally to having a service running on your kubernetes cluster access external services that are restricted by IP address, for example.

Best Answer

There is one known to me existing solution to your problem, currently still in alpha stage, and implemented as a Custom Resource Defitnion (CRD). Please refer to kube-static-egress-ip project on github.