Kubernetes, flannel and exposing services

coreosetcdkubernetes

I have a kubernetes setup running nicely, but I can't seem to expose services externally. I'm thinking my networking is not set up correctly:

  • kubernetes services addresses: --service-cluster-ip-range=172.16.0.1/16

  • flannel network config: etcdctl get /test.lan/network/config {"Network":"172.17.0.0/16"}

  • docker subnet setting: --bip=10.0.0.1/24

  • Hostnode IP: 192.168.4.57

I've got the nginx service running and I've tried to expose it like so:

[root@kubemaster ~]# kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
nginx-px6uy   1/1       Running   0          4m
[root@kubemaster ~]# kubectl get services
NAME         LABELS                                    SELECTOR    IP(S)           PORT(S)    AGE
kubernetes   component=apiserver,provider=kubernetes   <none>      172.16.0.1      443/TCP    31m
nginx        run=nginx                                 run=nginx   172.16.84.166   9000/TCP   3m

and then I exposed the service like this:

kubectl expose rc nginx --port=9000 --target-port=9000 --type=NodePort
NAME      LABELS      SELECTOR    IP(S)     PORT(S)    AGE
nginx     run=nginx   run=nginx             9000/TCP   292y

I'm expecting now to be able to get to the nginx container on the hostnodes IP (192.168.4.57) – have I misunderstood the networking? If I have, can explanation would be appreciated 🙁

Note: This is on physical hardware with no cloud provider provided load balancer, so NodePort is the only option I have, I think?

Best Answer

You don't have to use NodePort and you don't have to use external load balancer. Just dedicate some of your cluster nodes to be loadbalancer nodes. You put them in a different node group, give them some labels: mynodelabel/ingress: nginx, and than you host an nginx ingress daemonset on that node group.

Most important options are:

spec:
  restartPolicy: Always
  dnsPolicy: ClusterFirst
  hostNetwork: true
  nodeSelector:
    mynodelabel/ingress: nginx

and

      ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443

Optionally you can taint your loadbalancer nodes so that regular pods don't work on them and slow down the nginx.

Related Topic