Kubernetes Ingress – Fixing 502 Bad Gateway on DigitalOcean

I am trying to deploy a NestJS app with Kubernetes on DigitalOcean and I have followed this tutorial, but I am always getting a 502 Bad Gateway from the nginx-ingress-controller.

That's my deployment.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: nestjs-api
spec:
  ports:
    - port: 80
      targetPort: 3001
  selector:
    app: nestjs-api
---
# Create nestjs-api
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nestjs-api
  labels:
    app: nestjs-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nestjs-api
  template:
    metadata:
      labels:
        app: nestjs-api
    spec:
      containers:
        - name: nestjs-api
          image: registry.digitalocean.com/nestjs-registry/nestjs-api
          ports:
            - containerPort: 3001
          envFrom:
            - secretRef:
                name: api-env
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nestjs-ingress
spec:
  rules:
    - host: api.mydomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nestjs-api
                port:
                  number: 80

That's the Docker image on registry.digitalocean.com/nestjs-registry/nestjs-api

FROM node:14-alpine3.14 AS BUILD_IMAGE

RUN apk update && apk add yarn curl bash make && rm -rf /var/cache/apk/*

RUN curl -sfL https://install.goreleaser.com/github.com/tj/node-prune.sh | bash -s -- -b /usr/local/bin

WORKDIR /usr/src/app

# install dependencies
RUN yarn --frozen-lockfile

COPY . .
RUN yarn install
RUN yarn build

RUN npm prune --production

RUN /usr/local/bin/node-prune

FROM node:14-alpine3.14

USER 1000
RUN mkdir -p /home/node/app/
RUN mkdir -p /home/node/app/node_modules
RUN mkdir -p /home/node/app/dist

RUN chown -R 1000:1000 /home/node/app
RUN chown -R 1000:1000 /home/node/app/node_modules
RUN chown -R 1000:1000 /home/node/app/dist

WORKDIR /home/node/app

COPY --from=BUILD_IMAGE /usr/src/app/dist /home/node/app/dist
COPY --from=BUILD_IMAGE /usr/src/app/node_modules /home/node/app/node_modules

EXPOSE 3001
ENTRYPOINT ["node"]
CMD ["/home/node/app/dist/main.js"]

That's the log of my nginx-ingress-controller

2021/09/29 18:37:12 [error] 590#590: *147263 connect() failed (111: Connection refused) while connecting to upstream, client: MY_HOME_IP, server: api.mydomain.com, request: "GET / HTTP/1.1", upstream: "http://10.244.0.229:3001/", host: "api.mydomain.com"
2021/09/29 18:37:12 [error] 590#590: *147263 connect() failed (111: Connection refused) while connecting to upstream, client: MY_HOME_IP, server: api.mydomain.com, request: "GET / HTTP/1.1", upstream: "http://10.244.0.229:3001/", host: "api.mydomain.com"
2021/09/29 18:37:12 [error] 590#590: *147263 connect() failed (111: Connection refused) while connecting to upstream, client: MY_HOME_IP, server: api.mydomain.com, request: "GET / HTTP/1.1", upstream: "http://10.244.0.229:3001/", host: "api.mydomain.com"
MY_HOME_IP - - [29/Sep/2021:18:37:12 +0000] "GET / HTTP/1.1" 502 150 "-" "PostmanRuntime/7.28.4" 204 0.000 [default-nestjs-api-80] [] 10.244.0.229:3001, 10.244.0.229:3001, 10.244.0.229:3001 0, 0, 0 0.000, 0.000, 0.000 502, 502, 502 a54bfdae6e0b77bf894e53d8ac8fa29e

Here are some outputs from kubectl

$ kubectl get pods -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP             NODE                  NOMINATED NODE   READINESS GATES
nestjs-api-6bcccbdbd5-zmdqg   1/1     Running   0          61m   10.244.0.238   api-wn5e3n2u8-u3j8q   <none>           <none>

$ kubectl get service -o wide
NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE    SELECTOR
nestjs-api   ClusterIP   10.245.37.142   <none>        80/TCP    3h     app=nestjs-api
kubernetes     ClusterIP   10.245.0.1      <none>        443/TCP   5d6h   <none>


$ kubectl get pods -n ingress-nginx -o wide
NAME                                        READY   STATUS      RESTARTS   AGE     IP             NODE                  NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-psdn2        0/1     Completed   0          6h34m   10.244.0.251   api-wn5e3n2u8-u3j8q   <none>           <none>
ingress-nginx-admission-patch-x8vvs         0/1     Completed   1          6h34m   10.244.0.252   api-wn5e3n2u8-u3j8q   <none>           <none>
ingress-nginx-controller-68649d49b8-bj7vp   1/1     Running     0          6h34m   10.244.0.175   api-wn5e3n2u8-u3j8q   <none>           <none>

UPDATE

The log of my main.js shows that the app is listening on port 3001

{"message":"Application is running on: http://127.0.0.1:3001"}

Which comes from the following code

// Get Server IP and PORT from configuration
const ip = process.env.SERVER_IP;
const port = parseInt(process.env.SERVER_PORT, 10);

// Start server
await app.listen(port, ip);
logger.log(`Application is running on: ${await app.getUrl()}`);
logger.log(`Environment: ${environment}`);

These are the vars saved in api-env as a Secret on k8s.

SERVER_IP: 127.0.0.1
SERVER_PORT: 3001

Anyway as suggested by @mdaniel i have added the livenessProbe to the spec of my Deployment and the Pod crash.

    spec:
      containers:
        - name: nestjs-api
          image: registry.digitalocean.com/nestjs-registry/nestjs-api
          ports:
            - containerPort: 3001
          livenessProbe:
            httpGet:
              port: 3001
              path: '/'
          envFrom:
            - secretRef:
                name: api-env

Now I'm really confused. There is something wrong in my configuration but I have no idea what.

Thanks in advance.