Nginx Reverse Proxy Issue in Azure App Service – Solutions

I've been struggling for a few weeks now with an issue and I'm not knowledgeable enough to even know if what I'm trying to achieve is possible. FusionAuth forums haven't been very helpful, and neither has Azure support which can't do much outside of App service configuration help.

To give a brief explanation of what I am trying to achieve. I have FusionAuth running in an App Service Linux container. It has a domain similar like https://mycustomapp.azurewebsites.net, but I setup a custom domain https://auth.mycustomapp.com.

FusionAuth by default is listening on port 9011 within it's container. This is the http port. When run in production I receive this warning:

The recommended solution for this by FusionAuth is to have a reverse proxy sitting in front of the app to add the correct request header.

I've been trying to configure a Nginx reverse proxy in another App Service Linux instance in the same App Service plan. The container is using nginx:latest, and I'm copying into the container my custom config file as well as ssl certificates. I also setup a custom domain for this app, running at proxy.mycustomapp.com.

My configuration looks something like this:

server {
    if ($host = proxy.mycustomapp.com) {
        return 301 https://$host$request_uri;
    }
    listen 0.0.0.0:80;
    server_name proxy.mycustomapp.com;
    return 404;
}

server {
  listen 443 ssl;
  server_name proxy.mycustomapp.com;

  access_log /var/log/nginx/mycustomapp.com.log;

  ssl_certificate /etc/nginx/ssl/auth.mycustomapp.com-chain.pem;
  ssl_certificate_key /etc/nginx/ssl/auth.mycustomapp.com-key.pem;
  ssl_dhparam /etc/nginx/ssl/dhparam.pem;

  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers HIGH:!aNULL:!MD5;

  location / {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        X-Forwarded-Port $server_port;
    proxy_read_timeout      90;
    proxy_http_version      1.1;

    proxy_pass https://mycustomapp.azurewebsites.net/;
  }
}

My nginx server starts without issue but when navigating to proxy.mydomain.com I get a browser error that the page redirected me too many times.

Firstly is what I am trying to do possible, and if so what is wrong with my config?