Nginx failover with upstream load balancer not working, 504 when server 1 is down

I have an nginx docker container balancer that acts as a balancer for 3 nginx docker containers server_1, server_2, server_3.

Load balancing works fine, it hits all servers in a fashion depending on the specified method. However when I stop the first server using docker stop server_1, I get a 504, even though the other servers are still up.

My balancer default file looks like this:

# don't redirect proxy
proxy_redirect  off;

upstream app {
    least_conn;
    server webserver_1;
    server webserver_2;
    server webserver_3;
}

server {
    listen 80;
    location / {
        proxy_pass http://app;
        proxy_next_upstream error timeout invalid_header http_500 http_504;
        # proxy_set_header Host localhost;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

I tried adding fail_timeout=10s max_fails=1 to the three servers in the upstream directive, but no luck. I also tried adding the proxy_next_upstream_tries 2; directive, but this did not help either.

Please let me know if you need any more information.