NGINX SSL – How to Verify Upstream SSL with NGINX SSL Reverse Proxy

I have NGINX setup as a reverse proxy to host multiple websites using only one IP address. I have a Lets Encrypt certificate on the proxy and a different Lets Encrypt certificate on the upstream server. Basically, I need NGINX to forward traffic to the upstream server and verify that the upstream server has a valid TLS certificate. If I disable proxy_ssl_verify, it will work. If I go to https://app.local.example.com on my internal network, the app works fine.

Network Diagram:

https://app.example.com --> NGINX Reverse Proxy --> https://app.local.example.com (Local IP Address)

NGINX Reverse Proxy Configuration File:

server {
    listen 80;
    rewrite ^ https://$host$request_uri? permanent;
}

server {
        server_name app.example.com;

        listen                                  443 ssl;

        ssl_certificate                         /etc/letsencrypt/live/app.example.com/cert.pem;
        ssl_certificate_key                     /etc/letsencrypt/live/app.example.com/privkey.pem;
        ssl_trusted_certificate                 /etc/letsencrypt/live/app.example.com/chain.pem;

        location / {

                proxy_redirect off;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Ssl on;
                proxy_set_header X-Forwarded-Protocol $scheme;
                proxy_set_header X-Forwarded-HTTPS on;

                proxy_ssl_session_reuse        off;
                proxy_ssl_name                 app.local.example.com

                proxy_ssl_verify               on;
                proxy_ssl_verify_depth         2; # I've tried 1,2,3,4
                proxy_ssl_trusted_certificate /etc/ssl/certs/ca-certificates.crt; 
                            
                proxy_pass                     https://app.local.example.com
        }
}

Here's the error message I'm getting.

[error] 1087#1087: *2 upstream SSL certificate verify error: (20:unable to get local issuer certificate) while SSL handshaking to upstream, client: [Client IP], server: app.example.com, request: "GET / HTTP/1.1", upstream: "https://192.168.1.5:443/", host: "app.local.example.com", referrer: "https://app.example.com/">

OpenSSL Version: OpenSSL 1.1.1f 31 Mar 2020

nginx -v: nginx version: nginx/1.18.0 (Ubuntu)

cat /etc/issue: Ubuntu 20.04.1 LTS \n \l

Output of openssl s_client -connect app.local.example.com:443

CONNECTED(00000003)
depth=0 CN = app.local.example.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = app.local.example.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:CN = app.local.example.com
   i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFeDCCBGCgAwIBAgISA8NkbZ6wz2EnKcedXujKT9AmMA0GCSqGSIb3DQEBCwUA
...
-----END CERTIFICATE-----
...
SSL-Session:
    Protocol  : TLSv1.3
    ...
    Verify return code: 21 (unable to verify the first certificate)