Nginx SSL – SSL Doesn’t Work with Python Requests but Works with Browsers

nginxpythonssl

I have a tiny web server with Python (v3.6), Flask (v1.1.1) & NginX (v1.16.1). I have got and set an SSL certificate from sslforfree.com. Everything works fine when I visit pages using any browser, but I cannot access it using Python scripts and requests lib:

import requests
print(requests.get('https://www.some-project.com'))
Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 441, in wrap_socket
    cnx.do_handshake()
  File "/anaconda3/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1907, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/anaconda3/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1639, in _raise_ssl_error
    _raise_current_error()
  File "/anaconda3/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/anaconda3/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/anaconda3/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 448, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/anaconda3/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.some-project.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aivanf/Desktop/some-project/py-API/test.py", line 55, in <module>
    main()
  File "/Users/aivanf/Desktop/some-project/py-API/test.py", line 49, in main
    print(requests.get(MAIN_URL))
  File "/anaconda3/lib/python3.6/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/anaconda3/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.some-project.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

I tried to do the same from different computers, to update OpenSSL, root certificates, but nothing helped… What's the problem? Why all the browsers works fine with the website, but Python does not? How to solve it?

Also, accessing https://sslforfree.com itself works well, though it's certificates is very similar to mine. So, I guess the problem is with my NginX config:

Here is my NginX config:

ssl_certificate /var/www/ssl/ca.crt;
ssl_certificate_key /var/www/ssl/private.key;

# Redirect @ to www
server {
    listen 80 default_server;
    listen 443 ssl default_server;
    server_name  some-project.com;
    rewrite ^ https://www.some-project.com$request_uri?;
}

# Redirect www HTTP to HTTPS
server {
    listen 80;
    server_name  www.some-project.com;
    rewrite ^ https://www.some-project.com$request_uri? permanent;
}

server {
    listen 443 ssl;
    server_name  www.some-project.com;

    include /etc/nginx/vhosts-includes/*.conf;

    # Static Content
    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /var/www/some-project/;
    }

    location / {
        default_type "text/html";
        alias /var/www/some-project/static/;
    }
}

server {
    listen 443 ssl;
    server_name  api.some-project.com;

    # Reverse Proxy
    location / {
        proxy_pass http://localhost:5926;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-Ip $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Best Answer

The "wrong" certificates were used, as Nginx needs the full chain + key. Solved by combining the domain and CA-bundle certificates.

Related Topic