I'm using SSH port forwarding – as marvelously illustrated and elaborated here – to access my server.
I've spent more time that I'd like to admit, troubleshooting why access to my Python web server 404s. Full length question for completion here.
My issue is, that if I use port forwarding to directly forward my requests to the port, where the Python server is running (e.g. 8282), all requests 404.
The Python server, for simplicity's sake (serving an index.html, containing the string hello
):
python -m http.server 8282
The SSH port forwarding from the host, where I want to gain access:
ssh -L 8181:<farawayhost-ip>:8282 <sshuser>@<remotehost-ip>
For the subsequent requests on the host with curl
, I've defined the environment variable http_proxy
:
export http_proxy=127.0.0.1:8181
Apparently the requests are not correct, as they're prepended by the IP address. For comparison here's the Python http.server log, first accessing it locally from the server, then from the external host, using SSH port forwarding.
Serving HTTP on 0.0.0.0 port 8282 ...
<server-public-ip> - - [<timestamp>] "GET /index.html HTTP/1.1" 200 -
<proxy-ip> - - [<timestamp>] code 404, message File not found
<proxy-ip> - - [<timestamp>] "GET http://<server-public-ip>:8282/index.html HTTP/1.1" 404 -
I was able to gain access to the Python web service from the external host, by placing an Nginx proxy between the Python web server and the external host.
Nginx is installed on the same server as the Python web service is and simply listens on port 80 and proxies to localhost:8282
.
Nginx configuration:
server {
listen 80 default_server;
location / {
proxy_pass http://127.0.0.1:8282;
}
On the external host, the SSH port forwarding is configured to port 80, instead of 8282.
ssh -L 8181:<farawayhost-ip>:80 <sshuser>@<remotehost-ip>
export http_proxy=127.0.0.1:8181 # proxy for curl
Access to the web service now succeeds.
Nginx log:
<remotehost-ip> - - [<timestamp>] "GET http://<server-ip>/ HTTP/1.1" 200 6 "-" "<user-agent>"
Python http.server log:
127.0.0.1 - - [<timestamp>] "GET / HTTP/1.0" 200 -
To my question:
My initial problem is solved, but I want to understand why.
How do these requests differ, why does the direct access to the Python server with SSH port forwarding fail and the access via the Nginx proxy succeeds?
Best Answer
Your problem is that SSH tunnel is not a HTTP proxy (while
nginx
is) so settinghttp_proxy
to point to SSH tunnel end point causescurl
to send HTTP requests in format suitable for HTTP proxy while SSH tunnel sends these requests directly to your python app without any special processing as HTTP proxy would do. With SSH tunnel you need to removehttp_proxy
variable and access your app directly withcurl http://localhost:8181/
.