HTTP Server – Reasons for Different Results Using Direct IP vs Domain

httphttpsnetworkingPROXY

Some context, there's an HTTP(s) server that has some assets I'm trying to proxy and I need to do so using the IP address and not the domain. After some troubleshooting I realized that I'm getting the response of if I make an HTTP request to https://202.100.200.152/sushi/ and only get the response that i want if I use the domain https://sp.water.contoso.com/sushi/.

I know for sure the HTTP(s) server has a proxy in front of it to route connections all over the place. I can't access this server, so I have consider it a black box. I thought maybe it checks the domain via host header but it still didn't work when I overrode it.

I'm wondering besides HOST header what are some other factors for why I'm not getting the response that I want.

I simulated the BAD response with CURL:

curl -k -v -I -H 'Host: sp.water.contoso.com'  https://202.100.200.152/sushi/
*   Trying 202.100.200.152...
* TCP_NODELAY set
* Connected to 202.100.200.152 (202.100.200.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.water.contoso.com
*  start date: Feb 11 18:53:34 2020 GMT
*  expire date: Feb 10 18:53:35 2022 GMT
*  issuer: CN=ingress-operator@3582449223
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> HEAD /sushi/ HTTP/1.1
> Host: sp.water.contoso.com
> User-Agent: curl/7.64.1
> Accept: */*
> 
* HTTP 1.0, assume close after body
< HTTP/1.0 503 Service Unavailable
HTTP/1.0 503 Service Unavailable
< Pragma: no-cache
Pragma: no-cache
< Cache-Control: private, max-age=0, no-cache, no-store
Cache-Control: private, max-age=0, no-cache, no-store
< Connection: close
Connection: close
< Content-Type: text/html
Content-Type: text/html

< 
* Excess found in a non pipelined read: excess = 3131 url = /sushi/ (zero-length body)
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):

And then the GOOD response with CURL

curl -v -I -k  https://sp.water.contoso.com/sushi/
*   Trying 202.100.200.152...
* TCP_NODELAY set
* Connected to sp.water.contoso.com (202.100.200.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=CA; L=Silicon Valley; O=Cupcake; OU=contoso Data Platform; emailAddress=contoso-adp@us.contoso.com; CN=contoso-Data-and-AI
*  start date: Oct 29 04:33:35 2019 GMT
*  expire date: Jan 30 04:33:35 2022 GMT
*  issuer: C=US; ST=CA; L=Silicon Valley; O=Cupcake; OU=contoso Data Platform; emailAddress=contoso-adp@us.contoso.com; CN=contoso-Data-and-AI
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> HEAD /sushi/ HTTP/1.1
> Host: sp.water.contoso.com
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: openresty
Server: openresty
< Date: Wed, 17 Jun 2020 17:46:01 GMT
Date: Wed, 17 Jun 2020 17:46:01 GMT
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8
< Content-Length: 266
Content-Length: 266
< Connection: keep-alive
Connection: keep-alive
< X-Powered-By: Express
X-Powered-By: Express
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Cache-Control: public, max-age=0
Cache-Control: public, max-age=0
< Last-Modified: Tue, 02 Jun 2020 21:26:35 GMT
Last-Modified: Tue, 02 Jun 2020 21:26:35 GMT
< ETag: W/"10a-17276edcaf8"
ETag: W/"10a-17276edcaf8"
< X-Frame-Options: DENY
X-Frame-Options: DENY

< 
* Connection #0 to host sp.water.contoso.com left intact
* Closing connection 0

As you can see from CURL output, both are using the same IP and the > parts indicate they send same headers. What are some possible reasons why the server is returning me undesired pages?

Best Answer

The reason here is that when you make the request to IP address, TLS Server Name Indication field contains the IP address of the host, not the domain.

The server you are connecting has different virtual hosts defined for IP address and different domain names. The virtual host defined for the IP address does not provide the service you are looking for.

In order to send proper TLS Server Name Indication field with curl, you need to use the --resolve parameter:

curl --resolve sp.water.contoso.com:443:209.100.200.152 https://sp.water.contoso.com/sushi/

This will tell the server that the TLS connection should be made to the sp.water.contoso.com virtual host instead if the IP address.

Adding HTTP Host header is only effective with HTTP protocol.

Related Topic