No matter what URL I specify for curl
I always get the same HTML 404 Error page back.
If I use the --verbose
option, it looks like curl
always connects to the same IP address.
$ curl --verbose http://www.edgeoftheweb.co.uk
* About to connect() to www.edgeoftheweb.co.uk port 80
* Trying ::ffff:74.117.222.24... connected
* Connected to www.edgeoftheweb.co.uk (::ffff:74.117.222.24) port 80
> GET / HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: www.edgeoftheweb.co.uk
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 15 Sep 2011 13:52:07 GMT
< Server: Apache/2.2.3 (CentOS)
< X-Powered-By: PHP/5.2.11
< Content-Length: 519
< Connection: close
< Content-Type: text/html; charset=UTF-8
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<TITLE>www.edgeoftheweb.co.uk</TITLE>
</HEAD>
<FRAMESET rows="100%,*" border="0" frameborder="0" framespacing="0">
<FRAME name=top src="http://www.searchnut.com/?domain=edgeoftheweb.co.uk®istrar=directnicexpired&aff_txt=This+domain+is+expired%2C+please+renew+it.&aff_url=https%3A%2F%2Fsecure.directnic.com%2Fmyaccount%2Frenewals%2F" noresize>
</FRAMESET>
Closing connection #0
$ curl --verbose http://api.twitter.com
* About to connect() to api.twitter.com port 80
* Trying ::ffff:74.117.222.24... connected
* Connected to api.twitter.com (::ffff:74.117.222.24) port 80
> GET / HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: api.twitter.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 15 Sep 2011 13:53:25 GMT
< Server: Apache/2.2.3 (CentOS)
< X-Powered-By: PHP/5.2.11
< Content-Length: 505
< Connection: close
< Content-Type: text/html; charset=UTF-8
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<TITLE>api.twitter.com</TITLE>
</HEAD>
<FRAMESET rows="100%,*" border="0" frameborder="0" framespacing="0">
<FRAME name=top src="http://www.searchnut.com/?domain=twitter.com®istrar=directnicexpired&aff_txt=This+domain+is+expired%2C+please+renew+it.&aff_url=https%3A%2F%2Fsecure.directnic.com%2Fmyaccount%2Frenewals%2F" noresize>
</FRAMESET>
Closing connection #0
The output of curl --version
is:
curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
If I use wget
instead, then I retrieve the correct pages back.
Any ideas how to get curl
to resolve the URL's correctly?
Thanks.
Best Answer
Seems to me curl is using ipv6 to connect while wget is using ipv4
Try the following