Python – How to make urllib2 requests through Tor in Python


I'm trying to crawl websites using a crawler written in Python. I want to integrate Tor with Python meaning I want to crawl the site anonymously using Tor.

I tried doing this. It doesn't seem to work. I checked my IP it is still the same as the one before I used tor. I checked it via python.

import urllib2
proxy_handler = urllib2.ProxyHandler({"tcp":""})
opener = urllib2.build_opener(proxy_handler)

Best Answer

You are trying to connect to a SOCKS port - Tor rejects any non-SOCKS traffic. You can connect through a middleman - Privoxy - using Port 8118.


proxy_support = urllib2.ProxyHandler({"http" : ""})
opener = urllib2.build_opener(proxy_support) 
opener.addheaders = [('User-agent', 'Mozilla/5.0')]

Also please note properties passed to ProxyHandler, no http prefixing the ip:port