Python – How to set up the own proxy server

I don't think that there is a way to explicitly invalidate cached items, but here is an example of how to do the rest. Update: As mentioned by Piotr in another answer, there is a cache purge module that you can use. You can also force a refresh of a cached item using nginx's proxy_cache_bypass - see Cherian's answer for more information.

In this configuration, items that aren't cached will be retrieved from example.net and stored. The cached versions will be served up to future clients until they are no longer valid (60 minutes).

Your Cache-Control and Expires HTTP headers will be honored, so if you want to explicitly set an expiration date, you can do that by setting the correct headers in whatever you are proxying to.

There are lots of parameters that you can tune - see the nginx Proxy module documentation for more information about all of this including details on the meaning of the different settings/parameters: http://nginx.org/r/proxy_cache_path

http {
  proxy_cache_path  /var/www/cache levels=1:2 keys_zone=my-cache:8m max_size=1000m inactive=600m;
  proxy_temp_path /var/www/cache/tmp; 


  server {
    location / {
      proxy_pass http://example.net;
      proxy_cache my-cache;
      proxy_cache_valid  200 302  60m;
      proxy_cache_valid  404      1m;
    }
  }
}

I agree w/ pjz-- it sounds like you're looking for a VPN.

OpenVPN is a great, no cost method of getting started w/ VPNs. It's stable and ready for production use, but even if you don't end up using it, it's a good tool with which to get familiar with VPNs. It's really easy to setup with static keys (for playing around), and only marginally more difficult to setup with certificates (for production use).

• http://openvpn.net/static.html - Static key mini-HOWTO. Not recommended for security, but a great way to get a feel for how OpenVPN works before diving into simple PKI.
• http://openvpn.net/howto.html

You say "Internet traffic" in your question, but it's unclear if that just means browsing web sites, or literally all IP traffic to the Internet. You can pass a "default gateway" route down to the client w/ OpenVPN such that their Internet-bound traffic will route down the OpenVPN "pipe" to the server, which could then put it onto the Internet.

If you only wanted HTTP/HTTPS to be routed down the OpenVPN (i.e. if they PING, run Skype, etc, that traffic can go straight to the Internet), you might consider deploying something like Squid Cache, too, and then configuring client browsers to use that proxy server such that traffic to the proxy was routed down the OpenVPN "pipe" only (i.e. put the proxy on a VPN-accessible IP address, but leave the client's default gateway alone). (You could even do a 'push “dhcp-option 252 ...' to push out a proxy autoconfiguraiton URL to clients via OpenVPN, I believe.)

You've got some options, depending on what you want to do.

re: your comment to pjz about Intranet site access

You're going to have to "pay the piper" on this somehow.

If you're just routing all their Internet-bound traffic down the VPN via a default gateway changeup any traffic to web servers on-subnet with them would still "go direct". If the Intranet web server was on a different subnet, though, their traffic to that subnet would go down the OpenVPN pipe instead of to the on-site router. That'd be bad.

If you did my suggestion above of pushing down a proxy-autoconfiguration script to clients via OpenVPN (or some other means) you could put "exceptions" in that file to cause the clients to "go direct". I typically do that in proxy-autoconfiguration files with:

if ( isPlainHostName(host) ) { return "DIRECT"; }

This causes host names w/o any dots in them to be accessed directly.

If you know a particular host (or wildcard matching pattern) that needs to be directly accessed:

if ( shExpMatch(url,"http://*.customer.com")) { return "DIRECT"; }
if ( shExpMatch(url,"http://known-intranet-server.customer.com")) { return "DIRECT"; }

If you know where your users were going to be working you can put in exceptions into the proxy-autoconfiguraiion file prior to the fact. If not, though, you're going to have to reactively deal with such issues. If you don't know beforehand, though, you're asking for a solution that can "do the right thing" automatically. Unfortunately, computers do a horrible job with that. >smile<

I'd take the extra time with whatever you deploy to use proxy-autoconfiguration files. It gives you a centralized method (that can be updated "on the fly" w/o touching client computers) to control diverting HTTP traffic to a proxy server or letting it go directly to the Internet. They're amazingly handy for this kind of application.