Iis – Creating arbitrary urls that work like redirecting virtual directories

iisurlvirtualdirectories

I'm a complete newbie to server administration, so I'm not even sure if I'm posting this in the right place. If this seems to be a bad question, please respond correctly. Thank you.

I have been wondering for some time how sites such as URL shorteners work. For example, a shortened url with the popular service bit.ly looks like: http://bit.ly/2bgrkx

My question is, how are their servers set up to handle these urls with unique shortcodes at the end? And also, how could this handling be implemented in an IIS environment, with a different purpose of course, not URL shortening. For example, I also saw what twtvite.com was doing with these urls.

I've considered that they could be possibly creating virtual directories that redirect, but that seems a little too out of the ordinary.

Thank you very much, I hope this question isn't too out-of-place.

Best Answer

Most URl shorteners work like this - all requests, who are not requesting CSS / or other static files are routed to script, which found corresponding real URL from short URL identifier.

Then there are 2 ways: a)Sending HTTP header, which orders browser to redirect. Most popular choice is 302 Found . According to HTTP specs, its temporary redirect, but most browsers implement that as permament redirect. Better choice is 301 Moved permamently. b)Sending html document, with META tag, which says browser to redirect. Mostly, thats bad idea, because all redirects will be temporary and valid HTML doc is more bandwith consuming than simple HTTP header.And correct way is to do such things at protocol, not content level.

If you want to see how specific URL shortening services works, you can use Firebug (https://addons.mozilla.org/en-US/firefox/addon/1843) to see all requests, and responses with HTTP headers.

In apache you can use mod_rewrite, to rewrite urls, in IIS - http://www.isapirewrite.com/ (proabaly there is better solution, but I am not active IIS user any more).

Related Solutions

Iis – Way Around Empty HTTP_REFERRER From 301 Redirect

Here's how 301/302 redirects work:

User access the shortened URL
Shortened URL service returns a redirect header
The User's BROWSER gets the redirect header and loads the page without using the HTTP_REFERRER

As you can see, its not the service that's at fault, its the client-side browser. This is just how redirects work. You'll have to either (a) start using a URL shortening service that allows for traffic statistics (i.e., DwarfURL), or (b) force all your users to use specific browsers that DO enforce the returning of the referral.

Also see this post for more info.

Good luck!

Iis – Block Bots with IIS 7.5 and 8.0

Normally you use robots.txt. It will work on all well behaved bots.

For bots that are not well behaved there is often little you can do. You can limit connection counts or bandwidth in your firewall or webserver, but major bots will typically use multiple IP addresses. Limiting based on user-agent strings is usually not a good idea, as those are trivial for the bot to spoof, and bots that does not care about robots.txt have a tendency to spoof useragent strings as well. It works in the specific case when the bot sends a correct user agent, but does not obey the robots.txt.

Edit: If you really want to block based on useragent instead of pushing it back to your firewall or similar I think the easiest way is to use URLScan. You write a rule that looks something like this:

[Options]
 RuleList=DenyYandex

[DenyYandex]
 DenyDataSection=Agents
 ScanHeaders=User-Agent

[Agents]
 Yandex

Best Answer

Related Solutions

Iis – Way Around Empty HTTP_REFERRER From 301 Redirect

Iis – Block Bots with IIS 7.5 and 8.0

Related Topic