I am looking for any kind of solution to properly get an IIS request such as https://stackoverflow.com/% and http://bing.com/% to not display a 400 Bad Request page, but display a custom error page similar to how http://google.com/% and http://facebook.com/% do (obviously those examples are not on IIS).
I believe I have tried setting all the applicable http.sys registry settings (AllowRestrictedChars, PercentUAllowed) per http://support.microsoft.com/kb/820129 but that has not helped. Setting AllowRestrictedChars and a custom 400 page has fixed urls such as https://stackoverflow.com/%12 but not /%.
Best Answer
This is blocked right in the IIS kernel level. As a test I pulled out every module in IIS so that it didn't even have a static page handler, and it still displayed the 400 error message.
I don't believe it's possible with IIS to get around that. The registry settings you mentioned are for other types of restricted characters. I haven't seen a lever to change that functionality.
What's your goal is avoiding that? It opens your attack surface wider, and I can't imagine a legit visitor being lost as a result of blocking incomplete URL escape sequences.
Update2: Here are three great links on this. Both Nazim Lala and Wade Hilmo from the IIS team have blogged about this because of discussion around your question. Also Scott Hanselman has a great post on the querystring part within .NET:
Update: I checked with a member of the IIS team to get an authoritative answer. He mentioned that the % is considered an unsafe character according to RFC 1738 (http://www.ietf.org/rfc/rfc1738.txt).
Here's the relevent text:
So IIS proactively blocks this up at the core level, a proactive security measure to minimize their attack surface.