R – MOSS web service to find out if a user has access to a document library

If inheriting doesn't work out, you could also use jQuery to look for the error message and replace it with something friendly.

There is info about setting up jQuery in SharePoint in this question.

Why not just deploy the webservice to Sharepoint using a Sharepoint solution and a Feature. That way it will be running under the sharepoint app pool and all authentication is done by sp.

Edit:

Seeing that SharePoint should not be in "control" (as stated in the comment), you should create the webservice, and run the application it's under in as using Windows Authentication. IMHO you should create a WCF Service. The, using the information found in this article you make the Service authenticate users against the AD usergroups they are in. see the "Security: Authentication" section of the article.

Then in Visual Studio you create a webpart and add a service reference to the project, pointing to your newly created Service. Have the webpart perform the needed logic (i.e. display data etc.) Deploy the webpart to SharePoint using a SharePoint solution (.wsp files, created with WSPBuilder). Google for SharePoint + wspbuilder + tutorials. The solution should contain 1 feature to deploy the webpart. WSPBuilder integrates with VS and allows for the creation of WSPBuilde project. add a webpart feature item to the project (it will create the xml (deployment related) and code file for the webpart.