ISA 2006 switch to kerberos causes authentication problems for some users

isa-serverkerberosntlmpacPROXY

In our large corporate environment we have 4 ISA 2006 servers set up. The users (WinXP IE8) are configured with an automatic proxy configuration script. Recently, the PAC was modified to return FQDN instead of IP addresses of the ISA servers. This was done to force Kerberos authentication instead of NTLM.

The change has been causing intermittent problems for some users. When accessing sites over SSL, they get multiple prompts for authentication from the proxy server. Not all users are affected. Different sites are affected. At one point, one of the proxy servers started spewing out a "502 Proxy Error. No buffer space is supported." It was rebooted and was back in business.

The best we can figure is that it has to do with the large Kerberos token size (we are a big operation with hundreds/thousands of AD security groups).

Some of users have MaxPacketSize and MaxTokenSize configured for Kerberos. Some don't. The two problem users I looked at both had these settings. 

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters]
"MaxPacketSize"=dword:00000001
"MaxTokenSize"=dword:0000ffff

Rolling back the PAC to use IP addresses (and NTLM) resolves the problem for the users. But the proxy admins still want Kerberos for these reasons: Why use Kerberos instead of NTLM in IIS?.

Will pushing out these registry settings to all users fix the problem, or are these settings the root of the problem?

Is there a setting on the ISA server that needs to be adjusted to match the token size settings on the desktops?

Thanks.

Best Answer

Could be a token size issue. All computers should have those settings at those values anyway.

Another possibility is if there are any policy filters that specify a maximum http header length.
As kerberos stores the group membership in the pac, when using integrated authentication that is encoded and inserted into every http request header. Any http that involves integrated authentication with kerberos needs to be very generous with maximum request header size.

There also is a hotfix for a symptom with that description.

An ISA Server 2006 Web Proxy client receives error code 502 when a user tries to visit certain Web sites
http://support.microsoft.com/kb/935693

http://www.isaserver.org/tutorials/configuring-isa-server-2006-http-filter.html

http://technet.microsoft.com/en-us/library/bb838827.aspx

Related Topic