IIS 7 on SBS 2008 – logging is going haywire

iisiis-7windows-sbs-2008windows-server-2008wsus

The C: drive on our SBS 2008 server just ran out of space. The culprit appears to be IIS, which is logging an incredible amount of activity in one particular log folder. Most of the IIS log folders look normal, but each of the daily files in C:\inetpub\logs\LogFiles\W3SVC1372222313 is at least 4.4MB, with the largest being yesterday's, at 1.67 GB!

The largest ones I can't even open on the server, but I've examined several of the smaller ones. They all show several dozen entries being made every few minutes that look like this:

2009-07-11 00:00:02 fe80::5558:434c:a610:405a%10 POST /ApiRemoting30/WebService.asmx - 8530 [DOMAINNAME]\[SERVERNAME$] fe80::5558:434c:a610:405a%10 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.4016) 200 0 0 3

Typically 40 or 50 of these entries will be made in the same second, with gaps of 2-5 minutes between each batch of entries. The other 1 percent of the entries in the file appear to involve WSUS.

I'm going to delete most of these files, because I don't really have a choice, but I'd like to know what's causing this out-of-control logging and how to put a lid on it in the future.

UPDATE: Okay, I've been able to examine a few more files. The bloat is apparently being caused by something going wrong when someone (i.e., me or another admin) logs in to WSUS interactively:

  1. The trouble begins with a single log
    entry with no username (just "-").
    It gets an HTTP status of 401.2 and
    a sc-win32-status code of 5.

  2. This is followed by a long stretch
    of entries that alternate between no
    username and my own username. The
    ones with no username have an HTTP
    status of 401.1 and a
    sc-win32-status of 2148074254. The
    ones with my username are normal
    HTTP 200 entries.

So as far as I can tell what appears to be happening is that when I log in to administer WSUS via the SBS console, NTLM authentication is not persisting behind the scenes, causing continual reauthentication attempts throughout the session, transparently to me. Hundreds of these entries are being created every second, adding about 70MB per hour to the log file. I have no idea why this is happening.

Best Answer

That's IPv6-based access to WSUS that you're seeing there.

Temporarily disable logging so that you don't fill the drive again:

  • Jump into IIS Manager
  • Locate the WSUS web site (it'll be the one listening on port 8530)
  • Bring up the Logging properties for the root of the site
  • Click "Disable" in the "Actions" pane.

That'll stop the logs from building up.

I can't say that I've seen WSUS-related traffic build up logs that big before. 4.4MB in a day isn't unheard of, but the 1.67GB in a day means that something has gone wrong.

Yesterday's log file is going to tell you lots about what was occurring. I find it hard to believe that it was all WSUS traffic. I wonder if something else didn't start banging on the server computer. Get that larger log file off of the machine and have a look at it.

Your log looks like it's in the W3C extended format. The format of that log file appears to be:

Date, Time, source IP address, HTTP request method, URI stem, probably URI query, server port, username, server IP address, user agent, HTTP result, probably Win32 status, and probably time taken

(The "probably" fields are because I can't be sure without seeing more of the file.) The header on the file will tell you the format for sure.

You need to get a look at that 1.67GB file-- it's gonna tell you what's up. Logging disabled on the site will prevent the hard drive from filling up again, but you want to know what's happening, behind the scenes, since it's going to be impacting server performance in some manner. Ultimately, you want to get to the bottom of the cause and then get logging enabled again (so that you have an audit trail if you have to track down strangeness again in the future).

Related Topic