Iis – Getting Concurrent Session Metrics from IIS Raw Logs

You seem to be very close to a request that I have on a frequent basis from various managers here. "Can you tell me what parts of the [insert name here] website are not visited?"

The answer is simply anything that isnt reported as visited. I have gone around the explanation that its not possible to report on anything that isnt in the log so many times!

Reporting on nil hits is essentially a report of your whole website (100's or 1000's of pages??) with the visited ones excluded. It becomes less practical as the size of the site increases.

What are you intentions for the data - your query just lists the data to screen - do you put this into any sort of file/archive/database for further processing eg csv file for Excel to produce graphs, SQL DB for activity reports..?

Given that you are running into issues trying to sort the data for a single day, I'd look to one of two strategies.

1. Find a better sort. See if you can get the windows sort tool to work for you. The logs are rigged with date and time first, in an ascii-text-sort friendly format for a reason. It uses a lot less memory and doesn't have to parse lines to sort. My bet is this works for you.

2. Write an interleave, that opens all 14 files and pulls the earliest line from the top of each, working its way through the 14 files simultaneously. I shudder to think of this but it wouldn't need but 64KB of memory for each file.

old answer:

Divide and conquer. Write one script that reads logs and puts them in new files by date, with a known filename that has the date in it (weblog-20110101.log). Run a sort on each file that sorts by time. Cat the files you need together.