R – asp.net, stateserver, NLB, session lost

asp.netcluster-computingnlbsessionstateserver

1st post on stackoverflow, hope to have great feedback 🙂

I'm currently trying to load balance our web site. We have set up a 2 cluster NLB on windows server 2003 with IIS 6.

While testing the setup, I found that sometimes, our session is lost. A day and a half later, here's the result:

  1. Yes, our machine.config both have the same encryption/decryption key.
  2. Yes, the id in iis metabase.xml are the same for both machine. Actually, the entire file are the same, except for "AdminACL".
  3. Both web application are set with "StateServer" and both pointing at the same machine.

From that point, searching on google gives less information and possible solutions.

From what I know, there's no particular pattern that cause this problem. It just happen once in a while.

While trying to find the problem, I've seen that a request sent the asp session id cookie to the server, but the server didn't map it to the user session.

So the request number x was sent from the client, with the cookie, session was mapped, and everything went smoothly.
The request number x+1 was sent from the client, with the cookie, but session was not found.

Both request were made on the same machine in the NLB.

Here's a snippet of the asp trace.axd:

1st request:

Request Details
Session Id: j2ffvy45updpc52uhw1mbg55 Request Type: GET
Time of Request: 11/26/2008 2:58:06 PM Status Code: 200
Request Encoding: Unicode (UTF-8) Response Encoding: Unicode (UTF-8)

Request Cookies Collection

Name Value Size

ASP.NET_SessionId j2ffvy45updpc52uhw1mbg55 42
AID 22 9

Response Cookies Collection

Name Value Size

Headers Collection

Name Value

Cookie ASP.NET_SessionId=j2ffvy45updpc52uhw1mbg55; AID=22

2nd Request:

Request Details
Session Id: Request Type: POST
Time of Request: 11/26/2008 2:58:08 PM Status Code:
Request Encoding: Unicode (UTF-8) Response Encoding:

Request Cookies Collection

Name Value Size

Response Cookies Collection

Name Value Size

Headers Collection
Name Value
Cookie ASP.NET_SessionId=j2ffvy45updpc52uhw1mbg55; AID=22

As you can see in the 2nd request, the cookie is sent from the client, but asp seems to never add the cookies in it's "Request Cookies Collection". I think that's why it doesn't find the session.

So why the cookie is not mapped to the session? Is that the problem? Is the problem elsewhere?

Feel free to ask any clarifications.

Thank you all for your feedback.

JF

Best Answer

I finally found the answer to my problem. It's origin are within the application code (like 99% of a programmer's 3rd party tools 'bugs'). I decided to post it anyway in case someone is in a similar scenario.

This code was part of WebServiceRequester class. The web service requester class was instanciated when session was created and it is saved in session. During creation, we initalizate the member 'm_webServiceURL', and this member is saved in session after. At which value was this member initialize was depending on a setting on the local machine.

The important part is the following: WebServiceRequester class contains a WebService objects. WebService objects can't be saved in session, they are not serializable in asp. The property had the [NonSerialized] attribute on it. So everytime we accessed the 'WebService' property of the object for the first during a page life cycle, we had to create a new one, and assigning ot it the url 'm_webServiceURL' which was saved in session. So you see, new webservice object, on possibly a different machine, meaning a different setting on each machine.

so here's what happened: box 29 was set to access Web Service at localhost

box 30 was set to access Web Service as 192.168.253.29.

Technically, they are both set on the same machine. But here's a scenario:

login on box 29. m_webServiceURL is set to localhost in session.

[some request on box 29 here]

NLB balancing bring us on box 30. box 30 loads it's session, create a new webservice obect with localhost as the web service address. box 30 made the request to the wrong web service leading to a Session Expired exception.

One of the problem during the debug, was that the local communication were not recorded with the network monitor.

What lead me on the trace, was that we never had an exception logged on the box 29 log trace, as it should have.

Thanks for you suggestions everyone, it was really appreciated.

Have a good day. JF