Configuring f5 to retry requests on timeout

One thing that sticks out at me is that your web servers are on the same machine using different IP addresses. You're not really gaining anything with load balancing if you go and put all the servers on one box.

Some of what you describe with the packets seems normal; have you used the 'Expert Info' and 'Follow TCP Stream' features of Wireshark? Those will show higher-level views of the TCP connections. Also keep in mind that Wireshark might not be capturing all packets (especially if you're on gigabit ethernet).

As for the problem, it's possible you're seeing a bug in the LB, exposed by the fact that all your web services are on the same box, which means that your three IP addresses all have the same MAC address. If the LB tracks connections using the MAC address, it could be confusing your three server IP addresses and directing packets to the wrong place.

I suggest posting on F5's iRules forum for this, devcentral.f5.com. Here's the iRules forum link:

http://devcentral.f5.com/Forums/tabid/1082223/asg/50/showtab/groupforums/aff/5/afv/topics/Default.aspx

To provide a suggestion, it would help to know what type of persistence you're using. In general, you could manually select a pool member, but then you'd also need to manually set the persistence for that request as LTM won't do it for you on manual selections from an iRule.

Aaron