Can please someone explain me on the below screenshot why host 192.168.1.200 on packet 9 sends an ACK for the packets 6 - 7 - 8? The total packet size does not correspond to the window size agreed. If I understood it correctly the window size is the amount o data that a device can send without acknowledging.
The window size is the maximum amount of unacknowledged data that can be outstanding in a socket; however, there is no requirement to fill this window before ACK-ing. In fact, most low-latency connections do not fill the window because stations acknowledge data so quickly. What you are seeing is normal, there is no problem.
Well. Since you've captured packets that show that your Trading Server is sending the TCP ACK's with a window size of 0, you at least know the problem is definitely on your side. Which is actually a good thing, because you are in a position to fix it. (There is one thing that might be the issue which would be a problem on their end, I'll talk about that later)
You've also traced the issue to happening during times of increased throughput, also a good thing.
You said the CPU/RAM usage on your Trading Server reported normal. The application you are using, is it by chance configured to use a limited amount of RAM on the host OS? Maybe a limited percent? Because it would stand to reason that if so, as you had more connections and more throughput, there was less RAM available to the application, and therefore less resources available for TCP.
Either way, what OS is your Trading Server using? If you haven't already, you should look into tuning the OS to dedicate more RAM to TCP. In Windows, there are Registry values you can modify. In Linux, there are config files you can edit.
It would also be wise to make sure your Firewall (and nothing else in between) is trying to proxy your TCP sessions. That way you know you are dealing with the full "client to server" TCP connection, and not something in between.
The last thing I can offer is to study the TCP packets being sent from the Stock Exchange to your server just before your server sends a Window Size of 0. In particular, look for the incoming packets to have the value 11 in the IP Header's ECN field (Explicit Congestion Notification -- the last two bits in what used to be DSCP, bits 14 and 15 if you're looking at an IP Header). There is a chance that if both the Client and Server in the communication supported ECN, and a router in transit detected congestion, that it turned these bits on to tell the client and server to slow down their transfers. (This is that thing I said that might be a problem on their end)
I think that (tries to) answer questions 0,1,3. I'll have to dig around a bit more to give you a reliable answer for 2. But I'm pretty confident there is a way.
Best Answer
So all you need to do is: Right Click one of the TCP segments => Protocol Preferences => uncheck Relative Sequence Numbers.
FYI: Ignore the filters, I just did that to make it more aesthetically pleasing. They have nothing to do with the option selected.
EDIT
So to address your comment:
In the older version of Ethereal it has the option to disable relative sequence numbers and window scaling. When the window scaling option is enabled in the older version you first mentioned (Ethereal aka old school Wireshark), it simply calculates the current window size against the current window scaling factor to give you a total. When it's disabled, it only displays the current window size without the scaling factor taken into account.
The picture above is a snippet from the new version of Wireshark sniffing a TCP segment. Here's how it correlates to the older version.
Window Size value This is the value present when window scaling is disabled in the older version. Calculated Window size This is the value present when window scaling is enabled in the older version.
The major difference being is that the newer version contains both values and not just one or the other, so disabling the window scaling option isn't necessary.