Low Latency Code – Does It Have to Be ‘Ugly’?

"Low Latency" refers to one of the possible sides of high performance, in particular minimizing the time it takes a system to produce an output from the instant where an event is detected.

While there is a common ground to all optimizations (make the most out of the hardware) the type of work and optimizations to perform is quite different than when optimizing for high throughput or for high loads, and in many cases they are contradictory goals: optimizing for latency will in many cases reduce the overall performance of a system in ensuring that the response is fast.

This type of goals is quite common in financial systems, where being the first is the difference between getting the best price in a trade, or not even being able to execute the order at all.

As of the technologies that are related to this context, they usually range from good knowledge of the platform (hardware and OS) where the code will run, knowledge of performance measuring and tuning, multithreading and usually knowledge of networks.

I've done a fair amount of work supporting HFT groups in IB and Hedge Fund settings. I'm going to answer from the sysadmin view, but some of this is applicable to programming in such environments as well.

There are a couple of things an employer is usually looking for when they refer to "Low Latency" support. Some of these are "raw speed" questions (do you know what type of 10g card to buy, and what slot to put it in?), but more of them are about the ways in which a High Frequency Trading environment differs from a traditional Unix environment. Some examples:

• Unix is traditionally tuned to support running a large number of processes without starving any of them for resources, but in an HFT environment, you are likely to want to run one application with an absolute minimum of overhead for context switching, and so on. As a classic small example, turning on hyperthreading on an Intel CPU allows more processes to run at once -- but has a significant performance impact on the speed at which each individual process is executed. As a programmer, you're likewise going to have to look at the cost of abstractions like threading and RPC, and figure out where a more monolithic solution -- while less clean -- will avoid overhead.

• TCP/IP is typically tuned to prevent connection drops and make efficient use of the bandwidth available. If your goal is to get the lowest latency possible out of a very fast link -- instead of to get the highest bandwidth possible out of a more constrained link -- you're going to want to adjust the tuning of the network stack. From a programming side, you'll likewise going to want to look at the available socket options, and figure out which ones have defaults more tuned for bandwidth and reliability than for reducing latency.

• As with networking, so with storage -- you're going to want to know how to tell a storage performance problem from an application problem, and learn what patterns of I/O usage are least likely to interfere with your program's performance (as an example, learn where the complexity of using asynchronous IO can pay off for you, and what the downsides are).

• Finally, and more painfully: we Unix admins want as much information on the state of the environments we monitor as possible, so we like to run tools like SNMP agents, active monitoring tools like Nagios, and data gathering tools like sar(1). In an environment where context switches need to be absolutely minimized and use of disk and network IO tightly controlled, though, we have to find the right tradeoff between the expense of monitoring and the bare-metal performance of the boxes monitored. Similarly, what techniques are you using that make coding easier but are costing you performance?

Finally, there are other things that just come with time; tricks and details that you learn with experience. But these are more specialized (when do I use epoll? why do two models of HP server with theoretically identical PCIe controllers perform so differently?), more tied to whatever your specific shop is using, and more likely to change from one year to another.