Linux – Low latency TCP settings on Ubuntu

latencylinuxnetworkingperformance-tuningUbuntu

There is a server for measurements running on Ubuntu in my lab. And there is C program, which receives data through TCP connection and should as soon as possible send a reply.

Configuration

  • CPUs: 2 processors x 4 cores – Intel(R) Xeon(R) CPU E5345 @ 2.33GHz
  • RAM: 12 GB
  • NIC: Intel Corporation 80003ES2LAN Gigabit Ethernet Controller / 82546EB Gigabit Ethernet Controller
  • Network switch: Cisco Catalyst 2960
  • Data info: Data blocks come approx. each 10 milliseconds. Data block size is approx. 1000 byte.

Network latency when receiving packets is very critical (tens of microseconds are important). I optimized the program to the maximum, but I have no experience tweaking Ubuntu.

What can be configured in Ubuntu to reduce the local delay of processing/sending packets?

Best Answer

Honestly, I wouldn't be using Ubuntu for this... but there are options that can be applied to any Linux variant.

You'll want to increate your network stack buffers:

net.core.rmem_default = 10000000
net.core.wmem_default = 10000000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

If the application is writing to disk, maybe a scheduler/elevator change would be necessary (e.g. the deadline elevator).

At the server level, you can modify the CPU governor and power and CPU frequency management (P-States, C-States).

At the OS level, you can change the realtime priority of your application (chrt), optimizing to reduce interrupts, pinning it to a CPU or group of CPUs (taskset), and stopping any unnecessary services or daemons.

You can also see some suggestions at: How to troubleshoot latency between 2 linux hosts

It's difficult to get more specific without knowing the hardware or networking equipment involved.

Related Topic