What could cause a not so busy redis server to close connection

linux-networkingredistcp

We have 17k tcp connection in CLOSING state on redis server. My understanding is that only client would initiate closing a connection, not from a redis server. So I should see 0 connection in CLOSING state on redis server. Does anyone have some experience and know the root cause?

The actual connection# in 'redis-cli info' is less than 200 and we have more than enough the memory and cpu idle time is more than 90%.

I went through redis doc on http://redis.io/topics/clients. None of the items there could help me finding the root cause.

redis version: 2.6.7

netstat -an|awk '/tcp/ {print $6}'|sort|uniq -c
  1 CLOSE_WAIT
  16980 CLOSING
  128 ESTABLISHED
  21 LAST_ACK
  12 LISTEN
  1 SYN_RECV

Best Answer

This is caused by the phpredis library we are using to connect to the redis server.

Basically phpredis sends a QUIT command to ask the redis server to close the connection. But right after doing that, phpredis closes the tcp socket itself, causing both sides trying to close the connection. Therefore the server has so many connections stuck at CLOSING state.

I've created a simple fix for that issue. https://github.com/phpredis/phpredis/issues/562

Related Topic