I am new to nginx, and I am using it currently on my server to serve videos to users. a few days ago the server started becoming extremely slow while have around 500-600 users online, note that I have a 1gbps line on the server so its not the bandwidth. it takes the server around a minute or 2 to respond but once the file starts streaming the speed is decent, so its taking the server long to spot the file I guess! I was wondering if anyone knows how I can fix this? I have already tried increasing the open file limit and worker limit, but still not luck!!!
I am also interested to hire someone to fix this for me if they know how to do it exactly and have previous experience with a similar situation.
Regards
ADDITIONAL INFO
total used free shared buffers cached
Mem: 7861 7695 166 0 3 4233
-/+ buffers/cache: 3459 4402
Swap: 4095 44 4051
Best Answer
So, you need to work from the bottom-up to find the bottleneck. I wrote a fairly comprehensive article on finding performance bottlenecks a couple of years ago, which should be helpful.
In short, you need to identify the exact system component that's the source of the problem (using tools like
sar
or it's many equivalents), then fix it. Once you fix that, something else will become the bottleneck, so you repeat the process until performance is acceptable.In your specific case, disk I/O might be the problem, but the complicating factor is the fact that it takes a long time for requests to be serviced but they stream OK. I'd be looking into where the holdup there is -- is the kernel establishing the TCP connection quickly (look for the three-way-handshake with
tcpdump
), is nginx getting anaccept
through quickly (strace
nginx and it's workers), is the HTTP request processing taking a while (nginx
log files at maximum debug, cap'n)? A methodical approach to problem-solving will cure many ills.(As an aside, whilst I don't do consulting, I do work for a hosting company that specialises in full-stack system support, so if you need paid help, it's always there)