NGINX Serving Large mp4 Files extremely inefficiently

hard drivenginxperformancestreaming

I am currently running nginx/1.0.15 on a Centos 6.6 OS. The server has the following specs:

  • Intel(R) Atom(TM) CPU C2750 @ 2.40GHz (8 cores)
  • 32GB Ram
  • 5 x 6000 GB 7200 RPM (Raid 10)

The Problem

The server has a 1Gbit/s connection, however it tops out and bottlenecks after 400-500 mbit/s. Service starts to decline at roughly 100 connections.. and the speed with the server drops dramatically (despite having 50% bandwidth still available)

The NGINX server is strictly for serving static .mp4 files. Each file is typically 400-1200MB (700MB being the average)

I have tried many many configurations and just about all of them give me the same results.. I am extremely frustrated..

Server load also never passes 0.3.

Is there anything blatantly wrong or misguided in my configuration? Anything might help.

The Configurations

/etc/nginx/nginx.conf

user              nginx;
worker_processes  9;

error_log  /var/log/nginx/error.log;


pid        /var/run/nginx.pid;


events {
    worker_connections  51200;
    use epoll;
 }

worker_rlimit_nofile 600000;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

#access_log  /var/log/nginx/access.log  main;
access_log off;

aio on;
sendfile        off;
tcp_nopush      off;
tcp_nodelay      on;

#keepalive_timeout  0;
keepalive_timeout  65;

output_buffers 1 3m;
#gzip  on;

include /etc/nginx/conf.d/*.conf;

open_file_cache          max=10000 inactive=5m;
open_file_cache_valid    2m;
open_file_cache_min_uses 1;
open_file_cache_errors   on;

}

/etc/nginx/conf.d/default.conf

server {
    listen       80 default_server sndbuf=32k;
    server_name  _;

    #charset koi8-r;

    #access_log  logs/host.access.log  main;

    include /etc/nginx/default.d/*.conf;


    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    location /Videos/ {
        root /home;
        gzip off;
        gzip_static off;

        mp4;
        mp4_max_buffer_size   300m;
    }

    location /stats {
        stub_status on;
    }

    error_page  404              /404.html;
    location = /404.html {
        root   /usr/share/nginx/html;
    }


    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

Best Answer

The better start can be the set of following rules:

  1. disable logging and accept_mutex
  2. enable sendfile
  3. set sendfile_max_chunk

Configuration:

events {
    accept_mutex off;
}

access_log off;
sendfile on;
sendfile_max_chunk 512k;

New Nginx (1.7.11 or newer) feature thread pool can be really helpful in your case:

location / {
    root /home;
    aio threads;
    mp4;
}

On test samples it dramatically helps you to increase bandwidth from 1Gbps up to 9Gbps. Nine times! You have only 1Gbps but it makes it all utilised.

See more details: https://www.nginx.com/blog/thread-pools-boost-performance-9x/