If your backend is using a virtual host and requires the Host header to contain the actual hostname of the site, you will need to add this to your load balancer location:
proxy_set_header Host $host;
This will forward whatever Host: header the client sent to the load balancer on to the back-end. This exact scenario is documented on the nginx wiki.
Not sure which way most of your traffic flows, if it's client -> server, or server -> client, but if it's the later, you might be interested in using (or checking out, at least) the Linux Virtual Server (LVS) which is
an advanced load balancing solution that can be used to build highly scalable and highly available network services, such as scalable web, cache, mail, ftp, media and VoIP services.
(Excerpt from the website).
To get the traffic of your backend servers directly, without "something in between" to the / your clients, use Direct Routing. You'll find more information in this link, but, to get an idea how this does look like:
Now you could say...well, this sounds nice and fancy in theory, but come on, the website reads "Latest Press News... Wednesday, August 8, 2012"...we're in 2016 now. Yes, you would be completely right then, besides that LVS is proven and rock solid...did I recommend to have a look at least?
The Wikimedia Foundation does use it, as many others do, and here is one last image showing the setup in 2010.
Best Answer
Instead of LVS you could try HAproxy. In http mode it recognizes headers like "Host" which handles domain name. This way you could proxy static files directly to "internal image server"
Another way is to request another external IP and run two copy of LVS with different config - one for dynamic, one for static files, which binds on different external IP.