Nginx – Slow connection times from remote server doing apache bench

I am trying to measure perfornace on nginx with ab. Doing the same ab from the local machine gives me about 10 times the requests per second as from a remote machine that is located on the same network.

Local results:
Server Software:        nginx
Server Hostname:        192.168.0.116
Server Port:            8000

Document Path:          /data/adpix/picture/test.jpg
Document Length:        4602 bytes

Concurrency Level:      1000
Time taken for tests:   1.475 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      49854651 bytes
HTML transferred:       46622862 bytes
Requests per second:    6778.38 [#/sec] (mean)
Time per request:       147.528 [ms] (mean)
Time per request:       0.148 [ms] (mean, across all concurrent requests)
Transfer rate:          33001.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       25   67  24.4     66     137
Processing:    28   73  24.9     71     146
Waiting:       13   42  19.7     40     110
Total:         75  140  17.9    139     182

And now from a remote machine:

Server Software:        nginx
Server Hostname:        192.168.0.116
Server Port:            80

Document Path:          /data/adpix/picture/test.jpg
Document Length:        4602 bytes

Concurrency Level:      1000
Time taken for tests:   14.422 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      51737663 bytes
HTML transferred:       48380826 bytes
Requests per second:    693.40 [#/sec] (mean)
Time per request:       1442.165 [ms] (mean)
Time per request:       1.442 [ms] (mean, across all concurrent requests)
Transfer rate:          3503.42 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       68  230 182.7    166    3052
Processing:   110 1113 2918.5    227   11862
Waiting:       20  648 2101.1    189   11451
Total:        246 1344 2898.0    387   11981

The problem seems to be within the connection times.

Network ist fast. Netperf returns 80 Mb/s
Both machines are VM hosted on 2 different physical machines running quad cores. connected via gigabit switch.

Has anybody an idea what could be the problem?

Thank you for any help.