Zabbix smtp check fails with working smtp server

I think that the bottleneck is the disc. Here are my reasons for this:

You have a pretty busy web server. Zabbix is slow, I suspect to be reads from the disk (can be from the network too).

Run again the strace, and find the file descriptor in Zabbix

Then find if the file descriptor is a file or a socket:

ls -l /prod/<PID_of_straced_process>/fd/<FD_from_strace>

EDIT1:
You should not change the TIME_WAIT timeouts. The problem with small HTTP keep-alive, or with no HTTP keep-alive is that you increase the latency and bandwidth. Instead you should increase a little bit the HTTP keep-alive and install/enable SPDY.

EDIT2: Use dstat -ta 10 and compare the first line with the rest. The first line is the average since boot. Next lines are 10 seconds average (the last parameter).

EDIT3: Check if you do not have packets lost, use something like smokeping to monitor the server and the website from outside your network. You have a significant number of connections in CLOSING, FIN_WAIT1, FIN_WAIT2, SYN_RECV, LAST_ACK. I think your network is congested or you have a lot of short lived connections (confirmed by the high TIME_WAIT/ESTABILISHED ratio). See: http://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation

Basic Zabbix concept - one item (check) = one value. Zabbix server doesn't provide parser functionality to parse multiple values, so it has to be always only one value.

If you want to return more value, than you have to use some workarounds, see zabbix UserParameter return 2 or more values

Zabbix sender example:

allow command execution in the zabbix-agent and create item (active or passive) to run your script:

system.run["myscript.sh > output_for_zabbix_sender.txt; zabbix_sender -s <host_name> -z <zabbix_server> -i output_for_zabbix_sender",nowait]

It will execute command in nowait mode. myscript.sh produces output in format:

hostname key1 value1
hostname key2 value2
...

And this output is processed (sent) to the zabbix server by zabbix_sender. Item type must be Zabbix trapper in this case (zabbix_sender). Eventually you don't need system.run item, you can execute script with zabbix_sender as cron job.

If you need perfect monitoring solution don't forget to handle errors, minimize IOPs, ...

You can use zabbix_sender also for discovering - just follow documentation about required format, e.g.:

hostname discovery_key {"data":[{"{#ID}": "/"},{"{#ID}":"/usr"},{"{#ID}":"/var"}]}