TCP is backed by acks and retries to make sure you data gets where it's going. UDP is connectionless and "fire and forget". UDP is mostly used for streaming type applications, where if you lose some data you don't need to try to send it again.
Which one you use depends on the application. For example, a web server uses TCP.
A quick back-of-my hand calculation gives the following data:
100 mbit bit rate is 100 000 000 bits/second (network, so not 1024)
A full length ethernet frame is 1518 bytes, 12 144 bits.
Transfer of a full packet takes 12144/100000000 seconds, around 0.12 ms.
A minimum length ethernet frame is 64 byes, 512 bits.
Transfer of a minimal packet takes 512/100000000 seconds, around 0.005 ms.
So the whole jitter can be explained with a single packet in queue at one of the switches. In practice you should be more interested in the distribution of jitter than single outliers. To get this data you need to do a lot more measurements. If this is your maximum latency you observe you are doing as well as Ethernet can.
For VoIP a maximum latency under 10 ms will place you easily in MOS 5. People start complaining around 50-100ms. Above 100ms it's significantly degraded. For grading of voice quality have a look at http://en.wikipedia.org/wiki/Mean_opinion_score
Best Answer
Scapy is a very good tool for forging packets in conjunction with wireshark, it may fit your needs.