Sorry if this is a newb question…
I've heard stories of Netflix and Twitter being able to duplicate web traffic amongst two separate infrastructures: one is the authoritative/trusted one that goes back to the user; and the other is a 'shadow' or test infrastructure that thinks it is returning to the user but doesn't. The point is to test the secondary infrastructure at real-life load and timing.
I'm pretty sure there's a word to describe this, but 'bridge' doesn't seem to be the right one, nor does 'replay'.
Can anyone help me with what this technique is called and/or what tools can be used to accomplish this?
I guess that I should add that I've heard about techniques that are effectively 'replaying logs', but that's really difficult to get at real speeds/distributions.
And, we're not trying to verify 'correctness' of the output, but just make sure that we don't see errors/stacktraces/etc in the new infrastructure.
Best Answer
I'd call it "load testing via session replaying", personally. I don't know of any simple catch-all term for this kind of testing technique.
The basic strategy that I've seen employed for this kind of load testing is to ingest log files from the production system and replay them on a test system.
You can use tools like JMeter or Apache Bench to replay requests from log files. If you're looking at replaying very complex client / server interactions (with specific timing details based on the original log stream) in hopes of really exercising the innards of your application (looking for race conditions, timing-related bugs, etc) you might look at writing application-specific testing tools that simulate clients at scale.
You're not going to be able to simply capture boatloads of raw network traffic and "replay" it with any TCP or IP-based protocol. TCP sequence numbers aren't going to match the original captured traffic and it's not going to work. IP-layer captures are going to be problematic because your simulated clients will need to answer for the captured sender's IP address. You'd be better off capturing traffic closer to layer 7 and using that to replay sessions because, otherwise, you're looking at writing a TCP simulator, too. (I could imagine using something like
tsharkto bust out the layer 7 data and timing from a TCP stream and replaying that, for example.)Simply replaying network traffic simulates load but doesn't necessarily capture defects. Your simulated client would need to receive responses from the test server and parse them for correctness if you wanted load-test any test that the application is responding properly. Since your application is going to generate dynamic response data it's unlikely that your simulated client can simply compare the test server's response to the logged response from the production server. This is where you're going to get into writing a test harness specific to your application and its output.