.NET Reflection – Is Using Reflection Still ‘Bad’ or ‘Slow’?

What you are working on is basically ETL. So at a high level you need an export component (get stuff) a transform component (map to known format) and a load (take known format and put stuff somewhere). If you are comfortable being tied to a RDBMS you could use something like SQL Server SSIS packages. What I would do is create a host application that managed common aspects of the overall process (errors, and pipeline processing). Then make the specifics of the E, T, and L pluggable. A low ceremony way to get this would be to host the powershell runtime and create each seesion with common context objects that the scripts will use to communicate. You get a built in pipe and filter model for scripts and easy, safe extensibility. This design has worked extremely for my team with a similar situation.

This depends on what the limiting factor is, doesnt it? If the bottleneck is the utility program, then sure, running more than one copy or using more threads will speed things up. If the network is the limiting factor, then adding multiple instances of the utility isnt going to help since you still will be stuck moving at most X bytes per second. In fact it might hurt because you have the additional overhead of a second copy of the app. Same with disk-IO. You can only copy as fast as either machine can read from and write to disk. If thats already max'd out, adding copies isnt going to help.

What you need to do is test to see what the bottleneck is, and go from there.