Just an FYI: Unit testing is not equivalent to TDD. TDD is a process of which unit testing is an element.
With that said, if you were looking to implement unit testing then there's a number of things you could do:
All new code/enhancements are tested
This way you don't have to go through and unit test everything that already exists, so the initial hump of implementing unit testing is much smaller.
Test individual pieces of data
Testing something that can contain large amounts of data can lead to many edge cases and gaps in the test coverage. Instead, consider the 0, 1, many option. Test a 'batch' with 0 elements, 1 element and many elements. In the case of 1 element, test the various permutations that the data for that element can be in.
From there, test the edge cases (upper bounds to the size of individual elements, and quantity of elements in the batch). If you run the tests regularly, and you have long running tests (large batches?), most test runners allow categorization so that you can run those test cases separately (nightly?).
That should give you a strong base.
Using actual data
Feeding in 'actual' previously used data like you're doing now isn't a bad idea. Just complement it with well formed test data so that you immediately know specific points of failure. On a failure to handle actual data, you can inspect the results of the batch process, produce a unit test to replicate the error, and then you're back into red/green/refactor with useful regression cases.
"If it's not in source control, it doesn't exist."
This is one of the few things in our profession that I'm borderline dogmatic about. For the following reasons:
- Even though the workstation is company property, let's face it - there is a bit of an unwritten rule that a programmer's own workstation is his/her castle. I'm just uneasy with a workplace culture where anyone can routinely log onto it and go through it.
- Everybody has their own flow (as you said as well). Trying to force all developers to organise their own local workspaces a certain way may go against their particular way of working, and break their flow and make them less efficient.
- Stuff that isn't in source control is half-baked code. If it's fully baked code that's ready for release, it should be in source control. Which comes back again to the main point....
- "If it's not in source control, it doesn't exist."
One possible way to mitigate the issue of wanting to look at code on people's workstations is to foster a culture of regular checkins. I worked at a company once where - even though there was no official mandate to do so - it was seen a sort of point of pride to always have everything checked in for the weekend. In maintenance and release candidate phases, CR items were deliberately very fine grained to allow for small, cleanly visible changes, and regular checkins to keep track of them.
Also, having everything checked in before you go on vacation was mandatory.
TL;DR version: Rifling through people's workstations is bad form. Rather than trying to foster a culture of making it easy to go through people's workstations to find what we want, it's better practice to foster a culture of sensible source control use and regular checkins. Possibly even hyper-regular checkins, and fine-grained tasks when in critical phases of projects.
Best Answer
As others (and yourself) have noted, developers should unit test their own code. However, after that, any nontrivial product should also be tested by independent person(s) (QA department and/or the client herself).
Developers normally work with the developer mindset of "how to make this work?". A good tester is thinking about "how to break this?" - a very different mindset. Unit testing and TDD does teach developers to change hats to some extent, but you shouldn't rely on it. Moreover, as others have noted, there is always a possibility of misunderstanding requirements. Therefore final acceptance tests should be conducted by someone as close to the client as possible.