Java TDD – How to Test State of the Object

I go by Kent Beck's rule of thumb:

Test everything that could possibly break.

Of course, that is subjective to some extent. To me, trivial getters/setters and one-liners like yours above usually aren't worth it. But then again, I spend most of my time writing unit tests for legacy code, only dreaming about a nice greenfield TDD project... On such projects, the rules are different. With legacy code, the main aim is to cover as much ground with as little effort as possible, so unit tests tend to be higher level and more complex, more like integration tests if one is pedantic about terminology. And when you are struggling to get overall code coverage up from 0%, or just managed to bump it over 25%, unit testing getters and setters is the least of your worries.

OTOH in a greenfield TDD project, it may be more matter-of-fact to write tests even for such methods. Especially as you have already written the test before you get the chance of starting to wonder "is this one line worth a dedicated test?". And at least these tests are trivial to write and fast to run, so it's not a big deal either way.

Units

I think I can pinpoint exactly where the problem started:

I figured, I'll need a method that finds all the non-numerical fields in a line.

This should be immediately followed with asking yourself "Will that be a separate testable unit to gatherNonNumericColumns or part of the same one?"

If the answer is "yes, separate", then your course of action is simple: that method needs to be public on an appropriate class, so it can be tested as a unit. Your mentality is something like "I need to test drive out one method and I also need to test drive out another method"

From what you say though, you figured that the answer is "no, part of the same". At this point, your plan should no longer be to fully write and test findNonNumericFields then write gatherNonNumericColumns. Instead, it should be simply to write gatherNonNumericColumns. For now, findNonNumericFields should just be a likely part of the destination you have in mind when you're choosing your next red test case and doing your refactoring. This time your mentality is "I need to test drive out one method, and while I do so I should keep in mind that my finished implementation will probably include this other method".

Keeping a short cycle

Doing the above should not lead to the problems you describe in your penultimate paragraph:

Because if I start out by writing a test for a public method, there will be several minutes (or hours, or even days in very complex cases) before I get all the details in the private methods to work so that the test testing the public method passes.

At no point does this technique require you to write a red test which will only turn green when you implement the entirety of findNonNumericFields from scratch. Much more likely, findNonNumericFields will start as some code in-line in the public method you're testing, which will be built up over the course of several cycles and eventually extracted during a refactoring.

Roadmap

To give an approximate roadmap for this particular example, I don't know the exact test cases you used, but say you were writing gatherNonNumericColumns as your public method. Then most likely the test cases would be the same as the ones you wrote for findNonNumericFields, each one using a table with only one row. When that one-row scenario was fully implemented, and you wanted to write a test to force you to extract out the method, you'd write a two-row case which would require you to add your iteration.