Testing Workflows – How to Test Time Sensitive Software

testingworkflows

By time sensitive I mean for example a script that only runs once a month or a script that runs continuously but gives a certain output only once a month. Obviously you can unit test for a lot of cases but there are exceptions ( in my understanding).

A recent example I ran into was setting up a cron job to run on the second to last day of every month. This required using a shell script with cron tab to get the correct day of the month for cron, something like:

1 0 [shell command] * * [my script]

I was unfamiliar with the script and unfamiliar with shell scripts in general and so had no good way to test it other than waiting for the end of the month to come and seeing if the script executed correctly (actually my solution was to find a co-worker who knew a lot more about cron and shell scripting that I did).

So I am curious if there are any useful work arounds for testing time sensitive scripts.

Best Answer

In addition to unit testing, there are two other strategies for setting up automated tests to deal with an OS-specific issue:

  • Virtualization: You set up several OS images (for example, using VMWare) with the exact configurations you require, set up a way to automatically pull the binary to test (usually by mounting a special directory into the VM's space), and then execute the test.

Or:

  • Instrumentation: Manually add special if conditions to your program that will make the program behave differently. Under Unix, this would be done by checking if a certain environment variable is set, like FOOBAR_TEST_TIME_WITH_T=500. Your automated tests will then just use different settings of the environment variables, and different environment variables, to execute what you need.

You can also link to different libraries, if your interactions can be expressed at the library level, which you can think of as a virtualization (if the "library" is the OS kernel) or as an instrumentation technique. Both terms may be used, although the term virtualization as-used today almost always means something like VMWare. A library specifically for returning canned values or re-running specific interactions would be a mock or stub approach.

There are also automatic instrumentation tools, which can rewrite your binaries to get other desired effects, like the file system being full.

Overall, your goal is to find bugs. To check for weird cases like the file system being overfull, it's easiest and still effective to just manually instrument your program, going the virtualization or manual machine configuration route rarely if ever.

Related Topic