Unit Testing – Why Writing Tests in Different Languages is Uncommon

interoperabilitylanguage-agnosticlanguage-choiceunit testing

When Microsoft released Visual Studio 2008, there was a thing they were talking a lot about at the conferences and in their online tutorials: the idea of writing the actual code in one language, and the unit tests in a different language. For instance, unit tests for C# code can be written in Visual Basic.

Six years later, I don't know any open source project which uses this practice and no proprietary project I've seen uses it. The subject is not discussed any longer at Microsoft's conferences, blogs or tutorials.

The benefit of this practice is that doing the same type of error in two different languages is more difficult. For instance, if I don't know that in C#, 10/3 equals 3 and expect it to be 3.333..., a unit test written in Python or JavaScript will fail and lead me to the discovery in integer division.

.NET Framework, specifically, makes it possible to use any .NET-enabled language to test another one. While testing C# code with Visual Basic tests doesn't look particularly interesting, given the similarity of both languages, I would imagine that, for example, testing F# code with unit tests written in C# could be useful for people who are not that familiar with functional programming.

Personally, I have no experience writing unit tests in a different language. Therefore, I don't know if there are actual drawbacks in this technique. I often write system and acceptance tests in a different language (for instance in Python for a Node.js web app, because Python's Requests library is so great), but this doesn't count, since those tests are truly language-agnostic: they just do black box testing through HTTP: I can rewrite an app from Node.js to ASP.NET MVC, and the tests won't even notice the change.

So what could be the drawbacks, and why is this practice not that popular?

Best Answer

Unit tests are white-box tests and usually written by the developers of the thing being tested. The first means that you need first-class interaction with the APIs and objects and values of the language the actual product is written in, which rules out almost all combinations of languages. In fact, I think the CLR is the only agglomeration of languages that each can call into each other perfectly. And as you say, most CLR languages are so similar that none of the advantages you suggest apply.

That leaves pretty much only the combination of F# and C#/VB.NET. I can imagine several reasons this hasn't taken off:

  • F# doesn't have that big of a community. Conversely, most C# developers don't know enough F# to even consider this.
  • The people who do know F# are either busy writing both unit tests and actual code in F#, or in a team where they can't use F# at all because nobody else knows it (well). As mentioned before, unit tests are usually not written by a separate team, so people who write C# unit tests for F# code probably still have to know F#. Plus the whole white-box thing: If you don't know a thing about functional programming, even the data structures and APIs exposed by (idiomatic) F# will be unnatural for you once you delve deeper than very simple libraries of the form "put in a URL here, receive IEnumerable<UrlParameter>.
  • There's no benefit (or people don't believe the benefit), but quite possibly some very real friction from switching languages constantly.
Related Topic