Programming Languages – Correlation Between Project Scale and Language Strictness

dynamic-typinglanguage-choiceprogramming-languagesstatic analysisstatic-typing

Explaining the difference between strictness of languages and paradigms to a colleague of mine, I ended up asserting that:

  • Tolerant languages, such as dynamic and interpreted languages, are used best for prototypes and small projects or medium-size web applications. When choosing elegant dynamic languages such as Python or JavaScript with Node.js, the benefits are:

    1. Fast development,

    2. Reduced boilerplate code,

    3. Ability to attract young, creative programmers who flee “corporate languages”  like Java.

  • Statically typed/compiled languages are best for applications which require higher strictness such as business-critical apps or apps for medium to large-size apps.

    1. Well-known paradigms and patterns developed for decades,

    2. Ease of static checking,

    3. Ability to find many professional developers with decades of experience.

  • Strict languages such as Haskell, Ada or techniques such as Code contracts in C# are better for systems which favor safety over flexibility (even if Haskell can be extremely flexible), such as life critical systems and systems which are expected to be extremely stable. The benefits are:

    1. Ability to catch as many bugs as possible at compile time,

    2. Ease of static checking,

    3. Ease of formal proofs.

However, by looking at the languages and technologies used for large-scale projects by large corporations, it seems that my assertion is wrong. For example, Python is successfully used for large systems such as YouTube or other Google applications which require an important amount of strictness.

Is there still a correlation between the scale of the project and the strictness of the language/paradigm which should be used?

Is there a third factor that I've forgotten to take in account?

Where am I wrong?

Best Answer

An interesting case study on the matters of scaling projects that use dynamic and interpreted language can be found in Beginning Scala by David Pollak.

I started searching for a way to express the code in my brain in a simpler, more direct way. I found Ruby and Rails. I felt liberated. Ruby allowed me to express concepts in far fewer lines of code. Rails was so much easier to use than Spring MVC, Hibernate, and the other “streamlined” Java web frameworks. With Ruby and Rails, I got to express a lot more of what was in my head in a shorter period of time. It was similar to the liberation I felt when I moved from C++ to Java...

As my Ruby and Rails projects grew beyond a few thousand lines of code and as I added team members to my projects, the challenges of dynamic languages became apparent.

We were spending more than half our coding time writing tests, and much of the productivity gains we saw were lost in test writing. Most of the tests would have been unnecessary in Java because most of them were geared toward making sure that we’d updated the callers when we refactored code by changing method names or parameter counts. Also, I found that working on teams where there were mind melds between two to four team members, things went well in Ruby, but as we tried to bring new members onto the team, the mental connections were hard to transmit to new team members.

I went looking for a new language and development environment. I was looking for a language that was as expressive as Ruby but as safe and high-performance as Java...

As you can see, major challenges in project scaling for author turned out to be in test development and knowledge transfer.

In particular, author goes into more details in explaining the differences in test writing between dynamically and statically typed languages in Chapter 7. In section "Poignantly Killing Bunnies: Dwemthy’s Stairs" author discusses Scala port of a particular Ruby example:

Why the Lucky Stiff... introduces some of Ruby’s metaprogramming concepts in Dwemthy’s Array in which a rabbit battles an array of creatures. N8han14 updated the example to work in Scala...

Compared to the Ruby code, the library parts of the Scala code were more complex. We had to do a lot of work to make sure our types were correct. We had to manually rewrite Creature’s properties in the DupMonster and the CreatureCons classes. This is more work than method_missing. We also had to do a fair amount of work to support immutability in our Creatures and Weapons.

On the other hand, the result was much more powerful than the Ruby version. If we had to write tests for our Ruby code to test what the Scala compiler assures us of, we’d need a lot more lines of code. For example, we can be sure that our Rabbit could not wield an Axe. To get this assurance in Ruby, we’d have to write a test that makes sure that invoking |^ on a Rabbit fails. Our Scala version ensures that only the Weapons defined for a given Creature can be used by that Creature, something that would require a lot of runtime reflection in Ruby...

Reading above can make one think that as projects grow even larger, test writing might become prohibitively cumbersome. This reasoning would be wrong, as evidenced by examples of successful very large projects mentioned in this very question ("Python is successfully used for... YouTube").

Thing is, scaling of the projects isn't really straightforward. Very large, long-living projects can "afford" different test development process, with production quality test suites, professional test dev teams and other heavyweight stuff.

Youtube test suites or Java Compatibility Kit sure live a different life than tests in a small tutorial project like Dwemthy’s Array.

Related Topic