Language Design – Why F#, Rust Use Option Type Over Nullable Types

language-designnull

AFAIK, Option type will have runtime overhead, while nullable types won't, because Option time is an enum (consuming memory).

Why not just mark optional references as optional, then the compiler can follow code execution and find whenever it can't more be null?

Edit: I see I was misunderstood. I understand and agree with the advantages of avoiding null pointers. I'm not talking about arbitrary pointers that accept null. I'm only asking why not use compile-time metadata, like C# 8's nullable reference types and TypeScript with strict null checks, where default pointers can't be null and there's a special syntax (mostly ?) to indicate a pointer that can accept null.

Edit 2:

Also, Some is strange, in my opinion. Implicit conversion would be better. But that a language feature and not relevant.

Best Answer

The purpose of Null Tracking in general (of which Nullable Types are only one of many different forms), is to somehow regain a modicum of safety (and sanity) in languages that have null references.

If you have the chance to eliminate null references altogether, that is a much better solution since the problems that null references cause simply will not exist in the first place. Sir Tony Hoare has famously said that he considers inventing the Null Reference his "Billion Dollar Mistake", which is actually a quite conservative estimate on the total costs that null references have caused until today. If even the person who invented them considers them a mistake, why would you willingly put them in a language?

C# has them because, well, they probably didn't know any better, and now they can't get rid of them because of backwards-compatibility. TypeScript has them because its semantics are based on ECMAScript's, which has them.

The real beauty of an Option type, though, is that it is isomorphic to a collection that can only hold from zero to one elements. Dealing with collections is one of the most important parts of programming, and thus every language in the world has powerful collections libraries. And you can apply all of the work that has gone into collections also to Option types.

For example, if you want to execute an action with an option, you don't need to check whether it is defined! Every collection library on the planet has a way of iterating over a collection and executing an action for each element. Now, what does "executing an action for each element" mean for an Option? Well, if there is no element, then no action is executed. And if there is one element, then the action is executed once with that element.

In other words, foreach acts exactly like a NULL check! You can just blindly do

mightExistOrMightNot.foreach(println)

and it will print out the value contained in the Option if it exists and do nothing if it doesn't exist. The same applies when you want to perform a computation with the value. Every collections library on the planet has a way of iteration over a collection and transforming each element. Again, for an Option "transforming each element" translates to "transform the value or do nothing". So you can just do

val squared: Option[Int] = mightExistOrMightNot.map(_ ** 2)

Also, collections libraries have ways to flatten nested collections. Imagine you have a long chain of references, each of which could be NULL, and you wanted to access the last reference in that chain. With nested Options, you just write

longListOfReferences.flatten

And if you want to get a value out of an Option, then you can simply write

mightExistOrMightNot.getOrElse(42)

and you will either get the value inside the option if it exists, or a default value of your choosing if it doesn't.

The only reason, really, for you to explicitly check for the existence of an Option is if you want to do something completely different in case the value is missing.

It turns out that Option is actually even more than "just" a collection. It is a monad. Languages like C#, Scala, and Haskell have built in syntax sugar for working with monads, and they have powerful libraries for working with monads. I will not go into details about what it means to be a monad, but e.g. one of the advantages is that there are some specific mathematical laws and properties associated with monads, and one can exploit those properties.

The fact that Java's Optional is not implemented as a monad, not even as a collection, is a significant design flaw, and I think is partially to blame for people not understanding the advantages of Options, simply because some of those advantages cannot be realized with Java's Optional.

There is also a more philosophical reason for choosing an Option type over NULL references. We can call this "language democracy". There is a major difference between those two: NULL references are a language feature whereas Option is a library type.

Everybody can write a library type, but only the language designer can write a language feature. That means that if for my code, I need to handle the absence of values in a slightly different manner, I can write a MyOption. But I cannot write a MYNULL reference without changing the language semantics and thus the compiler (or, for a language like C, C++, Java, Go, ECMAScript, Python, Ruby, PHP with multiple implementations, every single compiler and interpreter that exists, has existed, and will ever exist).

The more the language designer moves out of the language into libraries, the more the programmers can tailor the language (really, the library) to their needs.

Also, the more the language designer moves out of the language into libraries, the more the compiler writers are forced to make library code fast. If a compiler writer figures out some clever trick to make NULL references fast, that doesn't help our hypothetical programmer who has written their own abstraction. But if a compiler writer figures out some clever trick to make Option fast, it is highly likely the same trick will also apply to MyOption (and Try, Either, Result, and possibly even every collection).

Take Scala, for example. Unfortunately, because it is designed to interoperate and integrate deeply with the host environment (the Java platform, the ECMAScript platform, there is also an abandoned CLI implementation), it has null references and exceptions. But, it also has the Option type which replaces the former and Try which replaces the latter. And Try first appeared in a library of helpers released by Twitter. It was only later added to the standard library. Such innovation is much harder to do with language features.

I can write my own Scala Option type, and I don't need to change the compiler for it:

sealed trait Option[+A] extends IterableOnce[A]:
  override def iterator: Iterator[A]
  override def knownSize: Int

  def isEmpty: Boolean
  def getOrElse[B >: A](default: => B): B
  def foreach[U](f: A => U): Unit
  def map[B](f: A => B): Option[B]
  // … and so on

final case class Some[+A](value: A) extends Option[A]:
  override def iterator = collection.Iterator.single(value)
  override val isEmpty = false
  
  override val knownSize = 1
  override def getOrElse[B >: A](default: => B) = value
  override def foreach[U](f: A => U) = f(value)
  override def map[B](f: A => B) = Some(f(value))
  // … and so on

case object None extends Option[Nothing]:
  override def iterator = collection.Iterator.empty
  override val isEmpty = true

  override val knownSize = 0
  override def getOrElse[B](default: => B) = default
  override def foreach[U](f: Nothing => U) = ()
  override def map[B](f: Nothing => B) = None
  // … and so on

@main def test = Some(23).foreach(println)

Try it out here.

Related Solutions

Java Scala – Alternatives to Null Values and Option-Like Types

What is a type?

A type is meta data that can be used, at compile time or run time, to describe the shape of our data.

trait A { val b: X; def c(y: Y): Z}

So we know in our program that when ever we see an a: A, we know that a has the shape of A so we can call a.b safely.

So what shape does null have?

null has no data. null does not have a shape. String s = null is allowed as null has no shape, so it can fit any where. MikeFHay above says that an empty collection can be thought of as a null object. This is wrong, an empty collection has exactly the same shape as any other collection.

If a field can be null, then we can no longer rely on a.b to be safe. The shape of the data in this field is not truly A, only potentially A. The other shape it can be is null. That is the shape of this field is a union of A and null.

Most languages allow unions to be implemented as a tagged union. Scala's Option type is a tagged union of None | Some(a:A). Like wise Ceylon adds a ? postfix for types that say that the type may be null as in String? s = null.

Java – Why do “checked exceptions”, i.e., “value-or-error return values”, work well in Rust and Go but not in Java

From a scientific point of view, checked exceptions can be seen as alternative return values, e.g.

Exactly. They can be seen that way, and they should be but they aren't.

Using an Error type like is common in Rust, Elm, Haskell, and in some sub-communities in Scala or a special error value as in Go is just an alternative return value indicated in the type system. A checked exception is like an alternative return value, but it doesn't use the normal way of returning values, it is a completely separate, very different way of "returning values". It also sits outside of the type system, and bolts on a completely separate "checked exception" system onto the type system.

But most importantly, it is not just an alternative return value, it is also an alternative control flow.

Another problem with the specific way checked exceptions are implemented in Java, is that they are anti-modular. That is, however, not a fundamental problem of checked exceptions, unlike the ones I mentioned above. There is an idea of Modular Anchored Exceptions, for example, where you can specify something like

int foo() throws like bar { return bar(); }

And you don't have to know (and leak!) which precise exceptions bar can throw. You can even do something like throws like bar except ArrayOutOfBoundsException when you are handling some errors yourself.

Best Answer

Related Solutions

Java Scala – Alternatives to Null Values and Option-Like Types

Java – Why do “checked exceptions”, i.e., “value-or-error return values”, work well in Rust and Go but not in Java

Related Topic