Scala/Spark version compatibility

tl;dr

• class C defines a class, just as in Java or C++.
• object O creates a singleton object O as instance of some anonymous class; it can be used to hold static members that are not associated with instances of some class.
• object O extends T makes the object O an instance of trait T; you can then pass O anywhere, a T is expected.
• if there is a class C, then object C is the companion object of class C; note that the companion object is not automatically an instance of C.

Also see Scala documentation for object and class.

object as host of static members

Most often, you need an object to hold methods and values/variables that shall be available without having to first instantiate an instance of some class. This use is closely related to static members in Java.

object A {
  def twice(i: Int): Int = 2*i
}

You can then call above method using A.twice(2).

If twice were a member of some class A, then you would need to make an instance first:

class A() {
  def twice(i: Int): Int = 2 * i
}

val a = new A()
a.twice(2)

You can see how redundant this is, as twice does not require any instance-specific data.

object as a special named instance

You can also use the object itself as some special instance of a class or trait. When you do this, your object needs to extend some trait in order to become an instance of a subclass of it.

Consider the following code:

object A extends B with C {
  ...
}

This declaration first declares an anonymous (inaccessible) class that extends both B and C, and instantiates a single instance of this class named A.

This means A can be passed to functions expecting objects of type B or C, or B with C.

Additional Features of object

There also exist some special features of objects in Scala. I recommend to read the official documentation.

• def apply(...) enables the usual method name-less syntax of A(...)
• def unapply(...) allows to create custom pattern matching extractors
• if accompanying a class of the same name, the object assumes a special role when resolving implicit parameters

The ones I can think of are

Existential types

def foo(l: List[Option[_]]) = ...

Higher kinded type parameters

case class A[K[_],T](a: K[T])

Ignored variables

val _ = 5

Ignored parameters

List(1, 2, 3) foreach { _ => println("Hi") }

Ignored names of self types

trait MySeq { _: Seq[_] => }

Wildcard patterns

Some(5) match { case Some(_) => println("Yes") }

Wildcard patterns in interpolations

"abc" match { case s"a$_c" => }

Sequence wildcard in patterns

C(1, 2, 3) match { case C(vs @ _*) => vs.foreach(f(_)) }

Wildcard imports

import java.util._

Hiding imports

import java.util.{ArrayList => _, _}

Joining letters to operators

def bang_!(x: Int) = 5

Assignment operators

def foo_=(x: Int) { ... }

Placeholder syntax

List(1, 2, 3) map (_ + 2)

Method values

List(1, 2, 3) foreach println _

Converting call-by-name parameters to functions

def toFunction(callByName: => Int): () => Int = callByName _

Default initializer

var x: String = _   // unloved syntax may be eliminated

There may be others I have forgotten!

Example showing why foo(_) and foo _ are different:

This example comes from 0__:

trait PlaceholderExample {
  def process[A](f: A => Unit)

  val set: Set[_ => Unit]

  set.foreach(process _) // Error 
  set.foreach(process(_)) // No Error
}

In the first case, process _ represents a method; Scala takes the polymorphic method and attempts to make it monomorphic by filling in the type parameter, but realizes that there is no type that can be filled in for A that will give the type (_ => Unit) => ? (Existential _ is not a type).

In the second case, process(_) is a lambda; when writing a lambda with no explicit argument type, Scala infers the type from the argument that foreach expects, and _ => Unit is a type (whereas just plain _ isn't), so it can be substituted and inferred.

This may well be the trickiest gotcha in Scala I have ever encountered.

Note that this example compiles in 2.13. Ignore it like it was assigned to underscore.