Effective Scala

Introduction

Formatting

The specifics of code formatting — so long as they are practical — are of little consequence. By definition style cannot be inherently good or bad and almost everybody differs in personal preference. However the consistent application of the same formatting rules will almost always enhance readability. A reader already familiar with a particular style does not have to grasp yet another set of local conventions, or decipher yet another corner of the language grammar.

This is of particular importance to Scala, as its grammar has a high degree of overlap. One telling example is method invocation: Methods can be invoked with “.”, with whitespace, without parenthesis for nullary or unary methods, with parenthesis for these, and so on. Furthermore, the different styles of method invocations expose different ambiguities in its grammar! Surely the consistent application of a carefully chosen set of formatting rules will resolve a great deal of ambiguity for both man and machine.

Whitespace

Indent by two spaces. Try to avoid lines greater than 100 columns in length. Use one blank line between method, class, and object definitions.

Naming

Use short names for small scopes

is, js and ks are all but expected in loops.

Use longer names for larger scopes

External APIs should have longer and explanatory names that confer meaning. Future.collect not Future.all.

Use common abbreviations but eschew esoteric ones

Everyone knows ok, err or defn whereas sfri is not so common.

Don't rebind names for different uses

Use vals

Avoid using `s to overload reserved names.

typ instead of `type`

Use active names for operations with side effects

user.activate() not user.setActive()

Use descriptive names for methods that return values

src.isDefined not src.defined

Don't prefix getters with get

As per the previous rule, it's redundant: site.count not site.getCount

Don't repeat names that are already encapsulated in package or object name

Prefer:

object User {
  def get(id: Int): Option[User]
}

to

object User {
  def getUser(id: Int): Option[User]
}

They are redundant in use: User.getUser provides no more information than User.get.

Imports

Sort import lines alphabetically

This makes it easy to examine visually, and is simple to automate.

Use braces when importing several names from a package

import com.twitter.concurrent.{Broker, Offer}

Use wildcards when more than six names are imported

e.g.: import com.twitter.concurrent._
Don't apply this blindly: some packages export too many names

When using collections, qualify names by importing scala.collection.immutable and/or scala.collection.mutable

Mutable and immutable collections have dual names. Qualifiying the names makes is obvious to the reader which variant is being used (e.g. "immutable.Map")

Do not use relative imports from other packages

Avoid

import com.twitter
import concurrent

in favor of the unambiguous

import com.twitter.concurrent

Put imports at the top of the file

The reader can refer to all imports in one place.

Braces

Braces are used to create compound expressions (they serve other uses in the “module language”), where the value of the compound expression is the last expression in the list. Avoid using braces for simple expressions; write

def square(x: Int) = x*x

but not

def square(x: Int) = {
  x * x
}

even though it may be tempting to distinguish the method body syntactically. The first alternative has less clutter and is easier to read. Avoid syntactical ceremony unless it clarifies.

Pattern matching

Use pattern matching directly in function definitions whenever applicable; instead of

list map { item =>
  item match {
    case Some(x) => x
    case None => default
  }
}

collapse the match

list map {
  case Some(x) => x
  case None => default
}

it's clear that the list items are being mapped over — the extra indirection does not elucidate.

Comments

Types and Generics

The primary objective of a type system is to detect programming errors. The type system effectively provides a limited form of static verification, allowing us to express certain kinds of invariants about our code that the compiler can verify. Type systems provide other benefits too of course, but error checking is its Raison d’Être.

Our use of the type system should reflect this goal, but we must remain mindful of the reader: judicious use of types can serve to enhance clarity, being unduly clever only obfuscates.

Return type annotations

While Scala allows these to be omitted, such annotations provide good documentation: this is especially important for public methods. Where a method is not exposed and its return type obvious, omit them.

This is especially important when instantiating objects with mixins as the scala compiler creates singleton types for these. For example, make in:

trait Service
def make() = new Service {
  def getId = 123
}

does not have a return type of Service; the compiler creates the refinement type Object with Service{def getId: Int}. Instead use an explicit annotation:

def make(): Service = new Service{}

Now the author is free to mix in more traits without changing the public type of make, making it easier to manage backwards compatibility.

Variance

Variance arises when generics are combined with subtyping. They define how subtyping of the contained type relate to subtyping of the container type. Because Scala has declaration site variance annotations, authors of common libraries — especially collections — must be prolific annotators. Such annotations are important for the usability of shared code, but misapplication can be dangerous.

Invariants are an advanced but necessary aspect of Scala’s typesystem, and should be used widely (and correctly) as it aids the application of subtyping.

Immutable collections should be covariant. Methods that receive the contained type should “downgrade” the collection appropriately:

trait Collection[+T] {
  def add[U >: T](other: U): Collection[U]
}

Mutable collections should be invariant. Covariance is typically invalid with mutable collections. Consider

trait HashSet[+T] {
  def add[U >: T](item: U)
}

and the following type hierarchy:

trait Mammal
trait Dog extends Mammal
trait Cat extends Mammal

If I now have a hash set of dogs

val dogs: HashSet[Dog]

treat it as a set of Mammals and add a cat.

val mammals: HashSet[Mammal] = dogs
mammals.add(new Cat{})

This is no longer a HashSet of dogs!

Type aliases

Use type aliases when they provide convenient naming or clarify purpose, but do not alias types that are self-explanatory.

() => Int

is clearer than

type IntMaker = () => Int
IntMaker

since it is both short and uses a common type. However

class ConcurrentPool[K, V] {
  type Queue = ConcurrentLinkedQueue[V]
  type Map   = ConcurrentHashMap[K, Queue]
  ...
}

is helpful since it communicates purpose and enhances brevity.

Don’t use subclassing when an alias will do.

trait SocketFactory extends (SocketAddress => Socket)

a SocketFactory is a function that produces a Socket. Using a type alias

type SocketFactory = SocketAddress => Socket

is better. We may now provide function literals for values of type SocketFactory and also use function composition:

val addrToInet: SocketAddress => Long
val inetToSocket: Long => Socket

val factory: SocketFactory = addrToInet andThen inetToSocket

Type aliases are bound to toplevel names by using package objects:

package com.twitter
package object net {
  type SocketFactory = (SocketAddress) => Socket
}

Note that type aliases are not new types — they are equivalent to the syntactically substituting the aliased name for its type.

Implicits

Implicits are a powerful type system feature, but they should be used sparingly. They have complicated resolution rules and make it difficult — by simple lexical examination — to grasp what is actually happening. It’s definitely OK to use implicits in the following situations:

• Extending or adding a Scala-style collection
• Adapting or extending an object (“pimp my library” pattern)
• Use to enhance type safety by providing constraint evidence
• To provide type evidence (typeclassing)
• For Manifests

If you do find yourself using implicits, always ask yourself if there is a way to achieve the same thing without their help.

Do not use implicits to do automatic conversions between similar datatypes (for example, converting a list to a stream); these are better done explicitly because the types have different semantics, and the reader should beware of these implications.

Collections

Hierarchy

The collections library is large: in addition to an elaborate hierarchy — the root of which being Traversable[T] — there are immutable and mutable variants for most collections. Whatever the complexity, the following diagram contains the important distinctions for both immutable and mutable hierarchies

Iterable[T] is any collection that may be iterated over, they provide an iterator method (and thus foreach). Seq[T]s are collections that are ordered, Set[T]s are mathematical sets (unordered collections of unique items), and Map[T]s are associative arrays, also unordered.

Use

Prefer using immutable collections. They are applicable in most circumstances, and make programs easier to reason about since they are referentially transparent and are thus also threadsafe by default.

Use the mutable namespace explicitly. Don’t import scala.collection.mutable._ and refer to Set, instead

import scala.collection.mutable
val set = mutable.Set()

makes it clear that the mutable variant is being used.

Use the default constructor for the collection type. Whenever you need an ordered sequence (and not necessarily linked list semantics), use the Seq() constructor, and so on:

val seq = Seq(1, 2, 3)
val set = Set(1, 2, 3)
val map = Map(1 -> "one", 2 -> "two", 3 -> "three")

This style separates the semantics of the collection from its implementation, letting the collections library use the most appropriate type: you need a Map, not necessarily a Red-Black Tree. Furthermore, these default constructors will often use specialized representations: for example, Map() will use a 3-field object for maps with 3 keys.

The corollary to the above is: in your own methods and constructors, receive the most generic collection type appropriate. This typically boils down to one of the above: Iterable, Seq, Set, or Map. If your method needs a sequence, use Seq[T], not List[T]. (A word of caution: the default Traversable, Iterable and Seq types in scope – defined in scala.package – are the scala.collection versions, as opposed to Map and Set – defined in Predef.scala – which are the scala.collection.immutable versions. This means that, for example, the default Seq type can be both the immutable and mutable implementations. Thus, if your method relies on a collection parameter being immutable, and you are using Traversable, Iterable or Seq, you must specifically require/import the immutable variant, otherwise someone may pass you the mutable version.)

Style

Functional programming encourages pipelining transformations of an immutable collection to shape it to its desired result. This often leads to very succinct solutions, but can also be confusing to the reader — it is often difficult to discern the author’s intent, or keep track of all the intermediate results that are only implied. For example, let’s say we wanted to aggregate votes for different programming languages from a sequence of (language, num votes), showing them in order of most votes to least, we could write:

val votes = Seq(("scala", 1), ("java", 4), ("scala", 10), ("scala", 1), ("python", 10))
val orderedVotes = votes
  .groupBy(_._1)
  .map { case (which, counts) => 
    (which, counts.foldLeft(0)(_ + _._2))
  }.toSeq
  .sortBy(_._2)
  .reverse

this is both succinct and correct, but nearly every reader will have a difficult time recovering the original intent of the author. A strategy that often serves to clarify is to name intermediate results and parameters:

val votesByLang = votes groupBy { case (lang, _) => lang }
val sumByLang = votesByLang map { case (lang, counts) =>
  val countsOnly = counts map { case (_, count) => count }
  (lang, countsOnly.sum)
}
val orderedVotes = sumByLang.toSeq
  .sortBy { case (_, count) => count }
  .reverse

the code is nearly as succinct, but much more clearly expresses both the transformations take place (by naming intermediate values), and the structure of the data being operated on (by naming parameters). If you worry about namespace pollution with this style, group expressions with {}:

val orderedVotes = {
  val votesByLang = ...
  ...
}

Performance

High level collections libraries (as with higher level constructs generally) make reasoning about performance more difficult: the further you stray from instructing the computer directly — in other words, imperative style — the harder it is to predict the exact performance implications of a piece of code. Reasoning about correctness however, is typically easier; readability is also enhanced. With Scala the picture is further complicated by the Java runtime; Scala hides boxing/unboxing operations from you, which can incur severe performance or space penalties.

Java Collections

Use scala.collection.JavaConverters to interoperate with Java collections. These are a set of implicits that add conversion asJava and asScala conversion methods. The use of these ensures that such conversions are explicit, aiding the reader:

import scala.collection.JavaConverters._

val list: java.util.List[Int] = Seq(1,2,3,4).asJava
val buffer: scala.collection.mutable.Buffer[Int] = list.asScala

Concurrency

Modern services are highly concurrent — it is common for servers to coordinate 10s–100s of thousands of simultaneous operations — and handling the implied complexity is a central theme in authoring robust systems software.

Threads provide a means of expressing concurrency: they give you independent, heap-sharing execution contexts that are scheduled by the operating system. However, thread creation is expensive in Java and is a resource that must be managed, typically with the use of pools. This creates additional complexity for the programmer, and also a high degree of coupling: it’s difficult to divorce application logic from their use of the underlying resources.

This complexity is especially apparent when creating services that have a high degree of fan-out: each incoming request results in a multitude of requests to yet another tier of systems. In these systems, thread pools must be managed so that they are balanced according to the ratios of requests in each tier: mismanagement of one thread pool bleeds into another.

Robust systems must also consider timeouts and cancellation, both of which require the introduction of yet more “control” threads, complicating the problem further. Note that if threads were cheap these problems would be diminished: no pooling would be required, timed out threads could be discarded, and no additional resource management would be required.

Thus resource management compromises modularity.

Futures

Collections

The subject of concurrent collections is fraught with opinions, subtleties, dogma and FUD. In most practical situations they are a nonissue: Always start with the simplest, most boring, and most standard collection that serves the purpose. Don’t reach for a concurrent collection before you know that a synchronized one won’t do: the JVM has sophisticated machinery to make synchronization cheap, so their efficacy may surprise you.

If an immutable collection will do, use it — they are referentially transparent, so reasoning about them in a concurrent context is simple. Mutations in immutable collections are typically handled by updating a reference to the current value (in a var cell or an AtomicReference). Care must be taken to apply these correctly: atomics must be retried, and vars must be declared volatile in order for them to be published to other threads.

Mutable concurrent collections have complicated semantics, and make use of subtler aspects of the Java memory model, so make sure you understand the implications — especially with respect to publishing updates — before you use them. Synchronized collections also compose better: operations like getOrElseUpdate cannot be implemented correctly by concurrent collections, and creating composite collections is especially error prone.

Control structures

Programs in the functional style tend to require fewer traditional control structures, and read better when written in the declarative style. This typically implies breaking your logic up into several small methods or functions, and gluing them together with match expressions. Functional programs also tend to be more expression-oriented: branches of conditionals compute values of the same type, for (..) yield computes comprehensions, and recursion is commonplace.

Recursion

Phrasing your problem in recursive terms often simplifies it, and if the tail call optimization applies (which can be checked by the @tailrec annotation), the compiler will even translate your code into a regular loop.

Consider a fairly standard imperative version of heap fix-down:

def fixDown(heap: Array[T], m: Int, n: Int): Unit = {
  var k: Int = m
  while (n >= 2*k) {
    var j = 2*k
    if (j < n && heap(j) < heap(j + 1))
      j += 1
    if (heap(k) >= heap(j))
      return
    else {
      swap(heap, k, j)
      k = j
    }
  }
}

Returns

This is not to say that imperative structures are not also valuable. In many cases they are well suited to terminate computation early instead of having conditional branches for every possible point of termination: indeed in the above fixDown, a return is used to terminate early if we’re at the end of the heap.

Returns can be used to cut down on branching and establish invariants. This helps the reader by reducing nesting (how did I get here?) and making it easier to reason about the correctness of subsequent code (the array cannot be accessed out of bounds after this point). This is especially useful in “guard” clauses:

def compare(a: AnyRef, b: AnyRef): Int = {
  if (a eq b)
    return 0

  val d = System.identityHashCode(a) compare System.identityHashCode(b)
  if (d != 0)
    return d

  // slow path..
}

Use returns to clarify and enhance readability, but not as you would in an imperative language; avoid using them to return the results of a computation. Instead of

def suffix(i: Int) = {
  if      (i == 1) return "st"
  else if (i == 2) return "nd"
  else if (i == 3) return "rd"
  else             return "th"
}

prefer:

def suffix(i: Int) =
  if      (i == 1) "st"
  else if (i == 2) "nd"
  else if (i == 3) "rd"
  else             "th"

but using a match expression is superior to either:

def suffix(i: Int) = i match {
  case 1 => "st"
  case 2 => "nd"
  case 3 => "rd"
  case _ => "th"
}

Note that returns can have hidden costs: when used inside of a closure,

seq foreach { elem =>
  if (elem.isLast)
    return

  // process...
}

this is implemented in bytecode as an exception catching/throwing pair which, used in hot code, has performance implications.

for loops and comprehensions

for provides both succinct and natural expression for looping and aggregation. It is especially useful when flattening many sequences. The syntax of for belies the underlying mechanism as it allocates and dispatches closures. This can lead to both unexpected costs and semantics; for example

for (item <- container) {
  if (item != 2) return
}

may cause a runtime error if the container delays computation, making the return nonlocal!

For these reasons, it is often preferrable to call foreach, flatMap, map, and filter directly — but do use fors when they clarify.

require and assert

require and assert both serve as executable documentation. Both are useful for situations in which the type system cannot express the required invariants. assert is used for invariants that the code assumes (either internal or external), for example

val stream = getClass.getResourceAsStream("someclassdata")
assert(stream != null)

Whereas require is used to express API contracts:

def fib(n: Int) = {
  require(n > 0)
  ...
}

Functional programming

Value oriented programming confers many advantages, especially when used in conjunction with functional programming constructs. This style emphasizes the transformation of values over stateful mutation, yielding code that is referentially transparent, providing stronger invariants and thus also easier to reason about. Case classes, pattern matching, destructuring bindings, type inference, and lightweight closure- and method-creation syntax are the tools of this trade.

Case classes as algebraic data types

Case classes encode ADTs: they are useful for modelling a large number of data structures and provide for succinct code with strong invariants, especially when used in conjunction with pattern matching. The pattern matcher implements exhaustivity analysis providing even stronger static guarantees.

Use the following pattern when encoding ADTs with case classes:

sealed trait Tree[T]
case class Node[T](left: Tree[T], right: Tree[T]) extends Tree[T]
case class Leaf[T](value: T) extends Tree[T]

the type Tree[T] has two constructors: Node and Leaf. Declaring the type sealed allows the compiler to do exhaustivity analysis since constructors cannot be added outside the source file.

Together with pattern matching, such modelling results in code that is both succinct and “obviously correct”:

def findMin[T <: Ordered[T]](tree: Tree[T]) = tree match {
  case Node(left, right) => Seq(findMin(left), findMin(right)).min
  case Leaf(value) => value
}

While recursive structures like trees constitute classic applications of ADTs, their domain of usefulness is much larger. Disjoint unions in particular are readily modelled with ADTs; these occur frequently in state machines.

Options

The Option type is a container that is either empty (None) or full (Some(value)). They provide a safe alternative to the use of null, and should be used in their stead whenever possible. They are a collection (of at most one item) and they are embellished with collection operations — use them!

Write

var username: Option[String] = None
...
username = Some("foobar")

instead of

var username: String = null
...
username = "foobar"

since the former is safer: the Option type statically enforces that username must be checked for emptyness.

Conditional execution on an Option value should be done with foreach; instead of

if (opt.isDefined)
  operate(opt.get)

write

opt foreach { value =>
  operate(value)
}

The style may seem odd, but provides greater safety (we don’t call the exceptional get) and brevity. If both branches are taken, use pattern matching:

opt match {
  case Some(value) => operate(value)
  case None => defaultAction()
}

but if all that's missing is a default value, use getOrElse

operate(opt getOrElse defaultValue)

Pattern matching

Pattern matches (x match { ...) are pervasive in well written Scala code: they conflate conditional execution, destructuring, and casting into one construct. Used well they enhance both clarity and safety.

Use pattern matching to implement type switches:

obj match {
  case str: String => ...
  case addr: SocketAddress => ...

Pattern matching works best when also combined with destructuring (for example if you are matching case classes); instead of

animal match {
  case dog: Dog => "dog (%s)".format(dog.breed)
  case _ => animal.species
  }

write

animal match {
  case Dog(breed) => "dog (%s)".format(breed)
  case other => other.species
}

Partial functions

Scala provides syntactical shorthand for defining a PartialFunction:

val pf: PartialFunction[Int, String] = {
  case i if i%2 == 0 => "even"
}

and they may be composed with orElse

val tf: (Int => String) = pf orElse { case _ => "odd"}

tf(1) == "odd"
tf(2) == "even"

Partial functions arise in many situations and are effectively encoded with PartialFunction, for example as arguments to methods

trait Publisher[T] {
  def subscribe(f: PartialFunction[T, Unit])
}

val publisher: Publisher[Int] = ...
publisher.subscribe {
  case i if isPrime(i) => println("found prime", i)
  case i if i%2 == 0 => count += 2
  /* ignore the rest */
}

or in situations that might otherwise call for returning an Option:

// Attempt to classify the the throwable for logging.
type Classifier = Throwable => Option[java.util.logging.Level]

might be better expressed with a PartialFunction

type Classifier = PartialFunction[Throwable, java.util.Logging.Level]

as it affords greater composability:

val classifier1: Classifier
val classifier2: Classifier

val classifier = classifier1 orElse classifier2 orElse { _ => java.util.Logging.Level.FINEST }

Destructuring bindings

Destructuring value bindings are related to pattern matching; they use the same mechanism but are applicable when there is exactly one option (lest you accept the possibility of an exception). Destructuring binds are particularly useful for tuples and case classes.

val tuple = ('a', 1)
val (char, digit) = tuple

val tweet = Tweet("just tweeting", Time.now)
val Tweet(text, timestamp) = tweet

Laziness

Fields in scala are computed by need when val is prefixed with lazy. Because fields and methods are equivalent in Scala (lest the fields are private[this])

lazy val field = computation()

is (roughly) short-hand for

var _theField = None
def field = if (_theField.isDefined) _theField.get else {
  _theField = Some(computation())
  _theField.get
}

i.e., it computes a results and memoizes it. Use lazy fields for this purpose, but avoid using laziness when laziness is required by semantics. In these cases it's better to be explicit since it makes the cost model explicit, and side effects can be controlled more precisely.

Lazy fields are thread safe.

Call by name

Method parameters may be specified by-name, meaning the parameter is bound not to a value but to a computation that may be repeated. This feature must be applied with care; a caller expecting by-value semantics will be surprised. The motivation for this feature is to construct syntactically natural DSLs — new control constructs in particular can be made to look much like native language features.

Only use call-by-name for such control constructs, where it is obvious to the caller that what is being passed in is a “block” rather than the result of an unsuspecting computation. Only use call-by-name arguments in the last position of the last argument list. When using call-by-name, ensure that method is named so that it is obvious to the caller that its argument is call-by-name.

When you do want a value to be computed multiple times, and especially when this computation is side effecting, use explicit functions:

class SSLConnector(mkEngine: () => SSLEngine)

The intent remains obvious and caller is left without surprises.

flatMap

flatMap — the combination of map with flatten — deserves special attention, for it has subtle power and great utility. Like its brethren map, it is frequently available in nontraditional collections such as Future and Option. Its behavior is revealed by its signature; for some Container[A]

flatMap[B](f: A => Container[B]): Container[B]

flatMap invokes the function f for the element(s) of the collection producing a new collection, (all of) which are flattened into its result. For example, to get all permutations of two character strings that aren't the same character repeated twice:

val chars = 'a' to 'z'
val perms = chars flatMap { a => 
  chars flatMap { b => 
    if (a != b) Seq("%c%c".format(a, b)) 
    else Seq() 
  }
}

which is equivalent to the more concise for-comprehension (which is — roughly — syntactical sugar for the above):

val perms = for {
  a <- chars
  b <- chars
  if a != b
} yield "%c%c".format(a, b)

flatMap is frequently useful when dealing with Options — it will collapse chains of options down to one,

val host: Option[String] = ...
val port: Option[Int] = ...

val addr: Option[InetSocketAddress] =
  host flatMap { h =>
    port map { p =>
      new InetSocketAddress(h, p)
    }
  }

which is also made more succinct with for

val addr: Option[InetSocketAddress] = for {
  h <- host
  p <- port
} yield new InetSocketAddress(h, p)

Object oriented programming

Much of Scala’s vastness lies in its object system. Scala is a pure language in the sense that all values are objects; there is no distinction between primitive types and composite ones. Scala also features mixins allowing for more orthogonal and piecemeal construction of modules that can be flexibly put together at compile time with all the benefits of static type checking.

Dependency injection

In our use, however, we’ve found that Scala itself removes so much of the syntactical overhead of “classic” (constructor) dependency injection that we’d rather just use that: it is clearer, the dependencies are still encoded in the (constructor) type, and class construction is so syntactically trivial that it becomes a breeze. It’s boring and simple and it works. Use dependency injection for program modularization, and in particular, prefer composition over inheritance — for this leads to more modular and testable programs. When encountering a situation requiring inheritance, ask yourself: how would you structure the program if the language lacked support for inheritance? The answer may be compelling.

Dependency injection typically makes use of traits,

trait TweetStream {
  def subscribe(f: Tweet => Unit)
}
class HosebirdStream extends TweetStream ...
class FileStream extends TweetStream ...

class TweetCounter(stream: TweetStream) {
  stream.subscribe { tweet => count += 1 }
}

It is common to inject factories — objects that produce other objects. In these cases, favor the use of simple functions over specialized factory types.

class FilteredTweetCounter(mkStream: Filter => TweetStream) {
  mkStream(PublicTweets).subscribe { tweet => publicCount += 1 }
  mkStream(DMs).subscribe { tweet => dmCount += 1 }
}

Traits

Dependency injection does not at all preclude the use of common interfaces, or the implementation of common code in traits. Quite the contrary — the use of traits are highly encouraged for exactly this reason: multiple interfaces (traits) may be implemented by a concrete class, and common code can be reused across all such classes.

Keep traits short and orthogonal: don’t lump separable functionality into a trait, think of the smallest related ideas that fit together. For example, imagine you have an something that can do IO:

trait IOer {
  def write(bytes: Array[Byte])
  def read(n: Int): Array[Byte]
}

separate the two behaviors:

trait Reader {
  def read(n: Int): Array[Byte]
}
trait Writer {
  def write(bytes: Array[Byte])
}

and mix them together to form what was an IOer: new Reader with Writer… Interface minimalism leads to greater orthogonality and cleaner modularization.

Visibility

Scala has very expressive visibility modifiers. It’s important to use these as they define what constitutes the public API. Public APIs should be limited so users don’t inadvertently rely on implementation details and limit the author’s ability to change them: They are crucial to good modularity. As a rule, it’s much easier to expand public APIs than to contract them. Poor annotations can also compromise backwards binary compatibility of your code.

private[this]

A class member marked private,

private val x: Int = ...

is visible to all instances of that class (but not their subclasses). In most cases, you want private[this].

private[this] val: Int = ...

which limits visibility to the particular instance. The Scala compiler is also able to translate private[this] into a simple field access (since access is limited to the statically defined class) which can sometimes aid performance optimizations.

Singleton class types

It’s common in Scala to create singleton class types, for example

def foo() = new Foo with Bar with Baz {
  ...
}

In these situations, visibility can be constrained by declaring the returned type:

def foo(): Foo with Bar = new Foo with Bar with Baz {
  ...
}

where callers of foo() will see a restricted view (Foo with Bar) of the returned instance.

Structural typing

Do not use structural types in normal use. They are a convenient and powerful feature, but unfortunately do not have an efficient implementation on the JVM. However — due to an implementation quirk — they provide a very nice shorthand for doing reflection.

val obj: AnyRef
obj.asInstanceOf[{def close()}].close()

Error handling

Scala provides an exception facility, but do not use it for commonplace errors, when the programmer must handle errors properly for correctness. Instead, encode such errors explicitly: using Option or com.twitter.util.Try are good, idiomatic choices, as they harness the type system to ensure that the user is properly considering error handling.

For example, when designing a repository, the following API may be tempting:

trait Repository[Key, Value] {
  def get(key: Key): Value
}

but this would require the implementor to throw an exception when the key is absent. A better approach is to use an Option:

trait Repository[Key, Value] {
  def get(key: Key): Option[Value]
}

This interface makes it obvious that the repository may not contain every key, and that the programmer must handle missing keys. Furthermore, Option has a number of combinators to handle these cases. For example, getOrElse is used to supply a default value for missing keys:

val repo: Repository[Int, String]
repo.get(123) getOrElse "defaultString"

Handling exceptions

Because Scala’s exception mechanism isn’t checked — the compiler cannot statically tell whether the programmer has covered the set of possible exceptions — it is often tempting to cast a wide net when handling exceptions.

However, some exceptions are fatal and should never be caught; the code

try {
  operation()
} catch {
  case _ => ...
}

is almost always wrong, as it would catch fatal errors that need to be propagated. Instead, use the com.twitter.util.NonFatal extractor to handle only nonfatal exceptions.

try {
  operation()
} catch {
  case NonFatal(exc) => ...
}

Garbage collection

We spend a lot of time tuning garbage collection in production. The garbage collection concerns are largely similar to those of Java though idiomatic Scala code tends to generate more (short-lived) garbage than idiomatic Java code — a byproduct of the functional style. Hotspot’s generational garbage collection typically makes this a nonissue as short-lived garbage is effectively free in most circumstances.

Java compatibility

When we write code in Scala that is used from Java, we ensure that usage from Java remains idiomatic. Oftentimes this requires no extra effort — classes and pure traits are exactly equivalent to their Java counterpart — but sometimes separate Java APIs need to be provided. A good way to get a feel for your library’s Java API is to write a unittest in Java (just for compilation); this also ensures that the Java-view of your library remains stable over time as the Scala compiler can be volatile in this regard.

Traits that contain implementation are not directly usable from Java: extend an abstract class with the trait instead.

// Not directly usable from Java
trait Animal {
  def eat(other: Animal)
  def eatMany(animals: Seq[Animal) = animals foreach(eat(_))
}

// But this is:
abstract class JavaAnimal extends Animal

Twitter’s standard libraries

Futures

Offer/Broker

Acknowledgments

The lessons herein are those of Twitter’s Scala community — I hope I’ve been a faithful chronicler.

Blake Matheny, Nick Kallen, Steve Gury, and Raghavendra Prabhu provided much helpful guidance and many excellent suggestions.