Monads, flatMap, and for-comprehensions

Lesson 08 gave us map: transform a value inside a context (an Option, a List) without leaving the context. Lesson 09 drilled the discipline that combining operations must be lawful so refactorings stay safe. This lesson hits the wall map cannot climb: what happens when the function you map with itself returns a context. The answer — flatMap — is the entire content of the word "monad," and it is what lets you sequence steps that each produce a new context. for-comprehensions are just readable sugar over it.

1 · The wall: map produces a nested context

Recall the functor contract from lesson 08: map takes an F[A] and a function A => B and gives back an F[B] — the context stays, the inside changes. That works perfectly as long as the function you supply returns a plain value. The trouble starts when the natural function for the job returns a context of its own.

Take three lookups, each of which can fail, so each returns an Option — a context meaning "maybe absent" (lesson 08's running example):

def findUser(id: Int): Option[User]          // user might not exist
def findAccount(u: User): Option[Account]    // user might have no account
def findBalance(a: Account): Option[Money]   // account might be unfunded

Now try to go from an id to a balance using only map. The function we feed to map is findAccount, whose return type is Option[Account] — itself a context. So map faithfully wraps that result in the outer Option:

val u: Option[User]              = findUser(id)
val a: Option[Option[Account]]   = u.map(findAccount)      // map of (User => Option[Account])
val b: Option[Option[Option[Money]]] = a.map(_.map(findBalance))  // and it gets worse

Each step adds a layer. After three context-returning steps you are holding an Option[Option[Option[Money]]] — a value buried under a tower of contexts, where "is it actually present?" now means "peel three layers and check each." map is doing exactly its job (preserve the context, transform the inside); the inside just happened to be another context, so the towers stack. map cannot flatten — that is not its job — so it can never get you back to a single Option[Money]. That is the wall.

2 · flatten and flatMap collapse the tower

The missing piece is an operation that removes one redundant layer — turns an F[F[A]] back into an F[A]:

For Option, flatten says: Some(Some(x)) becomes Some(x); Some(None) and None both become None. For List, flatten concatenates a list of lists into one list. It is the inverse move to the nesting that map introduced.

The pattern "map with a context-returning function, then flatten the result" is so common it gets one name and one method, flatMap:

Its signature is the whole point — burn it in, because it is the definition of "monad":

Read it aloud: given a context-carrying F[A] and a function that takes a plain A and produces a new context F[B], give back a single F[B] — no nesting. That is exactly the shape our lookups have. Redo the chain with flatMap and it stays one level deep the whole way:

val balance: Option[Money] =
  findUser(id)            // Option[User]
    .flatMap(findAccount) //   User    => Option[Account]  ⇒ stays Option[Account]
    .flatMap(findBalance) //   Account => Option[Money]    ⇒ stays Option[Money]

Two bonuses fall out for free. First, no tower: the result is a plain Option[Money]. Second — and this is what makes Option's flatMap so useful — it short-circuits: the moment any step yields None, the rest of the chain is skipped and the whole expression is None. You wrote the happy path; the failure path threads itself.

3 · What a monad actually is

You now have everything. A monad is a functor (lesson 08) that additionally provides two operations:

In Scala you can sketch the abstraction as a trait extending the functor from lesson 08:

trait Functor[F[_]] {
  def map[A, B](fa: F[A])(f: A => B): F[B]
}

trait Monad[F[_]] extends Functor[F] {
  def unit[A](a: A): F[A]                              // pure
  def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B]      // sequence

  // map comes for free, proving Monad really IS a Functor:
  def map[A, B](fa: F[A])(f: A => B): F[B] =
    flatMap(fa)(a => unit(f(a)))
}

So the one-sentence definition to carry away: a monad is a functor plus a lawful way to sequence steps that each produce a new context. There is no box, no burrito — there is a type F[_] and a flatMap with that signature. Option, List, and (next lesson) Try and Either are all monads in exactly this sense.

4 · The three monad laws, and why they matter

Having the right signature is not enough — lesson 07 taught us that lawful composition is what makes patterns trustworthy. A monad's unit and flatMap must obey three laws:

Why care? Because these laws are what make a for-comprehension behave predictably and refactorings safe. Associativity, in particular, says you can extract a middle run of steps into a helper, or re-bracket them, without changing meaning — the same equational-reasoning power lesson 09 gave fold, now applied to control flow. It is the principled replacement for the brittle, deeply nested callbacks of lesson 07: there, re-ordering or factoring callbacks risked subtle bugs; here the law guarantees the rewrite is sound.

5 · for-comprehensions are sugar over flatMap/map

Chained flatMap reads backwards once you have more than two steps. Scala lets you write the identical computation as a for-comprehension:

// the chained flatMap form (the trailing .map(b => b) is identity — the final `yield`) …
findUser(id).flatMap(u => findAccount(u).flatMap(a => findBalance(a).map(b => b)))

// … is EXACTLY this for-comprehension:
for {
  u <- findUser(id)      // each `<-` is a flatMap
  a <- findAccount(u)
  b <- findBalance(a)
} yield b                // the final `yield` is a map

This is not a new feature — it is pure syntax. The compiler desugars it by a fixed mechanical rule:

So the for above desugars step-by-step to the nested flatMap/map beside it. Because Option is a lawful monad, all the §2 properties carry over verbatim: one level deep, and short-circuiting — if findAccount(u) is None, the comprehension yields None and never evaluates the rest. The for form is what you write in real code; the desugaring is what you must be able to recite in an interview.

The same sugar over a different monad does something that looks completely different — because the meaning is entirely the monad's flatMap. List's flatMap runs the body once per element and concatenates, so a for over two lists is a cartesian product:

for { x <- List(1, 2); y <- List("a", "b") } yield (x, y)
// = List(1,2).flatMap(x => List("a","b").map(y => (x, y)))
// = List((1,"a"), (1,"b"), (2,"a"), (2,"b"))     // every pair, not 2 iterations

6 · The unifying point: one interface, many contexts

Step back. "Optionality" (Option), "nondeterminism / many results" (List), "typed error" (Either, lesson 11), "a value arriving later" (Future), "an effect to be run" (IO, lesson 14) — these feel like unrelated problems. Yet each is a context F[_] for which "do the next context-producing step" is captured by the same flatMap signature and the same three laws. That is the entire payoff of naming "monad": once a type is a lawful monad, you already know how to sequence it, you can write it in a for-comprehension, and your equational reasoning transfers unchanged. You learn the interface once and reuse it across every effect you will meet for the rest of the series.

Common mistakes / failure modes

Checkpoint exercise

Where this points next

We now know what flatMap and for buy you, in the abstract. Lesson 11 cashes that out into the three monads you will actually use every day: Option for plain absence, Either for typed errors that carry a reason, and Try for capturing thrown exceptions as values. Because all three are lawful monads, everything from this lesson — short-circuiting, for-comprehension sugar, safe refactoring — applies immediately; lesson 11 is mostly "which one when," and how this replaces null and exceptions with typed failure that composes.

Interview prompts

• Why can't map handle a chain of operations that each return an Option? (§1 — the mapped function returns a context, so map wraps it, giving Option[Option[…]]; map preserves the context and cannot flatten, so layers stack into an unusable tower.)
• Define flatMap by signature and in terms of map. (§2 — flatMap: F[A] => (A => F[B]) => F[B]; equals map(f) then flatten: F[F[A]] => F[A], collapsing the redundant layer.)
• What, precisely, is a monad? (§3 — a functor with unit: A => F[A] and flatMap, obeying the three laws; not a box or burrito but a type with a lawful flatMap. map is derivable as flatMap(fa)(a => unit(f(a))).)
• State the monad laws and say why they matter. (§4 — left identity unit(a).flatMap(f)==f(a), right identity m.flatMap(unit)==m, associativity m.flatMap(f).flatMap(g)==m.flatMap(x=>f(x).flatMap(g)); they make for-comprehensions predictable and re-grouping/extracting steps safe.)
• How does the compiler desugar a for-comprehension? (§5 — each non-final <- becomes flatMap, the final yield becomes map, an if guard becomes withFilter.)
• Why does a for over two Lists give a cartesian product but over two Options give short-circuit? (§5, §2 — the meaning is whatever the monad's flatMap does: List.flatMap runs the body per element and concatenates; Option.flatMap stops at the first None.)
• Why does mixing an Option and a List in one for-comprehension fail to compile? (§6 mistakes — every generator must be over the same F[_]; you cannot flatMap an Option with a List-returning function. Combining effects needs monad transformers.)