“for” destructuring improvements (yay!)

So, there is a long-standing wart in the Scala language, which folks have known about for a long time. Haoyi brought it more into the public eye in a blog post a couple of years ago, and formally opened an issue for it not long after.

Haoyi’s posts have some examples, but let me show a simplified variant of one that happens all the time in the Akka world (live code here):

sealed trait MyResponse
case class UsualCase(i: Int) extends MyResponse
case class UnusualCase(error: String) extends MyResponse
def doStuff(msg: String): Future[MyResponse] = {
  Future.successful(UnusualCase("OMG!!!"))
}
val resultFut = for {
  UsualCase(i) <- doStuff("Fetch my slippers")
}
  yield i
  
resultFut
  .map(i => println(s"I got back $i"))
  .onFailure {
    case err: Throwable => println(s"I got a runtime error $err")
  }

What happens here? Points if you looked at it and said “it throws an Exception at runtime”, but it’s entirely forgivable if you didn’t find that obvious.

When you look at it all together, this code has a clear smell: doStuff is returning a relatively broad type MyResponse, but the for comprehension is only checking for the UsualCase subtype. Somewhat surprisingly, the compiler (deliberately) allows this to slip through silently.

This becomes a good deal less obvious when the code is spread-out, as is typically the case in Akka, where the response to an actor call such as myActor ? doStuff is of type Any, but you can casually forget that and write a for comprehension such as the above. It compiles fine and runs most of the time, but suddenly blows up at runtime when you get an unexpected response.

The language wart here is that for comprehensions casually and silently filter the value from the right-hand side. More precisely, the right-hand side gets applied to the filter function (or, in more modern Scala, the withFilter function) found in the monad you’re working with. So long as the type being destructured on the left is compatible with the type being returned on the right, that compiles just fine — the compiler just silently applies withFilter under the hood.

(Exactly what happens if the types don’t match depends on the monad in question. If it’s a Future, you’ll get a runtime error. If it’s a List, as shown in Haoyi’s examples, the non-matching values will be silently thrown away — arguably even worse.)

It’s an idea that made sense at the time, since it is very syntactically convenient, but we’ve come to regret it because it isn’t quite compatible with Scala expectations. We allow this sort of runtime destructuring, yes — we do it all the time in pattern-matching — but we require a bit of ceremony so that you have to think about the fact that you’re doing it. Everywhere else, you do this by using the keyword case, but not here.

Getting to the point: Martin has just submitted a PR to rectify this inconsistency. The plan is that, starting in -strict mode in Scala 3.0, then by default in Scala 3.1, you will need to apply the case keyword to apply filtering in for comprehensions, like this:

val resultFut = for {
  case UsualCase(i) <- doStuff("Fetch my slippers")
}
  yield i

It’s a little change, but a welcome one. As so often in Scala, the point isn’t to prevent you from doing something possibly unsafe, it’s to make you do so consciously. I expect the use of case in this way to be fairly unusual in general, so requiring you to do so makes you pause and think a little, the same way that pattern-matching does.

As usual, this is just a PR — not yet merged to Dotty master as of this writing, much less approved as a Scala 3 enhancement. But I think this one is a nice little win, and I expect it to be uncontroversial: a fair number of folks have been wanting this wart fixed for many years. So I’m optimistic that it will make it into Scala 3.