A Philosophy of Testing 4: Scenario Tests

  • On the one hand, there are unit tests — atomized little tests, typically focusing on a single class or just a few.
  • On the other hand, there are integration tests — “opaque” tests that run the application for real, often in the context of other running services, only looking at it from the outside with an external test driver.

Structure of a Scenario Test

The key idea for scenario tests is that you are mostly running the entire service normally. You generally don’t mock any of the internal components, you use the real ones.

Why?

To understand why scenario tests are usually better than unit tests for services, let’s turn it around and ask: where are the bugs?

Emulating External Services

Unlike the mocks in a typical unit test, you can build most of the infrastructure once, since it is generally just “doing the right thing”. The goal is to create simple but realistic emulators for all of the external services, that generally work as expected.

Using the Real Thing

There’s a reasonable case to be made that, if you’re trying to faithfully emulate an external component, why not just use that external component? You can’t get much more faithful than that.

When to Use Unit Tests

As mentioned last article, there are exceptions to the “don’t bother with unit tests” rule. Some particular examples:

  • Data structures that are genuinely complicated — in particular, while you shouldn’t usually be using mutable structures, even a complex immutable structure needs careful testing. (Use ScalaCheck here.)
  • Complex algorithms. In particular, if you have functions that are highly sensitive to their inputs, then it is typically worth writing some focused tests for those.
  • Hard-to-hit code. This tends to be obscure error pathways that are difficult to exercise except under extreme circumstances. Unit tests for those are sometimes worthwhile, although I would always ask whether there is a feasible way to hit them with a scenario test instead.

Conclusion

Putting this together:

  • Services tend to be mostly plumbing.
  • Bugs arise less often in plumbing classes, and more between them.
  • You should focus on testing the interactions between the components.
  • The best way to do that is by testing the entire application on realistic inputs.
  • Emulate the external services used by the components, with little simulators that are fast and under precise control.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mark "Justin" Waks

Mark "Justin" Waks

Lifelong programmer and software architect, specializing in online social tools and (nowadays) Scala. Architect of Querki (“leading the small data revolution”).