Notice that in the Either case, should any string fail to parse the entire traversal is considered a failure. Moreover, once it hits its first bad parse, it will not attempt to parse any others down the line (similar behavior would be found with using Option as the effect). Contrast this with Validated where even if one bad parse is hit, it will continue trying to parse the others, accumulating any and all errors as it goes. The behavior of traversal is closely tied with the Applicative behavior of the data type.

Going back to our Future example, we can write an Applicative instance for Future that runs each Future concurrently. Then when we traverse a List[A] with an A => Future[B], we can imagine the traversal as a scatter-gather. Each A creates a concurrent computation that will produce a B (the scatter), and as the Futures complete they will be gathered back into a List.

Playing with Reader

Another interesting effect we can use is Reader. Recall that a Reader[E, A] is a type alias for Kleisli[Id, E, A] which is a wrapper around E => A.

If we fix E to be some sort of environment or configuration, we can use the Reader applicative in our traverse.

import cats.data.Reader

trait Context
trait Topic
trait Result

type Job[A] = Reader[Context, A]

def processTopic(topic: Topic): Job[Result] = ???

We can imagine we have a data pipeline that processes a bunch of data, each piece of data being categorized by a topic. Given a specific topic, we produce a Job that processes that topic. (Note that since a Job is just a Reader/Kleisli, one could write many small Jobs and compose them together into one Job that is used/returned by processTopic.)

Corresponding to our bunches of data are bunches of topics, a List[Topic] if you will. Since Reader has an Applicative instance, we can traverse over this list with processTopic.

def processTopics(topics: List[Topic]) =
topics.traverse(processTopic)

Note the nice return type - Job[List[Result]]. We now have one aggregate Job that when run, will go through each topic and run the topic-specific job, collecting results as it goes. We say "when run" because a Job is some function that requires a Context before producing the value we want.

One example of a "context" can be found in the Spark project. In Spark, information needed to run a Spark job (where the master node is, memory allocated, etc.) resides in a SparkContext. Going back to the above example, we can see how one may define topic-specific Spark jobs (type Job[A] = Reader[SparkContext, A]) and then run several Spark jobs on a collection of topics via traverse. We then get back a Job[List[Result]], which is equivalent to SparkContext => List[Result]. When finally passed a SparkContext, we can run the job and get our results back.

Moreover, the fact that our aggregate job is not tied to any specific SparkContext allows us to pass in a SparkContext pointing to a production cluster, or (using the exact same job) pass in a test SparkContext that just runs locally across threads. This makes testing our large job nice and easy.

Finally, this encoding ensures that all the jobs for each topic run on the exact same cluster. At no point do we manually pass in or thread a SparkContext through - that is taken care for us by the (applicative) effect of Reader and therefore by traverse.

Sequencing

Sometimes you may find yourself with a collection of data, each of which is already in an effect, for instance a List[Option[A]]. To make this easier to work with, you want a Option[List[A]]. Given Option has an Applicative instance, we can traverse over the list with the identity function.

Choose your effect

The type signature of Traverse appears highly abstract, and indeed it is - what traverse does as it walks the F[A] depends on the effect of the function. Let's see some examples where F is taken to be List.

import cats.Semigroup
import cats.data.{NonEmptyList, OneAnd, Validated, ValidatedNel}
import cats.implicits._

def parseIntEither(s: String): Either[NumberFormatException, Int] =
Either.catchOnly[NumberFormatException](s.toInt)

def parseIntValidated(s: String): ValidatedNel[NumberFormatException, Int] =
Validated.catchOnly[NumberFormatException](s.toInt).toValidatedNel

We can now traverse structures that contain strings parsing them into integers and accumulating failures with Either.

We need proof that NonEmptyList[A] is a Semigroup for there to be an Applicative instance for ValidatedNel.

implicit def nelSemigroup[A]: Semigroup[NonEmptyList[A]] =
  OneAnd.oneAndSemigroupK[List].algebra[A]

Now that we've provided such evidence, we can use ValidatedNel as an applicative.

Traverse provides a convenience method sequence that does exactly this.

List(Option(1), Option(2), Option(3)).sequence
List(Option(1), None, Option(3)).sequence

Traversing for effect

Sometimes our effectful functions return a Unit value in cases where there is no interesting value to return (e.g. writing to some sort of store).

trait Data
def writeToStore(data: Data): Future[Unit] = ???

If we traverse using this, we end up with a funny type.

import cats.implicits._
import scala.concurrent.ExecutionContext.Implicits.global

def writeManyToStore(data: List[Data]) =
data.traverse(writeToStore)

We end up with a Future[List[Unit]]! A List[Unit] is not of any use to us, and communicates the same amount of information as a single Unit does.

Traversing solely for the sake of the effect (ignoring any values that may be produced, Unit or otherwise) is common, so Foldable (superclass of Traverse) provides traverse_ and sequence_ methods that do the same thing as traverse and sequence but ignores any value produced along the way, returning Unit at the end.