com.snowplowanalytics.snowplow.enrich.common
utils
General-purpose utils to help the ETL process along.
Wrapper for io.gatling.jsonpath for json4s and scalaz
Contains general purpose extractors and other utilities for JSONs.
The problem we're trying to solve: converting maps to classes in Scala is not very easy to do in a functional way, and it gets even harder if you have a class with >22 fields (so can't use case classes).
The problem we're trying to solve: converting maps to classes in Scala is not very easy to do in a functional way, and it gets even harder if you have a class with >22 fields (so can't use case classes).
For a discussion about this on Stack Overflow, see: http://stackoverflow.com/questions/4290955/instantiating-a-case-class-from-a-list-of-parameters
The idea is to use Java Reflection with a big ol' TransformMap:
("key in map" -> Tuple2(transformFunc, "field in class"), "another key" -> Tuple2(transformFunc, "field in class"), "a third key" -> Tuple2(transformFunc, "field in class"))
And then there would be an iteration through the source map which looks up the key in the TransformMap and then applies the transformFunc using Java Reflection, storing the answer in the class's specified field.
If I can get this working, then the next step is to generate a "results" map which contains the results of each transformFunc. Each transformFunc should either return a ValidationNel[String, Unit], or should be promoted (via an implicit) to the same.
Having this should allow me to do something like: resultsMap.foldLeft(Unit.success, |@|) to roll up any validation errors into one final ValidatioNel.
If I can get all that working, then the final step is to support transformFuncs which set multiple fields. To avoid the complexity spiralling, it would probably be simplest if any transformFunc that wanted to return multiple values returned a TupleN, and then we use the same TupleN for the target fields. Maybe there will be an implicit to convert "raw" target fields into Tuple1s.
Okay, let's start...
Contains general purpose extractors and other utilities for JSONs. Jackson-based.