Packages

•  package root

  Definition Classes

  root

•  package com

  Definition Classes

  root

•  package github

  Definition Classes

  com

•  package garyaiki

  Definition Classes

  github

•  package dendrites

  Utility functions for case classes and Properties files

  Utility functions for case classes and Properties files

  ccToMap

  Case class fields to map of field names and values

  val kvMap = ccToMap(cc)
   kvMap foreach {
    case (key, value) => gRecord.put(key, value)
  }

  isElementEqual

  Is case class field at index a specified type

  system.actorOf(props) ! GetCustomerAccountBalances(2, Set(Checking, Savings, MoneyMarket))
    receiveOne(2.seconds) match {
     case result: IndexedSeq[Product] ⇒ {
      assert(isElementEqual(result(0), 0, Checking))
      assert(isElementEqual(result(1), 0, Savings))
      assert(isElementEqual(result(2), 0, MoneyMarket))
     }
     case result ⇒ assert(false, s"Expect 3 AccountTypes, got $result")
    }

  loadProperties

  Load Properties file from classpath

  val prop: Properties = loadProperties("kafkaProducer.properties")

  Definition Classes

  garyaiki

•  package aggregator

  Aggregate functions

  Aggregate functions

  mean

  Mean of sequence of Numeric

  val doubles = List(1.0, 2.1, 3.2)
  val m: Either[String,Double] = mean(doubles)

•  package algebird

  Aggregation functions for Twitter Algebird.

  Aggregation functions for Twitter Algebird.

  Algebird provides implicit implementations of common types which are imported here. Class extraction methods in com.github.garyaiki.dendrites can be used to extract a field from case classes and tuples.

  AveragedValue

  Average a Sequence of values com.github.garyaiki.dendrites.algebird.AveragedSpec

  val bigDecimals: Seq[BigDecimal]
  val avg0 = avg(bigDecimals)

  Average a sequence of AveragedValues

  val bigDecimals2: Seq[BigDecimal]
  val avg1 = avg(bigDecimals2)
  val avgs = Vector[AveragedValue](avg0, avg1)
  val avgSum = sumAverageValues(avgs)

  BloomFilter

  Fast find if a word is in a dictionary OSX and Linux have dictionaries you can use to create BloomFilters com.github.garyaiki.dendrites.fixtures.SysProcessUtils for Paths to properNames, connectives, words and functions to read them com.github.garyaiki.dendrites.algebird.fixtures.BloomFilterBuilder for creation of BloomFilters for these dictionaries and select test words for them.

  Create a BloomFilter for OSX words dictionary com.github.garyaiki.dendrites.algebird.BloomFilterSpec

  val falsePositivepProb: Double = 0.01
  val words = readWords(wordsPath)
  val wordsBF = createBF(words, fpProb)

  Is word in BloomFilter

  val falsePositivepProb: Double = 0.01
  val word = "path"
  val inDict = wordsBF.contains(word).isTrue

  Is false positive rate acceptable

  val falsePositivepProb: Double = 0.01
  val wordsFalseWords: IndexedSeq[String]
  val falsePositives = for {
  i <- wordsFalseWords
  if wordsBF.contains(i).isTrue
  } yield i
  val acceptable = falsePositives.size < words.size * fpProb

  CountMinSketch

  Test data is IP addresses repeated a random number of times com.github.garyaiki.dendrites.algebird.CountMinSketchSpec Estimate total number of elements seen so far com.github.garyaiki.dendrites.fixtures.InetAddressesBuilder

  val addrs = inetAddresses(ipRange)
  val longZips = inetToLongZip(addrs)
  val longs = testLongs(longZips)
  implicit val m = createCMSMonoid[Long]()
  val cms = createCountMinSketch(longs)
  val estimatedCount = cms.totalCount

  Estimate count of elements with the same value as the one selected

  val estFreq = cms.frequency(longZips(5))

  Sum a Sequence of CountMinSketch then estimate combined total number of elements

  val cms1 = createCountMinSketch(longs)
  val cmss = Vector(cms, cms1)
  val cmsSum = sumCountMinSketch(cmss)
  val estimatedCount = cmsSum.totalCount

  From a Sequence of CountMinSketch estimate count of elements with the indexed same value

  val estFreq = cmsSum.frequency(longZips(5))

  DecayedValue

  Test data is a sine wave with a value for each of 360 degrees with a corresponding time value. The idea is a rising and falling value over a year com.github.garyaiki.dendrites.fixtures.TrigUtils

  Moving average from the initial value to specified index com.github.garyaiki.dendrites.algebird.DecayedValueSpec

  val sines = genSineWave(100, 0 to 360)
  val days = Range.Double(0.0, 361.0, 1.0)
  val sinesZip = sines.zip(days)
  val decayedValues = toDecayedValues(sinesZip, 10.0, None)
  val avgAt90 = decayedValues(90).average(10.0)

  Moving average from specified index to specified index

  val avg80to90 = decayedValues(90).averageFrom(10.0, 80.0, 90.0)

  HyperLogLog

  Create a HLL from a sequence of Int com.github.garyaiki.dendrites.algebird.HyperLogLogSpec

  implicit val ag = HyperLogLogAggregator(12)
  val ints: Seq[Int]
  val hll = createHLL(ints)

  Create a sequence of HLL

  val ints2: Seq[Int]
  val hll2 = createHLL(ints2)
  val hlls = Vector(hll, hll2)

  Create a HLL from a sequence of Long

  val longs: Seq[Long]
  val hll = createHLL(longs)

  Create a sequence of HLL

  val longs2: Seq[Long]
  val hll2 = createHLL(longs2)
  val hlls = Vector(hll, hll2)

  Sum a Sequence of HLL and estimate total size

  val sum = hlls.reduce(_ + _)
  val size = sum.estimatedSize

  Create a sequence of Approximate HHL approximate Map a sequence of HLL to a sequence of Approximate

  val hlls = Vector(hll, hll2)
  val approxs = mapHLL2Approximate(hlls)

  Sum a Sequence of Approximate and estimate total size

  val sum = approxs.reduce(_ + _)

  QTree

  Build QTree from a Sequence com.github.garyaiki.dendrites.algebird.fixtures.QTreeBuilder

  val level = 5
  implicit val qtBDSemigroup = new QTreeSemigroup[BigDecimal](level)
  val qtBD = buildQTree[BigDecimal](bigDecimals)

  Get its InterQuartileMean com.github.garyaiki.dendrites.algebird.QTreeSpec

  val iqm = qtBD.interQuartileMean
  // iqm._1 lower bound
  // iqm._2 upper bound

  Sum a Sequence of QTrees to a QTree

  val qTrees = Vector(qtBD, qtBD2)
  val sumQTree = sumQTrees(qTrees)

  Functor

  Map elements of a sequence.

  def wrapMax[Int](x: Int) = Max(x)
  val wm = SeqFunctor.map[Int, Max[Int]](List(1,2,3,4))(wrapMax)
  
  val bigDecimals: Seq[BigDecimal]
  val negBigDecimals = SeqFunctor.map[BigDecimal, BigDecimal](bigDecimals)(negate)
  val invertedBigDecimals = SeqFunctor.map[BigDecimal, BigDecimal](bigDecimals)(inverse)

  Map the mapped elements of a sequence: f() andThen g().

  val bigDecimals: Seq[BigDecimal]
  val invertedNegBigDecimals = andThen[BigDecimal, BigDecimal, BigDecimal](ap)( inverse)( negate)

  Max Min

  For Sequence types that have a Semigroup, Monoid and Ordering Get Max element of a sequence. com.github.garyaiki.dendrites.algebird.MaxSpec

  val iqm = qtBD.interQuartileMean
  
  val bigDecimals: Seq[BigDecimal]
  val max = max(bigDecimals)
  val optBigDecs: [Option[BigDecimal]]
  val max2 = max(optBigDecs.flatten)
  val eithBigInts = Seq[Either[String, BigInt]]
  val max3 = max(filterRight(eithBigInts)

  Get Min element of a sequence. com.github.garyaiki.dendrites.algebird.MinSpec

  val bigDecimals: Seq[BigDecimal]
  val min = min(bigDecimals)
  val optBigDecs: [Option[BigDecimal]]
  val min2 = min(optBigDecs.flatten)
  val eithBigInts = Seq[Either[String, BigInt]]
  val min3 = min(filterRight(eithBigInts)

•  package avro

  Avro serializer/deserializer functions

  Avro serializer/deserializer functions

  Load Avro Schema from file

  val schema = loadSchema(filename)

  Serialize case class to bytearray

  val bytes = ccToByteArray(schema, GetAccountBalances(1L))
  val record = new ProducerRecord[String, Array[Byte]](topic, key, bytes)
  val rm: RecordMetadata = producer.send(record).get()

  Map bytearray to Avro GenericRecord

  new GraphStageLogic(shape) {
  setHandler(in, new InHandler {
    override def onPush(): Unit = {
      val bytes = grab(in)
      val record = byteArrayToGenericRecord(schema, bytes)
      push(out, f(record))
    }
  })

•  package avro4s

  Extend Avro4sOps to serialize/deserialize case class

  Extend Avro4sOps to serialize/deserialize case class

  object Avro4sShoppingCartCmd  extends Avro4sOps[ShoppingCartCmd] {
    implicit val schemaFor = SchemaFor[ShoppingCartCmd]
    implicit val toRecord = ToRecord[ShoppingCartCmd]
    implicit val fromRecord = FromRecord[ShoppingCartCmd]

  implement case class serializer

  def toBytes(caseClass: ShoppingCartCmd): Array[Byte] = {
    val baos = new ByteArrayOutputStream()
    val output = AvroOutputStream.binary[ShoppingCartCmd](baos)
    output.write(caseClass)
    output.close()
    baos.toByteArray
  }

  implement deserializer

  def toCaseClass(bytes: Array[Byte]): ShoppingCartCmd = {
    val in = new ByteArrayInputStream(bytes)
    val input = AvroInputStream.binary[ShoppingCartCmd](in)
    val result = input.iterator.toSeq
    result(0)
  }

•  package cassandra

  Common functions for Cassandra Java Driver

  Common functions for Cassandra Java Driver

  Create single node Cassandra Cluster.

  val config = ConfigFactory.load()
  val ipAddress = config.getString("dendrites.cassandra.ipAddress")
  val cluster = createCluster(ipAddress)

  Create Cluster with multiple host nodes and a RetryPolicy

  val addresses = myConfig.getInetAddresses()
  val retryPolicy = new LoggingRetryPolicy(DefaultRetryPolicy.INSTANCE)
  cluster = createCluster(addresses, retryPolicy)

  Log cluster's metadata.

  logMetadata(cluster)

  Bind Cassandra's QueryLogger to cluster

  registerQueryLogger(cluster)

  Create a simple LoadBalancing policy

  val myConfig = PlaylistSongConfig
  val lbp = createLoadBalancingPolicy(myConfig.localDataCenter)

  Initialize LoadBalancingPolicy

  initLoadBalancingPolicy(cluster, lbp)

  Connect to Cassandra. Return a Session which is thread safe and may last app's lifetime

  session = connect(cluster)

  Create a Keyspace

  val schema = myConfig.keySpace
  val strategy = myConfig.replicationStrategy
  val createSchemaRS = createSchema(session, schema, strategy, 3)

  Create a PreparedStatement to return all rows of a table

  val plPreStmt = selectAll(session, schema, Playlists.table)

  Asynchronously execute a BoundStatement

  val selAllRS = executeBoundStmt(session, new BoundStatement(plPreStmt))

  Get every row in a result set

  val allRows = getAllRows(selAllRS)

  Drop schema

  dropSchema(session, schema)

  Asynchronously close Session and Cluster. Turns Cassandra's Java Futures into Scala Futures

  close(session, cluster)

•  package concurrent

  Functions for concurrency

  Functions for concurrency

  Transform a Java Guava ListenableFuture into a Scala Future

  val sessCloseF = cassandraSession.closeAsync()
  val scalaSessF = listenableFutureToScala[Unit](sessCloseF.asInstanceOf[ListenableFuture[Unit]])
  scalaSessF onComplete {
    case Success(x) => logger.debug("session closed")
    case Failure(t) => logger.error(t, "session closed failed {}", t.getMessage())
  }

  Calculate exponential backoff delay Constant params can be passed to first argument list on startup

  val min = config getInt("dendrites.timer.min-backoff")
  val minDuration = FiniteDuration(min, MILLISECONDS)
  val max = config getInt("dendrites.timer.max-backoff")
  val maxDuration = FiniteDuration(max, MILLISECONDS)
  val randomFactor = config getDouble("dendrites.timer.randomFactor")
  val curriedDelay = calculateDelay(minDuration, maxDuration, randomFactor) _
  //Second arg list can be curried
  val curriedDelay = consumerConfig.curriedDelay
  val duration = curriedDelay(retries)
  if(duration < maxDuration) {
    waitForTimer = true
    scheduleOnce(None, duration)
  } else {
    failStage(e) // too many retries
  }

•  package cqrs
•  package examples
•  package filters

  Filter and extractor functions for sequences, Either, Option, tuples and case classes

  Filter and extractor functions for sequences, Either, Option, tuples and case classes

  Filter Sequence of different types of case classes

  val mixCaseClasses = keyBigDecimal ++ keyBigInt ++ keyBoolean ++ keyDouble ++ keyFloat
  val filtered = filterProducts(mixCaseClasses, fieldFilter, isType[BigDecimal])

  Filter by type

  if(filtered.forall(isType[BigDecimal]))

  Filter by Option[type]

  val filtered = filterProducts(mixCaseClasses, fieldFilter, isOptionType[Int])

  Filter by Either where Left is a String or Right is the type

  val filtered = filterProducts(mixCaseClasses, fieldFilter, isEitherStringRight[BigInt])

  Extract a case class field by index

  val eithBigDecs = extractElementByIndex[Either[String, BigDecimal]](keyEithBigDec, 1)

  Filter Right values of Either

  def collectRightFlow[A, B]: Flow[Seq[Either[A, B]], Seq[B], NotUsed] =
        Flow[Seq[Either[A, B]]].collect(PartialFunction(filterRight))

•  package http

  Provides Class to create an HostConnectionPool.

  Provides Class to create an HostConnectionPool. Also functions to create requests and handle response

  Get Config, ip address, port as tuple3

  val hostConfig = getHostConfig("dendrites.checking-balances.http.interface","my.http.port")

  Append path to host URL

  val baseURL = configBaseUrl("my.http.path", hostConfig)

  Create StringBuilder with URL including path

  val url = createUrl(scheme, ipDomain, port, path)

  Create StringBuilder with request path i.e. "?", "key=value" for all fields in case class

  val balanceQuery = GetAccountBalances(goodId)
  val checkingPath = "/account/balances/checking/"
  val balancesQuery = caseClassToGetQuery(balanceQuery)()
  val q = checkingPath ++ balancesQuery

  See also

  Config API

•  package kafka

  Provides Classes to add fields to KafkaConsumer and KafkaProducer.

  Provides Classes to add fields to KafkaConsumer and KafkaProducer. Also Factories for KafkaConsumer, KafkaProducer from their Properties files

  Create KafkaConsumer with properties

  val consumer = createConsumer[Key, Value]("kafkaConsumer.properties")

  Create KafkaProducer with properties

  val producer = createProducer[Key, Value]("kafkaProducer.properties")

  See also

  Config API

•  package reflection

  Reflection utils for logging and debugging

  Reflection utils for logging and debugging

  Log type information

  val result = results.toIndexedSeq
  log.debug("result:{}", weakParamInfo(result))

  See also

  TypeTags and Manifests

•  package stream

  Akka Stream Flows

  Akka Stream Flows

  Map sequence of Option to sequence of values

  val (pub, sub) = TestSource.probe[Seq[Option[BigInt]]]
    .via(flattenFlow)
    .via(maxFlow)
    .toMat(TestSink.probe[BigInt])(Keep.both)
    .run()

  Map sequence of Either to sequence of values

  val (pub, sub) = TestSource.probe[Seq[Either[String, Double]]]
    .via(collectRightFlow)
    .via(maxFlow)
    .toMat(TestSink.probe[Double])(Keep.both)
    .run()

  Flow accepts tuple3 from zip stage, wraps tuple3LeftRight which maps 3 Eithers to Sequence of Lefts and a Sequence of Rights

  def zipper = ZipWith((in0: Either[String, AnyRef],
    in1: Either[String, AnyRef],
    in2: Either[String, AnyRef]) => (in0, in1, in2))
  val flowGraph = GraphDSL.create() { implicit builder =>
    val zip = builder.add(zipper)
    val fgLR = builder.add(leftRightFlow)

  See also

  Flow

com.github.garyaiki

dendrites 