location

Contains classes to provide preferred spark node selection when running RyftOne queries.

Overview

Selecting preferred location can be useful for data locality when to ensure REST api is present on the same box, rack or network. It can be applied by implementing trait com.ryft.spark.connector.preferred.location.PreferredLocationsHandler, as so

// Example preferred location handler that requests Spark Tasks to be assigned on same nodes
// where REST api is located based on their availability.
class NoPreferredLocation extends PreferredLocationsHandler {
  override def nodes(url: URL): Set[String] = Set(new URL(url).getHost)
}

Other classes in this package represent examples of implementation:

com.ryft.spark.connector.preferred.location.NoPreferredLocation - No preferred location rules applied. Default behavior.