This class holds the current Comet settings and an assembly of reference instances for core, shared services
Any Spark Job will inherit from this class.
Any Spark Job will inherit from this class. All properties defined in application conf file and prefixed by the "spark" key will be loaded into the Spark Job
Utilities methods to reference datasets paths Datasets paths are constructed as follows :
Utilities methods to reference datasets paths Datasets paths are constructed as follows :
After going through the data pipeline a dataset may be referenced through a Hive table in a Hive Database.
After going through the data pipeline a dataset may be referenced through a Hive table in a Hive Database. For each input domain, 3 Hive databases may be created :
This class holds the current Comet settings and an assembly of reference instances for core, shared services
SMELL: this may be the start of a Dependency Injection root (but at 2-3 objects, is DI justified? probably not quite yet) — cchepelov