gcp4zio-dp
API
Generated with
gcp4zio-dp/gcp4zio.dp/DPJobImpl

DPJobImpl

gcp4zio.dp.DPJobImpl
case class DPJobImpl(client: JobControllerClient, cluster: String, project: String, region: String) extends DPJob

Attributes

Graph
Supertypes
trait Serializable
trait Product
trait Equals
trait DPJob
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

def submitHiveJob(query: String): Task[Job]

Submits a Hive Job in Dataproc Cluster. (This API will not wait for Job Completion, to wait for Job completion use trackJobProgress)

Submits a Hive Job in Dataproc Cluster. (This API will not wait for Job Completion, to wait for Job completion use trackJobProgress)

Value parameters

query

Hive SQL query to run

Attributes

def submitSparkJob(args: List[String], mainClass: String, libs: List[String], conf: Map[String, String]): Task[Job]

Submits a Spark Job in Dataproc Cluster. (This API will not wait for Job Completion, to wait for Job completion use trackJobProgress)

Submits a Spark Job in Dataproc Cluster. (This API will not wait for Job Completion, to wait for Job completion use trackJobProgress)

Value parameters

args

command line arguments which will be passed to spark application

conf

Key value pair of the spark properties

libs

List of jar required to run this Spark Application (including application jar)

mainClass

Main class to run

Attributes

def trackJobProgress(job: Job, interval: Duration): Task[Unit]

This API will track job until Completion

This API will track job until Completion

Value parameters

interval

Specifies duration each status check repetition should be spaced from the last run

job

Dataproc Job which needs to be tracked

Attributes

Inherited methods

def productElementNames: Iterator[String]

Attributes

Inherited from:
Product
def productIterator: Iterator[Any]

Attributes

Inherited from:
Product
Generated with