org.apache.flink.runtime.minicluster

Class MiniCluster

java.lang.Object

org.apache.flink.runtime.minicluster.MiniCluster

All Implemented Interfaces:

AutoCloseable, JobExecutor, JobExecutorService, org.apache.flink.util.AutoCloseableAsync

public class MiniCluster
extends Object
implements JobExecutorService, org.apache.flink.util.AutoCloseableAsync

MiniCluster to execute Flink jobs locally.

Constructor Summary

Constructors

Constructor and Description
MiniCluster(MiniClusterConfiguration miniClusterConfiguration) Creates a new Flink mini cluster based on the given configuration.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Acknowledge>	cancelJob(org.apache.flink.api.common.JobID jobId)
CompletableFuture<Void>	closeAsync() Shuts down the mini cluster, failing all currently executing jobs.
protected MetricRegistryImpl	createMetricRegistry(org.apache.flink.configuration.Configuration config) Factory method to create the metric registry for the mini cluster.
protected RpcService	createRpcService(org.apache.flink.configuration.Configuration configuration, org.apache.flink.api.common.time.Time askTimeout, boolean remoteEnabled, String bindAddress) Factory method to instantiate the RPC service.
CompletableFuture<Acknowledge>	disposeSavepoint(String savepointPath)
org.apache.flink.api.common.JobExecutionResult	executeJobBlocking(JobGraph job) This method runs a job in blocking mode.
CompletableFuture<? extends AccessExecutionGraph>	getExecutionGraph(org.apache.flink.api.common.JobID jobId)
HighAvailabilityServices	getHighAvailabilityServices()
CompletableFuture<JobStatus>	getJobStatus(org.apache.flink.api.common.JobID jobId)
URI	getRestAddress()
boolean	isRunning() Checks if the mini cluster was started and is running.
CompletableFuture<Collection<JobStatusMessage>>	listJobs()
CompletableFuture<JobResult>	requestJobResult(org.apache.flink.api.common.JobID jobId)
void	runDetached(JobGraph job) This method executes a job in detached mode.
void	start() Starts the mini cluster, based on the configured properties.
protected ResourceManagerRunner	startResourceManager(org.apache.flink.configuration.Configuration configuration, HighAvailabilityServices haServices, HeartbeatServices heartbeatServices, MetricRegistry metricRegistry, RpcService resourceManagerRpcService, ClusterInformation clusterInformation)
protected TaskExecutor[]	startTaskManagers(org.apache.flink.configuration.Configuration configuration, HighAvailabilityServices haServices, HeartbeatServices heartbeatServices, MetricRegistry metricRegistry, BlobCacheService blobCacheService, int numTaskManagers, RpcService[] taskManagerRpcServices)
CompletableFuture<Acknowledge>	stopJob(org.apache.flink.api.common.JobID jobId)
CompletableFuture<org.apache.flink.api.common.JobSubmissionResult>	submitJob(JobGraph jobGraph)
CompletableFuture<String>	triggerSavepoint(org.apache.flink.api.common.JobID jobId, String targetDirectory, boolean cancelJob)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Constructor Detail

MiniCluster

public MiniCluster(MiniClusterConfiguration miniClusterConfiguration)

Creates a new Flink mini cluster based on the given configuration.

Parameters:

miniClusterConfiguration - The configuration for the mini cluster

Method Detail

getRestAddress

public URI getRestAddress()

getHighAvailabilityServices

public HighAvailabilityServices getHighAvailabilityServices()

isRunning

public boolean isRunning()

Checks if the mini cluster was started and is running.

start

public void start()
           throws Exception

Starts the mini cluster, based on the configured properties.

Throws:

Exception - This method passes on any exception that occurs during the startup of the mini cluster.

closeAsync

public CompletableFuture<Void> closeAsync()

Shuts down the mini cluster, failing all currently executing jobs. The mini cluster can be started again by calling the start() method again.

This method shuts down all started services and components, even if an exception occurs in the process of shutting down some component.

Specified by:

closeAsync in interface org.apache.flink.util.AutoCloseableAsync

Returns:

Future which is completed once the MiniCluster has been completely shut down

listJobs

public CompletableFuture<Collection<JobStatusMessage>> listJobs()

getJobStatus

public CompletableFuture<JobStatus> getJobStatus(org.apache.flink.api.common.JobID jobId)

cancelJob

public CompletableFuture<Acknowledge> cancelJob(org.apache.flink.api.common.JobID jobId)

stopJob

public CompletableFuture<Acknowledge> stopJob(org.apache.flink.api.common.JobID jobId)

triggerSavepoint

public CompletableFuture<String> triggerSavepoint(org.apache.flink.api.common.JobID jobId,
                                                  String targetDirectory,
                                                  boolean cancelJob)

disposeSavepoint

public CompletableFuture<Acknowledge> disposeSavepoint(String savepointPath)

getExecutionGraph

public CompletableFuture<? extends AccessExecutionGraph> getExecutionGraph(org.apache.flink.api.common.JobID jobId)

runDetached

public void runDetached(JobGraph job)
                 throws JobExecutionException,
                        InterruptedException

This method executes a job in detached mode. The method returns immediately after the job has been added to the

Parameters:

job - The Flink job to execute

Throws:

JobExecutionException - Thrown if anything went amiss during initial job launch, or if the job terminally failed.

InterruptedException

executeJobBlocking

public org.apache.flink.api.common.JobExecutionResult executeJobBlocking(JobGraph job)
                                                                  throws JobExecutionException,
                                                                         InterruptedException

This method runs a job in blocking mode. The method returns only after the job completed successfully, or after it failed terminally.

Specified by:

executeJobBlocking in interface JobExecutor

Parameters:

job - The Flink job to execute

Returns:

The result of the job execution

Throws:

JobExecutionException - Thrown if anything went amiss during initial job launch, or if the job terminally failed.

InterruptedException

submitJob

public CompletableFuture<org.apache.flink.api.common.JobSubmissionResult> submitJob(JobGraph jobGraph)

requestJobResult

public CompletableFuture<JobResult> requestJobResult(org.apache.flink.api.common.JobID jobId)

createMetricRegistry

protected MetricRegistryImpl createMetricRegistry(org.apache.flink.configuration.Configuration config)

Factory method to create the metric registry for the mini cluster.

Parameters:

config - The configuration of the mini cluster

createRpcService

protected RpcService createRpcService(org.apache.flink.configuration.Configuration configuration,
                                      org.apache.flink.api.common.time.Time askTimeout,
                                      boolean remoteEnabled,
                                      String bindAddress)

Factory method to instantiate the RPC service.

Parameters:

configuration - The configuration of the mini cluster

askTimeout - The default RPC timeout for asynchronous "ask" requests.

remoteEnabled - True, if the RPC service should be reachable from other (remote) RPC services.

bindAddress - The address to bind the RPC service to. Only relevant when "remoteEnabled" is true.

Returns:

The instantiated RPC service

startResourceManager

protected ResourceManagerRunner startResourceManager(org.apache.flink.configuration.Configuration configuration,
                                                     HighAvailabilityServices haServices,
                                                     HeartbeatServices heartbeatServices,
                                                     MetricRegistry metricRegistry,
                                                     RpcService resourceManagerRpcService,
                                                     ClusterInformation clusterInformation)
                                              throws Exception

Throws:

Exception

startTaskManagers

protected TaskExecutor[] startTaskManagers(org.apache.flink.configuration.Configuration configuration,
                                           HighAvailabilityServices haServices,
                                           HeartbeatServices heartbeatServices,
                                           MetricRegistry metricRegistry,
                                           BlobCacheService blobCacheService,
                                           int numTaskManagers,
                                           RpcService[] taskManagerRpcServices)
                                    throws Exception

Throws:

Exception