Package org.apache.flink.configuration

Class TaskManagerOptions

  • @PublicEvolving
public class TaskManagerOptions
extends Object
    The set of configuration options relating to TaskManager and Task settings.

    • Field Detail

      • MANAGED_MEMORY_CONSUMER_NAME_OPERATOR

        public static final String MANAGED_MEMORY_CONSUMER_NAME_OPERATOR
        See Also:
        Constant Field Values

      • MANAGED_MEMORY_CONSUMER_NAME_STATE_BACKEND

        public static final String MANAGED_MEMORY_CONSUMER_NAME_STATE_BACKEND
        See Also:
        Constant Field Values

      • KILL_ON_OUT_OF_MEMORY

        public static final ConfigOption<Boolean> KILL_ON_OUT_OF_MEMORY
        Whether to kill the TaskManager when the task thread throws an OutOfMemoryError.

      • HOST

        public static final ConfigOption<String> HOST
        The external address of the network interface where the TaskManager is exposed. Overrides HOST_BIND_POLICY automatic address binding.

      • BIND_HOST

        public static final ConfigOption<String> BIND_HOST
        The local address of the network interface that the task manager binds to.

      • RPC_PORT

        public static final ConfigOption<String> RPC_PORT
        The default network port range the task manager expects incoming IPC connections. The "0" means that the TaskManager searches for a free port.

      • RPC_BIND_PORT

        public static final ConfigOption<Integer> RPC_BIND_PORT
        The local port that the task manager binds to.

      • COLLECT_PORT

        public static final ConfigOption<Integer> COLLECT_PORT
        The default port that CollectSinkFunction$ServerThread is using.

      • REGISTRATION_TIMEOUT

        public static final ConfigOption<Duration> REGISTRATION_TIMEOUT
        Defines the timeout it can take for the TaskManager registration. If the duration is exceeded without a successful registration, then the TaskManager terminates.

      • NUM_TASK_SLOTS

        public static final ConfigOption<Integer> NUM_TASK_SLOTS
        The config parameter defining the number of task slots of a task manager.

      • SLOT_TIMEOUT

        public static final ConfigOption<Duration> SLOT_TIMEOUT
        Timeout for identifying inactive slots.

      • DEBUG_MEMORY_USAGE_LOG_INTERVAL_MS

        public static final ConfigOption<Duration> DEBUG_MEMORY_USAGE_LOG_INTERVAL_MS

      • MEMORY_SEGMENT_SIZE

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> MEMORY_SEGMENT_SIZE
        Size of memory buffers used by the network stack and the memory manager.

      • MIN_MEMORY_SEGMENT_SIZE

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> MIN_MEMORY_SEGMENT_SIZE
        Minimum possible size of memory buffers used by the network stack and the memory manager.

      • HOST_BIND_POLICY

        public static final ConfigOption<String> HOST_BIND_POLICY
        The config parameter for automatically defining the TaskManager's binding address, if HOST configuration option is not set.

      • TASK_MANAGER_RESOURCE_ID

        public static final ConfigOption<String> TASK_MANAGER_RESOURCE_ID
        The TaskManager's ResourceID. If not configured, the ResourceID will be generated with the RpcAddress:RpcPort and a 6-character random string. Notice that this option is not valid in Yarn and Native Kubernetes mode.

      • CPU_CORES

        public static final ConfigOption<Double> CPU_CORES
        This config option describes number of cpu cores of task executors. In case of Yarn / Kubernetes, it is used to launch a container for the task executor.

        DO NOT USE THIS CONFIG OPTION. This config option is currently only used internally, for passing cpu cores into task executors for dynamic fine grained slot resource management. The feature is not completed at the moment, and the config option is experimental and might be changed / removed in the future. Thus, we do not expose this config option to users.

        For configuring the cpu cores of container on Yarn / Kubernetes, please use YarnConfigOptions#VCORES and KubernetesConfigOptions#TASK_MANAGER_CPU.

      • TOTAL_PROCESS_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> TOTAL_PROCESS_MEMORY
        Total Process Memory size for the TaskExecutors.

      • TOTAL_FLINK_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> TOTAL_FLINK_MEMORY
        Total Flink Memory size for the TaskExecutors.

      • FRAMEWORK_HEAP_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> FRAMEWORK_HEAP_MEMORY
        Framework Heap Memory size for TaskExecutors.

      • FRAMEWORK_OFF_HEAP_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> FRAMEWORK_OFF_HEAP_MEMORY
        Framework Off-Heap Memory size for TaskExecutors.

      • TASK_HEAP_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> TASK_HEAP_MEMORY
        Task Heap Memory size for TaskExecutors.

      • TASK_OFF_HEAP_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> TASK_OFF_HEAP_MEMORY
        Task Off-Heap Memory size for TaskExecutors.

      • MANAGED_MEMORY_SIZE

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> MANAGED_MEMORY_SIZE
        Managed Memory size for TaskExecutors.

      • MANAGED_MEMORY_FRACTION

        public static final ConfigOption<Float> MANAGED_MEMORY_FRACTION
        Fraction of Total Flink Memory to be used as Managed Memory, if MANAGED_MEMORY_SIZE is not specified.

      • MANAGED_MEMORY_CONSUMER_WEIGHTS

        public static final ConfigOption<Map<String,​String>> MANAGED_MEMORY_CONSUMER_WEIGHTS
        Weights of managed memory consumers.

      • NETWORK_MEMORY_MIN

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> NETWORK_MEMORY_MIN
        Min Network Memory size for TaskExecutors.

      • NETWORK_MEMORY_MAX

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> NETWORK_MEMORY_MAX
        Max Network Memory size for TaskExecutors.

      • NETWORK_MEMORY_FRACTION

        public static final ConfigOption<Float> NETWORK_MEMORY_FRACTION
        Fraction of Total Flink Memory to be used as Network Memory.

      • BUFFER_DEBLOAT_PERIOD

        public static final ConfigOption<Duration> BUFFER_DEBLOAT_PERIOD
        The period between recalculation the relevant size of the buffer.

      • BUFFER_DEBLOAT_SAMPLES

        public static final ConfigOption<Integer> BUFFER_DEBLOAT_SAMPLES
        The number of samples requires for the buffer size adjustment.

      • BUFFER_DEBLOAT_TARGET

        public static final ConfigOption<Duration> BUFFER_DEBLOAT_TARGET
        The total time for which automated adjusted buffers should be fully consumed.

      • BUFFER_DEBLOAT_THRESHOLD_PERCENTAGES

        public static final ConfigOption<Integer> BUFFER_DEBLOAT_THRESHOLD_PERCENTAGES
        Difference between the new and the old buffer size for applying the new value(in percent).

      • NETWORK_BATCH_SHUFFLE_READ_MEMORY

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> NETWORK_BATCH_SHUFFLE_READ_MEMORY
        Size of direct memory used by batch shuffle for shuffle data read (currently only used by sort-shuffle and hybrid shuffle).

      • JVM_METASPACE

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> JVM_METASPACE
        JVM Metaspace Size for the TaskExecutors.

      • JVM_OVERHEAD_MIN

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> JVM_OVERHEAD_MIN
        Min JVM Overhead size for the TaskExecutors.

      • JVM_OVERHEAD_MAX

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> JVM_OVERHEAD_MAX
        Max JVM Overhead size for the TaskExecutors.

      • JVM_OVERHEAD_FRACTION

        public static final ConfigOption<Float> JVM_OVERHEAD_FRACTION
        Fraction of Total Process Memory to be reserved for JVM Overhead.

      • TASK_CANCELLATION_INTERVAL

        public static final ConfigOption<Duration> TASK_CANCELLATION_INTERVAL
        Time interval in milliseconds between two successive task cancellation attempts.

      • TASK_CANCELLATION_TIMEOUT

        public static final ConfigOption<Duration> TASK_CANCELLATION_TIMEOUT
        Timeout in milliseconds after which a task cancellation times out and leads to a fatal TaskManager error. A value of 0 deactivates the watch dog. Notice that a task cancellation is different from both a task failure and a clean shutdown. Task cancellation timeout only applies to task cancellation and does not apply to task closing/clean-up caused by a task failure or a clean shutdown.

      • TASK_CANCELLATION_TIMEOUT_TIMERS

        public static final ConfigOption<Duration> TASK_CANCELLATION_TIMEOUT_TIMERS
        This configures how long we wait for the timers in milliseconds to finish all pending timer threads when the stream task is cancelled.

      • TASK_MANAGER_SYSTEM_OUT_LOG_THREAD_NAME

        public static final ConfigOption<Boolean> TASK_MANAGER_SYSTEM_OUT_LOG_THREAD_NAME

      • TASK_MANAGER_SYSTEM_OUT_LOG_CACHE_SIZE

        public static final ConfigOption<org.apache.flink.configuration.MemorySize> TASK_MANAGER_SYSTEM_OUT_LOG_CACHE_SIZE

      • MINI_CLUSTER_NUM_TASK_MANAGERS

        public static final ConfigOption<Integer> MINI_CLUSTER_NUM_TASK_MANAGERS