Completes the batch of annotations.

Size of every batch (Default depends on model).

Closes the llama.cpp model backend freeing resources. The model is reloaded when used again.

requirement for annotators copies

This param is set internally once via loadSavedModel. That's why there is no setter

Override for additional custom schema checks

Size of every batch.

Get the metadata for the model

Gets annotation column name going to generate

Set how to split the model across GPUs

• NONE: No GPU split
• LAYER: Split the model across GPUs by layer
• ROW: Split the model across GPUs by rows

Annotator reference id. Used to identify elements in metadata or to refer to this annotator type

columns that contain annotations necessary to run this annotator AnnotatorType is used both as input and output columns if not specified

Set optimization strategies that help on some NUMA systems (if available)

Available Strategies:

• DISABLED: No NUMA optimizations
• DISTRIBUTE: Spread execution evenly over all
• ISOLATE: Only spawn threads on CPUs on the node that execution started on
• NUMA_CTL: Use the CPU map provided by numactl
• MIRROR: Mirrors the model across NUMA nodes

Set the RoPE frequency scaling method, defaults to linear unless specified by the model.

• UNSPECIFIED: Don't use any scaling
• LINEAR: Linear scaling
• YARN: YaRN RoPE scaling

Sets the value for a protected Param.

If the parameter was already set, it will not be set again. Default values do not count as a set value and can be overridden.

Size of every batch.

Whether to remember the prompt to avoid reprocessing it

The chat template to use

Set the KV cache defragmentation threshold

Set the token ids to disable in the completion. This corresponds to setTokenBias with a value of Float.NEGATIVE_INFINITY.

Set the dynamic temperature exponent

Set the dynamic temperature range

Whether to enable Flash Attention

Set the repetition alpha frequency penalty

Set how to split the model across GPUs

• NONE: No GPU split -LAYER: Split the model across GPUs by layer 2. ROW: Split the model across GPUs by rows

Set BNF-like grammar to constrain generations

Set whether to ignore end of stream token and continue generating (implies --logit-bias 2-inf)

Overrides required annotators column if different than default

Set the prompt to start generation with

Set a suffix for infilling

Set the verbosity threshold. Messages with a higher verbosity will be ignored.

Values map to the following:

• GGML_LOG_LEVEL_NONE = 0
• GGML_LOG_LEVEL_DEBUG = 1
• GGML_LOG_LEVEL_INFO = 2
• GGML_LOG_LEVEL_WARN = 3
• GGML_LOG_LEVEL_ERROR = 4
• GGML_LOG_LEVEL_CONT = 5 (continue previous log)

Set the GPU that is used for scratch and small tensors

Set the metadata for the model

Set the amount of tokens the samplers should return at least (0 = disabled)

Set min-p sampling

Set MiroStat sampling strategies.

• DISABLED: No MiroStat
• V1: MiroStat V1
• V2: MiroStat V2

Set the MiroStat learning rate, parameter eta

Set the MiroStat target entropy, parameter tau

Set the draft model for speculative decoding

Set the logical batch size for prompt processing (must be >=32 to use BLAS)

Set the size of the prompt context

Set the number of tokens to draft for speculative decoding

Set the number of layers to store in VRAM (-1 - use default)

Set the number of layers to store in VRAM for the draft model (-1 - use default)

Set the number of tokens to keep from the initial prompt

Sets the number of parallel processes for decoding. This is an alias for setBatchSize.

Set the number of tokens to predict

Set the amount top tokens probabilities to output if greater than 0.

Set the number of threads to use during generation

Set the number of threads to use during batch and prompt processing

Set the physical batch size for prompt processing (must be >=32 to use BLAS)

Whether to disable KV offload

Set optimization strategies that help on some NUMA systems (if available)

Available Strategies:

• DISABLED: No NUMA optimizations
• DISTRIBUTE: spread execution evenly over all
• ISOLATE: only spawn threads on CPUs on the node that execution started on
• NUMA_CTL: use the CPU map provided by numactl
• MIRROR: Mirrors the model across NUMA nodes

Overrides annotation column name when transforming

Set whether to penalize newline tokens

Override which part of the prompt is penalized for repetition.

Set the repetition alpha presence penalty

Set a thinking tag (e.g. think) to be removed from output. Will produce the regex (?s)<$TAG>.+?</$TAG>

Set the last n tokens to consider for penalties

Set the penalty of repeated sequences of tokens

Set the RoPE base frequency, used by NTK-aware scaling

Set the RoPE frequency scaling factor, expands context by a factor of 1/N

Set the RoPE frequency scaling method, defaults to linear unless specified by the model.

• NONE: Don't use any scaling
• LINEAR: Linear scaling
• YARN: YaRN RoPE scaling

Set which samplers to use for token generation in the given order .

Available Samplers are:

• TOP_K: Top-k sampling
• TFS_Z: Tail free sampling
• TYPICAL_P: Locally typical sampling p
• TOP_P: Top-p sampling
• MIN_P: Min-p sampling
• TEMPERATURE: Temperature sampling

Set the RNG seed

Set strings upon seeing which token generation is stopped

Set a system prompt to use

Set the temperature

Set tail free sampling, parameter z

Set the tokens to disable during completion. (Override for PySpark)

Set the tokens to disable during completion.

Set the token ids to disable in the completion. (Override for PySpark)

Set the token ids to disable in the completion.

Set top-k sampling

Set top-p sampling

Set locally typical sampling, parameter p

Set whether or not generate should apply a chat template

Whether to force the system to keep model in RAM rather than swapping or compressing

Whether to use memory-map model (faster load but may increase pageouts if not using mlock)

Set the YaRN scale sqrt(t) or attention magnitude

Set the YaRN low correction dim or beta

Set the YaRN high correction dim or alpha

Set the YaRN extrapolation mix factor

Set the YaRN original context size of model

Given requirements are met, this applies ML transformation within a Pipeline or stand-alone Output annotation will be generated as a new column, previous annotations are still available separately metadata is built at schema level to record annotations structural information outside its content

requirement for pipeline transformation validation. It is called on fit()

takes a Dataset and checks to see if all the required annotation types are present.