Interface ModelProvider.Context

  • Enclosing interface:
    ModelProvider

    @PublicEvolving
    public static interface ModelProvider.Context
    Context for creating runtime providers.
    • Method Detail

      • runtimeConfig

        org.apache.flink.configuration.ReadableConfig runtimeConfig()
        Runtime config provided to provider. The config can be used by planner or model provider at runtime. For example, async options can be used by planner to choose async inference. Other config such as http timeout or retry can be used to configure model provider runtime http client when calling external model providers such as OpenAI.