package proto
Type Members
- final class AddArtifactsRequest extends GeneratedMessage with AddArtifactsRequestOrBuilder
Request to transfer client-local artifacts.
Request to transfer client-local artifacts.
Protobuf type
spark.connect.AddArtifactsRequest
- trait AddArtifactsRequestOrBuilder extends MessageOrBuilder
- final class AddArtifactsResponse extends GeneratedMessage with AddArtifactsResponseOrBuilder
Response to adding an artifact. Contains relevant metadata to verify successful transfer of artifact(s). Next ID: 4
Response to adding an artifact. Contains relevant metadata to verify successful transfer of artifact(s). Next ID: 4
Protobuf type
spark.connect.AddArtifactsResponse
- trait AddArtifactsResponseOrBuilder extends MessageOrBuilder
- final class Aggregate extends GeneratedMessage with AggregateOrBuilder
Relation of type [[Aggregate]].
Relation of type [[Aggregate]].
Protobuf type
spark.connect.Aggregate
- trait AggregateOrBuilder extends MessageOrBuilder
- final class AnalyzePlanRequest extends GeneratedMessage with AnalyzePlanRequestOrBuilder
Request to perform plan analyze, optionally to explain the plan.
Request to perform plan analyze, optionally to explain the plan.
Protobuf type
spark.connect.AnalyzePlanRequest
- trait AnalyzePlanRequestOrBuilder extends MessageOrBuilder
- final class AnalyzePlanResponse extends GeneratedMessage with AnalyzePlanResponseOrBuilder
Response to performing analysis of the query. Contains relevant metadata to be able to reason about the performance. Next ID: 16
Response to performing analysis of the query. Contains relevant metadata to be able to reason about the performance. Next ID: 16
Protobuf type
spark.connect.AnalyzePlanResponse
- trait AnalyzePlanResponseOrBuilder extends MessageOrBuilder
- final class ApplyInPandasWithState extends GeneratedMessage with ApplyInPandasWithStateOrBuilder
Protobuf type
spark.connect.ApplyInPandasWithState
- trait ApplyInPandasWithStateOrBuilder extends MessageOrBuilder
- final class ArtifactStatusesRequest extends GeneratedMessage with ArtifactStatusesRequestOrBuilder
Request to get current statuses of artifacts at the server side.
Request to get current statuses of artifacts at the server side.
Protobuf type
spark.connect.ArtifactStatusesRequest
- trait ArtifactStatusesRequestOrBuilder extends MessageOrBuilder
- final class ArtifactStatusesResponse extends GeneratedMessage with ArtifactStatusesResponseOrBuilder
Response to checking artifact statuses. Next ID: 4
Response to checking artifact statuses. Next ID: 4
Protobuf type
spark.connect.ArtifactStatusesResponse
- trait ArtifactStatusesResponseOrBuilder extends MessageOrBuilder
- final class AsOfJoin extends GeneratedMessage with AsOfJoinOrBuilder
Relation of type [[AsOfJoin]]. `left` and `right` must be present.
Relation of type [[AsOfJoin]]. `left` and `right` must be present.
Protobuf type
spark.connect.AsOfJoin
- trait AsOfJoinOrBuilder extends MessageOrBuilder
- final class Base extends AnyRef
- final class Bools extends GeneratedMessage with BoolsOrBuilder
Protobuf type
spark.connect.Bools
- trait BoolsOrBuilder extends MessageOrBuilder
- final class CacheTable extends GeneratedMessage with CacheTableOrBuilder
See `spark.catalog.cacheTable`
See `spark.catalog.cacheTable`
Protobuf type
spark.connect.CacheTable
- trait CacheTableOrBuilder extends MessageOrBuilder
- final class CachedLocalRelation extends GeneratedMessage with CachedLocalRelationOrBuilder
A local relation that has been cached already.
A local relation that has been cached already.
Protobuf type
spark.connect.CachedLocalRelation
- trait CachedLocalRelationOrBuilder extends MessageOrBuilder
- final class CachedRemoteRelation extends GeneratedMessage with CachedRemoteRelationOrBuilder
Represents a remote relation that has been cached on server.
Represents a remote relation that has been cached on server.
Protobuf type
spark.connect.CachedRemoteRelation
- trait CachedRemoteRelationOrBuilder extends MessageOrBuilder
- final class CallFunction extends GeneratedMessage with CallFunctionOrBuilder
Protobuf type
spark.connect.CallFunction
- trait CallFunctionOrBuilder extends MessageOrBuilder
- final class Catalog extends GeneratedMessage with CatalogOrBuilder
Catalog messages are marked as unstable.
Catalog messages are marked as unstable.
Protobuf type
spark.connect.Catalog
- trait CatalogOrBuilder extends MessageOrBuilder
- final class CatalogOuterClass extends AnyRef
- final class CheckpointCommand extends GeneratedMessage with CheckpointCommandOrBuilder
Protobuf type
spark.connect.CheckpointCommand
- trait CheckpointCommandOrBuilder extends MessageOrBuilder
- final class CheckpointCommandResult extends GeneratedMessage with CheckpointCommandResultOrBuilder
Protobuf type
spark.connect.CheckpointCommandResult
- trait CheckpointCommandResultOrBuilder extends MessageOrBuilder
- final class ClearCache extends GeneratedMessage with ClearCacheOrBuilder
See `spark.catalog.clearCache`
See `spark.catalog.clearCache`
Protobuf type
spark.connect.ClearCache
- trait ClearCacheOrBuilder extends MessageOrBuilder
- final class CoGroupMap extends GeneratedMessage with CoGroupMapOrBuilder
Protobuf type
spark.connect.CoGroupMap
- trait CoGroupMapOrBuilder extends MessageOrBuilder
- final class CollectMetrics extends GeneratedMessage with CollectMetricsOrBuilder
Collect arbitrary (named) metrics from a dataset.
Collect arbitrary (named) metrics from a dataset.
Protobuf type
spark.connect.CollectMetrics
- trait CollectMetricsOrBuilder extends MessageOrBuilder
- final class Command extends GeneratedMessage with CommandOrBuilder
A [[Command]] is an operation that is executed by the server that does not directly consume or produce a relational result.
A [[Command]] is an operation that is executed by the server that does not directly consume or produce a relational result.
Protobuf type
spark.connect.Command
- trait CommandOrBuilder extends MessageOrBuilder
- final class Commands extends AnyRef
- final class Common extends AnyRef
- final class CommonInlineUserDefinedDataSource extends GeneratedMessage with CommonInlineUserDefinedDataSourceOrBuilder
Protobuf type
spark.connect.CommonInlineUserDefinedDataSource
- trait CommonInlineUserDefinedDataSourceOrBuilder extends MessageOrBuilder
- final class CommonInlineUserDefinedFunction extends GeneratedMessage with CommonInlineUserDefinedFunctionOrBuilder
Protobuf type
spark.connect.CommonInlineUserDefinedFunction
- trait CommonInlineUserDefinedFunctionOrBuilder extends MessageOrBuilder
- final class CommonInlineUserDefinedTableFunction extends GeneratedMessage with CommonInlineUserDefinedTableFunctionOrBuilder
Protobuf type
spark.connect.CommonInlineUserDefinedTableFunction
- trait CommonInlineUserDefinedTableFunctionOrBuilder extends MessageOrBuilder
- final class ConfigRequest extends GeneratedMessage with ConfigRequestOrBuilder
Request to update or fetch the configurations.
Request to update or fetch the configurations.
Protobuf type
spark.connect.ConfigRequest
- trait ConfigRequestOrBuilder extends MessageOrBuilder
- final class ConfigResponse extends GeneratedMessage with ConfigResponseOrBuilder
Response to the config request. Next ID: 5
Response to the config request. Next ID: 5
Protobuf type
spark.connect.ConfigResponse
- trait ConfigResponseOrBuilder extends MessageOrBuilder
- final class CreateDataFrameViewCommand extends GeneratedMessage with CreateDataFrameViewCommandOrBuilder
A command that can create DataFrame global temp view or local temp view.
A command that can create DataFrame global temp view or local temp view.
Protobuf type
spark.connect.CreateDataFrameViewCommand
- trait CreateDataFrameViewCommandOrBuilder extends MessageOrBuilder
- final class CreateExternalTable extends GeneratedMessage with CreateExternalTableOrBuilder
See `spark.catalog.createExternalTable`
See `spark.catalog.createExternalTable`
Protobuf type
spark.connect.CreateExternalTable
- trait CreateExternalTableOrBuilder extends MessageOrBuilder
- final class CreateResourceProfileCommand extends GeneratedMessage with CreateResourceProfileCommandOrBuilder
Command to create ResourceProfile
Command to create ResourceProfile
Protobuf type
spark.connect.CreateResourceProfileCommand
- trait CreateResourceProfileCommandOrBuilder extends MessageOrBuilder
- final class CreateResourceProfileCommandResult extends GeneratedMessage with CreateResourceProfileCommandResultOrBuilder
Response for command 'CreateResourceProfileCommand'.
Response for command 'CreateResourceProfileCommand'.
Protobuf type
spark.connect.CreateResourceProfileCommandResult
- trait CreateResourceProfileCommandResultOrBuilder extends MessageOrBuilder
- final class CreateTable extends GeneratedMessage with CreateTableOrBuilder
See `spark.catalog.createTable`
See `spark.catalog.createTable`
Protobuf type
spark.connect.CreateTable
- trait CreateTableOrBuilder extends MessageOrBuilder
- final class CurrentCatalog extends GeneratedMessage with CurrentCatalogOrBuilder
See `spark.catalog.currentCatalog`
See `spark.catalog.currentCatalog`
Protobuf type
spark.connect.CurrentCatalog
- trait CurrentCatalogOrBuilder extends MessageOrBuilder
- final class CurrentDatabase extends GeneratedMessage with CurrentDatabaseOrBuilder
See `spark.catalog.currentDatabase`
See `spark.catalog.currentDatabase`
Protobuf type
spark.connect.CurrentDatabase
- trait CurrentDatabaseOrBuilder extends MessageOrBuilder
- final class DataType extends GeneratedMessage with DataTypeOrBuilder
This message describes the logical [[DataType]] of something. It does not carry the value itself but only describes it.
This message describes the logical [[DataType]] of something. It does not carry the value itself but only describes it.
Protobuf type
spark.connect.DataType
- trait DataTypeOrBuilder extends MessageOrBuilder
- final class DatabaseExists extends GeneratedMessage with DatabaseExistsOrBuilder
See `spark.catalog.databaseExists`
See `spark.catalog.databaseExists`
Protobuf type
spark.connect.DatabaseExists
- trait DatabaseExistsOrBuilder extends MessageOrBuilder
- final class Deduplicate extends GeneratedMessage with DeduplicateOrBuilder
Relation of type [[Deduplicate]] which have duplicate rows removed, could consider either only the subset of columns or all the columns.
Relation of type [[Deduplicate]] which have duplicate rows removed, could consider either only the subset of columns or all the columns.
Protobuf type
spark.connect.Deduplicate
- trait DeduplicateOrBuilder extends MessageOrBuilder
- final class Doubles extends GeneratedMessage with DoublesOrBuilder
Protobuf type
spark.connect.Doubles
- trait DoublesOrBuilder extends MessageOrBuilder
- final class Drop extends GeneratedMessage with DropOrBuilder
Drop specified columns.
Drop specified columns.
Protobuf type
spark.connect.Drop
- final class DropGlobalTempView extends GeneratedMessage with DropGlobalTempViewOrBuilder
See `spark.catalog.dropGlobalTempView`
See `spark.catalog.dropGlobalTempView`
Protobuf type
spark.connect.DropGlobalTempView
- trait DropGlobalTempViewOrBuilder extends MessageOrBuilder
- trait DropOrBuilder extends MessageOrBuilder
- final class DropTempView extends GeneratedMessage with DropTempViewOrBuilder
See `spark.catalog.dropTempView`
See `spark.catalog.dropTempView`
Protobuf type
spark.connect.DropTempView
- trait DropTempViewOrBuilder extends MessageOrBuilder
- final class ExamplePluginCommand extends GeneratedMessage with ExamplePluginCommandOrBuilder
Protobuf type
spark.connect.ExamplePluginCommand
- trait ExamplePluginCommandOrBuilder extends MessageOrBuilder
- final class ExamplePluginExpression extends GeneratedMessage with ExamplePluginExpressionOrBuilder
Protobuf type
spark.connect.ExamplePluginExpression
- trait ExamplePluginExpressionOrBuilder extends MessageOrBuilder
- final class ExamplePluginRelation extends GeneratedMessage with ExamplePluginRelationOrBuilder
Protobuf type
spark.connect.ExamplePluginRelation
- trait ExamplePluginRelationOrBuilder extends MessageOrBuilder
- final class ExamplePlugins extends AnyRef
- final class ExecuteExternalCommand extends GeneratedMessage with ExecuteExternalCommandOrBuilder
Execute an arbitrary string command inside an external execution engine
Execute an arbitrary string command inside an external execution engine
Protobuf type
spark.connect.ExecuteExternalCommand
- trait ExecuteExternalCommandOrBuilder extends MessageOrBuilder
- final class ExecutePlanRequest extends GeneratedMessage with ExecutePlanRequestOrBuilder
A request to be executed by the service.
A request to be executed by the service.
Protobuf type
spark.connect.ExecutePlanRequest
- trait ExecutePlanRequestOrBuilder extends MessageOrBuilder
- final class ExecutePlanResponse extends GeneratedMessage with ExecutePlanResponseOrBuilder
The response of a query, can be one or more for each request. Responses belonging to the same input query, carry the same `session_id`. Next ID: 17
The response of a query, can be one or more for each request. Responses belonging to the same input query, carry the same `session_id`. Next ID: 17
Protobuf type
spark.connect.ExecutePlanResponse
- trait ExecutePlanResponseOrBuilder extends MessageOrBuilder
- final class ExecutorResourceRequest extends GeneratedMessage with ExecutorResourceRequestOrBuilder
An executor resource request.
An executor resource request.
Protobuf type
spark.connect.ExecutorResourceRequest
- trait ExecutorResourceRequestOrBuilder extends MessageOrBuilder
- final class Expression extends GeneratedMessage with ExpressionOrBuilder
Expression used to refer to fields, functions and similar. This can be used everywhere expressions in SQL appear.
Expression used to refer to fields, functions and similar. This can be used everywhere expressions in SQL appear.
Protobuf type
spark.connect.Expression
- final class ExpressionCommon extends GeneratedMessage with ExpressionCommonOrBuilder
Protobuf type
spark.connect.ExpressionCommon
- trait ExpressionCommonOrBuilder extends MessageOrBuilder
- trait ExpressionOrBuilder extends MessageOrBuilder
- final class Expressions extends AnyRef
- final class Fetch extends GeneratedMessage with FetchOrBuilder
Message for fetching attribute from object on the server side. Fetch can be represented as a Relation or a ML command Command: model.coefficients, model.summary.weightedPrecision which returns the final literal result Relation: model.summary.roc which returns a DataFrame (Relation)
Message for fetching attribute from object on the server side. Fetch can be represented as a Relation or a ML command Command: model.coefficients, model.summary.weightedPrecision which returns the final literal result Relation: model.summary.roc which returns a DataFrame (Relation)
Protobuf type
spark.connect.Fetch
- final class FetchErrorDetailsRequest extends GeneratedMessage with FetchErrorDetailsRequestOrBuilder
Protobuf type
spark.connect.FetchErrorDetailsRequest
- trait FetchErrorDetailsRequestOrBuilder extends MessageOrBuilder
- final class FetchErrorDetailsResponse extends GeneratedMessage with FetchErrorDetailsResponseOrBuilder
Next ID: 5
Next ID: 5
Protobuf type
spark.connect.FetchErrorDetailsResponse
- trait FetchErrorDetailsResponseOrBuilder extends MessageOrBuilder
- trait FetchOrBuilder extends MessageOrBuilder
- final class Filter extends GeneratedMessage with FilterOrBuilder
Relation that applies a boolean expression `condition` on each row of `input` to produce the output result.
Relation that applies a boolean expression `condition` on each row of `input` to produce the output result.
Protobuf type
spark.connect.Filter
- trait FilterOrBuilder extends MessageOrBuilder
- final class Floats extends GeneratedMessage with FloatsOrBuilder
Protobuf type
spark.connect.Floats
- trait FloatsOrBuilder extends MessageOrBuilder
- final class FunctionExists extends GeneratedMessage with FunctionExistsOrBuilder
See `spark.catalog.functionExists`
See `spark.catalog.functionExists`
Protobuf type
spark.connect.FunctionExists
- trait FunctionExistsOrBuilder extends MessageOrBuilder
- final class GetDatabase extends GeneratedMessage with GetDatabaseOrBuilder
See `spark.catalog.getDatabase`
See `spark.catalog.getDatabase`
Protobuf type
spark.connect.GetDatabase
- trait GetDatabaseOrBuilder extends MessageOrBuilder
- final class GetFunction extends GeneratedMessage with GetFunctionOrBuilder
See `spark.catalog.getFunction`
See `spark.catalog.getFunction`
Protobuf type
spark.connect.GetFunction
- trait GetFunctionOrBuilder extends MessageOrBuilder
- final class GetResourcesCommand extends GeneratedMessage with GetResourcesCommandOrBuilder
Command to get the output of 'SparkContext.resources'
Command to get the output of 'SparkContext.resources'
Protobuf type
spark.connect.GetResourcesCommand
- trait GetResourcesCommandOrBuilder extends MessageOrBuilder
- final class GetResourcesCommandResult extends GeneratedMessage with GetResourcesCommandResultOrBuilder
Response for command 'GetResourcesCommand'.
Response for command 'GetResourcesCommand'.
Protobuf type
spark.connect.GetResourcesCommandResult
- trait GetResourcesCommandResultOrBuilder extends MessageOrBuilder
- final class GetTable extends GeneratedMessage with GetTableOrBuilder
See `spark.catalog.getTable`
See `spark.catalog.getTable`
Protobuf type
spark.connect.GetTable
- trait GetTableOrBuilder extends MessageOrBuilder
- final class GroupMap extends GeneratedMessage with GroupMapOrBuilder
Protobuf type
spark.connect.GroupMap
- trait GroupMapOrBuilder extends MessageOrBuilder
- final class Hint extends GeneratedMessage with HintOrBuilder
Specify a hint over a relation. Hint should have a name and optional parameters.
Specify a hint over a relation. Hint should have a name and optional parameters.
Protobuf type
spark.connect.Hint
- trait HintOrBuilder extends MessageOrBuilder
- final class HtmlString extends GeneratedMessage with HtmlStringOrBuilder
Compose the string representing rows for output. It will invoke 'Dataset.htmlString' to compute the results.
Compose the string representing rows for output. It will invoke 'Dataset.htmlString' to compute the results.
Protobuf type
spark.connect.HtmlString
- trait HtmlStringOrBuilder extends MessageOrBuilder
- final class InterruptRequest extends GeneratedMessage with InterruptRequestOrBuilder
Protobuf type
spark.connect.InterruptRequest
- trait InterruptRequestOrBuilder extends MessageOrBuilder
- final class InterruptResponse extends GeneratedMessage with InterruptResponseOrBuilder
Next ID: 4
Next ID: 4
Protobuf type
spark.connect.InterruptResponse
- trait InterruptResponseOrBuilder extends MessageOrBuilder
- final class Ints extends GeneratedMessage with IntsOrBuilder
Protobuf type
spark.connect.Ints
- trait IntsOrBuilder extends MessageOrBuilder
- final class IsCached extends GeneratedMessage with IsCachedOrBuilder
See `spark.catalog.isCached`
See `spark.catalog.isCached`
Protobuf type
spark.connect.IsCached
- trait IsCachedOrBuilder extends MessageOrBuilder
- final class JavaUDF extends GeneratedMessage with JavaUDFOrBuilder
Protobuf type
spark.connect.JavaUDF
- trait JavaUDFOrBuilder extends MessageOrBuilder
- final class Join extends GeneratedMessage with JoinOrBuilder
Relation of type [[Join]]. `left` and `right` must be present.
Relation of type [[Join]]. `left` and `right` must be present.
Protobuf type
spark.connect.Join
- trait JoinOrBuilder extends MessageOrBuilder
- final class JvmOrigin extends GeneratedMessage with JvmOriginOrBuilder
Protobuf type
spark.connect.JvmOrigin
- trait JvmOriginOrBuilder extends MessageOrBuilder
- final class KeyValue extends GeneratedMessage with KeyValueOrBuilder
The key-value pair for the config request and response.
The key-value pair for the config request and response.
Protobuf type
spark.connect.KeyValue
- trait KeyValueOrBuilder extends MessageOrBuilder
- final class LateralJoin extends GeneratedMessage with LateralJoinOrBuilder
Relation of type [[LateralJoin]]. `left` and `right` must be present.
Relation of type [[LateralJoin]]. `left` and `right` must be present.
Protobuf type
spark.connect.LateralJoin
- trait LateralJoinOrBuilder extends MessageOrBuilder
- final class Limit extends GeneratedMessage with LimitOrBuilder
Relation of type [[Limit]] that is used to `limit` rows from the input relation.
Relation of type [[Limit]] that is used to `limit` rows from the input relation.
Protobuf type
spark.connect.Limit
- trait LimitOrBuilder extends MessageOrBuilder
- final class ListCatalogs extends GeneratedMessage with ListCatalogsOrBuilder
See `spark.catalog.listCatalogs`
See `spark.catalog.listCatalogs`
Protobuf type
spark.connect.ListCatalogs
- trait ListCatalogsOrBuilder extends MessageOrBuilder
- final class ListColumns extends GeneratedMessage with ListColumnsOrBuilder
See `spark.catalog.listColumns`
See `spark.catalog.listColumns`
Protobuf type
spark.connect.ListColumns
- trait ListColumnsOrBuilder extends MessageOrBuilder
- final class ListDatabases extends GeneratedMessage with ListDatabasesOrBuilder
See `spark.catalog.listDatabases`
See `spark.catalog.listDatabases`
Protobuf type
spark.connect.ListDatabases
- trait ListDatabasesOrBuilder extends MessageOrBuilder
- final class ListFunctions extends GeneratedMessage with ListFunctionsOrBuilder
See `spark.catalog.listFunctions`
See `spark.catalog.listFunctions`
Protobuf type
spark.connect.ListFunctions
- trait ListFunctionsOrBuilder extends MessageOrBuilder
- final class ListTables extends GeneratedMessage with ListTablesOrBuilder
See `spark.catalog.listTables`
See `spark.catalog.listTables`
Protobuf type
spark.connect.ListTables
- trait ListTablesOrBuilder extends MessageOrBuilder
- final class LocalRelation extends GeneratedMessage with LocalRelationOrBuilder
A relation that does not need to be qualified by name.
A relation that does not need to be qualified by name.
Protobuf type
spark.connect.LocalRelation
- trait LocalRelationOrBuilder extends MessageOrBuilder
- final class Longs extends GeneratedMessage with LongsOrBuilder
Protobuf type
spark.connect.Longs
- trait LongsOrBuilder extends MessageOrBuilder
- final class MapPartitions extends GeneratedMessage with MapPartitionsOrBuilder
Protobuf type
spark.connect.MapPartitions
- trait MapPartitionsOrBuilder extends MessageOrBuilder
- final class MergeAction extends GeneratedMessage with MergeActionOrBuilder
Protobuf type
spark.connect.MergeAction
- trait MergeActionOrBuilder extends MessageOrBuilder
- final class MergeIntoTableCommand extends GeneratedMessage with MergeIntoTableCommandOrBuilder
Protobuf type
spark.connect.MergeIntoTableCommand
- trait MergeIntoTableCommandOrBuilder extends MessageOrBuilder
- final class Ml extends AnyRef
- final class MlCommand extends GeneratedMessage with MlCommandOrBuilder
Command for ML
Command for ML
Protobuf type
spark.connect.MlCommand
- trait MlCommandOrBuilder extends MessageOrBuilder
- final class MlCommandResult extends GeneratedMessage with MlCommandResultOrBuilder
The result of MlCommand
The result of MlCommand
Protobuf type
spark.connect.MlCommandResult
- trait MlCommandResultOrBuilder extends MessageOrBuilder
- final class MlCommon extends AnyRef
- final class MlOperator extends GeneratedMessage with MlOperatorOrBuilder
MLOperator represents the ML operators like (Estimator, Transformer or Evaluator)
MLOperator represents the ML operators like (Estimator, Transformer or Evaluator)
Protobuf type
spark.connect.MlOperator
- trait MlOperatorOrBuilder extends MessageOrBuilder
- final class MlParams extends GeneratedMessage with MlParamsOrBuilder
MlParams stores param settings for ML Estimator / Transformer / Evaluator
MlParams stores param settings for ML Estimator / Transformer / Evaluator
Protobuf type
spark.connect.MlParams
- trait MlParamsOrBuilder extends MessageOrBuilder
- final class MlRelation extends GeneratedMessage with MlRelationOrBuilder
Relation to represent ML world
Relation to represent ML world
Protobuf type
spark.connect.MlRelation
- trait MlRelationOrBuilder extends MessageOrBuilder
- final class NADrop extends GeneratedMessage with NADropOrBuilder
Drop rows containing null values. It will invoke 'Dataset.na.drop' (same as 'DataFrameNaFunctions.drop') to compute the results.
Drop rows containing null values. It will invoke 'Dataset.na.drop' (same as 'DataFrameNaFunctions.drop') to compute the results.
Protobuf type
spark.connect.NADrop
- trait NADropOrBuilder extends MessageOrBuilder
- final class NAFill extends GeneratedMessage with NAFillOrBuilder
Replaces null values. It will invoke 'Dataset.na.fill' (same as 'DataFrameNaFunctions.fill') to compute the results. Following 3 parameter combinations are supported: 1, 'values' only contains 1 item, 'cols' is empty: replaces null values in all type-compatible columns. 2, 'values' only contains 1 item, 'cols' is not empty: replaces null values in specified columns. 3, 'values' contains more than 1 items, then 'cols' is required to have the same length: replaces each specified column with corresponding value.
Replaces null values. It will invoke 'Dataset.na.fill' (same as 'DataFrameNaFunctions.fill') to compute the results. Following 3 parameter combinations are supported: 1, 'values' only contains 1 item, 'cols' is empty: replaces null values in all type-compatible columns. 2, 'values' only contains 1 item, 'cols' is not empty: replaces null values in specified columns. 3, 'values' contains more than 1 items, then 'cols' is required to have the same length: replaces each specified column with corresponding value.
Protobuf type
spark.connect.NAFill
- trait NAFillOrBuilder extends MessageOrBuilder
- final class NAReplace extends GeneratedMessage with NAReplaceOrBuilder
Replaces old values with the corresponding values. It will invoke 'Dataset.na.replace' (same as 'DataFrameNaFunctions.replace') to compute the results.
Replaces old values with the corresponding values. It will invoke 'Dataset.na.replace' (same as 'DataFrameNaFunctions.replace') to compute the results.
Protobuf type
spark.connect.NAReplace
- trait NAReplaceOrBuilder extends MessageOrBuilder
- final class NamedArgumentExpression extends GeneratedMessage with NamedArgumentExpressionOrBuilder
Protobuf type
spark.connect.NamedArgumentExpression
- trait NamedArgumentExpressionOrBuilder extends MessageOrBuilder
- final class ObjectRef extends GeneratedMessage with ObjectRefOrBuilder
Represents a reference to the cached object which could be a model or summary evaluated by a model
Represents a reference to the cached object which could be a model or summary evaluated by a model
Protobuf type
spark.connect.ObjectRef
- trait ObjectRefOrBuilder extends MessageOrBuilder
- final class Offset extends GeneratedMessage with OffsetOrBuilder
Relation of type [[Offset]] that is used to read rows staring from the `offset` on the input relation.
Relation of type [[Offset]] that is used to read rows staring from the `offset` on the input relation.
Protobuf type
spark.connect.Offset
- trait OffsetOrBuilder extends MessageOrBuilder
- final class Origin extends GeneratedMessage with OriginOrBuilder
Protobuf type
spark.connect.Origin
- trait OriginOrBuilder extends MessageOrBuilder
- final class Parse extends GeneratedMessage with ParseOrBuilder
Protobuf type
spark.connect.Parse
- trait ParseOrBuilder extends MessageOrBuilder
- final class Plan extends GeneratedMessage with PlanOrBuilder
A [[Plan]] is the structure that carries the runtime information for the execution from the client to the server. A [[Plan]] can either be of the type [[Relation]] which is a reference to the underlying logical plan or it can be of the [[Command]] type that is used to execute commands on the server.
A [[Plan]] is the structure that carries the runtime information for the execution from the client to the server. A [[Plan]] can either be of the type [[Relation]] which is a reference to the underlying logical plan or it can be of the [[Command]] type that is used to execute commands on the server.
Protobuf type
spark.connect.Plan
- trait PlanOrBuilder extends MessageOrBuilder
- final class Project extends GeneratedMessage with ProjectOrBuilder
Projection of a bag of expressions for a given input relation. The input relation must be specified. The projected expression can be an arbitrary expression.
Projection of a bag of expressions for a given input relation. The input relation must be specified. The projected expression can be an arbitrary expression.
Protobuf type
spark.connect.Project
- trait ProjectOrBuilder extends MessageOrBuilder
- final class PythonDataSource extends GeneratedMessage with PythonDataSourceOrBuilder
Protobuf type
spark.connect.PythonDataSource
- trait PythonDataSourceOrBuilder extends MessageOrBuilder
- final class PythonOrigin extends GeneratedMessage with PythonOriginOrBuilder
Protobuf type
spark.connect.PythonOrigin
- trait PythonOriginOrBuilder extends MessageOrBuilder
- final class PythonUDF extends GeneratedMessage with PythonUDFOrBuilder
Protobuf type
spark.connect.PythonUDF
- trait PythonUDFOrBuilder extends MessageOrBuilder
- final class PythonUDTF extends GeneratedMessage with PythonUDTFOrBuilder
Protobuf type
spark.connect.PythonUDTF
- trait PythonUDTFOrBuilder extends MessageOrBuilder
- final class Range extends GeneratedMessage with RangeOrBuilder
Relation of type [[Range]] that generates a sequence of integers.
Relation of type [[Range]] that generates a sequence of integers.
Protobuf type
spark.connect.Range
- trait RangeOrBuilder extends MessageOrBuilder
- final class Read extends GeneratedMessage with ReadOrBuilder
Relation that reads from a file / table or other data source. Does not have additional inputs.
Relation that reads from a file / table or other data source. Does not have additional inputs.
Protobuf type
spark.connect.Read
- trait ReadOrBuilder extends MessageOrBuilder
- final class ReattachExecuteRequest extends GeneratedMessage with ReattachExecuteRequestOrBuilder
Protobuf type
spark.connect.ReattachExecuteRequest
- trait ReattachExecuteRequestOrBuilder extends MessageOrBuilder
- final class ReattachOptions extends GeneratedMessage with ReattachOptionsOrBuilder
Protobuf type
spark.connect.ReattachOptions
- trait ReattachOptionsOrBuilder extends MessageOrBuilder
- final class RecoverPartitions extends GeneratedMessage with RecoverPartitionsOrBuilder
See `spark.catalog.recoverPartitions`
See `spark.catalog.recoverPartitions`
Protobuf type
spark.connect.RecoverPartitions
- trait RecoverPartitionsOrBuilder extends MessageOrBuilder
- final class RefreshByPath extends GeneratedMessage with RefreshByPathOrBuilder
See `spark.catalog.refreshByPath`
See `spark.catalog.refreshByPath`
Protobuf type
spark.connect.RefreshByPath
- trait RefreshByPathOrBuilder extends MessageOrBuilder
- final class RefreshTable extends GeneratedMessage with RefreshTableOrBuilder
See `spark.catalog.refreshTable`
See `spark.catalog.refreshTable`
Protobuf type
spark.connect.RefreshTable
- trait RefreshTableOrBuilder extends MessageOrBuilder
- final class Relation extends GeneratedMessage with RelationOrBuilder
The main [[Relation]] type. Fundamentally, a relation is a typed container that has exactly one explicit relation type set. When adding new relation types, they have to be registered here.
The main [[Relation]] type. Fundamentally, a relation is a typed container that has exactly one explicit relation type set. When adding new relation types, they have to be registered here.
Protobuf type
spark.connect.Relation
- final class RelationCommon extends GeneratedMessage with RelationCommonOrBuilder
Common metadata of all relations.
Common metadata of all relations.
Protobuf type
spark.connect.RelationCommon
- trait RelationCommonOrBuilder extends MessageOrBuilder
- trait RelationOrBuilder extends MessageOrBuilder
- final class Relations extends AnyRef
- final class ReleaseExecuteRequest extends GeneratedMessage with ReleaseExecuteRequestOrBuilder
Protobuf type
spark.connect.ReleaseExecuteRequest
- trait ReleaseExecuteRequestOrBuilder extends MessageOrBuilder
- final class ReleaseExecuteResponse extends GeneratedMessage with ReleaseExecuteResponseOrBuilder
Next ID: 4
Next ID: 4
Protobuf type
spark.connect.ReleaseExecuteResponse
- trait ReleaseExecuteResponseOrBuilder extends MessageOrBuilder
- final class ReleaseSessionRequest extends GeneratedMessage with ReleaseSessionRequestOrBuilder
Protobuf type
spark.connect.ReleaseSessionRequest
- trait ReleaseSessionRequestOrBuilder extends MessageOrBuilder
- final class ReleaseSessionResponse extends GeneratedMessage with ReleaseSessionResponseOrBuilder
Next ID: 3
Next ID: 3
Protobuf type
spark.connect.ReleaseSessionResponse
- trait ReleaseSessionResponseOrBuilder extends MessageOrBuilder
- final class RemoveCachedRemoteRelationCommand extends GeneratedMessage with RemoveCachedRemoteRelationCommandOrBuilder
Command to remove `CashedRemoteRelation`
Command to remove `CashedRemoteRelation`
Protobuf type
spark.connect.RemoveCachedRemoteRelationCommand
- trait RemoveCachedRemoteRelationCommandOrBuilder extends MessageOrBuilder
- final class Repartition extends GeneratedMessage with RepartitionOrBuilder
Relation repartition.
Relation repartition.
Protobuf type
spark.connect.Repartition
- final class RepartitionByExpression extends GeneratedMessage with RepartitionByExpressionOrBuilder
Protobuf type
spark.connect.RepartitionByExpression
- trait RepartitionByExpressionOrBuilder extends MessageOrBuilder
- trait RepartitionOrBuilder extends MessageOrBuilder
- final class ResourceInformation extends GeneratedMessage with ResourceInformationOrBuilder
ResourceInformation to hold information about a type of Resource. The corresponding class is 'org.apache.spark.resource.ResourceInformation'
ResourceInformation to hold information about a type of Resource. The corresponding class is 'org.apache.spark.resource.ResourceInformation'
Protobuf type
spark.connect.ResourceInformation
- trait ResourceInformationOrBuilder extends MessageOrBuilder
- final class ResourceProfile extends GeneratedMessage with ResourceProfileOrBuilder
Protobuf type
spark.connect.ResourceProfile
- trait ResourceProfileOrBuilder extends MessageOrBuilder
- final class SQL extends GeneratedMessage with SQLOrBuilder
Relation that uses a SQL query to generate the output.
Relation that uses a SQL query to generate the output.
Protobuf type
spark.connect.SQL
- trait SQLOrBuilder extends MessageOrBuilder
- final class Sample extends GeneratedMessage with SampleOrBuilder
Relation of type [[Sample]] that samples a fraction of the dataset.
Relation of type [[Sample]] that samples a fraction of the dataset.
Protobuf type
spark.connect.Sample
- trait SampleOrBuilder extends MessageOrBuilder
- final class ScalarScalaUDF extends GeneratedMessage with ScalarScalaUDFOrBuilder
Protobuf type
spark.connect.ScalarScalaUDF
- trait ScalarScalaUDFOrBuilder extends MessageOrBuilder
- final class SetCurrentCatalog extends GeneratedMessage with SetCurrentCatalogOrBuilder
See `spark.catalog.setCurrentCatalog`
See `spark.catalog.setCurrentCatalog`
Protobuf type
spark.connect.SetCurrentCatalog
- trait SetCurrentCatalogOrBuilder extends MessageOrBuilder
- final class SetCurrentDatabase extends GeneratedMessage with SetCurrentDatabaseOrBuilder
See `spark.catalog.setCurrentDatabase`
See `spark.catalog.setCurrentDatabase`
Protobuf type
spark.connect.SetCurrentDatabase
- trait SetCurrentDatabaseOrBuilder extends MessageOrBuilder
- final class SetOperation extends GeneratedMessage with SetOperationOrBuilder
Relation of type [[SetOperation]]
Relation of type [[SetOperation]]
Protobuf type
spark.connect.SetOperation
- trait SetOperationOrBuilder extends MessageOrBuilder
- final class ShowString extends GeneratedMessage with ShowStringOrBuilder
Compose the string representing rows for output. It will invoke 'Dataset.showString' to compute the results.
Compose the string representing rows for output. It will invoke 'Dataset.showString' to compute the results.
Protobuf type
spark.connect.ShowString
- trait ShowStringOrBuilder extends MessageOrBuilder
- final class Sort extends GeneratedMessage with SortOrBuilder
Relation of type [[Sort]].
Relation of type [[Sort]].
Protobuf type
spark.connect.Sort
- trait SortOrBuilder extends MessageOrBuilder
- final class SparkConnectServiceGrpc extends AnyRef
Main interface for the SparkConnect service.
Main interface for the SparkConnect service.
- Annotations
- @Generated() @GrpcGenerated()
- final class SqlCommand extends GeneratedMessage with SqlCommandOrBuilder
A SQL Command is used to trigger the eager evaluation of SQL commands in Spark. When the SQL provide as part of the message is a command it will be immediately evaluated and the result will be collected and returned as part of a LocalRelation. If the result is not a command, the operation will simply return a SQL Relation. This allows the client to be almost oblivious to the server-side behavior.
A SQL Command is used to trigger the eager evaluation of SQL commands in Spark. When the SQL provide as part of the message is a command it will be immediately evaluated and the result will be collected and returned as part of a LocalRelation. If the result is not a command, the operation will simply return a SQL Relation. This allows the client to be almost oblivious to the server-side behavior.
Protobuf type
spark.connect.SqlCommand
- trait SqlCommandOrBuilder extends MessageOrBuilder
- final class StackTraceElement extends GeneratedMessage with StackTraceElementOrBuilder
A message to hold a [[java.lang.StackTraceElement]].
A message to hold a [[java.lang.StackTraceElement]].
Protobuf type
spark.connect.StackTraceElement
- trait StackTraceElementOrBuilder extends MessageOrBuilder
- final class StatApproxQuantile extends GeneratedMessage with StatApproxQuantileOrBuilder
Calculates the approximate quantiles of numerical columns of a DataFrame. It will invoke 'Dataset.stat.approxQuantile' (same as 'StatFunctions.approxQuantile') to compute the results.
Calculates the approximate quantiles of numerical columns of a DataFrame. It will invoke 'Dataset.stat.approxQuantile' (same as 'StatFunctions.approxQuantile') to compute the results.
Protobuf type
spark.connect.StatApproxQuantile
- trait StatApproxQuantileOrBuilder extends MessageOrBuilder
- final class StatCorr extends GeneratedMessage with StatCorrOrBuilder
Calculates the correlation of two columns of a DataFrame. Currently only supports the Pearson Correlation Coefficient. It will invoke 'Dataset.stat.corr' (same as 'StatFunctions.pearsonCorrelation') to compute the results.
Calculates the correlation of two columns of a DataFrame. Currently only supports the Pearson Correlation Coefficient. It will invoke 'Dataset.stat.corr' (same as 'StatFunctions.pearsonCorrelation') to compute the results.
Protobuf type
spark.connect.StatCorr
- trait StatCorrOrBuilder extends MessageOrBuilder
- final class StatCov extends GeneratedMessage with StatCovOrBuilder
Calculate the sample covariance of two numerical columns of a DataFrame. It will invoke 'Dataset.stat.cov' (same as 'StatFunctions.calculateCov') to compute the results.
Calculate the sample covariance of two numerical columns of a DataFrame. It will invoke 'Dataset.stat.cov' (same as 'StatFunctions.calculateCov') to compute the results.
Protobuf type
spark.connect.StatCov
- trait StatCovOrBuilder extends MessageOrBuilder
- final class StatCrosstab extends GeneratedMessage with StatCrosstabOrBuilder
Computes a pair-wise frequency table of the given columns. Also known as a contingency table. It will invoke 'Dataset.stat.crosstab' (same as 'StatFunctions.crossTabulate') to compute the results.
Computes a pair-wise frequency table of the given columns. Also known as a contingency table. It will invoke 'Dataset.stat.crosstab' (same as 'StatFunctions.crossTabulate') to compute the results.
Protobuf type
spark.connect.StatCrosstab
- trait StatCrosstabOrBuilder extends MessageOrBuilder
- final class StatDescribe extends GeneratedMessage with StatDescribeOrBuilder
Computes basic statistics for numeric and string columns, including count, mean, stddev, min, and max. If no columns are given, this function computes statistics for all numerical or string columns.
Computes basic statistics for numeric and string columns, including count, mean, stddev, min, and max. If no columns are given, this function computes statistics for all numerical or string columns.
Protobuf type
spark.connect.StatDescribe
- trait StatDescribeOrBuilder extends MessageOrBuilder
- final class StatFreqItems extends GeneratedMessage with StatFreqItemsOrBuilder
Finding frequent items for columns, possibly with false positives. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Finding frequent items for columns, possibly with false positives. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Protobuf type
spark.connect.StatFreqItems
- trait StatFreqItemsOrBuilder extends MessageOrBuilder
- final class StatSampleBy extends GeneratedMessage with StatSampleByOrBuilder
Returns a stratified sample without replacement based on the fraction given on each stratum. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Returns a stratified sample without replacement based on the fraction given on each stratum. It will invoke 'Dataset.stat.freqItems' (same as 'StatFunctions.freqItems') to compute the results.
Protobuf type
spark.connect.StatSampleBy
- trait StatSampleByOrBuilder extends MessageOrBuilder
- final class StatSummary extends GeneratedMessage with StatSummaryOrBuilder
Computes specified statistics for numeric and string columns. It will invoke 'Dataset.summary' (same as 'StatFunctions.summary') to compute the results.
Computes specified statistics for numeric and string columns. It will invoke 'Dataset.summary' (same as 'StatFunctions.summary') to compute the results.
Protobuf type
spark.connect.StatSummary
- trait StatSummaryOrBuilder extends MessageOrBuilder
- final class StorageLevel extends GeneratedMessage with StorageLevelOrBuilder
StorageLevel for persisting Datasets/Tables.
StorageLevel for persisting Datasets/Tables.
Protobuf type
spark.connect.StorageLevel
- trait StorageLevelOrBuilder extends MessageOrBuilder
- final class StreamingForeachFunction extends GeneratedMessage with StreamingForeachFunctionOrBuilder
Protobuf type
spark.connect.StreamingForeachFunction
- trait StreamingForeachFunctionOrBuilder extends MessageOrBuilder
- final class StreamingQueryCommand extends GeneratedMessage with StreamingQueryCommandOrBuilder
Commands for a streaming query.
Commands for a streaming query.
Protobuf type
spark.connect.StreamingQueryCommand
- trait StreamingQueryCommandOrBuilder extends MessageOrBuilder
- final class StreamingQueryCommandResult extends GeneratedMessage with StreamingQueryCommandResultOrBuilder
Response for commands on a streaming query.
Response for commands on a streaming query.
Protobuf type
spark.connect.StreamingQueryCommandResult
- trait StreamingQueryCommandResultOrBuilder extends MessageOrBuilder
- sealed final class StreamingQueryEventType extends Enum[StreamingQueryEventType] with ProtocolMessageEnum
The enum used for client side streaming query listener event There is no QueryStartedEvent defined here, it is added as a field in WriteStreamOperationStartResult
The enum used for client side streaming query listener event There is no QueryStartedEvent defined here, it is added as a field in WriteStreamOperationStartResult
Protobuf enum
spark.connect.StreamingQueryEventType
- final class StreamingQueryInstanceId extends GeneratedMessage with StreamingQueryInstanceIdOrBuilder
A tuple that uniquely identifies an instance of streaming query run. It consists of `id` that persists across the streaming runs and `run_id` that changes between each run of the streaming query that resumes from the checkpoint.
A tuple that uniquely identifies an instance of streaming query run. It consists of `id` that persists across the streaming runs and `run_id` that changes between each run of the streaming query that resumes from the checkpoint.
Protobuf type
spark.connect.StreamingQueryInstanceId
- trait StreamingQueryInstanceIdOrBuilder extends MessageOrBuilder
- final class StreamingQueryListenerBusCommand extends GeneratedMessage with StreamingQueryListenerBusCommandOrBuilder
The protocol for client-side StreamingQueryListener. This command will only be set when either the first listener is added to the client, or the last listener is removed from the client. The add_listener_bus_listener command will only be set true in the first case. The remove_listener_bus_listener command will only be set true in the second case.
The protocol for client-side StreamingQueryListener. This command will only be set when either the first listener is added to the client, or the last listener is removed from the client. The add_listener_bus_listener command will only be set true in the first case. The remove_listener_bus_listener command will only be set true in the second case.
Protobuf type
spark.connect.StreamingQueryListenerBusCommand
- trait StreamingQueryListenerBusCommandOrBuilder extends MessageOrBuilder
- final class StreamingQueryListenerEvent extends GeneratedMessage with StreamingQueryListenerEventOrBuilder
The protocol for the returned events in the long-running response channel.
The protocol for the returned events in the long-running response channel.
Protobuf type
spark.connect.StreamingQueryListenerEvent
- trait StreamingQueryListenerEventOrBuilder extends MessageOrBuilder
- final class StreamingQueryListenerEventsResult extends GeneratedMessage with StreamingQueryListenerEventsResultOrBuilder
Protobuf type
spark.connect.StreamingQueryListenerEventsResult
- trait StreamingQueryListenerEventsResultOrBuilder extends MessageOrBuilder
- final class StreamingQueryManagerCommand extends GeneratedMessage with StreamingQueryManagerCommandOrBuilder
Commands for the streaming query manager.
Commands for the streaming query manager.
Protobuf type
spark.connect.StreamingQueryManagerCommand
- trait StreamingQueryManagerCommandOrBuilder extends MessageOrBuilder
- final class StreamingQueryManagerCommandResult extends GeneratedMessage with StreamingQueryManagerCommandResultOrBuilder
Response for commands on the streaming query manager.
Response for commands on the streaming query manager.
Protobuf type
spark.connect.StreamingQueryManagerCommandResult
- trait StreamingQueryManagerCommandResultOrBuilder extends MessageOrBuilder
- final class Strings extends GeneratedMessage with StringsOrBuilder
Protobuf type
spark.connect.Strings
- trait StringsOrBuilder extends MessageOrBuilder
- final class SubqueryAlias extends GeneratedMessage with SubqueryAliasOrBuilder
Relation alias.
Relation alias.
Protobuf type
spark.connect.SubqueryAlias
- trait SubqueryAliasOrBuilder extends MessageOrBuilder
- final class SubqueryExpression extends GeneratedMessage with SubqueryExpressionOrBuilder
Protobuf type
spark.connect.SubqueryExpression
- trait SubqueryExpressionOrBuilder extends MessageOrBuilder
- final class TableExists extends GeneratedMessage with TableExistsOrBuilder
See `spark.catalog.tableExists`
See `spark.catalog.tableExists`
Protobuf type
spark.connect.TableExists
- trait TableExistsOrBuilder extends MessageOrBuilder
- final class Tail extends GeneratedMessage with TailOrBuilder
Relation of type [[Tail]] that is used to fetch `limit` rows from the last of the input relation.
Relation of type [[Tail]] that is used to fetch `limit` rows from the last of the input relation.
Protobuf type
spark.connect.Tail
- trait TailOrBuilder extends MessageOrBuilder
- final class TaskResourceRequest extends GeneratedMessage with TaskResourceRequestOrBuilder
A task resource request.
A task resource request.
Protobuf type
spark.connect.TaskResourceRequest
- trait TaskResourceRequestOrBuilder extends MessageOrBuilder
- final class ToDF extends GeneratedMessage with ToDFOrBuilder
Rename columns on the input relation by the same length of names.
Rename columns on the input relation by the same length of names.
Protobuf type
spark.connect.ToDF
- trait ToDFOrBuilder extends MessageOrBuilder
- final class ToSchema extends GeneratedMessage with ToSchemaOrBuilder
Protobuf type
spark.connect.ToSchema
- trait ToSchemaOrBuilder extends MessageOrBuilder
- final class Transpose extends GeneratedMessage with TransposeOrBuilder
Transpose a DataFrame, switching rows to columns. Transforms the DataFrame such that the values in the specified index column become the new columns of the DataFrame.
Transpose a DataFrame, switching rows to columns. Transforms the DataFrame such that the values in the specified index column become the new columns of the DataFrame.
Protobuf type
spark.connect.Transpose
- trait TransposeOrBuilder extends MessageOrBuilder
- final class TypedAggregateExpression extends GeneratedMessage with TypedAggregateExpressionOrBuilder
Protobuf type
spark.connect.TypedAggregateExpression
- trait TypedAggregateExpressionOrBuilder extends MessageOrBuilder
- final class Types extends AnyRef
- final class UncacheTable extends GeneratedMessage with UncacheTableOrBuilder
See `spark.catalog.uncacheTable`
See `spark.catalog.uncacheTable`
Protobuf type
spark.connect.UncacheTable
- trait UncacheTableOrBuilder extends MessageOrBuilder
- final class Unknown extends GeneratedMessage with UnknownOrBuilder
Used for testing purposes only.
Used for testing purposes only.
Protobuf type
spark.connect.Unknown
- trait UnknownOrBuilder extends MessageOrBuilder
- final class Unpivot extends GeneratedMessage with UnpivotOrBuilder
Unpivot a DataFrame from wide format to long format, optionally leaving identifier columns set.
Unpivot a DataFrame from wide format to long format, optionally leaving identifier columns set.
Protobuf type
spark.connect.Unpivot
- trait UnpivotOrBuilder extends MessageOrBuilder
- final class UnresolvedTableValuedFunction extends GeneratedMessage with UnresolvedTableValuedFunctionOrBuilder
Protobuf type
spark.connect.UnresolvedTableValuedFunction
- trait UnresolvedTableValuedFunctionOrBuilder extends MessageOrBuilder
- final class UserContext extends GeneratedMessage with UserContextOrBuilder
User Context is used to refer to one particular user session that is executing queries in the backend.
User Context is used to refer to one particular user session that is executing queries in the backend.
Protobuf type
spark.connect.UserContext
- trait UserContextOrBuilder extends MessageOrBuilder
- final class WithColumns extends GeneratedMessage with WithColumnsOrBuilder
Adding columns or replacing the existing columns that have the same names.
Adding columns or replacing the existing columns that have the same names.
Protobuf type
spark.connect.WithColumns
- trait WithColumnsOrBuilder extends MessageOrBuilder
- final class WithColumnsRenamed extends GeneratedMessage with WithColumnsRenamedOrBuilder
Rename columns on the input relation by a map with name to name mapping.
Rename columns on the input relation by a map with name to name mapping.
Protobuf type
spark.connect.WithColumnsRenamed
- trait WithColumnsRenamedOrBuilder extends MessageOrBuilder
- final class WithRelations extends GeneratedMessage with WithRelationsOrBuilder
Relation of type [[WithRelations]]. This relation contains a root plan, and one or more references that are used by the root plan. There are two ways of referencing a relation, by name (through a subquery alias), or by plan_id (using RelationCommon.plan_id). This relation can be used to implement CTEs, describe DAGs, or to reduce tree depth.
Relation of type [[WithRelations]]. This relation contains a root plan, and one or more references that are used by the root plan. There are two ways of referencing a relation, by name (through a subquery alias), or by plan_id (using RelationCommon.plan_id). This relation can be used to implement CTEs, describe DAGs, or to reduce tree depth.
Protobuf type
spark.connect.WithRelations
- trait WithRelationsOrBuilder extends MessageOrBuilder
- final class WithWatermark extends GeneratedMessage with WithWatermarkOrBuilder
Protobuf type
spark.connect.WithWatermark
- trait WithWatermarkOrBuilder extends MessageOrBuilder
- final class WriteOperation extends GeneratedMessage with WriteOperationOrBuilder
As writes are not directly handled during analysis and planning, they are modeled as commands.
As writes are not directly handled during analysis and planning, they are modeled as commands.
Protobuf type
spark.connect.WriteOperation
- trait WriteOperationOrBuilder extends MessageOrBuilder
- final class WriteOperationV2 extends GeneratedMessage with WriteOperationV2OrBuilder
As writes are not directly handled during analysis and planning, they are modeled as commands.
As writes are not directly handled during analysis and planning, they are modeled as commands.
Protobuf type
spark.connect.WriteOperationV2
- trait WriteOperationV2OrBuilder extends MessageOrBuilder
- final class WriteStreamOperationStart extends GeneratedMessage with WriteStreamOperationStartOrBuilder
Starts write stream operation as streaming query. Query ID and Run ID of the streaming query are returned.
Starts write stream operation as streaming query. Query ID and Run ID of the streaming query are returned.
Protobuf type
spark.connect.WriteStreamOperationStart
- trait WriteStreamOperationStartOrBuilder extends MessageOrBuilder
- final class WriteStreamOperationStartResult extends GeneratedMessage with WriteStreamOperationStartResultOrBuilder
Protobuf type
spark.connect.WriteStreamOperationStartResult
- trait WriteStreamOperationStartResultOrBuilder extends MessageOrBuilder