Packages

abstract class FileTable extends Table with SupportsRead with SupportsWrite

Linear Supertypes
SupportsWrite, SupportsRead, Table, AnyRef, Any
Known Subclasses
CSVTable, JsonTable, OrcTable, ParquetTable, TextTable
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FileTable
  2. SupportsWrite
  3. SupportsRead
  4. Table
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new FileTable(sparkSession: SparkSession, options: CaseInsensitiveStringMap, paths: Seq[String], userSpecifiedSchema: Option[StructType])

Abstract Value Members

  1. abstract def fallbackFileFormat: Class[_ <: FileFormat]

    Returns a V1 FileFormat class of the same file data source.

    Returns a V1 FileFormat class of the same file data source. This is a solution for the following cases: 1. File datasource V2 implementations cause regression. Users can disable the problematic data source via SQL configuration and fall back to FileFormat. 2. Catalog support is required, which is still under development for data source V2.

  2. abstract def formatName: String

    The string that represents the format that this data source provider uses.

    The string that represents the format that this data source provider uses. This is overridden by children to provide a nice alias for the data source. For example:

    override def formatName(): String = "ORC"
  3. abstract def inferSchema(files: Seq[FileStatus]): Option[StructType]

    When possible, this method should return the schema of the given files.

    When possible, this method should return the schema of the given files. When the format does not support inference, or no valid files are given should return None. In these cases Spark will require that user specify the schema manually.

  4. abstract def name(): String
    Definition Classes
    Table
  5. abstract def newScanBuilder(arg0: CaseInsensitiveStringMap): ScanBuilder
    Definition Classes
    SupportsRead
  6. abstract def newWriteBuilder(arg0: LogicalWriteInfo): WriteBuilder
    Definition Classes
    SupportsWrite

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def capabilities(): Set[TableCapability]
    Definition Classes
    FileTable → Table
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  7. lazy val dataSchema: StructType
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  10. lazy val fileIndex: PartitioningAwareFileIndex
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. def partitioning(): Array[Transform]
    Definition Classes
    FileTable → Table
  19. def properties(): Map[String, String]
    Definition Classes
    FileTable → Table
  20. lazy val schema: StructType
    Definition Classes
    FileTable → Table
  21. def supportsDataType(dataType: DataType): Boolean

    Returns whether this format supports the given DataType in read/write path.

    Returns whether this format supports the given DataType in read/write path. By default all data types are supported.

  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. def toString(): String
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from SupportsWrite

Inherited from SupportsRead

Inherited from Table

Inherited from AnyRef

Inherited from Any

Ungrouped