eels

Type Members

1. case class FilePattern(pattern: String, filter: (Path) ⇒ Boolean = _ => true) extends Logging with Product with Serializable

   

2. trait Frame extends AnyRef

   

3. class FrameSource extends Frame with Logging with Using

   

4. trait Part extends AnyRef

   

   A Part represents part of the source data.

   A Part represents part of the source data. Eg a single file in a multi-file source, or a single table in a multi-table source. Splitting sources into parts allows them to be read concurrently, therefore, implementations must ensure that different parts can be safely read in parallel. A single part is always read by a single thread.

5. case class Row(schema: Schema, values: Vector[Any]) extends Product with Serializable

   

6. trait SchemaInferrer extends AnyRef

   

7. case class SchemaRule(pattern: String, fieldType: FieldType, nullable: Boolean = true) extends Product with Serializable

   

8. trait Sink extends AnyRef

   

9. trait SinkWriter extends AnyRef

   

10. trait Source extends Logging

    

    A Source is a provider of data.

    A Source is a provider of data.

    A source implementation must provide two methods:

    1: schema() which returns an eel schema for the data source.

    2: parts() which returns zero or more Part instances representing the data.

    A part instance is a subset of the data in a Source, and allows for concurrent reading of that data. For example a part could be a single file in a multi-file source, or a partition in a partitioned source.