Packages

p

io.github.setl

internal

package internal

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. class BenchmarkInvocationHandler extends InvocationHandler

    BenchmarkInvocationHandler is used to handle the @Benchmark annotation.

    BenchmarkInvocationHandler is used to handle the @Benchmark annotation. It measure the elapsed time of the method having the annotation.

  2. trait CanCreate extends AnyRef

    Connectors that inherit CanCreate should be able to create a table in a database or a file/folder in a file system

  3. trait CanDelete extends AnyRef

    Connectors that inherit CanDelete should be able to delete records for a given query string

  4. trait CanDrop extends AnyRef

    Connectors that inherit CanDrop should be able to drop the entire data table

  5. trait CanPartition extends AnyRef

    Connectors that inherit CanPartition should be able to partition the output by the given columns on the file system

  6. trait CanUpdate extends AnyRef

    Connectors that inherit CanUpdate should be able to update the data store with a new data frame and a given matching columns.

  7. trait CanVacuum extends AnyRef

    Connectors that inherit CanVacuum should be able to recursively delete files and directories in the table that are not needed by the table for maintaining older versions up to the given retention threshold

  8. trait CanWait extends AnyRef

    Connectors that inherit CanWait should be able to wait for the execution to stop

  9. trait Configurable extends AnyRef
    Annotations
    @Evolving()
  10. trait HasBenchmark extends AnyRef

    HasBenchmark should be used for object having an aggregated benchmark.

    HasBenchmark should be used for object having an aggregated benchmark. Typically a Pipeline or a Stage

  11. trait HasDescription extends AnyRef
    Annotations
    @Evolving()
  12. trait HasDiagram extends AnyRef
  13. trait HasReader extends AnyRef
  14. trait HasReaderWriter extends HasReader with HasWriter
  15. trait HasRegistry[T <: Identifiable] extends AnyRef

    HasUUIDRegistry provide a UUID registry and methods to check if an io.github.setl.internal.Identifiable object already exists in its registry

    HasUUIDRegistry provide a UUID registry and methods to check if an io.github.setl.internal.Identifiable object already exists in its registry

    Annotations
    @Evolving()
  16. trait HasType extends AnyRef

    HasType should be used on classed having a payload

    HasType should be used on classed having a payload

    Annotations
    @Evolving()
  17. trait HasWriter extends AnyRef
  18. trait Identifiable extends AnyRef

    Identifiable generates an UUID for any object that implement the trait

    Identifiable generates an UUID for any object that implement the trait

    Annotations
    @Evolving()
  19. trait Writable extends AnyRef

    Indicate that users can activate or deactivate the write of the class

Value Members

  1. object SchemaConverter extends Logging

    SchemaConverter will rename the column of a dataset/dataframe according to the given case class T.

    SchemaConverter will rename the column of a dataset/dataframe according to the given case class T.

    import io.github.setl.annotations.colName
case class MyObject(@colName("col1") column1: String, column2: String)

val ds: Dataset[MyObject] = Seq(MyObject("a", "A"), MyObject("b", "B")).toDS()
// +-------+-------+
// |column1|column2|
// +-------+-------+
// |      a|      A|
// |      b|      B|
// +-------+-------+

val df = SchemaConverter.toDF(ds)
// +----+-------+
// |col1|column2|
// +----+-------+
// |   a|      A|
// |   b|      B|
// +----+-------+

val ds2 = SchemaConverter.fromDF[MyObject](df)
// +-------+-------+
// |column1|column2|
// +-------+-------+
// |      a|      A|
// |      b|      B|
// +-------+-------+
  2. object StructAnalyser extends Logging

    StructAnalyser will analyse the schema for a given case class.

    StructAnalyser will analyse the schema for a given case class. It will register the information about

Ungrouped