io

Type Members

1. final case class BinaryIO(path: String) extends ScioIO[Array[Byte]] with Product with Serializable

   

   A ScioIO class for writing raw bytes to files.

   A ScioIO class for writing raw bytes to files. Like TextIO, but without newline delimiters and operating over Array[Byte] instead of String.

   path

   a path to write to.

2. final case class ClosedTap[T] extends Product with Serializable

   

3. final case class CustomIO[T](id: String) extends TestIO[T] with Product with Serializable

   

   Special version of ScioIO for use with ScioContext.customInput and SCollection.saveAsCustomOutput.

4. final case class DatastoreIO(projectId: String) extends ScioIO[Entity] with Product with Serializable

   

5. final class EmptyTapOf[A] extends TapT[A]

   

6. sealed trait PubsubIO[T] extends ScioIO[T]

   

7. final case class ReadIO[T](id: String) extends TestIO[T] with Product with Serializable

   

   Special version of ScioIO for use with SCollection.readAll and SCollection.readAllBytes.

8. trait ScioIO[T] extends AnyRef

   

   Base trait for all Read/Write IO classes.

   Base trait for all Read/Write IO classes. Every IO connector must implement this. This trait has two abstract implicit methods #read, #write that need to be implemented in every subtype. Look at the com.spotify.scio.io.TextIO subclass for a reference implementation. IO connectors can choose to override #readTest and #writeTest if custom test logic is necessary.

9. trait Tap[T] extends Serializable

   

   Placeholder to an external data set that can either be load into memory as an iterator or opened in a new ScioContext as an SCollection.

10. class TapNotAvailableException extends Exception

    

    Exception for when a tap is not available.

11. final class TapOf[A] extends TapT[A]

    

12. sealed trait TapT[A] extends Serializable

    

13. trait Taps extends AnyRef

    

    Utility for managing Future[Tap[T]]s.

14. trait TestIO[T] extends ScioIO[T]

    

    Base trait for ScioIO without business logic, for stubbing mock data with JobTest.

15. final case class TextIO(path: String) extends ScioIO[String] with Product with Serializable

    

16. final case class TextTap(path: String) extends Tap[String] with Product with Serializable

    

    Tap for text files on local file system or GCS.

17. case class UnsupportedTap[T](msg: String) extends Tap[T] with Product with Serializable