ScioContextOps

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   Any

2. final def ##(): Int

   

   Definition Classes

   Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def bigtable(bigtableOptions: BigtableOptions, tableId: String, keyRange: ByteKeyRange, rowFilter: RowFilter): SCollection[Row]

   

   Get an SCollection for a Bigtable table.

6. def bigtable(projectId: String, instanceId: String, tableId: String, keyRange: ByteKeyRange = ..., rowFilter: RowFilter = ...): SCollection[Row]

   

   Get an SCollection for a Bigtable table.

7. def ensureTables(bigtableOptions: BigtableOptions, tablesAndColumnFamilies: Map[String, List[String]]): Unit

   

   Ensure that tables and column families exist.

   Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

   tablesAndColumnFamilies

   A map of tables and column families. Keys are table names. Values are a list of column family names.

8. def ensureTables(projectId: String, instanceId: String, tablesAndColumnFamilies: Map[String, List[String]]): Unit

   

   Ensure that tables and column families exist.

   Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

   tablesAndColumnFamilies

   A map of tables and column families. Keys are table names. Values are a list of column family names.

9. def ensureTablesWithExpiration(bigtableOptions: BigtableOptions, tablesAndColumnFamiliesWithExpiration: Map[String, List[(String, Option[Duration])]]): Unit

   

   Ensure that tables and column families exist.

   Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

   tablesAndColumnFamiliesWithExpiration

   A map of tables and column families. Keys are table names. Values are a list of column family names along with the desired cell expiration. Cell expiration is the duration before which garbage collection of a cell may occur. Note: minimum granularity is second.

10. def ensureTablesWithExpiration(projectId: String, instanceId: String, tablesAndColumnFamiliesWithExpiration: Map[String, List[(String, Option[Duration])]]): Unit

    

    Ensure that tables and column families exist.

    Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

    tablesAndColumnFamiliesWithExpiration

    A map of tables and column families. Keys are table names. Values are a list of column family names along with the desired cell expiration. Cell expiration is the duration before which garbage collection of a cell may occur. Note: minimum granularity is second.

11. def ensureTablesWithGcRules(bigtableOptions: BigtableOptions, tablesAndColumnFamiliesWithGcRule: Map[String, List[(String, Option[GcRule])]]): Unit

    

    Ensure that tables and column families exist.

    Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

    tablesAndColumnFamiliesWithGcRule

    A map of tables and column families. Keys are table names. Values are a list of column family names along with the desired cell expiration. Cell expiration is the duration before which garbage collection of a cell may occur. Note: minimum granularity is second.

12. def ensureTablesWithGcRules(projectId: String, instanceId: String, tablesAndColumnFamiliesWithGcRules: Map[String, List[(String, Option[GcRule])]]): Unit

    

    Ensure that tables and column families exist.

    Ensure that tables and column families exist. Checks for existence of tables or creates them if they do not exist. Also checks for existence of column families within each table and creates them if they do not exist.

    tablesAndColumnFamiliesWithGcRules

    A map of tables and column families. Keys are table names. Values are a list of column family names along with the desired GcRule.

13. def getBigtableClusterSizes(projectId: String, instanceId: String): Map[String, Int]

    

    Get size of all clusters for specified Bigtable instance.

    Get size of all clusters for specified Bigtable instance.

    returns

    map of clusterId to its number of nodes

14. def getClass(): Class[_ <: AnyVal]

    

    Definition Classes

    AnyVal → Any

15. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

16. def toString(): String

    

    Definition Classes

    Any

17. def updateNumberOfBigtableNodes(bigtableOptions: BigtableOptions, numberOfNodes: Int, sleepDuration: Duration): Unit

    

    Updates all clusters within the specified Bigtable instance to a specified number of nodes.

    Updates all clusters within the specified Bigtable instance to a specified number of nodes. Useful for increasing the number of nodes at the beginning of a job and decreasing it at the end to lower costs yet still get high throughput during bulk ingests/dumps.

    sleepDuration

    How long to sleep after updating the number of nodes. Google recommends at least 20 minutes before the new nodes are fully functional

18. def updateNumberOfBigtableNodes(projectId: String, instanceId: String, numberOfNodes: Int, sleepDuration: Duration = DefaultSleepDuration): Unit

    

    Updates all clusters within the specified Bigtable instance to a specified number of nodes.

    Updates all clusters within the specified Bigtable instance to a specified number of nodes. Useful for increasing the number of nodes at the beginning of a job and decreasing it at the end to lower costs yet still get high throughput during bulk ingests/dumps.

    sleepDuration

    How long to sleep after updating the number of nodes. Google recommends at least 20 minutes before the new nodes are fully functional