Package org.datavec.api.split.partition

Interface Partitioner

    • Method Detail

      • totalRecordsWritten

        int totalRecordsWritten()
        Returns the total records written
        Returns:

      • numRecordsWritten

        int numRecordsWritten()
        Number of records written so far
        Returns:

      • numPartitions

        int numPartitions()
        Returns the number of partitions
        Returns:

      • init

        void init​(InputSplit inputSplit)
        Initializes this partitioner with the given configuration and input split
        Parameters:
        inputSplit - the input split to use with this partitioner

      • init

        void init​(Configuration configuration,
          InputSplit split)
        Initializes this partitioner with the given configuration and input split
        Parameters:
        configuration - the configuration to configure this partitioner with
        split - the input split to use with this partitioner

      • updatePartitionInfo

        void updatePartitionInfo​(PartitionMetaData metadata)
        Updates the metadata for this partitioner (to indicate whether the next partition is needed or not)
        Parameters:
        metadata -

      • needsNewPartition

        boolean needsNewPartition()
        Returns true if the partition needs to be moved to the next. This is controlled with updatePartitionInfo(PartitionMetaData) which handles incrementing counters and the like to determine whether the current partition has been exhausted.
        Returns:

      • openNewStream

        OutputStream openNewStream()
        "Increment" to the next stream
        Returns:
        the new opened output stream

      • currentOutputStream

        OutputStream currentOutputStream()
        Get the current output stream
        Returns: