Class ParquetRowBuffer
- java.lang.Object
-
- net.snowflake.ingest.streaming.internal.ParquetRowBuffer
-
public class ParquetRowBuffer extends Object
The buffer in the Streaming Ingest channel that holds the un-flushed rows, these rows will be converted to Parquet format for faster processing
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close(String name)
Close the row buffer and release allocated memory for the channel.Flusher<ParquetChunkData>
createFlusher()
net.snowflake.ingest.streaming.internal.ChannelData<T>
flush(String filePath)
Flush the data in the row buffer by taking the ownership of the old vectors and pass all the required info back to the flush service to build the blobfloat
getSize()
Get the current buffer sizeInsertValidationResponse
insertRows(Iterable<Map<String,Object>> rows, String offsetToken)
Insert a batch of rows into the row buffervoid
setupSchema(List<net.snowflake.ingest.streaming.internal.ColumnMetadata> columns)
-
-
-
Method Detail
-
setupSchema
public void setupSchema(List<net.snowflake.ingest.streaming.internal.ColumnMetadata> columns)
-
createFlusher
public Flusher<ParquetChunkData> createFlusher()
-
getSize
public float getSize()
Get the current buffer size- Returns:
- the current buffer size
-
insertRows
public InsertValidationResponse insertRows(Iterable<Map<String,Object>> rows, String offsetToken)
Insert a batch of rows into the row buffer- Parameters:
rows
- input rowoffsetToken
- offset token of the latest row in the batch- Returns:
- insert response that possibly contains errors because of insertion failures
-
flush
public net.snowflake.ingest.streaming.internal.ChannelData<T> flush(String filePath)
Flush the data in the row buffer by taking the ownership of the old vectors and pass all the required info back to the flush service to build the blob- Parameters:
filePath
- the name of the file the data will be written in- Returns:
- A ChannelData object that contains the info needed by the flush service to build a blob
-
close
public void close(String name)
Close the row buffer and release allocated memory for the channel.
-
-