Package net.snowflake.ingest.streaming.internal

Class ParquetRowBuffer

java.lang.Object

net.snowflake.ingest.streaming.internal.ParquetRowBuffer

public class ParquetRowBuffer
extends Object

The buffer in the Streaming Ingest channel that holds the un-flushed rows, these rows will be converted to Parquet format for faster processing

Method Summary

Modifier and Type	Method	Description
void	close(String name)	Close the row buffer and release allocated memory for the channel.
Flusher<ParquetChunkData>	createFlusher()
net.snowflake.ingest.streaming.internal.ChannelData<T>	flush(String filePath)	Flush the data in the row buffer by taking the ownership of the old vectors and pass all the required info back to the flush service to build the blob
float	getSize()	Get the current buffer size
InsertValidationResponse	insertRows(Iterable<Map<String,Object>> rows, String startOffsetToken, String endOffsetToken)	Insert a batch of rows into the row buffer
void	setupSchema(List<net.snowflake.ingest.streaming.internal.ColumnMetadata> columns)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setupSchema

public void setupSchema(List<net.snowflake.ingest.streaming.internal.ColumnMetadata> columns)

createFlusher

public Flusher<ParquetChunkData> createFlusher()

getSize

public float getSize()

Get the current buffer size

Returns:

the current buffer size

insertRows

public InsertValidationResponse insertRows(Iterable<Map<String,Object>> rows,
                                           String startOffsetToken,
                                           String endOffsetToken)

Insert a batch of rows into the row buffer

Parameters:

rows - input row

startOffsetToken - start offset token of the batch

endOffsetToken - offset token of the latest row in the batch

Returns:

insert response that possibly contains errors because of insertion failures

flush

public net.snowflake.ingest.streaming.internal.ChannelData<T> flush(String filePath)

Flush the data in the row buffer by taking the ownership of the old vectors and pass all the required info back to the flush service to build the blob

Parameters:

filePath - the name of the file the data will be written in

Returns:

A ChannelData object that contains the info needed by the flush service to build a blob

close

public void close(String name)

Close the row buffer and release allocated memory for the channel.