public class ArrowStreamWriter extends ArrowWriter
LOGGER, option, out, schema
Constructor and Description |
---|
ArrowStreamWriter(VectorSchemaRoot root,
DictionaryProvider provider,
OutputStream out)
Construct an ArrowStreamWriter with an optional DictionaryProvider for the OutputStream.
|
ArrowStreamWriter(VectorSchemaRoot root,
DictionaryProvider provider,
WritableByteChannel out)
Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.
|
ArrowStreamWriter(VectorSchemaRoot root,
DictionaryProvider provider,
WritableByteChannel out,
IpcOption option)
Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.
|
ArrowStreamWriter(VectorSchemaRoot root,
DictionaryProvider provider,
WritableByteChannel out,
IpcOption option,
CompressionCodec.Factory compressionFactory,
CompressionUtil.CodecType codecType)
Construct an ArrowStreamWriter with compression enabled.
|
ArrowStreamWriter(VectorSchemaRoot root,
DictionaryProvider provider,
WritableByteChannel out,
IpcOption option,
CompressionCodec.Factory compressionFactory,
CompressionUtil.CodecType codecType,
Optional<Integer> compressionLevel)
Construct an ArrowStreamWriter with compression enabled.
|
Modifier and Type | Method and Description |
---|---|
void |
close() |
protected void |
endInternal(WriteChannel out) |
protected void |
ensureDictionariesWritten(DictionaryProvider provider,
Set<Long> dictionaryIdsUsed)
Write dictionaries after schema and before recordBatches, dictionaries won't be
written if empty stream (only has schema data in IPC).
|
static void |
writeEndOfStream(WriteChannel out,
IpcOption option)
Write an EOS identifier to the WriteChannel.
|
bytesWritten, end, start, startInternal, writeBatch, writeDictionaryBatch, writeDictionaryBatch, writeRecordBatch
public ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, OutputStream out)
root
- Existing VectorSchemaRoot with vectors to be written.provider
- DictionaryProvider for any vectors that are dictionary encoded.
(Optional, can be null)out
- OutputStream for writing.public ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out)
public ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option)
root
- Existing VectorSchemaRoot with vectors to be written.provider
- DictionaryProvider for any vectors that are dictionary encoded.
(Optional, can be null)option
- IPC write optionsout
- WritableByteChannel for writing.public ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType)
root
- Existing VectorSchemaRoot with vectors to be written.provider
- DictionaryProvider for any vectors that are dictionary encoded.
(Optional, can be null)option
- IPC write optionscompressionFactory
- Compression codec factorycodecType
- Codec typeout
- WritableByteChannel for writing.public ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType, Optional<Integer> compressionLevel)
root
- Existing VectorSchemaRoot with vectors to be written.provider
- DictionaryProvider for any vectors that are dictionary encoded.
(Optional, can be null)option
- IPC write optionscompressionFactory
- Compression codec factorycodecType
- Codec typecompressionLevel
- Compression levelout
- WritableByteChannel for writing.public static void writeEndOfStream(WriteChannel out, IpcOption option) throws IOException
out
- Open WriteChannel with an active Arrow stream.option
- IPC write optionIOException
- on errorprotected void endInternal(WriteChannel out) throws IOException
endInternal
in class ArrowWriter
IOException
protected void ensureDictionariesWritten(DictionaryProvider provider, Set<Long> dictionaryIdsUsed) throws IOException
ArrowWriter
ensureDictionariesWritten
in class ArrowWriter
IOException
public void close()
close
in interface AutoCloseable
close
in class ArrowWriter
Copyright © 2023 The Apache Software Foundation. All rights reserved.