public abstract class BaseVariableWidthVector extends BaseValueVector implements VariableWidthVector, FieldVector, VectorDefinitionSetter
修飾子とタイプ | フィールドと説明 |
---|---|
protected static byte[] |
emptyByteArray |
protected Field |
field |
protected int |
lastSet |
static int |
OFFSET_WIDTH |
protected ArrowBuf |
offsetBuffer |
protected ArrowBuf |
validityBuffer |
protected ArrowBuf |
valueBuffer |
protected int |
valueCount |
allocator, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE, MAX_ALLOCATION_SIZE_PROPERTY, name
コンストラクタと説明 |
---|
BaseVariableWidthVector(String name,
BufferAllocator allocator,
FieldType fieldType) |
修飾子とタイプ | メソッドと説明 |
---|---|
void |
allocateNew()
Same as
allocateNewSafe() . |
void |
allocateNew(int totalBytes,
int valueCount)
Allocate memory for the vector to support storing at least the provided number of
elements in the vector.
|
boolean |
allocateNewSafe()
Allocate memory for the vector.
|
void |
clear()
Same as
close() |
void |
close()
Close the vector and release the associated buffers.
|
void |
fillEmpties(int index)
Create holes in the vector upto the given index (exclusive).
|
protected void |
fillHoles(int index)
*
helper methods for setters *
*
|
static byte[] |
get(ArrowBuf data,
ArrowBuf offset,
int index)
Method used by Json Writer to read a variable width element from
the variable width vector and write to Json.
|
ArrowBuf[] |
getBuffers(boolean clear)
Return the underlying buffers associated with this vector.
|
int |
getBufferSize()
Get the size (number of bytes) of underlying buffers used by this
vector
|
int |
getBufferSizeFor(int valueCount)
Get the potential buffer size for a particular number of records.
|
int |
getByteCapacity()
Get the size (number of bytes) of underlying data buffer.
|
List<FieldVector> |
getChildrenFromFields()
Get the inner child vectors.
|
int |
getCurrentSizeInBytes() |
ArrowBuf |
getDataBuffer()
Get the buffer that stores the data for elements in the vector.
|
long |
getDataBufferAddress()
Get the memory address of buffer that stores the data for elements
in the vector.
|
double |
getDensity()
Get the density of this ListVector
|
Field |
getField()
Get information about how this field is materialized.
|
List<ArrowBuf> |
getFieldBuffers()
Get the buffers belonging to this vector
|
List<BufferBacked> |
getFieldInnerVectors()
非推奨です。
|
int |
getLastSet()
Get the index of last non-null element in the vector.
|
int |
getNullCount()
Get the number of elements that are null in the vector
|
ArrowBuf |
getOffsetBuffer()
buffer that stores the offsets for elements
in the vector.
|
long |
getOffsetBufferAddress()
Get the memory address of buffer that stores the offsets for elements
in the vector.
|
long |
getStartEnd(int index)
Get the starting position (offset) in the data stream for a given
element in the vector.
|
protected int |
getstartOffset(int index) |
TransferPair |
getTransferPair(BufferAllocator allocator)
Construct a transfer pair of this vector and another vector of same type.
|
abstract TransferPair |
getTransferPair(String ref,
BufferAllocator allocator)
Construct a transfer pair of this vector and another vector of same type.
|
TransferPair |
getTransferPair(String ref,
BufferAllocator allocator,
CallBack callBack)
Construct a transfer pair of this vector and another vector of same type.
|
ArrowBuf |
getValidityBuffer()
Get buffer that manages the validity (NULL or NON-NULL nature) of
elements in the vector.
|
long |
getValidityBufferAddress()
Get the memory address of buffer that manages the validity
(NULL or NON-NULL nature) of elements in the vector.
|
int |
getValueCapacity()
Get the current value capacity for the vector
|
int |
getValueCount()
Get the value count of vector.
|
int |
getValueLength(int index)
Get the variable length element at specified index as Text.
|
protected void |
handleSafe(int index,
int dataLength) |
void |
initializeChildrenFromFields(List<Field> children)
Initialize the children in schema for this Field.
|
boolean |
isNull(int index)
Check if element at given index is null.
|
boolean |
isSafe(int index)
Check if the given index is within the current value capacity
of the vector
|
int |
isSet(int index)
Same as
isNull(int) . |
void |
loadFieldBuffers(ArrowFieldNode fieldNode,
List<ArrowBuf> ownBuffers)
Load the buffers of this vector with provided source buffers.
|
void |
reAlloc()
Resize the vector to increase the capacity.
|
void |
reallocDataBuffer()
Reallocate the data buffer.
|
void |
reallocValidityAndOffsetBuffers()
Reallocate the validity and offset buffers for this vector.
|
void |
reset()
Reset the vector to initial state.
|
static ArrowBuf |
set(ArrowBuf buffer,
BufferAllocator allocator,
int valueCount,
int index,
int value)
Method used by Json Reader to explicitly set the offsets of the variable
width vector data.
|
void |
set(int index,
byte[] value)
Set the variable length element at the specified index to the supplied
byte array.
|
void |
set(int index,
byte[] value,
int start,
int length)
Set the variable length element at the specified index to the supplied
byte array.
|
void |
set(int index,
ByteBuffer value,
int start,
int length)
Set the variable length element at the specified index to the
content in supplied ByteBuffer
|
void |
set(int index,
int start,
int length,
ArrowBuf buffer)
Store the given value at a particular position in the vector. isSet indicates
whether the value is NULL or not.
|
void |
set(int index,
int isSet,
int start,
int end,
ArrowBuf buffer)
Store the given value at a particular position in the vector. isSet indicates
whether the value is NULL or not.
|
protected void |
setBytes(int index,
byte[] value,
int start,
int length) |
void |
setIndexDefined(int index)
Mark the particular position in the vector as non-null.
|
void |
setInitialCapacity(int valueCount)
Sets the desired value capacity for the vector.
|
void |
setInitialCapacity(int valueCount,
double density)
Sets the desired value capacity for the vector.
|
void |
setLastSet(int value)
Set the index of last non-null element in the vector.
|
void |
setNull(int index)
Set the element at the given index to null.
|
void |
setSafe(int index,
byte[] value)
Same as
set(int, byte[]) except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector. |
void |
setSafe(int index,
byte[] value,
int start,
int length)
Same as
set(int, byte[], int, int) except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector. |
void |
setSafe(int index,
ByteBuffer value,
int start,
int length)
Same as
set(int, ByteBuffer, int, int) except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector. |
void |
setSafe(int index,
int start,
int length,
ArrowBuf buffer)
Same as
set(int, int, int, int, ArrowBuf) except that it handles the case
when index is greater than or equal to current value capacity of the
vector. |
void |
setSafe(int index,
int isSet,
int start,
int end,
ArrowBuf buffer)
Same as
set(int, int, int, int, ArrowBuf) except that it handles the case
when index is greater than or equal to current value capacity of the
vector. |
void |
setValueCount(int valueCount)
Sets the value count for the vector
|
void |
setValueLengthSafe(int index,
int length)
Sets the value length for an element.
|
void |
splitAndTransferTo(int startIndex,
int length,
BaseVariableWidthVector target)
Slice this vector at desired index and length and transfer the
corresponding data to the target vector.
|
void |
transferTo(BaseVariableWidthVector target)
Transfer this vector'data to another vector.
|
void |
zeroVector()
zero out the vector and the data in associated buffers.
|
checkBufRefs, compareTypes, getAllocator, getValidityBufferSizeFromCount, iterator, releaseBuffer, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
getAllocator, getMinorType, getObject, getReader, makeTransferPair
forEach, iterator, spliterator
public static final int OFFSET_WIDTH
protected static final byte[] emptyByteArray
protected ArrowBuf validityBuffer
protected ArrowBuf valueBuffer
protected ArrowBuf offsetBuffer
protected int valueCount
protected int lastSet
protected final Field field
public BaseVariableWidthVector(String name, BufferAllocator allocator, FieldType fieldType)
public ArrowBuf getValidityBuffer()
getValidityBuffer
インタフェース内 ValueVector
public ArrowBuf getDataBuffer()
getDataBuffer
インタフェース内 ValueVector
public ArrowBuf getOffsetBuffer()
getOffsetBuffer
インタフェース内 ValueVector
public long getOffsetBufferAddress()
getOffsetBufferAddress
インタフェース内 FieldVector
public long getValidityBufferAddress()
getValidityBufferAddress
インタフェース内 FieldVector
public long getDataBufferAddress()
getDataBufferAddress
インタフェース内 FieldVector
public void setInitialCapacity(int valueCount)
setInitialCapacity
インタフェース内 ValueVector
valueCount
- desired number of elements in the vectorpublic void setInitialCapacity(int valueCount, double density)
setInitialCapacity
インタフェース内 DensityAwareVector
valueCount
- desired number of elements in the vectordensity
- average number of bytes per variable width elementpublic double getDensity()
public int getValueCapacity()
getValueCapacity
インタフェース内 ValueVector
public void zeroVector()
public void reset()
zeroVector()
.
Note that this method doesn't release any memory.reset
インタフェース内 ValueVector
public void close()
close
インタフェース内 Closeable
close
インタフェース内 AutoCloseable
close
インタフェース内 ValueVector
close
クラス内 BaseValueVector
public void clear()
close()
clear
インタフェース内 ValueVector
clear
クラス内 BaseValueVector
@Deprecated public List<BufferBacked> getFieldInnerVectors()
getFieldInnerVectors
インタフェース内 FieldVector
public void initializeChildrenFromFields(List<Field> children)
initializeChildrenFromFields
インタフェース内 FieldVector
children
- the schemaIllegalArgumentException
- if children is a non-empty list for scalar types.public List<FieldVector> getChildrenFromFields()
getChildrenFromFields
インタフェース内 FieldVector
public void loadFieldBuffers(ArrowFieldNode fieldNode, List<ArrowBuf> ownBuffers)
loadFieldBuffers
インタフェース内 FieldVector
fieldNode
- the fieldNode indicating the value countownBuffers
- the buffers for this Field (own buffers only, children not included)public List<ArrowBuf> getFieldBuffers()
getFieldBuffers
インタフェース内 FieldVector
public void allocateNew()
allocateNewSafe()
.allocateNew
インタフェース内 ValueVector
public boolean allocateNewSafe()
allocateNew(int, int)
for allocating memory for specific
number of elements in the vector.allocateNewSafe
インタフェース内 ValueVector
public void allocateNew(int totalBytes, int valueCount)
allocateNew
インタフェース内 VariableWidthVector
totalBytes
- desired total memory capacityvalueCount
- the desired number of elements in the vectorOutOfMemoryException
public void reAlloc()
reAlloc
インタフェース内 ValueVector
public void reallocDataBuffer()
OversizedAllocationException
- if the desired new size is more than
max allowedOutOfMemoryException
- if the internal memory allocation failspublic void reallocValidityAndOffsetBuffers()
OversizedAllocationException
- if the desired new size is more than
max allowedOutOfMemoryException
- if the internal memory allocation failspublic int getByteCapacity()
getByteCapacity
インタフェース内 VariableWidthVector
public int getCurrentSizeInBytes()
getCurrentSizeInBytes
インタフェース内 VariableWidthVector
public int getBufferSize()
getBufferSize
インタフェース内 ValueVector
public int getBufferSizeFor(int valueCount)
getBufferSizeFor
インタフェース内 ValueVector
valueCount
- desired number of elements in the vectorpublic Field getField()
getField
インタフェース内 ValueVector
public ArrowBuf[] getBuffers(boolean clear)
getBuffers
インタフェース内 ValueVector
clear
- Whether to clear vector before returning; the buffers will still be refcounted
but the returned array will be the only reference to thembuffers
that is used by this
vector instance.public TransferPair getTransferPair(String ref, BufferAllocator allocator, CallBack callBack)
getTransferPair
インタフェース内 ValueVector
ref
- name of the target vectorallocator
- allocator for the target vectorcallBack
- public TransferPair getTransferPair(BufferAllocator allocator)
getTransferPair
インタフェース内 ValueVector
getTransferPair
クラス内 BaseValueVector
allocator
- allocator for the target vectorpublic abstract TransferPair getTransferPair(String ref, BufferAllocator allocator)
getTransferPair
インタフェース内 ValueVector
ref
- name of the target vectorallocator
- allocator for the target vectorpublic void transferTo(BaseVariableWidthVector target)
target
- destination vector for transferpublic void splitAndTransferTo(int startIndex, int length, BaseVariableWidthVector target)
startIndex
- start position of the split in source vector.length
- length of the split.target
- destination vectorpublic int getNullCount()
getNullCount
インタフェース内 ValueVector
public boolean isSafe(int index)
index
- position to checkpublic boolean isNull(int index)
isNull
インタフェース内 ValueVector
index
- position of elementpublic int isSet(int index)
isNull(int)
.index
- position of elementpublic int getValueCount()
getValueCount
インタフェース内 ValueVector
public void setValueCount(int valueCount)
setValueCount
インタフェース内 ValueVector
valueCount
- value countpublic void fillEmpties(int index)
index
- target indexpublic void setLastSet(int value)
setValueCount(int)
.value
- desired index of last non-null element.public int getLastSet()
public long getStartEnd(int index)
index
- position of the element in the vectorpublic void setIndexDefined(int index)
setIndexDefined
インタフェース内 VectorDefinitionSetter
index
- position of the element.public void setValueLengthSafe(int index, int length)
index
- position of the element to setlength
- length of the elementpublic int getValueLength(int index)
index
- position of element to getpublic void set(int index, byte[] value)
set(int, byte[], int, int)
with start as 0 and length as value.lengthindex
- position of the element to setvalue
- array of bytes to writepublic void setSafe(int index, byte[] value)
set(int, byte[])
except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector.index
- position of the element to setvalue
- array of bytes to writepublic void set(int index, byte[] value, int start, int length)
index
- position of the element to setvalue
- array of bytes to writestart
- start index in array of byteslength
- length of data in array of bytespublic void setSafe(int index, byte[] value, int start, int length)
set(int, byte[], int, int)
except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector.index
- position of the element to setvalue
- array of bytes to writestart
- start index in array of byteslength
- length of data in array of bytespublic void set(int index, ByteBuffer value, int start, int length)
index
- position of the element to setvalue
- ByteBuffer with datastart
- start index in ByteBufferlength
- length of data in ByteBufferpublic void setSafe(int index, ByteBuffer value, int start, int length)
set(int, ByteBuffer, int, int)
except that it handles the
case where index and length of new element are beyond the existing
capacity of the vector.index
- position of the element to setvalue
- ByteBuffer with datastart
- start index in ByteBufferlength
- length of data in ByteBufferpublic void setNull(int index)
index
- position of elementpublic void set(int index, int isSet, int start, int end, ArrowBuf buffer)
index
- position of the new valueisSet
- 0 for NULL value, 1 otherwisestart
- start position of data in bufferend
- end position of data in bufferbuffer
- data buffer containing the variable width element to be stored
in the vectorpublic void setSafe(int index, int isSet, int start, int end, ArrowBuf buffer)
set(int, int, int, int, ArrowBuf)
except that it handles the case
when index is greater than or equal to current value capacity of the
vector.index
- position of the new valueisSet
- 0 for NULL value, 1 otherwisestart
- start position of data in bufferend
- end position of data in bufferbuffer
- data buffer containing the variable width element to be stored
in the vectorpublic void set(int index, int start, int length, ArrowBuf buffer)
index
- position of the new valuestart
- start position of data in bufferlength
- length of data in bufferbuffer
- data buffer containing the variable width element to be stored
in the vectorpublic void setSafe(int index, int start, int length, ArrowBuf buffer)
set(int, int, int, int, ArrowBuf)
except that it handles the case
when index is greater than or equal to current value capacity of the
vector.index
- position of the new valuestart
- start position of data in bufferlength
- length of data in bufferbuffer
- data buffer containing the variable width element to be stored
in the vectorprotected final void fillHoles(int index)
protected final void setBytes(int index, byte[] value, int start, int length)
protected final int getstartOffset(int index)
protected final void handleSafe(int index, int dataLength)
public static byte[] get(ArrowBuf data, ArrowBuf offset, int index)
data
- buffer storing the variable width vector elementsoffset
- buffer storing the offsets of variable width vector elementsindex
- position of the element in the vectorpublic static ArrowBuf set(ArrowBuf buffer, BufferAllocator allocator, int valueCount, int index, int value)
buffer
- ArrowBuf to store offsets for variable width elementsallocator
- memory allocatorvalueCount
- number of elementsindex
- position of the elementvalue
- offset of the elementCopyright © 2018 The Apache Software Foundation. All rights reserved.