me.lemire.integercompression

Class IntegratedFastPFOR

java.lang.Object

me.lemire.integercompression.IntegratedFastPFOR

All Implemented Interfaces:

IntegerCODEC, IntegratedIntegerCODEC

public final class IntegratedFastPFOR
extends Object
implements IntegratedIntegerCODEC

This is an integrated version of FastPFOR meaning that it computes differential coding as part of the compression.

It encodes integers in blocks of 128 integers within pages of up to 65536 integers. Note that it is important, to get good compression and good performance, to use sizeable blocks (greater than 1024 integers). For arrays containing a number of integers that is not divisible by 128, you should use it in conjunction with another CODEC:

IntegratedIntegerCODEC is =  
  new IntegratedComposition(new IntegratedFastPFOR(),
               new IntegratedVariableByte())

For details, please see

Daniel Lemire and Leonid Boytsov, Decoding billions of integers per second through vectorization Software: Practice & Experience http://onlinelibrary.wiley.com/doi/10.1002/spe.2203/abstract http://arxiv.org/abs/1209.2137

Daniel Lemire, Leonid Boytsov, Nathan Kurz, SIMD Compression and the Intersection of Sorted Integers http://arxiv.org/abs/1401.6399

For multi-threaded applications, each thread should use its own IntegratedFastPFOR object.

You should only use this scheme on sorted arrays. Use FastPFOR if you have unsorted arrays.

Author:

Daniel Lemire

Constructor Summary

Constructors

Constructor and Description
IntegratedFastPFOR() Construct the fastPFOR CODEC with default parameters.
IntegratedFastPFOR(int pagesize) Construct the FastPFOR CODEC.

Method Summary

Modifier and Type	Method and Description
void	compress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos) Compress data in blocks of 128 integers (if fewer than 128 integers are provided, nothing is done).
String	toString()
void	uncompress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos) Uncompress data in blocks of 128 integers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

IntegratedFastPFOR

public IntegratedFastPFOR(int pagesize)

Construct the FastPFOR CODEC.

Parameters:

pagesize - the desired page size (for expert use)

IntegratedFastPFOR

public IntegratedFastPFOR()

Construct the fastPFOR CODEC with default parameters.

Method Detail

compress

public void compress(int[] in,
                     IntWrapper inpos,
                     int inlength,
                     int[] out,
                     IntWrapper outpos)

Compress data in blocks of 128 integers (if fewer than 128 integers are provided, nothing is done).

Specified by:

compress in interface IntegerCODEC

Parameters:

in - input array

inpos - location in the input array

inlength - how many integers to compress

out - output array

outpos - where to write in the output array

See Also:

IntegerCODEC.compress(int[], IntWrapper, int, int[], IntWrapper)

uncompress

public void uncompress(int[] in,
                       IntWrapper inpos,
                       int inlength,
                       int[] out,
                       IntWrapper outpos)

Uncompress data in blocks of 128 integers. In this particular case, the inlength parameter is ignored: it is deduced from the compressed data.

Specified by:

uncompress in interface IntegerCODEC

Parameters:

in - array containing data in compressed form

inpos - where to start reading in the array

inlength - length of the compressed data (ignored by some schemes)

out - array where to write the compressed output

outpos - where to write the compressed output in out

See Also:

IntegerCODEC.compress(int[], IntWrapper, int, int[], IntWrapper)

toString

public String toString()

Overrides:

toString in class Object