This gives you a number of bits to use to have a given standard error
The true error is distributed like a Gaussian with this standard deviation.
The true error is distributed like a Gaussian with this standard deviation. let m = 2^bits. The size of the HLL is m bytes.
bits | size | error 9 512 0.0460 10 1024 0.0325 11 2048 0.0230 12 4096 0.0163 13 8192 0.0115 14 16384 0.0081 15 32768 0.0057 16 65536 0.0041 17 131072 0.0029 18 262144 0.0020 19 524288 0.0014 20 1048576 0.0010
Keep in mind, to store N distinct longs, you only need 8N bytes. See SetSizeAggregator for an approach that uses an exact set when the cardinality is small, and switches to HLL after we have enough items. Ideally, you would keep an exact set until it is smaller to store the HLL (but actually since we use sparse vectors to store the HLL, a small HLL takes a lot less than the size above).
the value 'j' is equal to <w_0, w_1 .
the value 'j' is equal to <w_0, w_1 ... w_(bits-1)> TODO: We could read in a byte at a time.
We are computing j and \rho(w) from the paper, sorry for the name, but it allows someone to compare to the paper extremely low probability rhow (position of the leftmost one bit) is > 127, so we use a Byte to store it Given a hash <w_0, w_1, w_2 .
We are computing j and \rho(w) from the paper, sorry for the name, but it allows someone to compare to the paper extremely low probability rhow (position of the leftmost one bit) is > 127, so we use a Byte to store it Given a hash <w_0, w_1, w_2 ... w_n> the value 'j' is equal to <w_0, w_1 ... w_(bits-1)> and the value 'w' is equal to <w_bits ... w_n>. The function rho counts the number of leading zeroes in 'w'. We can calculate rho(w) at once with the method rhoW.
The value 'w' is equal to <w_bits .
The value 'w' is equal to <w_bits ... w_n>. The function rho counts the number of leading zeroes in 'w'. We can calculate rho(w) at once with the method rhoW.
Implementation of the HyperLogLog approximate counting as a Monoid