org.apache.accumulo.core.iterators.user
Class IntersectingIterator

java.lang.Object
  extended by org.apache.accumulo.core.iterators.user.IntersectingIterator
All Implemented Interfaces:
SortedKeyValueIterator<Key,Value>
Direct Known Subclasses:
IndexedDocIterator, IntersectingIterator

public class IntersectingIterator
extends Object
implements SortedKeyValueIterator<Key,Value>

This iterator facilitates document-partitioned indexing. It involves grouping a set of documents together and indexing those documents into a single row of an Accumulo table. This allows a tablet server to perform boolean AND operations on terms in the index. The table structure should have the following form: row: shardID, colfam: term, colqual: docID When you configure this iterator with a set of terms (column families), it will return only the docIDs that appear with all of the specified terms. The result will have an empty column family, as follows: row: shardID, colfam: (empty), colqual: docID This iterator is commonly used with BatchScanner or AccumuloInputFormat, to parallelize the search over all shardIDs. README.shard in docs/examples shows an example of using the IntersectingIterator.


Nested Class Summary
protected static class IntersectingIterator.TermSource
           
 
Field Summary
protected  org.apache.hadoop.io.Text currentDocID
           
protected  org.apache.hadoop.io.Text currentPartition
           
protected  boolean inclusive
           
protected static org.apache.log4j.Logger log
           
protected  org.apache.hadoop.io.Text nullText
           
protected  Collection<ByteSequence> seekColumnFamilies
           
protected  Key topKey
           
protected  Value value
           
 
Constructor Summary
IntersectingIterator()
           
 
Method Summary
 void addSource(SortedKeyValueIterator<Key,Value> source, IteratorEnvironment env, org.apache.hadoop.io.Text term, boolean notFlag)
           
protected  void advanceToIntersection()
           
protected  Key buildFollowingPartitionKey(Key key)
           
protected  Key buildKey(org.apache.hadoop.io.Text partition, org.apache.hadoop.io.Text term)
           
protected  Key buildKey(org.apache.hadoop.io.Text partition, org.apache.hadoop.io.Text term, org.apache.hadoop.io.Text docID)
           
static boolean[] decodeBooleans(String flags)
          Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).
protected static org.apache.hadoop.io.Text[] decodeColumns(String columns)
           
 SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment env)
          Creates a deep copy of this iterator as though seek had not yet been called.
static String encodeBooleans(boolean[] flags)
          Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).
static String encodeColumns(org.apache.hadoop.io.Text[] columns)
          Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).
protected  org.apache.hadoop.io.Text getDocID(Key key)
           
protected  org.apache.hadoop.io.Text getPartition(Key key)
           
protected  org.apache.hadoop.io.Text getTerm(Key key)
           
 Key getTopKey()
          Returns top key.
 Value getTopValue()
          Returns top value.
 boolean hasTop()
          Returns true if the iterator has more elements.
 void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> options, IteratorEnvironment env)
          Initializes the iterator.
 void next()
          Advances to the next K,V pair.
 void seek(Range range, Collection<ByteSequence> seekColumnFamilies, boolean inclusive)
          Seeks to the first key in the Range, restricting the resulting K,V pairs to those with the specified columns.
static void setColumnFamilies(IteratorSetting cfg, org.apache.hadoop.io.Text[] columns)
          Encode the columns to be used when iterating.
static void setColumnFamilies(IteratorSetting cfg, org.apache.hadoop.io.Text[] columns, boolean[] notFlags)
          Encode columns and NOT flags indicating which columns should be negated (docIDs will be excluded if matching negated columns, instead of included).
static String stringTopKey(SortedKeyValueIterator<Key,Value> iter)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

nullText

protected org.apache.hadoop.io.Text nullText

log

protected static final org.apache.log4j.Logger log

currentPartition

protected org.apache.hadoop.io.Text currentPartition

currentDocID

protected org.apache.hadoop.io.Text currentDocID

topKey

protected Key topKey

value

protected Value value

seekColumnFamilies

protected Collection<ByteSequence> seekColumnFamilies

inclusive

protected boolean inclusive
Constructor Detail

IntersectingIterator

public IntersectingIterator()
Method Detail

getPartition

protected org.apache.hadoop.io.Text getPartition(Key key)

getTerm

protected org.apache.hadoop.io.Text getTerm(Key key)

getDocID

protected org.apache.hadoop.io.Text getDocID(Key key)

buildKey

protected Key buildKey(org.apache.hadoop.io.Text partition,
                       org.apache.hadoop.io.Text term)

buildKey

protected Key buildKey(org.apache.hadoop.io.Text partition,
                       org.apache.hadoop.io.Text term,
                       org.apache.hadoop.io.Text docID)

buildFollowingPartitionKey

protected Key buildFollowingPartitionKey(Key key)

deepCopy

public SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment env)
Description copied from interface: SortedKeyValueIterator
Creates a deep copy of this iterator as though seek had not yet been called. init should be called on an iterator before deepCopy is called. init should not need to be called on the copy that is returned by deepCopy; that is, when necessary init should be called in the deepCopy method on the iterator it returns. The behavior is unspecified if init is called after deepCopy either on the original or the copy.

Specified by:
deepCopy in interface SortedKeyValueIterator<Key,Value>
Parameters:
env - IteratorEnvironment environment in which iterator is being run.
Returns:
SortedKeyValueIterator a copy of this iterator (with the same source and settings).

getTopKey

public Key getTopKey()
Description copied from interface: SortedKeyValueIterator
Returns top key. Can be called 0 or more times without affecting behavior of next() or hasTop(). Note that in minor compaction scope and in non-full major compaction scopes the iterator may see deletion entries. These entries should be preserved by all iterators except ones that are strictly scan-time iterators that will never be configured for the minc or majc scopes. Deletion entries are only removed during full major compactions.

Specified by:
getTopKey in interface SortedKeyValueIterator<Key,Value>
Returns:
K

getTopValue

public Value getTopValue()
Description copied from interface: SortedKeyValueIterator
Returns top value. Can be called 0 or more times without affecting behavior of next() or hasTop().

Specified by:
getTopValue in interface SortedKeyValueIterator<Key,Value>
Returns:
V

hasTop

public boolean hasTop()
Description copied from interface: SortedKeyValueIterator
Returns true if the iterator has more elements.

Specified by:
hasTop in interface SortedKeyValueIterator<Key,Value>
Returns:
true if the iterator has more elements.

next

public void next()
          throws IOException
Description copied from interface: SortedKeyValueIterator
Advances to the next K,V pair. Note that in minor compaction scope and in non-full major compaction scopes the iterator may see deletion entries. These entries should be preserved by all iterators except ones that are strictly scan-time iterators that will never be configured for the minc or majc scopes. Deletion entries are only removed during full major compactions.

Specified by:
next in interface SortedKeyValueIterator<Key,Value>
Throws:
IOException - if an I/O error occurs.

advanceToIntersection

protected void advanceToIntersection()
                              throws IOException
Throws:
IOException

stringTopKey

public static String stringTopKey(SortedKeyValueIterator<Key,Value> iter)

encodeColumns

public static String encodeColumns(org.apache.hadoop.io.Text[] columns)
Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).

Parameters:
columns -
Returns:
encoded columns

encodeBooleans

public static String encodeBooleans(boolean[] flags)
Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).

Parameters:
flags -
Returns:
encoded flags

decodeColumns

protected static org.apache.hadoop.io.Text[] decodeColumns(String columns)

decodeBooleans

public static boolean[] decodeBooleans(String flags)
Deprecated. since 1.4. To be made protected. Do not interact with flags string directly, just use setColumnFamilies(IteratorSetting, Text[], boolean[]).

to be made protected

Parameters:
flags -
Returns:
decoded flags

init

public void init(SortedKeyValueIterator<Key,Value> source,
                 Map<String,String> options,
                 IteratorEnvironment env)
          throws IOException
Description copied from interface: SortedKeyValueIterator
Initializes the iterator. Data should not be read from the source in this method.

Specified by:
init in interface SortedKeyValueIterator<Key,Value>
Parameters:
source - SortedKeyValueIterator source to read data from.
options - Map map of string option names to option values.
env - IteratorEnvironment environment in which iterator is being run.
Throws:
IOException - unused.

seek

public void seek(Range range,
                 Collection<ByteSequence> seekColumnFamilies,
                 boolean inclusive)
          throws IOException
Description copied from interface: SortedKeyValueIterator
Seeks to the first key in the Range, restricting the resulting K,V pairs to those with the specified columns. An iterator does not have to stop at the end of the range. The whole range is provided so that iterators can make optimizations.

Specified by:
seek in interface SortedKeyValueIterator<Key,Value>
Parameters:
range - Range of keys to iterate over.
seekColumnFamilies - Collection of column families to include or exclude.
inclusive - boolean that indicates whether to include (true) or exclude (false) column families.
Throws:
IOException - if an I/O error occurs.

addSource

public void addSource(SortedKeyValueIterator<Key,Value> source,
                      IteratorEnvironment env,
                      org.apache.hadoop.io.Text term,
                      boolean notFlag)

setColumnFamilies

public static void setColumnFamilies(IteratorSetting cfg,
                                     org.apache.hadoop.io.Text[] columns)
Encode the columns to be used when iterating.

Parameters:
cfg -
columns -

setColumnFamilies

public static void setColumnFamilies(IteratorSetting cfg,
                                     org.apache.hadoop.io.Text[] columns,
                                     boolean[] notFlags)
Encode columns and NOT flags indicating which columns should be negated (docIDs will be excluded if matching negated columns, instead of included).

Parameters:
cfg -
columns -
notFlags -


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.