lsh

Type Members

1. class BitSet extends Serializable

   

   A simple, fixed-size bit set implementation.

   A simple, fixed-size bit set implementation. This implementation is fast because it avoids safety/bound checking.

2. trait Joiner extends AnyRef

   

3. class Lsh extends Joiner with Serializable

   

   Lsh implementation as described in 'Randomized Algorithms and NLP: Using Locality Sensitive Hash Function for High Speed Noun Clustering' by Ravichandran et al.

   Lsh implementation as described in 'Randomized Algorithms and NLP: Using Locality Sensitive Hash Function for High Speed Noun Clustering' by Ravichandran et al. See original publication for a detailed description of the parameters.

   See also

   http://dl.acm.org/citation.cfm?id=1219917

4. class NearestNeighbours extends Joiner with Serializable

   

   Brute force O(n^{2) method to compute exact nearest neighbours. As this is a very expensive computation O(n}2) an additional sample parameter may be passed such that neighbours are just computed for a random fraction.

5. class QueryHamming extends QueryJoiner with Serializable

   

   Implementation based on approximated cosine distances.

   Implementation based on approximated cosine distances. The cosine distances are approximated using hamming distances which are way faster to compute. Either the catalog matrix or the query matrix is broadcasted. This implementation is therefore suited for tasks where one of the matrices is very small (in order to be broadcastet) compared to the query matrix.

6. trait QueryJoiner extends AnyRef

   

7. class QueryLsh extends QueryJoiner with Serializable

   

   Standard Lsh implementation.

   Standard Lsh implementation. The queryMatrix is hashed multiple times and exact hash matches are searched for in the dbMatrix. These candidates are used to compute the cosine distance.

8. class QueryNearestNeighbours extends QueryJoiner with Serializable

   

   Brute force O(size(query) * size(catalog)) method to compute exact nearest neighbours for rows in the query matrix.

   Brute force O(size(query) * size(catalog)) method to compute exact nearest neighbours for rows in the query matrix. As this is a very expensive computation, additional sample parameters may be passed such that neighbours are just computed for a random fraction of the data set.

9. final case class Signature(index: Long, vector: Vector, bitSet: BitSet) extends Ordered[Signature] with Product with Serializable

   

   An id with it's hash encoding and original vector.

10. class SlidingRDD[T] extends RDD[Array[T]]

    

    Represents an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them.

    Represents an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them. The ordering is first based on the partition index and then the ordering of items within each partition. This is similar to sliding in Scala collections, except that it becomes an empty RDD if the window size is greater than the total number of items. It needs to trigger a Spark job if the parent RDD has more than one partitions. To make this operation efficient, the number of items per partition should be larger than the window size and the window size should be small, e.g., 2.

    See also

    Int)*

    Int)*

11. class SlidingRDDPartition[T] extends Partition with Serializable

    

    NOTE: both classes are copied from mllib and slightly modified since these classes are mllib private! Modified lines are marked with comments

12. final case class SparseSignature(index: Long, bitSet: BitSet) extends Ordered[SparseSignature] with Product with Serializable

    

    An id with it's hash encoding.

13. case class SubBucket(bucketHash: Int, subBucketId: Int = 1) extends Product with Serializable

    

14. trait VectorDistance extends Serializable

    

    interface defining similarity measurement between 2 vectors