org.deeplearning4j.text.invertedindex

Interface InvertedIndex

All Superinterfaces:

Serializable

All Known Implementing Classes:

LuceneInvertedIndex

public interface InvertedIndex
extends Serializable

An inverted index for mapping words to documents and documents to words

Method Summary

Modifier and Type	Method and Description
void	addWordsToDoc(int doc, List<VocabWord> words) Adds words to the given document
void	addWordToDoc(int doc, VocabWord word) Add word to a document
int[]	allDocs() Returns a list of all documents
int	batchSize() For word vectors, this is the batch size for which to train on
List<VocabWord>	document(int index) Returns a list of words for a document
int[]	documents(VocabWord vocabWord) Returns the list of documents a vocab word is in
void	eachDoc(com.google.common.base.Function<List<VocabWord>,Void> func, ExecutorService exec) Iterate over each document
void	finish() Finishes saving data
Iterator<List<VocabWord>>	miniBatches() Iterates over mini batches
int	numDocuments() Returns the number of documents
double	sample() Sampling for creating mini batches
int	totalWords() Total number of words in the index

Method Detail

sample

double sample()

Sampling for creating mini batches

Returns:

the sampling for mini batches

miniBatches

Iterator<List<VocabWord>> miniBatches()

Iterates over mini batches

Returns:

the mini batches created by this vectorizer

document

List<VocabWord> document(int index)

Returns a list of words for a document

Parameters:

index -

Returns:

documents

int[] documents(VocabWord vocabWord)

Returns the list of documents a vocab word is in

Parameters:

vocabWord - the vocab word to get documents for

Returns:

the documents for a vocab word

numDocuments

int numDocuments()

Returns the number of documents

Returns:

allDocs

int[] allDocs()

Returns a list of all documents

Returns:

the list of all documents

addWordToDoc

void addWordToDoc(int doc,
                  VocabWord word)

Add word to a document

Parameters:

doc - the document to add to

word - the word to add

addWordsToDoc

void addWordsToDoc(int doc,
                   List<VocabWord> words)

Adds words to the given document

Parameters:

doc - the document to add to

words - the words to add

finish

void finish()

Finishes saving data

totalWords

int totalWords()

Total number of words in the index

Returns:

the total number of words in the index

batchSize

int batchSize()

For word vectors, this is the batch size for which to train on

Returns:

the batch size for which to train on

eachDoc

void eachDoc(com.google.common.base.Function<List<VocabWord>,Void> func,
             ExecutorService exec)

Iterate over each document

Parameters:

func - the function to apply

exec - exectuor service for execution