org.deeplearning4j.bagofwords.vectorizer

Class TfidfVectorizer

java.lang.Object

org.deeplearning4j.bagofwords.vectorizer.BaseTextVectorizer

org.deeplearning4j.bagofwords.vectorizer.TfidfVectorizer

All Implemented Interfaces:

Serializable, TextVectorizer, Vectorizer

public class TfidfVectorizer
extends BaseTextVectorizer

Author:

[email protected]

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TfidfVectorizer.Builder

Field Summary

Fields inherited from class org.deeplearning4j.bagofwords.vectorizer.BaseTextVectorizer

index, isParallel, iterator, labelsSource, minWordFrequency, stopWords, tokenizerFactory, vocabCache

Constructor Summary

Constructors

Constructor and Description
TfidfVectorizer()

Method Summary

Modifier and Type	Method and Description
double	tfidfWord(String word, long wordCount, long documentLength)
org.nd4j.linalg.api.ndarray.INDArray	transform(List<String> tokens) Transforms the matrix
org.nd4j.linalg.api.ndarray.INDArray	transform(String text) Transforms the matrix
org.nd4j.linalg.dataset.DataSet	vectorize() Vectorizes the input source in to a dataset
org.nd4j.linalg.dataset.DataSet	vectorize(File input, String label)
org.nd4j.linalg.dataset.DataSet	vectorize(InputStream is, String label) Text coming from an input stream considered as one document
org.nd4j.linalg.dataset.DataSet	vectorize(String text, String label) Vectorizes the passed in text treating it as one document

Methods inherited from class org.deeplearning4j.bagofwords.vectorizer.BaseTextVectorizer

buildVocab, fit, getLabelsSource, numWordsEncountered

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.bagofwords.vectorizer.TextVectorizer

getIndex, getVocabCache

Constructor Detail

TfidfVectorizer

public TfidfVectorizer()

Method Detail

vectorize

public org.nd4j.linalg.dataset.DataSet vectorize(InputStream is,
                                                 String label)

Text coming from an input stream considered as one document

Parameters:

is - the input stream to read from

label - the label to assign

Returns:

a dataset with a applyTransformToDestination of weights(relative to impl; could be word counts or tfidf scores)

vectorize

public org.nd4j.linalg.dataset.DataSet vectorize(String text,
                                                 String label)

Vectorizes the passed in text treating it as one document

Parameters:

text - the text to vectorize

label - the label of the text

Returns:

a dataset with a transform of weights(relative to impl; could be word counts or tfidf scores)

vectorize

public org.nd4j.linalg.dataset.DataSet vectorize(File input,
                                                 String label)

Parameters:

input - the text to vectorize

label - the label of the text

Returns:

DataSet with a applyTransformToDestination of weights(relative to impl; could be word counts or tfidf scores)

transform

public org.nd4j.linalg.api.ndarray.INDArray transform(String text)

Transforms the matrix

Parameters:

text - text to transform

Returns:

INDArray

transform

public org.nd4j.linalg.api.ndarray.INDArray transform(List<String> tokens)

Description copied from interface: TextVectorizer

Transforms the matrix

Returns:

tfidfWord

public double tfidfWord(String word,
                        long wordCount,
                        long documentLength)

vectorize

public org.nd4j.linalg.dataset.DataSet vectorize()

Vectorizes the input source in to a dataset

Returns:

Adam Gibson