Class SequenceVectors<T extends SequenceElement>
- java.lang.Object
-
- org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl<T>
-
- org.deeplearning4j.models.sequencevectors.SequenceVectors<T>
-
- All Implemented Interfaces:
Serializable
,WordVectors
,org.deeplearning4j.nn.weights.embeddings.EmbeddingInitializer
public class SequenceVectors<T extends SequenceElement> extends WordVectorsImpl<T> implements WordVectors
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected class
SequenceVectors.AsyncSequencer
This class is used to fetch data from iterator in background thread, and convert it to ListIt becomes very usefull if text processing pipeline behind iterator is complex, and we're not loading data from simple text file with whitespaces as separator. static class
SequenceVectors.Builder<T extends SequenceElement>
-
Field Summary
Fields Modifier and Type Field Description protected VectorsConfiguration
configuration
protected boolean
configured
protected ElementsLearningAlgorithm<T>
elementsLearningAlgorithm
protected boolean
enableScavenger
protected Set<VectorsListener<T>>
eventListeners
protected WordVectors
existingModel
protected WordVectors
intersectModel
protected SequenceIterator<T>
iterator
protected boolean
lockFactor
protected static org.slf4j.Logger
log
protected org.nd4j.shade.guava.util.concurrent.AtomicDouble
scoreElements
protected org.nd4j.shade.guava.util.concurrent.AtomicDouble
scoreSequences
protected SequenceLearningAlgorithm<T>
sequenceLearningAlgorithm
protected T
unknownElement
protected int
vocabLimit
-
Fields inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl
batchSize, DEFAULT_UNK, layerSize, learningRate, learningRateDecayWords, lookupTable, minLearningRate, minWordFrequency, modelUtils, negative, numEpochs, numIterations, resetModel, sampling, seed, stopWords, trainElementsVectors, trainSequenceVectors, useAdeGrad, useUnknown, variableWindows, vocab, window, workers
-
-
Constructor Summary
Constructors Constructor Description SequenceVectors()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
buildVocab()
Builds vocabulary from provided SequenceIterator instancevoid
fit()
Starts training overdouble
getElementsScore()
double
getSequencesScore()
String
getUNK()
org.nd4j.linalg.api.ndarray.INDArray
getWordVectorMatrix(String word)
Get the word vector for a given matrixprotected void
initLearners()
void
setUNK(String UNK)
protected void
trainSequence(@NonNull Sequence<T> sequence, AtomicLong nextRandom, double alpha)
-
Methods inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl
accuracy, getLayerSize, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, jsonSerializable, loadWeightsInto, lookupTable, outOfVocabularySupported, setLookupTable, setModelUtils, setVocab, similarity, similarWordsInVocabTo, update, update, vectorSize, vocab, vocabSize, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.deeplearning4j.nn.weights.embeddings.EmbeddingInitializer
jsonSerializable, loadWeightsInto, vectorSize, vocabSize
-
Methods inherited from interface org.deeplearning4j.models.embeddings.wordvectors.WordVectors
accuracy, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, lookupTable, outOfVocabularySupported, setModelUtils, similarity, similarWordsInVocabTo, vocab, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum
-
-
-
-
Field Detail
-
iterator
protected transient SequenceIterator<T extends SequenceElement> iterator
-
elementsLearningAlgorithm
protected transient ElementsLearningAlgorithm<T extends SequenceElement> elementsLearningAlgorithm
-
sequenceLearningAlgorithm
protected transient SequenceLearningAlgorithm<T extends SequenceElement> sequenceLearningAlgorithm
-
configuration
protected VectorsConfiguration configuration
-
log
protected static final org.slf4j.Logger log
-
existingModel
protected transient WordVectors existingModel
-
intersectModel
protected transient WordVectors intersectModel
-
unknownElement
protected transient T extends SequenceElement unknownElement
-
scoreElements
protected transient org.nd4j.shade.guava.util.concurrent.AtomicDouble scoreElements
-
scoreSequences
protected transient org.nd4j.shade.guava.util.concurrent.AtomicDouble scoreSequences
-
configured
protected transient boolean configured
-
lockFactor
protected transient boolean lockFactor
-
enableScavenger
protected boolean enableScavenger
-
vocabLimit
protected int vocabLimit
-
eventListeners
protected transient Set<VectorsListener<T extends SequenceElement>> eventListeners
-
-
Method Detail
-
getUNK
public String getUNK()
- Specified by:
getUNK
in interfaceWordVectors
-
setUNK
public void setUNK(String UNK)
- Specified by:
setUNK
in interfaceWordVectors
-
getElementsScore
public double getElementsScore()
-
getSequencesScore
public double getSequencesScore()
-
getWordVectorMatrix
public org.nd4j.linalg.api.ndarray.INDArray getWordVectorMatrix(String word)
Description copied from interface:WordVectors
Get the word vector for a given matrix- Specified by:
getWordVectorMatrix
in interfaceWordVectors
- Overrides:
getWordVectorMatrix
in classWordVectorsImpl<T extends SequenceElement>
- Parameters:
word
- the word to get the matrix for- Returns:
- the ndarray for this word
-
buildVocab
public void buildVocab()
Builds vocabulary from provided SequenceIterator instance
-
initLearners
protected void initLearners()
-
fit
public void fit()
Starts training over
-
trainSequence
protected void trainSequence(@NonNull @NonNull Sequence<T> sequence, AtomicLong nextRandom, double alpha)
-
-