Package org.deeplearning4j.models.word2vec.wordstore.inmemory

Class InMemoryLookupCache

    • Field Detail

      • wordFrequencies

        public org.nd4j.common.primitives.Counter<String> wordFrequencies
        Deprecated.

      • docFrequencies

        public org.nd4j.common.primitives.Counter<String> docFrequencies
        Deprecated.

    • Constructor Detail

      • InMemoryLookupCache

        public InMemoryLookupCache()
        Deprecated.

      • InMemoryLookupCache

        @Deprecated
public InMemoryLookupCache​(boolean addUnk)
        Deprecated.

    • Method Detail

      • setWordFrequencies

        public void setWordFrequencies​(org.nd4j.common.primitives.Counter<String> cnt)
        Deprecated.

      • getWordFrequencies

        public org.nd4j.common.primitives.Counter<String> getWordFrequencies()
        Deprecated.

      • incrementWordCount

        public void incrementWordCount​(String word)
        Deprecated.
        Increment the count for the given word
        Specified by:
        incrementWordCount in interface VocabCache<VocabWord>
        Parameters:
        word - the word to increment the count for

      • incrementWordCount

        public void incrementWordCount​(String word,
                               int increment)
        Deprecated.
        Increment the count for the given word by the amount increment
        Specified by:
        incrementWordCount in interface VocabCache<VocabWord>
        Parameters:
        word - the word to increment the count for
        increment - the amount to increment by

      • wordFrequency

        public int wordFrequency​(String word)
        Deprecated.
        Returns the number of times the word has occurred
        Specified by:
        wordFrequency in interface VocabCache<VocabWord>
        Parameters:
        word - the word to retrieve the occurrence frequency for
        Returns:
        0 if hasn't occurred or the number of times the word occurs

      • containsWord

        public boolean containsWord​(String word)
        Deprecated.
        Returns true if the cache contains the given word
        Specified by:
        containsWord in interface VocabCache<VocabWord>
        Parameters:
        word - the word to check for
        Returns:

      • wordAtIndex

        public String wordAtIndex​(int index)
        Deprecated.
        Returns the word contained at the given index or null
        Specified by:
        wordAtIndex in interface VocabCache<VocabWord>
        Parameters:
        index - the index of the word to get
        Returns:
        the word at the given index

      • indexOf

        public int indexOf​(String word)
        Deprecated.
        Returns the index of a given word
        Specified by:
        indexOf in interface VocabCache<VocabWord>
        Parameters:
        word - the index of a given word
        Returns:
        the index of a given word or -1 if not found

      • totalWordOccurrences

        public long totalWordOccurrences()
        Deprecated.
        The total number of word occurrences
        Specified by:
        totalWordOccurrences in interface VocabCache<VocabWord>
        Returns:
        the total number of word occurrences

      • putVocabWord

        @Deprecated
public void putVocabWord​(String word)
        Deprecated.
        Description copied from interface: VocabCache
        Inserts the word as a vocab word (it gets the vocab word from the internal token store). Note that the index must be set on the token.
        Specified by:
        putVocabWord in interface VocabCache<VocabWord>
        Parameters:
        word -

      • numWords

        public int numWords()
        Deprecated.
        Returns the number of words in the cache
        Specified by:
        numWords in interface VocabCache<VocabWord>
        Returns:
        the number of words in the cache

      • docAppearedIn

        public int docAppearedIn​(String word)
        Deprecated.
        Description copied from interface: VocabCache
        Count of documents a word appeared in
        Specified by:
        docAppearedIn in interface VocabCache<VocabWord>
        Parameters:
        word - the number of documents the word appeared in
        Returns:

      • incrementDocCount

        public void incrementDocCount​(String word,
                              long howMuch)
        Deprecated.
        Description copied from interface: VocabCache
        Increment the document count
        Specified by:
        incrementDocCount in interface VocabCache<VocabWord>
        Parameters:
        word - the word to increment by

      • setCountForDoc

        public void setCountForDoc​(String word,
                           long count)
        Deprecated.
        Description copied from interface: VocabCache
        Set the count for the number of documents the word appears in
        Specified by:
        setCountForDoc in interface VocabCache<VocabWord>
        Parameters:
        word - the word to set the count for
        count - the count of the word

      • totalNumberOfDocs

        public long totalNumberOfDocs()
        Deprecated.
        Description copied from interface: VocabCache
        Returns the total of number of documents encountered in the corpus
        Specified by:
        totalNumberOfDocs in interface VocabCache<VocabWord>
        Returns:
        the total number of docs in the corpus

      • incrementTotalDocCount

        public void incrementTotalDocCount​(long by)
        Deprecated.
        Description copied from interface: VocabCache
        Increment the doc count
        Specified by:
        incrementTotalDocCount in interface VocabCache<VocabWord>
        Parameters:
        by - the number to increment by

      • addToken

        public boolean addToken​(VocabWord word)
        Deprecated.
        Description copied from interface: VocabCache
        Adds a token to the cache
        Specified by:
        addToken in interface VocabCache<VocabWord>
        Parameters:
        word - the word to add
        Returns:
        true if token was added, false if updated

      • tokenFor

        public VocabWord tokenFor​(String word)
        Deprecated.
        Description copied from interface: VocabCache
        Returns the token (again not necessarily in the vocab) for this word
        Specified by:
        tokenFor in interface VocabCache<VocabWord>
        Parameters:
        word - the word to get the token for
        Returns:
        the vocab word for this token

      • hasToken

        public boolean hasToken​(String token)
        Deprecated.
        Description copied from interface: VocabCache
        Returns whether the cache contains this token or not
        Specified by:
        hasToken in interface VocabCache<VocabWord>
        Parameters:
        token - the token to tes
        Returns:
        whether the token exists in the cache or not

      • removeElement

        public void removeElement​(String label)
        Deprecated.
        Description copied from interface: VocabCache
        Removes element with specified label from vocabulary Please note: Huffman index should be updated after element removal
        Specified by:
        removeElement in interface VocabCache<VocabWord>
        Parameters:
        label - label of the element to be removed

      • removeElement

        public void removeElement​(VocabWord element)
        Deprecated.
        Description copied from interface: VocabCache
        Removes specified element from vocabulary Please note: Huffman index should be updated after element removal
        Specified by:
        removeElement in interface VocabCache<VocabWord>
        Parameters:
        element - SequenceElement to be removed

      • saveVocab

        public void saveVocab()
        Deprecated.
        Description copied from interface: VocabCache
        Saves the vocab: this allow for reuse of word frequencies
        Specified by:
        saveVocab in interface VocabCache<VocabWord>

      • load

        public static InMemoryLookupCache load​(InputStream from)
        Deprecated.
        Load a look up cache from an input stream delimited by \n
        Parameters:
        from - the input stream to read from
        Returns:
        the in memory lookup cache

      • equals

        public boolean equals​(Object o)
        Deprecated.
        Overrides:
        equals in class Object

      • hashCode

        public int hashCode()
        Deprecated.
        Overrides:
        hashCode in class Object