org.deeplearning4j.models.embeddings.loader

Class WordVectorSerializer

java.lang.Object

org.deeplearning4j.models.embeddings.loader.WordVectorSerializer

public class WordVectorSerializer
extends Object

Loads word 2 vec models

Author:

Adam Gibson

Constructor Summary

Constructors

Constructor and Description
WordVectorSerializer()

Method Summary

Modifier and Type	Method and Description
static WordVectors	fromPair(Pair<WeightLookupTable,VocabCache> pair) Load word vectors from the given pair
static WordVectors	fromTableAndVocab(WeightLookupTable table, VocabCache vocab) Load word vectors for the given vocab and table
static Word2Vec	loadGoogleModel(String path, boolean binary) Loads the google model
static Pair<WeightLookupTable,VocabCache>	loadTxt(File path) Loads an in memory cache from the given path (sets syn0 and the vocab)
static WordVectors	loadTxtVectors(File path) Loads an in memory cache from the given path (sets syn0 and the vocab)
static void	writeTsneFormat(Glove vec, org.nd4j.linalg.api.ndarray.INDArray tsne, File csv) Write the tsne format
static void	writeTsneFormat(Word2Vec vec, org.nd4j.linalg.api.ndarray.INDArray tsne, File csv) Write the tsne format
static void	writeWordVectors(InMemoryLookupTable l, InMemoryLookupCache cache, String path) Writes the word vectors to the given path.
static void	writeWordVectors(Word2Vec vec, String path) Writes the word vectors to the given path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordVectorSerializer

public WordVectorSerializer()

Method Detail

loadGoogleModel

public static Word2Vec loadGoogleModel(String path,
                                       boolean binary)
                                throws IOException

Loads the google model

Parameters:

path - the path to the google model

Returns:

the loaded model

Throws:

IOException

writeWordVectors

public static void writeWordVectors(InMemoryLookupTable l,
                                    InMemoryLookupCache cache,
                                    String path)
                             throws IOException

Writes the word vectors to the given path. Note that this assumes an in memory cache

Parameters:

path - the path to write

Throws:

IOException

writeWordVectors

public static void writeWordVectors(Word2Vec vec,
                                    String path)
                             throws IOException

Writes the word vectors to the given path. Note that this assumes an in memory cache

Parameters:

vec - the word2vec to write

path - the path to write

Throws:

IOException

fromTableAndVocab

public static WordVectors fromTableAndVocab(WeightLookupTable table,
                                            VocabCache vocab)

Load word vectors for the given vocab and table

Parameters:

table - the weights to use

vocab - the vocab to use

Returns:

wordvectors based on the given parameters

fromPair

public static WordVectors fromPair(Pair<WeightLookupTable,VocabCache> pair)

Load word vectors from the given pair

Parameters:

pair - the given pair

Returns:

a read only word vectors impl based on the given lookup table and vocab

loadTxtVectors

public static WordVectors loadTxtVectors(File path)
                                  throws FileNotFoundException

Loads an in memory cache from the given path (sets syn0 and the vocab)

Parameters:

path - the path of the file to load\

Returns:

the

Throws:

FileNotFoundException

loadTxt

public static Pair<WeightLookupTable,VocabCache> loadTxt(File path)
                                                  throws FileNotFoundException

Loads an in memory cache from the given path (sets syn0 and the vocab)

Parameters:

path - the path of the file to load

Returns:

the

Throws:

FileNotFoundException

writeTsneFormat

public static void writeTsneFormat(Glove vec,
                                   org.nd4j.linalg.api.ndarray.INDArray tsne,
                                   File csv)
                            throws Exception

Write the tsne format

Parameters:

vec - the word vectors to use for labeling

tsne - the tsne array to write

csv - the file to use

Throws:

Exception

writeTsneFormat

public static void writeTsneFormat(Word2Vec vec,
                                   org.nd4j.linalg.api.ndarray.INDArray tsne,
                                   File csv)
                            throws Exception

Write the tsne format

Parameters:

vec - the word vectors to use for labeling

tsne - the tsne array to write

csv - the file to use

Throws:

Exception