org.deeplearning4j.models.embeddings.loader

Class WordVectorSerializer

java.lang.Object

org.deeplearning4j.models.embeddings.loader.WordVectorSerializer

public class WordVectorSerializer
extends Object

Loads word 2 vec models

Author:

Adam Gibson

Constructor Summary

Constructors

Constructor and Description
WordVectorSerializer()

Method Summary

Modifier and Type	Method and Description
static float	getFloat(byte[] b) Read float from byte array, credit to: Credit to: https://github.com/NLPchina/Word2VEC_java/blob/master/src/com/ansj/vec/Word2VEC.java
static Word2Vec	loadGoogleModel(String path, boolean binary) Loads the google model
static Pair<WeightLookupTable,VocabCache>	loadTxt(File path) Loads an in memory cache from the given path (sets syn0 and the vocab)
static WordVectors	loadTxtVectors(File path) Loads an in memory cache from the given path (sets syn0 and the vocab)
static float	readFloat(InputStream is) Credit to: https://github.com/NLPchina/Word2VEC_java/blob/master/src/com/ansj/vec/Word2VEC.java
static void	writeTsneFormat(Glove vec, org.nd4j.linalg.api.ndarray.INDArray tsne, File csv) Write the tsne format
static void	writeTsneFormat(Word2Vec vec, org.nd4j.linalg.api.ndarray.INDArray tsne, File csv) Write the tsne format
static void	writeWordVectors(InMemoryLookupTable l, InMemoryLookupCache cache, String path) Writes the word vectors to the given path.
static void	writeWordVectors(Word2Vec vec, String path) Writes the word vectors to the given path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordVectorSerializer

public WordVectorSerializer()

Method Detail

loadGoogleModel

public static Word2Vec loadGoogleModel(String path,
                                       boolean binary)
                                throws IOException

Loads the google model

Parameters:

path - the path to the google model

Returns:

the loaded model

Throws:

IOException

readFloat

public static float readFloat(InputStream is)
                       throws IOException

Credit to: https://github.com/NLPchina/Word2VEC_java/blob/master/src/com/ansj/vec/Word2VEC.java

Parameters:

is -

Returns:

Throws:

IOException

getFloat

public static float getFloat(byte[] b)

Read float from byte array, credit to: Credit to: https://github.com/NLPchina/Word2VEC_java/blob/master/src/com/ansj/vec/Word2VEC.java

Parameters:

b -

Returns:

writeWordVectors

public static void writeWordVectors(InMemoryLookupTable l,
                                    InMemoryLookupCache cache,
                                    String path)
                             throws IOException

Writes the word vectors to the given path. Note that this assumes an in memory cache

Parameters:

path - the path to write

Throws:

IOException

writeWordVectors

public static void writeWordVectors(Word2Vec vec,
                                    String path)
                             throws IOException

Writes the word vectors to the given path. Note that this assumes an in memory cache

Parameters:

vec - the word2vec to write

path - the path to write

Throws:

IOException

loadTxtVectors

public static WordVectors loadTxtVectors(File path)
                                  throws FileNotFoundException

Loads an in memory cache from the given path (sets syn0 and the vocab)

Parameters:

path - the path of the file to load

Returns:

the

Throws:

FileNotFoundException

loadTxt

public static Pair<WeightLookupTable,VocabCache> loadTxt(File path)
                                                  throws FileNotFoundException

Loads an in memory cache from the given path (sets syn0 and the vocab)

Parameters:

path - the path of the file to load

Returns:

the

Throws:

FileNotFoundException

writeTsneFormat

public static void writeTsneFormat(Glove vec,
                                   org.nd4j.linalg.api.ndarray.INDArray tsne,
                                   File csv)
                            throws Exception

Write the tsne format

Parameters:

vec - the word vectors to use for labeling

tsne - the tsne array to write

csv - the file to use

Throws:

Exception

writeTsneFormat

public static void writeTsneFormat(Word2Vec vec,
                                   org.nd4j.linalg.api.ndarray.INDArray tsne,
                                   File csv)
                            throws Exception

Write the tsne format

Parameters:

vec - the word vectors to use for labeling

tsne - the tsne array to write

csv - the file to use

Throws:

Exception