org.apache.spark.mllib.feature

Class Word2VecModel

java.lang.Object

org.apache.spark.mllib.feature.Word2VecModel

All Implemented Interfaces:

java.io.Serializable, Saveable

public class Word2VecModel
extends java.lang.Object
implements scala.Serializable, Saveable

:: Experimental :: Word2Vec model param: wordIndex maps each word to an index, which can retrieve the corresponding vector from wordVectors param: wordVectors array of length numWords * vectorSize, vector corresponding to the word mapped with index i can be retrieved by the slice (i * vectorSize, i * vectorSize + vectorSize)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Word2VecModel(scala.collection.immutable.Map<java.lang.String,float[]> model)

Method Summary

Methods

Modifier and Type	Method and Description
scala.Tuple2<java.lang.String,java.lang.Object>[]	findSynonyms(java.lang.String word, int num) Find synonyms of a word
scala.Tuple2<java.lang.String,java.lang.Object>[]	findSynonyms(Vector vector, int num) Find synonyms of the vector representation of a word
protected java.lang.String	formatVersion() Current version of model save/load format.
scala.collection.immutable.Map<java.lang.String,float[]>	getVectors() Returns a map of words to their vector representations.
static Word2VecModel	load(SparkContext sc, java.lang.String path)
void	save(SparkContext sc, java.lang.String path) Save this model to the given path.
Vector	transform(java.lang.String word) Transforms a word to its vector representation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Word2VecModel

public Word2VecModel(scala.collection.immutable.Map<java.lang.String,float[]> model)

Method Detail

load

public static Word2VecModel load(SparkContext sc,
                 java.lang.String path)

formatVersion

protected java.lang.String formatVersion()

Description copied from interface: Saveable

Current version of model save/load format.

Specified by:

formatVersion in interface Saveable

save

public void save(SparkContext sc,
        java.lang.String path)

Description copied from interface: Saveable

Save this model to the given path.

This saves: - human-readable (JSON) model metadata to path/metadata/ - Parquet formatted data to path/data/

The model may be loaded using Loader.load.

Specified by:

save in interface Saveable

Parameters:

sc - Spark context used to save model data.

path - Path specifying the directory in which to save this model. If the directory already exists, this method throws an exception.

transform

public Vector transform(java.lang.String word)

Transforms a word to its vector representation

Parameters:

word - a word

Returns:

vector representation of word

findSynonyms

public scala.Tuple2<java.lang.String,java.lang.Object>[] findSynonyms(java.lang.String word,
                                                             int num)

Find synonyms of a word

Parameters:

word - a word

num - number of synonyms to find

Returns:

array of (word, cosineSimilarity)

findSynonyms

public scala.Tuple2<java.lang.String,java.lang.Object>[] findSynonyms(Vector vector,
                                                             int num)

Find synonyms of the vector representation of a word

Parameters:

vector - vector representation of a word

num - number of synonyms to find

Returns:

array of (word, cosineSimilarity)

getVectors

public scala.collection.immutable.Map<java.lang.String,float[]> getVectors()

Returns a map of words to their vector representations.

Returns:

(undocumented)