trait Operators extends AnyRef
High level NLP operators.
- Alphabetic
- By Inheritance
- Operators
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
- def +(other: String): String
- def ->[B](y: B): (Operators, B)
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
bigram(p: Double, minFreq: Int, text: String*): Array[BigramCollocation]
Identify bigram collocations whose p-value is less than the given threshold.
Identify bigram collocations whose p-value is less than the given threshold.
- p
the p-value threshold
- minFreq
the minimum frequency of collocation.
- text
input text.
- returns
significant bigram collocations in descending order of likelihood ratio.
-
def
bigram(k: Int, minFreq: Int, text: String*): Array[BigramCollocation]
Identify bigram collocations (words that often appear consecutively) within corpora.
Identify bigram collocations (words that often appear consecutively) within corpora. They may also be used to find other associations between word occurrences.
Finding collocations requires first calculating the frequencies of words and their appearance in the context of other words. Often the collection of words will then requiring filtering to only retain useful content terms. Each n-gram of words may then be scored according to some association measure, in order to determine the relative likelihood of each n-gram being a collocation.
- k
finds top k bigram.
- minFreq
the minimum frequency of collocation.
- text
input text.
- returns
significant bigram collocations in descending order of likelihood ratio.
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )
-
def
corpus(text: Seq[String]): SimpleCorpus
Creates an in-memory text corpus.
Creates an in-memory text corpus.
- text
a set of text.
-
def
df(terms: Array[String], corpus: Array[Map[String, Int]]): Array[Int]
Returns the document frequencies, i.e.
Returns the document frequencies, i.e. the number of documents that contain term.
- terms
the token list used as features.
- corpus
the training corpus.
- returns
the array of document frequencies.
- def ensuring(cond: (Operators) ⇒ Boolean, msg: ⇒ Any): Operators
- def ensuring(cond: (Operators) ⇒ Boolean): Operators
- def ensuring(cond: Boolean, msg: ⇒ Any): Operators
- def ensuring(cond: Boolean): Operators
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
- def formatted(fmtstr: String): String
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
ngram(maxNGramSize: Int, minFreq: Int, text: String*): Seq[Seq[NGram]]
An Apiori-like algorithm to extract n-gram phrases.
An Apiori-like algorithm to extract n-gram phrases.
- maxNGramSize
The maximum length of n-gram
- minFreq
The minimum frequency of n-gram in the sentences.
- text
input text.
- returns
An array of sets of n-grams. The i-th entry is the set of i-grams.
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
postag(sentence: Array[String]): Array[PennTreebankPOS]
Part-of-speech taggers.
Part-of-speech taggers.
- sentence
a sentence that is already segmented to words.
- returns
the pos tags.
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
tfidf(bag: Array[Double], n: Int, df: Array[Int]): Array[Double]
Converts a bag of words to a feature vector by TF-IDF, which is normalized to L2 norm 1.
Converts a bag of words to a feature vector by TF-IDF, which is normalized to L2 norm 1.
- bag
the bag-of-words feature vector of a document.
- n
the number of documents in training corpus.
- df
the number of documents containing the given term in the corpus.
- returns
TF-IDF feature vector
-
def
tfidf(corpus: Array[Array[Double]]): Array[Array[Double]]
Converts a corpus to TF-IDF feature vectors, which are normalized to L2 norm 1.
Converts a corpus to TF-IDF feature vectors, which are normalized to L2 norm 1.
- corpus
the corpus of documents in bag-of-words representation.
- returns
a matrix of which each row is the TF-IDF feature vector.
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
vectorize(terms: Array[String], bag: Set[String]): Array[Int]
Converts a binary bag of words to a sparse feature vector.
Converts a binary bag of words to a sparse feature vector.
- terms
the token list used as features.
- bag
the bag of words.
- returns
an integer vector, which elements are the indices of presented feature tokens in ascending order.
-
def
vectorize(terms: Array[String], bag: Map[String, Int]): Array[Double]
Converts a bag of words to a feature vector.
Converts a bag of words to a feature vector.
- terms
the token list used as features.
- bag
the bag of words.
- returns
a vector of frequency of feature tokens in the bag.
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )
- def →[B](y: B): (Operators, B)
High level Smile operators in Scala.