eu.project.ttc.api

Class TerminoExtractor

java.lang.Object

eu.project.ttc.api.TerminoExtractor

public class TerminoExtractor
extends java.lang.Object

A builder and launcher class for execute a terminology extraction pipeline from raw text files or from TermSuite preprocessed files.

See Also:

TermSuitePreprocessor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TerminoExtractor.ContextualizerMode

Constructor Summary

Constructors

Constructor and Description
TerminoExtractor()

Method Summary

Modifier and Type	Method and Description
TerminoExtractor	dynamicMaxSizeFilter(int maxTermIndexSize) Filters the TermIndex dynamically during the term spotting phase (RegexSpotter) of terminology extraction by cleaning by frequency whenever the number of terms in-memory exceeds a max number of terms allowed.
TermIndex	execute()
static TerminoExtractor	fromDocumentCollection(Lang lang, java.util.Collection<Document> documents)
static TerminoExtractor	fromDocumentStream(Lang lang, java.util.stream.Stream<Document> documentStream, long streamSize)
static TerminoExtractor	fromPreprocessedDocumentStream(Lang lang, java.util.stream.Stream<org.apache.uima.jcas.JCas> casStream, long streamSize)
static TerminoExtractor	fromPreprocessedJsonFiles(Lang lang, java.lang.String directory)
static TerminoExtractor	fromPreprocessedJsonFiles(Lang lang, java.lang.String directory, java.lang.String encoding)
static TerminoExtractor	fromPreprocessedXmiFiles(Lang lang, java.lang.String directory) WARNING : encoding of XMI file must be UTF-8.
static TerminoExtractor	fromSingleDocument(Lang lang, Document document)
static TerminoExtractor	fromSinglePreprocessedDocument(Lang lang, org.apache.uima.jcas.JCas cas)
static TerminoExtractor	fromTextString(Lang lang, java.lang.String text)
static TerminoExtractor	fromTxtCorpus(Lang lang, java.lang.String directory, java.lang.String pattern)
static TerminoExtractor	fromTxtCorpus(Lang lang, java.lang.String directory, java.lang.String pattern, java.lang.String encoding)
TerminoExtractor	postFilter(TerminoFilterConfig filterConfig) Filters the TermIndex at the end of the pipeline, i.e. after the term variant detection phase.
TerminoExtractor	preFilter(TerminoFilterConfig filterConfig) Filters the TermIndex before the term variant detection phase.
TerminoExtractor	setTreeTaggerHome(java.lang.String treeTaggerHome)
TerminoExtractor	setWatcher(TermHistory history)
TerminoExtractor	useContextualizer(int scope, TerminoExtractor.ContextualizerMode contextualizerMode)
TerminoExtractor	usingCustomResources(java.lang.String resourceDir)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TerminoExtractor

public TerminoExtractor()

Method Detail

fromTextString

public static TerminoExtractor fromTextString(Lang lang,
                                              java.lang.String text)

fromSingleDocument

public static TerminoExtractor fromSingleDocument(Lang lang,
                                                  Document document)

fromPreprocessedJsonFiles

public static TerminoExtractor fromPreprocessedJsonFiles(Lang lang,
                                                         java.lang.String directory)

fromPreprocessedJsonFiles

public static TerminoExtractor fromPreprocessedJsonFiles(Lang lang,
                                                         java.lang.String directory,
                                                         java.lang.String encoding)

fromPreprocessedXmiFiles

public static TerminoExtractor fromPreprocessedXmiFiles(Lang lang,
                                                        java.lang.String directory)

WARNING : encoding of XMI file must be UTF-8.

fromPreprocessedDocumentStream

public static TerminoExtractor fromPreprocessedDocumentStream(Lang lang,
                                                              java.util.stream.Stream<org.apache.uima.jcas.JCas> casStream,
                                                              long streamSize)

fromDocumentStream

public static TerminoExtractor fromDocumentStream(Lang lang,
                                                  java.util.stream.Stream<Document> documentStream,
                                                  long streamSize)

fromDocumentCollection

public static TerminoExtractor fromDocumentCollection(Lang lang,
                                                      java.util.Collection<Document> documents)

fromTxtCorpus

public static TerminoExtractor fromTxtCorpus(Lang lang,
                                             java.lang.String directory,
                                             java.lang.String pattern)

fromTxtCorpus

public static TerminoExtractor fromTxtCorpus(Lang lang,
                                             java.lang.String directory,
                                             java.lang.String pattern,
                                             java.lang.String encoding)

fromSinglePreprocessedDocument

public static TerminoExtractor fromSinglePreprocessedDocument(Lang lang,
                                                              org.apache.uima.jcas.JCas cas)

setTreeTaggerHome

public TerminoExtractor setTreeTaggerHome(java.lang.String treeTaggerHome)

usingCustomResources

public TerminoExtractor usingCustomResources(java.lang.String resourceDir)

useContextualizer

public TerminoExtractor useContextualizer(int scope,
                                          TerminoExtractor.ContextualizerMode contextualizerMode)

preFilter

public TerminoExtractor preFilter(TerminoFilterConfig filterConfig)

Filters the TermIndex before the term variant detection phase. This early-stage filtering will result in missing several low-frequency variations during the term variation detection but is often necessary when detecting variant takes too long.

Parameters:

filterConfig - The filtering configuration

Returns:

this TerminoExtractor launcher class

See Also:

postFilter(TerminoFilterConfig)

dynamicMaxSizeFilter

public TerminoExtractor dynamicMaxSizeFilter(int maxTermIndexSize)

Filters the TermIndex dynamically during the term spotting phase (RegexSpotter) of terminology extraction by cleaning by frequency whenever the number of terms in-memory exceeds a max number of terms allowed.

Parameters:

maxTermIndexSize - the maximum number of Term instances allowed to be kept in memory during the terminology extraction process.

Returns:

this TerminoExtractor launcher class

See Also:

TermSuitePipeline.aeMaxSizeThresholdCleaner(TermProperty, int)

postFilter

public TerminoExtractor postFilter(TerminoFilterConfig filterConfig)

Filters the TermIndex at the end of the pipeline, i.e. after the term variant detection phase. This filtering is loss-less when configured with TerminoFilterConfig#keepVariants(true).

Parameters:

filterConfig - The filtering configuration

Returns:

this TerminoExtractor launcher class

See Also:

preFilter(TerminoFilterConfig)

execute

public TermIndex execute()

setWatcher

public TerminoExtractor setWatcher(TermHistory history)