Tokenizer

textmogrify.lucene.Tokenizer
object Tokenizer

Attributes

Source
Tokenizer.scala
Graph
Supertypes
class Object
trait Matchable
class Any
Self type
Tokenizer.type

Members list

Value members

Concrete methods

def vectorTokenizer[F[_]](analyzer: Resource[F, Analyzer])(implicit F: Sync[F]): Resource[F, String => F[Vector[String]]]

Build a tokenizing function that runs its input through the Analyzer and collects all tokens into a Vector

Build a tokenizing function that runs its input through the Analyzer and collects all tokens into a Vector

Attributes

Source
Tokenizer.scala