textmogrify-docs/textmogrify.lucene/Tokenizer

Tokenizer

object Tokenizer
Source:
Tokenizer.scala
class Object
trait Matchable
class Any

Value members

Concrete methods

def vectorTokenizer[F[_]](analyzer: Analyzer)(implicit F: Sync[F]): String => F[Vector[String]]

Build a tokenizing function that runs its input through the Analyzer and collects all tokens into a Vector

Build a tokenizing function that runs its input through the Analyzer and collects all tokens into a Vector

Source:
Tokenizer.scala