org.clulab.processors.clu.tokenizer
Tokenizes some hyphenated prefixes, which are better handled downstream as separate tokens For example: "mid-July" is separated into "mid" and "July", which is better for date recognition
Tokenizes some hyphenated prefixes, which are better handled downstream as separate tokens For example: "mid-July" is separated into "mid" and "July", which is better for date recognition