icu_tokenizer.c File Reference

ICU tokenization - using ubrk_-functions from ICU. More...

Go to the source code of this file.

Detailed Description

ICU tokenization - using ubrk_-functions from ICU.

Definition in file icu_tokenizer.c.