Commit Graph

2 Commits

Author SHA1 Message Date
Ines Montani f12a46472c Remove unicode declarations 2020-03-26 15:18:32 +01:00
adrianeboyd 86c43e55fa
Improve Lithuanian tokenization (#5205)
* Improve Lithuanian tokenization

Modify Lithuanian tokenization to improve performance for
UD_Lithuanian-ALKSNIS.

* Update Lithuanian tokenizer tests
2020-03-25 11:28:12 +01:00