spaCy

Commit Graph

Author	SHA1	Message	Date
Adriane Boyd	1d59fdbd39	Update Vietnamese tokenizer (#8099 ) * Adapt tokenization methods from `pyvi` to preserve text encoding and whitespace * Add serialization support similar to Chinese and Japanese Note: as for Chinese and Japanese, some settings are duplicated in `config.cfg` and `tokenizer/cfg`.	2021-05-17 18:16:20 +10:00
Adriane Boyd	31ec9a906e	Clean up 3rd party license info (#6478 ) Move scikit-learn license from `Scorer` to `licenses/3rd_party_licenses.txt`.	2020-12-02 10:15:23 +01:00
Adriane Boyd	caf23462eb	Add 3rd party licenses (#5959 )	2020-08-26 15:23:59 +02:00

Author

SHA1

Message

Date

Adriane Boyd

1d59fdbd39

Update Vietnamese tokenizer (#8099 )

* Adapt tokenization methods from `pyvi` to preserve text encoding and
whitespace
* Add serialization support similar to Chinese and Japanese

Note: as for Chinese and Japanese, some settings are duplicated in
`config.cfg` and `tokenizer/cfg`.

2021-05-17 18:16:20 +10:00

Adriane Boyd

31ec9a906e

Clean up 3rd party license info (#6478 )

Move scikit-learn license from `Scorer` to
`licenses/3rd_party_licenses.txt`.

2020-12-02 10:15:23 +01:00

Adriane Boyd

caf23462eb

Add 3rd party licenses (#5959 )

2020-08-26 15:23:59 +02:00

3 Commits