spaCy/spacy
Koichi Yasuoka 0afb54ac93
JapaneseTokenizer.pipe added (#6515)
* JapaneseTokenizer.pipe added

For [spacymoji](https://spacy.io/universe/project/spacymoji)  with `Japanese()`.

* DummyTokenizer.pipe added instead
2020-12-08 20:02:23 +01:00
..
cli Remove non-working --use-chars from train CLI 2020-12-08 08:30:00 +01:00
data
displacy Fix on EntityRendered to support break lines (after last entity) (closes #5838) 2020-07-29 18:48:39 +02:00
lang Added Multext-East V5 tagset for Croatian language (#6248) 2020-11-05 12:19:22 +01:00
matcher Add SPACY as a Matcher attribute (#6463) 2020-11-30 09:34:50 +08:00
ml Reproducibility for TextCat and Tok2Vec (#6218) 2020-10-08 00:43:46 +02:00
pipeline Fix subsequent pipe detection in EntityRuler 2020-12-08 10:01:30 +01:00
syntax Restore cleanup_beam method (#6446) 2020-11-25 13:21:48 +01:00
tests Only set NORM on Token in retokenizer (#6464) 2020-11-30 09:35:42 +08:00
tokens Only set NORM on Token in retokenizer (#6464) 2020-11-30 09:35:42 +08:00
__init__.pxd
__init__.py Simplify warnings 2020-04-28 13:37:37 +02:00
__main__.py Use latest wasabi 2019-11-04 02:38:45 +01:00
_ml.py Bugfix textcat reproducibility on GPU (#6411) 2020-11-23 12:29:35 +01:00
about.py Set version to v2.3.4 2020-11-26 08:48:52 +01:00
analysis.py Simplify warnings 2020-04-28 13:37:37 +02:00
attrs.pxd Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
attrs.pyx Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
compat.py Replace function registries with catalogue (#4584) 2019-11-07 11:45:22 +01:00
errors.py Add warning when Matcher subpattern is discarded (#5873) 2020-08-05 14:56:14 +02:00
glossary.py Update tag maps and docs for English and German (#4501) 2019-10-24 12:56:05 +02:00
gold.pxd Merge changes from master 2019-08-21 14:18:52 +02:00
gold.pyx Updates to docstrings (#5589) 2020-06-15 14:58:36 +02:00
kb.pxd Tidy up and avoid absolute spacy imports in core 2020-05-21 20:05:03 +02:00
kb.pyx Merge pull request #5264 from lfiedler/issue-5230 2020-05-22 00:31:07 +02:00
language.py Move max_length to nlp.make_doc() (#6512) 2020-12-08 14:24:02 +08:00
lemmatizer.py Fix lemmatizer is_base_form for python2.7 (#5734) 2020-07-09 22:11:24 +02:00
lexeme.pxd Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
lexeme.pyx Fix polarity of Token.is_oov and Lexeme.is_oov (#5634) 2020-06-23 13:29:51 +02:00
lookups.py Reduce memory usage of Lookup's BloomFilter (#5606) 2020-06-26 14:09:10 +02:00
morphology.pxd annotate kb_id through ents in doc 2019-03-22 11:36:44 +01:00
morphology.pyx Improve tag map initialization and updating (#5768) 2020-07-19 11:13:39 +02:00
parts_of_speech.pxd
parts_of_speech.pyx
scorer.py Fix GoldParse init when token count differs (#5191) 2020-03-26 10:46:23 +01:00
strings.pxd
strings.pyx Merge branch 'master' into feature/lemmatizer 2019-03-16 13:44:22 +01:00
structs.pxd Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
symbols.pxd Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
symbols.pyx Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
tokenizer.pxd Rename to url_match 2020-05-22 12:41:03 +02:00
tokenizer.pyx Rename to url_match 2020-05-22 12:41:03 +02:00
typedefs.pxd Use int8_t instead of char in Matcher (#6413) 2020-11-23 10:26:47 +01:00
typedefs.pyx
util.py JapaneseTokenizer.pipe added (#6515) 2020-12-08 20:02:23 +01:00
vectors.pyx fix deserialization order 2020-05-30 12:53:32 +02:00
vocab.pxd Use int8_t instead of char in Matcher (#6413) 2020-11-23 10:26:47 +01:00
vocab.pyx Add all vectors to vocab before pruning (#6408) 2020-11-23 10:00:59 +01:00