spaCy/spacy
Adriane Boyd 0e7f94b247
Update Tokenizer.explain with special matches (#7749)
* Update Tokenizer.explain with special matches

Update `Tokenizer.explain` and the pseudo-code in the docs to include
the processing of special cases that contain affixes or whitespace.

* Handle optional settings in explain

* Add test for special matches in explain

Add test for `Tokenizer.explain` for special cases containing affixes.
2021-04-19 19:08:20 +10:00
..
cli assemble CLI command (#7783) 2021-04-19 18:39:11 +10:00
displacy
lang
matcher
ml Register CharEmbed layer (#7805) 2021-04-19 18:39:34 +10:00
pipeline Bugfix/nel crossing sentence (#7630) 2021-04-12 18:08:01 +10:00
tests Update Tokenizer.explain with special matches (#7749) 2021-04-19 19:08:20 +10:00
tokens
training Fix parser sourcing in NER converter (#7631) 2021-04-08 12:25:03 +02:00
__init__.pxd
__init__.py
__main__.py
about.py
attrs.pxd
attrs.pyx
compat.py
default_config.cfg
default_config_pretraining.cfg
errors.py Improve checks for sourced components (#7490) 2021-04-19 18:36:32 +10:00
glossary.py
kb.pxd
kb.pyx
language.py Improve checks for sourced components (#7490) 2021-04-19 18:36:32 +10:00
lexeme.pxd
lexeme.pyx
lookups.py
morphology.pxd
morphology.pyx
parts_of_speech.pxd
parts_of_speech.pyx
pipe_analysis.py
py.typed
schemas.py
scorer.py
strings.pxd
strings.pyx Make vocab update in get_docs deterministic (#7603) 2021-04-09 11:53:13 +02:00
structs.pxd
symbols.pxd
symbols.pyx
tokenizer.pxd
tokenizer.pyx Update Tokenizer.explain with special matches (#7749) 2021-04-19 19:08:20 +10:00
typedefs.pxd
typedefs.pyx
util.py Set catalogue lower pin to v2.0.3 (#7762) 2021-04-19 18:37:17 +10:00
vectors.pyx Fix vectors data on GPU (#7626) 2021-04-19 18:30:03 +10:00
vocab.pxd
vocab.pyx Fix vectors data on GPU (#7626) 2021-04-19 18:30:03 +10:00