mirror of https://github.com/explosion/spaCy.git
bb911e5f4e
* Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| entityruler.py | ||
| functions.py | ||
| hooks.py | ||
| pipes.pyx | ||