mirror of https://github.com/explosion/spaCy.git
bb911e5f4e
* Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training |
||
|---|---|---|
| .. | ||
| converters | ||
| __init__.py | ||
| _schemas.py | ||
| convert.py | ||
| debug_data.py | ||
| download.py | ||
| evaluate.py | ||
| info.py | ||
| init_model.py | ||
| link.py | ||
| package.py | ||
| pretrain.py | ||
| profile.py | ||
| train.py | ||
| validate.py | ||