mirror of https://github.com/explosion/spaCy.git
bb911e5f4e
* Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training |
||
|---|---|---|
| .. | ||
| __init__.pxd | ||
| __init__.py | ||
| _beam_utils.pxd | ||
| _beam_utils.pyx | ||
| _parser_model.pxd | ||
| _parser_model.pyx | ||
| _state.pxd | ||
| _state.pyx | ||
| arc_eager.pxd | ||
| arc_eager.pyx | ||
| ner.pxd | ||
| ner.pyx | ||
| nn_parser.pxd | ||
| nn_parser.pyx | ||
| nonproj.pxd | ||
| nonproj.pyx | ||
| stateclass.pxd | ||
| stateclass.pyx | ||
| transition_system.pxd | ||
| transition_system.pyx | ||