spaCy

Commit Graph

Author	SHA1	Message	Date
Ines Montani	3d8fd4b461	Revert #4334	2019-09-29 17:32:12 +02:00
Ines Montani	c9cd516d96	Move tests out of package (#4334 ) * Move tests out of package * Fix typo	2019-09-28 18:05:00 +02:00
Matthew Honnibal	bb911e5f4e	Fix #3830 : 'subtok' label being added even if learn_tokens=False (#4188 ) * Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training	2019-08-23 17:54:00 +02:00

Author

SHA1

Message

Date

Ines Montani

3d8fd4b461

Revert #4334

2019-09-29 17:32:12 +02:00

Ines Montani

c9cd516d96

Move tests out of package (#4334 )

* Move tests out of package

* Fix typo

2019-09-28 18:05:00 +02:00

Matthew Honnibal

bb911e5f4e

Fix #3830 : 'subtok' label being added even if learn_tokens=False (#4188 )

* Prevent subtok label if not learning tokens

The parser introduces the subtok label to mark tokens that should be
merged during post-processing. Previously this happened even if we did
not have the --learn-tokens flag set. This patch passes the config
through to the parser, to prevent the problem.

* Make merge_subtokens a parser post-process if learn_subtokens

* Fix train script

* Add test for 3830: subtok problem

* Fix handlign of non-subtok in parser training

2019-08-23 17:54:00 +02:00

3 Commits