spaCy/spacy/tests/regression
adrianeboyd 392c4880d9 Restructure Example with merged sents as default (#4632)
* Switch to train_dataset() function in train CLI

* Fixes for pipe() methods in pipeline components

* Don't clobber `examples` variable with `as_example` in pipe() methods
* Remove unnecessary traversals of `examples`

* Update Parser.pipe() for Examples

* Add `as_examples` kwarg to `pipe()` with implementation to return
`Example`s

* Accept `Doc` or `Example` in `pipe()` with `_get_doc()` (copied from
`Pipe`)

* Fixes to Example implementation in spacy.gold

* Move `make_projective` from an attribute of Example to an argument of
`Example.get_gold_parses()`

* Head of 0 are not treated as unset

* Unset heads are set to self rather than `None` (which causes problems
while projectivizing)

* Check for `Doc` (not just not `None`) when creating GoldParses for
pre-merged example

* Don't clobber `examples` variable in `iter_gold_docs()`

* Add/modify gold tests for handling projectivity

* In JSON roundtrip compare results from `dev_dataset` rather than
`train_dataset` to avoid projectivization (and other potential
modifications)

* Add test for projective train vs. nonprojective dev versions of the
same `Doc`

* Handle ignore_misaligned as arg rather than attr

Move `ignore_misaligned` from an attribute of `Example` to an argument
to `Example.get_gold_parses()`, which makes it parallel to
`make_projective`.

Add test with old and new align that checks whether `ignore_misaligned`
errors are raised as expected (only for new align).

* Remove unused attrs from gold.pxd

Remove `ignore_misaligned` and `make_projective` from `gold.pxd`

* Restructure Example with merged sents as default

An `Example` now includes a single `TokenAnnotation` that includes all
the information from one `Doc` (=JSON `paragraph`). If required, the
individual sentences can be returned as a list of examples with
`Example.split_sents()` with no raw text available.

* Input/output a single `Example.token_annotation`

* Add `sent_starts` to `TokenAnnotation` to handle sentence boundaries

* Replace `Example.merge_sents()` with `Example.split_sents()`

* Modify components to use a single `Example.token_annotation`

  * Pipeline components
  * conllu2json converter

* Rework/rename `add_token_annotation()` and `add_doc_annotation()` to
`set_token_annotation()` and `set_doc_annotation()`, functions that set
rather then appending/extending.

* Rename `morphology` to `morphs` in `TokenAnnotation` and `GoldParse`

* Add getters to `TokenAnnotation` to supply default values when a given
attribute is not available

* `Example.get_gold_parses()` in `spacy.gold._make_golds()` is only
applied on single examples, so the `GoldParse` is returned saved in the
provided `Example` rather than creating a new `Example` with no other
internal annotation

* Update tests for API changes and `merge_sents()` vs. `split_sents()`

* Refer to Example.goldparse in iter_gold_docs()

Use `Example.goldparse` in `iter_gold_docs()` instead of `Example.gold`
because a `None` `GoldParse` is generated with ignore_misaligned and
generating it on-the-fly can raise an unwanted AlignmentError

* Fix make_orth_variants()

Fix bug in make_orth_variants() related to conversion from multiple to
one TokenAnnotation per Example.

* Add basic test for make_orth_variants()

* Replace try/except with conditionals

* Replace default morph value with set
2019-11-25 16:03:28 +01:00
..
__init__.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue1-1000.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue1001-1500.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue1501-2000.py Restructure Example with merged sents as default (#4632) 2019-11-25 16:03:28 +01:00
test_issue2001-2500.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue2501-3000.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue3001-3500.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3521.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3526.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3531.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3540.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3549.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3555.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3611.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue3625.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3803.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3830.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3839.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3869.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3879.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3880.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3882.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3951.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue3959.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3962.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue3972.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue4002.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue4030.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue4042.py Tidy up and auto-format 2019-10-18 11:27:38 +02:00
test_issue4054.py Fix test imports 2019-09-29 17:34:56 +02:00
test_issue4120.py Implement new API for {Phrase}Matcher.add (backwards-compatible) (#4522) 2019-10-25 22:21:08 +02:00
test_issue4133.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue4190.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue4267.py Tidy up and auto-format 2019-10-18 11:27:38 +02:00
test_issue4272.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue4278.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue4313.py Revert #4334 2019-09-29 17:32:12 +02:00
test_issue4348.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue4367.py Tidy up and auto-format 2019-10-18 11:27:38 +02:00
test_issue4373.py Make PhraseMatcher.vocab consistent with Matcher.vocab (closes #4373) 2019-10-04 12:18:41 +02:00
test_issue4402.py Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
test_issue4528.py Fix serialization of extension attr values in DocBin (#4540) 2019-10-28 16:02:13 +01:00
test_issue4529.py [#4529] fix: gold pyx (#4530) 2019-10-27 13:50:07 +01:00
test_issue4590.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00