spaCy/spacy/pipeline
Raphael Mitsch c0fd8a2e71
find-threshold: CLI command for multi-label classifier threshold tuning (#11280)
* Add foundation for find-threshold CLI functionality.

* Finish first draft for find-threshold.

* Add tests.

* Revert adjusted import statements.

* Fix mypy errors.

* Fix imports.

* Harmonize arguments with spacy evaluate command.

* Generalize component and threshold handling. Harmonize arguments with 'spacy evaluate' CLI.

* Fix Spancat test.

* Add beta parameter to Scorer and PRFScore.

* Make beta a component scorer setting.

* Remove beta.

* Update nlp.config (workaround).

* Reload pipeline on threshold change. Adjust tests. Remove confection reference.

* Remove assumption of component being a Pipe object or having a .cfg attribute.

* Adjust test output and reference values.

* Remove beta references. Delete universe.json.

* Reverting unnecessary changes. Removing unused default values. Renaming variables in find-cli tests.

* Update spacy/cli/find_threshold.py

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Remove adding labels in tests.

* Remove unused error

* Undo changes to PRFScorer

* Change default value for n_trials. Log table iteratively.

* Add warnings for pointless applications of find_threshold().

* Fix imports.

* Adjust type check of TextCategorizer to exclude subclasses.

* Change check of if there's only one unique value in scores.

* Update spacy/cli/find_threshold.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Incorporate feedback.

* Fix test issue. Update docstring.

* Update docs & docstring.

* Update spacy/tests/test_cli.py

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Add examples to docs. Rename _nlp to nlp in tests.

* Update spacy/cli/find_threshold.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/cli/find_threshold.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-11-25 11:44:55 +01:00
..
_edit_tree_internals Refactor error messages to remove hardcoded strings (#10729) 2022-05-02 13:38:46 +02:00
_parser_internals account for NER labels with a hyphen in the name (#10960) 2022-06-17 20:02:37 +01:00
legacy Refactor KB for easier customization (#11268) 2022-09-08 10:38:07 +02:00
__init__.py Add SpanRuler component (#9880) 2022-06-02 13:12:53 +02:00
attributeruler.py Merge remote-tracking branch 'upstream/master' into chore/update-develop-from-master-v3.2-1 2021-10-26 11:53:50 +02:00
dep_parser.pyx account for NER labels with a hyphen in the name (#10960) 2022-06-17 20:02:37 +01:00
edit_tree_lemmatizer.py Auto-format code with black (#11649) 2022-10-14 18:04:55 +09:00
entity_linker.py Refactor KB for easier customization (#11268) 2022-09-08 10:38:07 +02:00
entityruler.py update mypy to latest version (#11546) 2022-09-29 14:24:40 +02:00
functions.py Add doc_cleaner component (#9659) 2021-11-23 15:33:33 +01:00
lemmatizer.py Merge remote-tracking branch 'upstream/master' into chore/update-develop-from-master-v3.2-1 2021-10-26 11:53:50 +02:00
morphologizer.pyx Tagger: use unnormalized probabilities for inference (#10197) 2022-03-15 14:15:31 +01:00
multitask.pyx Replace negative rows with 0 in StaticVectors (#7674) 2021-04-22 18:04:15 +10:00
ner.pyx account for NER labels with a hyphen in the name (#10960) 2022-06-17 20:02:37 +01:00
pipe.pxd TrainablePipe (#6213) 2020-10-08 21:33:49 +02:00
pipe.pyi Add Pipe.hide_labels to omit labels from pipeline meta (#10175) 2022-02-05 17:59:24 +01:00
pipe.pyx Fix config validation failures caused by NVTX pipeline wrappers (#11460) 2022-09-12 14:55:41 +02:00
sentencizer.pyx Add overwrite settings for more components (#9050) 2021-09-30 15:35:55 +02:00
senter.pyx Tagger: use unnormalized probabilities for inference (#10197) 2022-03-15 14:15:31 +01:00
span_ruler.py Add SpanRuler component (#9880) 2022-06-02 13:12:53 +02:00
spancat.py find-threshold: CLI command for multi-label classifier threshold tuning (#11280) 2022-11-25 11:44:55 +01:00
tagger.pyx Tagger: use unnormalized probabilities for inference (#10197) 2022-03-15 14:15:31 +01:00
textcat.py Auto-format code with black (#11826) 2022-11-18 18:24:13 +09:00
textcat_multilabel.py Check textcat values for validity (#11763) 2022-11-17 10:25:01 +01:00
tok2vec.py Prevent tok2vec to broadcast to listeners when predicting (#11385) 2022-09-12 15:36:48 +02:00
trainable_pipe.pxd Refactor scoring methods to use registered functions (#8766) 2021-08-10 15:13:39 +02:00
trainable_pipe.pyx Fix config validation failures caused by NVTX pipeline wrappers (#11460) 2022-09-12 14:55:41 +02:00
transition_parser.pxd Parser: use C saxpy/sgemm provided by the Ops implementation (#10773) 2022-05-27 11:20:52 +02:00
transition_parser.pyx precompute_hiddens/Parser: do not look up CPU ops (3.4) (#11069) 2022-07-05 10:53:42 +02:00