spaCy/spacy/cli
Daniël de Kok 319eb508b5
Add a `spacy benchmark speed` subcommand (#11902)
* Add a `spacy evaluate speed` subcommand

This subcommand reports the mean batch performance of a model on a data set with
a 95% confidence interval. For reliability, it first performs some warmup
rounds. Then it will measure performance on batches with randomly shuffled
documents.

To avoid having too many spaCy commands, `speed` is a subcommand of `evaluate`
and accuracy evaluation is moved to its own `evaluate accuracy` subcommand.

* Fix import cycle

* Restore `spacy evaluate`, make `spacy benchmark speed` an alias

* Add documentation for `spacy benchmark`

* CREATES -> PRINTS

* WPS -> words/s

* Disable formatting of benchmark speed arguments

* Fail with an error message when trying to speed bench empty corpus

* Make it clearer that `benchmark accuracy` is a replacement for `evaluate`

* Fix docstring webpage reference

* tests: check `evaluate` output against `benchmark accuracy`
2023-01-12 11:55:21 +01:00
..
project Refactor kwargs in CLI msg for future wasabi compatibility (#11918) 2022-12-05 10:00:00 +01:00
templates Config generation fails for GPU without transformers (#11899) 2022-12-02 10:17:11 +01:00
__init__.py Add a `spacy benchmark speed` subcommand (#11902) 2023-01-12 11:55:21 +01:00
_util.py Add a `spacy benchmark speed` subcommand (#11902) 2023-01-12 11:55:21 +01:00
apply.py Auto-format code with black (#12019) 2022-12-23 12:44:07 +01:00
assemble.py Tidy up with flake8: imports, comparisons, etc. 2021-06-28 12:08:15 +02:00
benchmark_speed.py Add a `spacy benchmark speed` subcommand (#11902) 2023-01-12 11:55:21 +01:00
convert.py fix processing of "auto" in convert (#12050) 2023-01-05 10:21:00 +01:00
debug_config.py Fix references to config file in the docs & UX (#9961) 2022-01-04 14:31:26 +01:00
debug_data.py Replace Pipe type with Callable in Language (#11803) 2022-11-29 13:20:08 +01:00
debug_diff.py Add debug diff command in spaCy CLI (#10502) 2022-04-07 10:48:45 +02:00
debug_model.py Tidy up with flake8: imports, comparisons, etc. 2021-06-28 12:08:15 +02:00
download.py Add a way to get the URL to download a pipeline to the CLI (#11175) 2022-09-02 11:58:21 +02:00
evaluate.py Add a `spacy benchmark speed` subcommand (#11902) 2023-01-12 11:55:21 +01:00
find_threshold.py find-threshold: CLI command for multi-label classifier threshold tuning (#11280) 2022-11-25 11:44:55 +01:00
info.py Auto-format code with black (#11468) 2022-09-09 11:21:17 +02:00
init_config.py Workaround for Typer optional default values with Python calls (#10788) 2022-06-17 12:15:36 +02:00
init_pipeline.py Add support for floret vectors (#8909) 2021-10-27 14:08:31 +02:00
package.py Allow overriding spacy_version in spacy package meta (#11552) 2022-09-29 10:44:06 +02:00
pretrain.py Add parent argument (#11210) 2022-07-26 14:35:18 +02:00
profile.py 🏷 Add Mypy check to CI and ignore all existing Mypy errors (#9167) 2021-10-14 15:21:40 +02:00
train.py Add docs section for spacy.cli.train.train (#9545) 2021-10-29 10:36:34 +02:00
validate.py 🏷 Add Mypy check to CI and ignore all existing Mypy errors (#9167) 2021-10-14 15:21:40 +02:00