Commit Graph

2 Commits

Author SHA1 Message Date
Daniël de Kok 319eb508b5
Add a `spacy benchmark speed` subcommand (#11902)
* Add a `spacy evaluate speed` subcommand

This subcommand reports the mean batch performance of a model on a data set with
a 95% confidence interval. For reliability, it first performs some warmup
rounds. Then it will measure performance on batches with randomly shuffled
documents.

To avoid having too many spaCy commands, `speed` is a subcommand of `evaluate`
and accuracy evaluation is moved to its own `evaluate accuracy` subcommand.

* Fix import cycle

* Restore `spacy evaluate`, make `spacy benchmark speed` an alias

* Add documentation for `spacy benchmark`

* CREATES -> PRINTS

* WPS -> words/s

* Disable formatting of benchmark speed arguments

* Fail with an error message when trying to speed bench empty corpus

* Make it clearer that `benchmark accuracy` is a replacement for `evaluate`

* Fix docstring webpage reference

* tests: check `evaluate` output against `benchmark accuracy`
2023-01-12 11:55:21 +01:00
Sofie Van Landeghem 7f6c638c3a
fix processing of "auto" in convert (#12050)
* fix processing of "auto" in walk_directory

* add check for None

* move AUTO check to convert and fix verification of args

* add specific CLI test with CliRunner

* cleanup

* more cleanup

* update docstring
2023-01-05 10:21:00 +01:00