From b132cb3036a44dc6ccf8093befb53591dc978e3c Mon Sep 17 00:00:00 2001 From: svlandeg Date: Thu, 21 Jan 2021 20:24:05 +0100 Subject: [PATCH 1/4] update accuracies for new a1 models --- website/docs/usage/_benchmarks-models.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/website/docs/usage/_benchmarks-models.md b/website/docs/usage/_benchmarks-models.md index 1e755e39d..81c29d297 100644 --- a/website/docs/usage/_benchmarks-models.md +++ b/website/docs/usage/_benchmarks-models.md @@ -4,12 +4,14 @@ import { Help } from 'components/typography'; import Link from 'components/link' | Pipeline | Parser | Tagger | NER | | ---------------------------------------------------------- | -----: | -----: | ---: | -| [`en_core_web_trf`](/models/en#en_core_web_trf) (spaCy v3) | 95.5 | 98.3 | 89.4 | -| [`en_core_web_lg`](/models/en#en_core_web_lg) (spaCy v3) | 92.2 | 97.4 | 85.4 | +| [`en_core_web_trf`](/models/en#en_core_web_trf) (spaCy v3) | 95.2 | 97.8 | 89.9 | +| [`en_core_web_lg`](/models/en#en_core_web_lg) (spaCy v3) | 91.9 | 97.4 | 85.5 | | `en_core_web_lg` (spaCy v2) | 91.9 | 97.2 | 85.5 |
+ + **Full pipeline accuracy and speed** on the [OntoNotes 5.0](https://catalog.ldc.upenn.edu/LDC2013T19) corpus (reported on the development set). From a071279bc763811347ae02a798ac76788d0335db Mon Sep 17 00:00:00 2001 From: svlandeg Date: Fri, 22 Jan 2021 18:46:35 +0100 Subject: [PATCH 2/4] add speed comparison to docs --- website/docs/usage/facts-figures.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/website/docs/usage/facts-figures.md b/website/docs/usage/facts-figures.md index 269ac5e17..845f292cc 100644 --- a/website/docs/usage/facts-figures.md +++ b/website/docs/usage/facts-figures.md @@ -92,6 +92,30 @@ results. Project template: +### Speed comparison {#benchmarks-speed} + +We compare the speed of different NLP libraries, measured in words per second. +The evaluation was run on 10,000 Reddit comments. + +
+ +| Library | Pipeline | WPS CPU words per second on CPU, higher is better | WPS GPU words per second on GPU, higher is better | +| ------- | ----------------------------------------------- | -------------------------------------------------------------: | -------------------------------------------------------------: | +| spaCy | [`en_core_web_lg`](/models/en#en_core_web_lg) | 10,014 | 14,954 | +| spaCy | [`en_core_web_trf`](/models/en#en_core_web_trf) | 684 | 3,768 | +| Stanza | `en_ewt` | 878 | 2,180 | +| Flair | `pos`(`-fast`) & `ner`(`-fast`) | 323 | 1,184 | +| UDPipe | `english-ewt-ud-2.5` | 1,101 | NA | + +
+ +**End-to-end processing speed** on raw unannotated text. Project template: +[`benchmarks/speed`](%%GITHUB_PROJECTS/benchmarks/speed). + +
+ +
+ From d7c0f40a96723b3e7f3f60a1ae256683dbc24402 Mon Sep 17 00:00:00 2001 From: svlandeg Date: Fri, 22 Jan 2021 18:55:18 +0100 Subject: [PATCH 3/4] update comment --- website/docs/usage/facts-figures.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/usage/facts-figures.md b/website/docs/usage/facts-figures.md index 845f292cc..1fb932889 100644 --- a/website/docs/usage/facts-figures.md +++ b/website/docs/usage/facts-figures.md @@ -94,8 +94,8 @@ results. Project template: ### Speed comparison {#benchmarks-speed} -We compare the speed of different NLP libraries, measured in words per second. -The evaluation was run on 10,000 Reddit comments. +We compare the speed of different NLP libraries, measured in words per second +(WPS) - higher is better. The evaluation was performed on 10,000 Reddit comments.
From 56064faed99843520f4a6dc15998f4b77e8342e5 Mon Sep 17 00:00:00 2001 From: svlandeg Date: Sat, 23 Jan 2021 00:57:00 +0100 Subject: [PATCH 4/4] update caption --- website/docs/usage/_benchmarks-models.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/website/docs/usage/_benchmarks-models.md b/website/docs/usage/_benchmarks-models.md index 81c29d297..be49406bc 100644 --- a/website/docs/usage/_benchmarks-models.md +++ b/website/docs/usage/_benchmarks-models.md @@ -10,9 +10,7 @@ import { Help } from 'components/typography'; import Link from 'components/link'
- - -**Full pipeline accuracy and speed** on the +**Full pipeline accuracy** on the [OntoNotes 5.0](https://catalog.ldc.upenn.edu/LDC2013T19) corpus (reported on the development set).