mirror of https://github.com/explosion/spaCy.git
115 lines
4.5 KiB
Markdown
115 lines
4.5 KiB
Markdown
---
|
||
title: What's New in v3.1
|
||
teaser: New features and how to upgrade
|
||
menu:
|
||
- ['New Features', 'features']
|
||
- ['Upgrading Notes', 'upgrading']
|
||
---
|
||
|
||
## New Features {#features hidden="true"}
|
||
|
||
<!-- TODO: intro -->
|
||
|
||
### Using predicted annotations during training {#predicted-annotations-training}
|
||
|
||
<!-- TODO: write -->
|
||
|
||
<Project id="pipelines/tagger_parser_predicted_annotations">
|
||
|
||
This project shows how to use the `token.dep` attribute predicted by the parser
|
||
as a feature for a subsequent tagger component in the pipeline.
|
||
|
||
</Project>
|
||
|
||
### SpanCategorizer for predicting arbitrary and overlapping spans {#spancategorizer tag="experimental"}
|
||
|
||
A common task in applied NLP is extracting spans of texts from documents,
|
||
including longer phrases or nested expressions. Named entity recognition isn't
|
||
the right tool for this problem, since an entity recognizer typically predicts
|
||
single token-based tags that are very sensitive to boundaries. This is effective
|
||
for proper nouns and self-contained expressions, but less useful for other types
|
||
of phrases or overlapping spans. The new
|
||
[`SpanCategorizer`](/api/spancategorizer) component and
|
||
[SpanCategorizer](/api/architectures#spancategorizer) architecture let you label
|
||
arbitrary and potentially overlapping spans of texts. A span categorizer
|
||
consists of two parts: a [suggester function](/api/spancategorizer#suggesters)
|
||
that proposes candidate spans, which may or may not overlap, and a labeler model
|
||
that predicts zero or more labels for each candidate. The predicted spans are
|
||
available via the [`Doc.spans`](/api/doc#spans) container.
|
||
|
||
<!-- TODO: example, getting started (init config?), maybe project template -->
|
||
|
||
<Infobox title="Tip: Create data with Prodigy's new span annotation UI">
|
||
|
||
<!-- TODO: screenshot -->
|
||
|
||
The upcoming version of our annotation tool [Prodigy](https://prodi.gy)
|
||
(currently available as a [pre-release](https://support.prodi.gy/t/3861) for all
|
||
users) features a [new workflow and UI](https://support.prodi.gy/t/3861) for
|
||
annotating overlapping and nested spans. You can use it to create training data
|
||
for spaCy's `SpanCategorizer` component.
|
||
|
||
</Infobox>
|
||
|
||
### Update the entity recognizer with partial incorrect annotations {#negative-samples}
|
||
|
||
> #### config.cfg (excerpt)
|
||
>
|
||
> ```ini
|
||
> [components.ner]
|
||
> factory = "ner"
|
||
> incorrect_spans_key = "incorrect_spans"
|
||
> moves = null
|
||
> update_with_oracle_cut_size = 100
|
||
> ```
|
||
|
||
The [`EntityRecognizer`](/api/entityrecognizer) can now be updated with known
|
||
incorrect annotations, which lets you take advantage of partial and sparse data.
|
||
For example, you'll be able to use the information that certain spans of text
|
||
are definitely **not** `PERSON` entities, without having to provide the
|
||
complete-gold standard annotations for the given example. The incorrect span
|
||
annotations can be added via the [`Doc.spans`](/api/doc#spans) in the training
|
||
data under the key defined as
|
||
[`incorrect_spans_key`](/api/entityrecognizer#init) in the component config.
|
||
|
||
<!-- TODO: more details and/or example project? -->
|
||
|
||
### New pipeline packages for Catalan and Danish {#pipeline-packages}
|
||
|
||
<!-- TODO: intro and update with final numbers -->
|
||
|
||
| Package | Language | Tagger | Parser | NER |
|
||
| ------------------------------------------------- | -------- | -----: | -----: | ---: |
|
||
| [`ca_core_news_sm`](/models/ca#ca_core_news_sm) | Catalan | | | |
|
||
| [`ca_core_news_md`](/models/ca#ca_core_news_md) | Catalan | | | |
|
||
| [`ca_core_news_lg`](/models/ca#ca_core_news_lg) | Catalan | | | |
|
||
| [`ca_core_news_trf`](/models/ca#ca_core_news_trf) | Catalan | | | |
|
||
| [`da_core_news_trf`](/models/da#da_core_news_trf) | Danish | | | |
|
||
|
||
### Resizable text classification architectures {#resizable-textcat}
|
||
|
||
<!-- TODO: write -->
|
||
|
||
### CLI command to assemble pipeline from config {#assemble}
|
||
|
||
The [`spacy assemble`](/api/cli#assemble) command lets you assemble a pipeline
|
||
from a config file without additional training. It can be especially useful for
|
||
creating a blank pipeline with a custom tokenizer, rule-based components or word
|
||
vectors.
|
||
|
||
```cli
|
||
$ python -m spacy assemble config.cfg ./output
|
||
```
|
||
|
||
### Support for streaming large or infinite corpora {#streaming-corpora}
|
||
|
||
<!-- TODO: write -->
|
||
|
||
### New lemmatizers for Catalan and Italian {#pos-lemmatizers}
|
||
|
||
<!-- TODO: write -->
|
||
|
||
## Notes about upgrading from v3.0 {#upgrading}
|
||
|
||
<!-- TODO: this could just be a bullet-point list mentioning stuff like the spacy_version, vectors initialization etc. -->
|