mirror of https://github.com/explosion/spaCy.git
39 lines
2.4 KiB
Markdown
39 lines
2.4 KiB
Markdown
|
spaCy's models are **statistical** and every "decision" they make – for example,
|
|||
|
|
which part-of-speech tag to assign, or whether a word is a named entity – is a
|
|||
|
|
**prediction**. This prediction is based on the examples the model has seen
|
|||
|
|
during **training**. To train a model, you first need training data – examples
|
|||
|
|
of text, and the labels you want the model to predict. This could be a
|
|||
|
|
part-of-speech tag, a named entity or any other information.
|
|||
|
|
|
|||
|
|
The model is then shown the unlabelled text and will make a prediction. Because
|
|||
|
|
we know the correct answer, we can give the model feedback on its prediction in
|
|||
|
|
the form of an **error gradient** of the **loss function** that calculates the
|
|||
|
|
difference between the training example and the expected output. The greater the
|
|||
|
|
difference, the more significant the gradient and the updates to our model.
|
|||
|
|
|
|||
|
|
> - **Training data:** Examples and their annotations.
|
|||
|
|
> - **Text:** The input text the model should predict a label for.
|
|||
|
|
> - **Label:** The label the model should predict.
|
|||
|
|
> - **Gradient:** Gradient of the loss function calculating the difference
|
|||
|
|
> between input and expected output.
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
When training a model, we don't just want it to memorize our examples – we want
|
|||
|
|
it to come up with theory that can be **generalized across other examples**.
|
|||
|
|
After all, we don't just want the model to learn that this one instance of
|
|||
|
|
"Amazon" right here is a company – we want it to learn that "Amazon", in
|
|||
|
|
contexts _like this_, is most likely a company. That's why the training data
|
|||
|
|
should always be representative of the data we want to process. A model trained
|
|||
|
|
on Wikipedia, where sentences in the first person are extremely rare, will
|
|||
|
|
likely perform badly on Twitter. Similarly, a model trained on romantic novels
|
|||
|
|
will likely perform badly on legal text.
|
|||
|
|
|
|||
|
|
This also means that in order to know how the model is performing, and whether
|
|||
|
|
it's learning the right things, you don't only need **training data** – you'll
|
|||
|
|
also need **evaluation data**. If you only test the model with the data it was
|
|||
|
|
trained on, you'll have no idea how well it's generalizing. If you want to train
|
|||
|
|
a model from scratch, you usually need at least a few hundred examples for both
|
|||
|
|
training and evaluation. To update an existing model, you can already achieve
|
|||
|
|
decent results with very few examples – as long as they're representative.
|