spaCy/website/docs/usage/training-ner.jade

include ../../_includes/_mixins

p
    |  All #[+a("/docs/usage/models") spaCy models] support online learning, so
    |  you can update a pre-trained model with new examples. You can even add
    |  new classes to an existing model, to recognise a new entity type,
    |  part-of-speech, or syntactic relation. Updating an existing model is
    |  particularly useful as a "quick and dirty solution", if you have only a
    |  few corrections or annotations.

+under-construction

+h(2, "improving-accuracy") Improving accuracy on existing entity types

p
    |  To update the model, you first need to create an instance of
    |  #[+api("goldparse") #[code spacy.gold.GoldParse]], with the entity labels
    |  you want to learn. You will then pass this instance to the
    |  #[+api("entityrecognizer#update") #[code EntityRecognizer.update()]]
    |  method.

p
    |  You'll usually need to provide many examples to meaningfully improve the
    |  system — a few hundred is a good start, although more is better. You
    |  should avoid iterating over the same few examples multiple times, or the
    |  model is likely to "forget" how to annotate other examples. If you
    |  iterate over the same few examples, you're effectively changing the loss
    |  function. The optimizer will find a way to minimize the loss on your
    |  examples, without regard for the consequences on the examples it's no
    |  longer paying attention to.

p
    |  One way to avoid this "catastrophic forgetting" problem is to "remind"
    |  the model of other examples by augmenting your annotations with sentences
    |  annotated with entities automatically recognised by the original model.
    |  Ultimately, this is an empirical process: you'll need to
    |  #[strong experiment on your own data] to find a solution that works best
    |  for you.

+h(2, "saving-loading") Saving and loading

p
    |  After training our model, you'll usually want to save its state, and load
    |  it back later. You can do this with the
    |  #[+api("language#to_disk") #[code Language.to_disk()]] method:

+code.
    nlp.to_disk('/home/me/data/en_technology')

p
    |  To make the model more convenient to deploy, we recommend wrapping it as
    |  a Python package, so that you can install it via pip and load it as a
    |  module. spaCy comes with a handy #[+api("cli#package") #[code package]]
    |  CLI command to create all required files and directories.

+code(false, "bash").
    python -m spacy package /home/me/data/en_technology /home/me/my_models

p
    |  To build the package and create a #[code .tar.gz] archive, run
    |  #[code python setup.py sdist] from within its directory.

+infobox("Saving and loading models")
    |  For more information and a detailed guide on how to package your model,
    |  see the documentation on
    |  #[+a("/docs/usage/saving-loading#models") saving and loading models].