spaCy/website/docs/usage/transformers.md

title	teaser
Transformers	Using transformer models like BERT in spaCy

spaCy v3.0 lets you use almost any statistical model to power your pipeline. You can use models implemented in a variety of frameworks, including TensorFlow, PyTorch and MXNet. To keep things sane, spaCy expects models from these frameworks to be wrapped with a common interface, using our machine learning library Thinc. A transformer model is just a statistical model, so the spacy-transformers package actually has very little work to do: we just have to provide a few functions that do the required plumbing. We also provide a pipeline component, Transformer, that lets you do multi-task learning and lets you save the transformer outputs for later use.

Try out a BERT-based model pipeline using this project template: swap in your data, edit the settings and hyperparameters and train, evaluate, package and visualize your model.