diff --git a/website/docs/usage/embeddings-transformers.md b/website/docs/usage/embeddings-transformers.md index aaa5fde10..7792ce124 100644 --- a/website/docs/usage/embeddings-transformers.md +++ b/website/docs/usage/embeddings-transformers.md @@ -578,7 +578,12 @@ def MyCustomVectors( ## Pretraining {#pretraining} - +- explain general concept and idea (short!) +- present it as a separate lightweight mechanism for pretraining the tok2vec + layer +- advantages (could also be pros/cons table) +- explain how it generates a separate file (!) and how it depends on the same + vectors > #### Raw text format >