mirror of https://github.com/explosion/spaCy.git
67 lines
2.9 KiB
Plaintext
67 lines
2.9 KiB
Plaintext
//- 💫 DOCS > USAGE > DEEP LEARNING > THINC
|
||
|
||
p
|
||
| #[+a(gh("thinc")) Thinc] is the machine learning library powering spaCy.
|
||
| It's a practical toolkit for implementing models that follow the
|
||
| #[+a("https://explosion.ai/blog/deep-learning-formula-nlp", true) "Embed, encode, attend, predict"]
|
||
| architecture. It's designed to be easy to install, efficient for CPU
|
||
| usage and optimised for NLP and deep learning with text – in particular,
|
||
| hierarchically structured input and variable-length sequences.
|
||
|
||
p
|
||
| spaCy's built-in pipeline components can all be powered by any object
|
||
| that follows Thinc's #[code Model] API. If a wrapper is not yet available
|
||
| for the library you're using, you should create a
|
||
| #[code thinc.neural.Model] subclass that implements a #[code begin_update]
|
||
| method. You'll also want to implement #[code to_bytes], #[code from_bytes],
|
||
| #[code to_disk] and #[code from_disk] methods, to save and load your
|
||
| model. Here's the tempate you'll need to fill in:
|
||
|
||
+code("Thinc Model API").
|
||
class ThincModel(thinc.neural.Model):
|
||
def __init__(self, *args, **kwargs):
|
||
pass
|
||
|
||
def begin_update(self, X, drop=0.):
|
||
def backprop(dY, sgd=None):
|
||
return dX
|
||
return Y, backprop
|
||
|
||
def to_disk(self, path, **exclude):
|
||
return None
|
||
|
||
def from_disk(self, path, **exclude):
|
||
return self
|
||
|
||
def to_bytes(self, **exclude):
|
||
return bytes
|
||
|
||
def from_bytes(self, msgpacked_bytes, **exclude):
|
||
return self
|
||
|
||
p
|
||
| The #[code begin_update] method should return a callback, that takes the
|
||
| gradient with respect to the output, and returns the gradient with
|
||
| respect to the input. It's usually convenient to implement the callback
|
||
| as a nested function, so you can refer to any intermediate variables from
|
||
| the forward computation in the enclosing scope.
|
||
|
||
+h(3, "how-thinc-works") How Thinc works
|
||
|
||
p
|
||
| Neural networks are all about composing small functions that we know how
|
||
| to differentiate into larger functions that we know how to differentiate.
|
||
| To differentiate a function efficiently, you usually need to store
|
||
| intermediate results, computed during the "forward pass", to reuse them
|
||
| during the backward pass. Most libraries require the data passed through
|
||
| the network to accumulate these intermediate result. This is the "tape"
|
||
| in tape-based differentiation.
|
||
|
||
p
|
||
| In Thinc, a model that computes #[code y = f(x)] is required to also
|
||
| return a callback that computes #[code dx = f'(dy)]. The same
|
||
| intermediate state needs to be tracked, but this becomes an
|
||
| implementation detail for the model to take care of – usually, the
|
||
| callback is implemented as a closure, so the intermediate results can be
|
||
| read from the enclosing scope.
|