mirror of https://github.com/explosion/spaCy.git
210 lines
9.9 KiB
Markdown
210 lines
9.9 KiB
Markdown
---
|
|
title: Cython Classes
|
|
menu:
|
|
- ['Doc', 'doc']
|
|
- ['Token', 'token']
|
|
- ['Span', 'span']
|
|
- ['Lexeme', 'lexeme']
|
|
- ['Vocab', 'vocab']
|
|
- ['StringStore', 'stringstore']
|
|
---
|
|
|
|
## Doc {#doc tag="cdef class" source="spacy/tokens/doc.pxd"}
|
|
|
|
The `Doc` object holds an array of [`TokenC`](/api/cython-structs#tokenc)
|
|
structs.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see [`Doc`](/api/doc).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#doc_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| ------------ | ------------ | ----------------------------------------------------------------------------------------- |
|
|
| `mem` | `cymem.Pool` | A memory pool. Allocated memory will be freed once the `Doc` object is garbage collected. |
|
|
| `vocab` | `Vocab` | A reference to the shared `Vocab` object. |
|
|
| `c` | `TokenC*` | A pointer to a [`TokenC`](/api/cython-structs#tokenc) struct. |
|
|
| `length` | `int` | The number of tokens in the document. |
|
|
| `max_length` | `int` | The underlying size of the `Doc.c` array. |
|
|
|
|
### Doc.push_back {#doc_push_back tag="method"}
|
|
|
|
Append a token to the `Doc`. The token can be provided as a
|
|
[`LexemeC`](/api/cython-structs#lexemec) or
|
|
[`TokenC`](/api/cython-structs#tokenc) pointer, using Cython's
|
|
[fused types](http://cython.readthedocs.io/en/latest/src/userguide/fusedtypes.html).
|
|
|
|
> #### Example
|
|
>
|
|
> ```python
|
|
> from spacy.tokens cimport Doc
|
|
> from spacy.vocab cimport Vocab
|
|
>
|
|
> doc = Doc(Vocab())
|
|
> lexeme = doc.vocab.get("hello")
|
|
> doc.push_back(lexeme, True)
|
|
> assert doc.text == "hello "
|
|
> ```
|
|
|
|
| Name | Type | Description |
|
|
| ------------ | --------------- | ----------------------------------------- |
|
|
| `lex_or_tok` | `LexemeOrToken` | The word to append to the `Doc`. |
|
|
| `has_space` | `bint` | Whether the word has trailing whitespace. |
|
|
|
|
## Token {#token tag="cdef class" source="spacy/tokens/token.pxd"}
|
|
|
|
A Cython class providing access and methods for a
|
|
[`TokenC`](/api/cython-structs#tokenc) struct. Note that the `Token` object does
|
|
not own the struct. It only receives a pointer to it.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see [`Token`](/api/token).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#token_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| ------- | --------- | ------------------------------------------------------------- |
|
|
| `vocab` | `Vocab` | A reference to the shared `Vocab` object. |
|
|
| `c` | `TokenC*` | A pointer to a [`TokenC`](/api/cython-structs#tokenc) struct. |
|
|
| `i` | `int` | The offset of the token within the document. |
|
|
| `doc` | `Doc` | The parent document. |
|
|
|
|
### Token.cinit {#token_cinit tag="method"}
|
|
|
|
Create a `Token` object from a `TokenC*` pointer.
|
|
|
|
> #### Example
|
|
>
|
|
> ```python
|
|
> token = Token.cinit(&doc.c[3], doc, 3)
|
|
> ```
|
|
|
|
| Name | Type | Description |
|
|
| -------- | --------- | ------------------------------------------------------------ |
|
|
| `vocab` | `Vocab` | A reference to the shared `Vocab`. |
|
|
| `c` | `TokenC*` | A pointer to a [`TokenC`](/api/cython-structs#tokenc)struct. |
|
|
| `offset` | `int` | The offset of the token within the document. |
|
|
| `doc` | `Doc` | The parent document. |
|
|
|
|
## Span {#span tag="cdef class" source="spacy/tokens/span.pxd"}
|
|
|
|
A Cython class providing access and methods for a slice of a `Doc` object.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see [`Span`](/api/span).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#span_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| ------------ | -------------------------------------- | ------------------------------------------------------- |
|
|
| `doc` | `Doc` | The parent document. |
|
|
| `start` | `int` | The index of the first token of the span. |
|
|
| `end` | `int` | The index of the first token after the span. |
|
|
| `start_char` | `int` | The index of the first character of the span. |
|
|
| `end_char` | `int` | The index of the last character of the span. |
|
|
| `label` | <Abbr title="uint64_t">`attr_t`</Abbr> | A label to attach to the span, e.g. for named entities. |
|
|
|
|
## Lexeme {#lexeme tag="cdef class" source="spacy/lexeme.pxd"}
|
|
|
|
A Cython class providing access and methods for an entry in the vocabulary.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see [`Lexeme`](/api/lexeme).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#lexeme_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| ------- | -------------------------------------- | --------------------------------------------------------------- |
|
|
| `c` | `LexemeC*` | A pointer to a [`LexemeC`](/api/cython-structs#lexemec) struct. |
|
|
| `vocab` | `Vocab` | A reference to the shared `Vocab` object. |
|
|
| `orth` | <Abbr title="uint64_t">`attr_t`</Abbr> | ID of the verbatim text content. |
|
|
|
|
## Vocab {#vocab tag="cdef class" source="spacy/vocab.pxd"}
|
|
|
|
A Cython class providing access and methods for a vocabulary and other data
|
|
shared across a language.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see [`Vocab`](/api/vocab).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#vocab_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| --------- | ------------- | ------------------------------------------------------------------------------------------- |
|
|
| `mem` | `cymem.Pool` | A memory pool. Allocated memory will be freed once the `Vocab` object is garbage collected. |
|
|
| `strings` | `StringStore` | A `StringStore` that maps string to hash values and vice versa. |
|
|
| `length` | `int` | The number of entries in the vocabulary. |
|
|
|
|
### Vocab.get {#vocab_get tag="method"}
|
|
|
|
Retrieve a [`LexemeC*`](/api/cython-structs#lexemec) pointer from the
|
|
vocabulary.
|
|
|
|
> #### Example
|
|
>
|
|
> ```python
|
|
> lexeme = vocab.get(vocab.mem, "hello")
|
|
> ```
|
|
|
|
| Name | Type | Description |
|
|
| ----------- | ---------------- | ------------------------------------------------------------------------------------------- |
|
|
| `mem` | `cymem.Pool` | A memory pool. Allocated memory will be freed once the `Vocab` object is garbage collected. |
|
|
| `string` | str | The string of the word to look up. |
|
|
| **RETURNS** | `const LexemeC*` | The lexeme in the vocabulary. |
|
|
|
|
### Vocab.get_by_orth {#vocab_get_by_orth tag="method"}
|
|
|
|
Retrieve a [`LexemeC*`](/api/cython-structs#lexemec) pointer from the
|
|
vocabulary.
|
|
|
|
> #### Example
|
|
>
|
|
> ```python
|
|
> lexeme = vocab.get_by_orth(doc[0].lex.norm)
|
|
> ```
|
|
|
|
| Name | Type | Description |
|
|
| ----------- | -------------------------------------- | ------------------------------------------------------------------------------------------- |
|
|
| `mem` | `cymem.Pool` | A memory pool. Allocated memory will be freed once the `Vocab` object is garbage collected. |
|
|
| `orth` | <Abbr title="uint64_t">`attr_t`</Abbr> | ID of the verbatim text content. |
|
|
| **RETURNS** | `const LexemeC*` | The lexeme in the vocabulary. |
|
|
|
|
## StringStore {#stringstore tag="cdef class" source="spacy/strings.pxd"}
|
|
|
|
A lookup table to retrieve strings by 64-bit hashes.
|
|
|
|
<Infobox variant="warning">
|
|
|
|
This section documents the extra C-level attributes and methods that can't be
|
|
accessed from Python. For the Python documentation, see
|
|
[`StringStore`](/api/stringstore).
|
|
|
|
</Infobox>
|
|
|
|
### Attributes {#stringstore_attributes}
|
|
|
|
| Name | Type | Description |
|
|
| ------ | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------ |
|
|
| `mem` | `cymem.Pool` | A memory pool. Allocated memory will be freed once the`StringStore` object is garbage collected. |
|
|
| `keys` | <Abbr title="vector[uint64_t]">`vector[hash_t]`</Abbr> | A list of hash values in the `StringStore`. |
|