spaCy/spacy/tokens/doc.pxd

from cymem.cymem cimport Pool
cimport numpy as np
from preshed.counter cimport PreshCounter

from ..vocab cimport Vocab
from ..structs cimport TokenC, LexemeC
from ..typedefs cimport attr_t
from ..attrs cimport attr_id_t


cdef attr_t get_token_attr(const TokenC* token, attr_id_t feat_name) nogil


ctypedef const LexemeC* const_Lexeme_ptr
ctypedef const TokenC* const_TokenC_ptr

ctypedef fused LexemeOrToken:
    const_Lexeme_ptr
    const_TokenC_ptr


cdef int token_by_start(const TokenC* tokens, int length, int start_char) except -2


cdef int token_by_end(const TokenC* tokens, int length, int end_char) except -2


cdef class Doc:
    cdef readonly Pool mem
    cdef readonly Vocab vocab

    cdef public object _vector
    cdef public object _vector_norm

    cdef TokenC* c

    cdef public bint is_tagged
    cdef public bint is_parsed

    cdef public list _py_tokens

    cdef int length
    cdef int max_length

    cdef int push_back(self, LexemeOrToken lex_or_tok, bint trailing_space) except -1

    cpdef np.ndarray to_array(self, object features)

    cdef int set_parse(self, const TokenC* parsed) except -1
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00			`from cymem.cymem cimport Pool`
			`cimport numpy as np`
			`from preshed.counter cimport PreshCounter`

			`from ..vocab cimport Vocab`
			`from ..structs cimport TokenC, LexemeC`
* Gazetteer stuff working, now need to wire up to API 2015-08-05 22:35:40 +00:00			`from ..typedefs cimport attr_t`
			`from ..attrs cimport attr_id_t`


			`cdef attr_t get_token_attr(const TokenC* token, attr_id_t feat_name) nogil`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00

			`ctypedef const LexemeC* const_Lexeme_ptr`
* More work on language-generic parsing 2015-08-28 00:02:33 +00:00			`ctypedef const TokenC* const_TokenC_ptr`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00
			`ctypedef fused LexemeOrToken:`
			`const_Lexeme_ptr`
* More work on language-generic parsing 2015-08-28 00:02:33 +00:00			`const_TokenC_ptr`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00

* Rework the Span-merge patch, to avoid extending the interface of Doc, and avoid virtualizing the Span.start and Span.end indices, to keep Span usage efficient 2015-11-06 21:55:34 +00:00			`cdef int token_by_start(const TokenC* tokens, int length, int start_char) except -2`


			`cdef int token_by_end(const TokenC* tokens, int length, int end_char) except -2`


* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00			`cdef class Doc:`
* Make mem and vocab python-visible in Doc 2015-07-28 18:46:59 +00:00			`cdef readonly Pool mem`
			`cdef readonly Vocab vocab`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00
* Try giving Doc and Span objects vector and vector_norm attributes, and .similarity functions. Turns out to be bad idea. 2015-09-17 01:50:11 +00:00			`cdef public object _vector`
			`cdef public object _vector_norm`

* Rename Doc.data to Doc.c 2015-11-03 13:15:14 +00:00			`cdef TokenC* c`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00
			`cdef public bint is_tagged`
			`cdef public bint is_parsed`

* Restore _py_tokens cache, to handle orphan tokens. 2015-07-13 20:28:10 +00:00			`cdef public list _py_tokens`

* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00			`cdef int length`
			`cdef int max_length`

* Refactor tokenizer, to set the 'spacy' field on TokenC instead of passing a string 2015-07-13 19:46:02 +00:00			`cdef int push_back(self, LexemeOrToken lex_or_tok, bint trailing_space) except -1`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 17:58:26 +00:00
			`cpdef np.ndarray to_array(self, object features)`

			`cdef int set_parse(self, const TokenC* parsed) except -1`