spaCy/spacy/structs.pxd

from libc.stdint cimport uint8_t, uint32_t, int32_t

from .typedefs cimport flags_t, attr_t, id_t, hash_t
from .parts_of_speech cimport univ_pos_t


cdef struct LexemeC:
    const float* repvec

    flags_t flags

    attr_t id
    attr_t length

    attr_t orth
    attr_t lower
    attr_t norm
    attr_t shape
    attr_t prefix
    attr_t suffix

    attr_t cluster

    float prob
    float sentiment
    float l2_norm


cdef struct Morphology:
    uint8_t number
    uint8_t tenspect # Tense/aspect/voice
    uint8_t mood
    uint8_t gender
    uint8_t person
    uint8_t case
    uint8_t misc


cdef struct PosTag:
    Morphology morph
    int id
    univ_pos_t pos


cdef struct Entity:
    int start
    int end
    int label


cdef struct Constituent:
    const TokenC* head
    const Constituent* parent
    const Constituent* first
    const Constituent* last
    int label
    int length


cdef struct TokenC:
    const LexemeC* lex
    Morphology morph
    const Constituent* ctnt
    univ_pos_t pos
    bint spacy
    int tag
    int idx
    int lemma
    int sense
    int head
    int dep
    bint sent_start

    uint32_t l_kids
    uint32_t r_kids
    uint32_t l_edge
    uint32_t r_edge

    int ent_iob
    int ent_type


cdef struct Utf8Str:
    id_t i
    hash_t key
    unsigned char* chars
    int length


cdef struct UniStr:
    Py_UNICODE* chars
    size_t n
    hash_t key
* Tmp commit of NER refactoring 2015-02-18 09:41:06 +00:00			`from libc.stdint cimport uint8_t, uint32_t, int32_t`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00
* Move POS tag definitions to parts_of_speech.pxd 2015-01-25 05:31:07 +00:00			`from .typedefs cimport flags_t, attr_t, id_t, hash_t`
			`from .parts_of_speech cimport univ_pos_t`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00

* Tmp commit. Refactoring to create a Python Lexeme class. 2015-01-11 23:26:22 +00:00			`cdef struct LexemeC:`
* Rename vec to repvec 2015-01-21 15:04:24 +00:00			`const float* repvec`
* Tmp commit. Refactoring to create a Python Lexeme class. 2015-01-11 23:26:22 +00:00
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`flags_t flags`
Remove trailing whitespace 2015-04-19 08:31:31 +00:00
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`attr_t id`
* Tmp. Refactoring, introducing a Lexeme PyObject. 2015-01-12 00:23:44 +00:00			`attr_t length`

* Rename sic to orth 2015-01-22 15:08:25 +00:00			`attr_t orth`
* Rename NORM1 and NORM2 attrs to lower and norm 2015-01-23 19:17:03 +00:00			`attr_t lower`
			`attr_t norm`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`attr_t shape`
			`attr_t prefix`
			`attr_t suffix`
Remove trailing whitespace 2015-04-19 08:31:31 +00:00
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`attr_t cluster`

			`float prob`
			`float sentiment`
* Add L2 norm field to LexemeC struct 2015-02-07 13:43:17 +00:00			`float l2_norm`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00

			`cdef struct Morphology:`
			`uint8_t number`
			`uint8_t tenspect # Tense/aspect/voice`
			`uint8_t mood`
			`uint8_t gender`
			`uint8_t person`
			`uint8_t case`
			`uint8_t misc`


			`cdef struct PosTag:`
			`Morphology morph`
			`int id`
* Move POS tag definitions to parts_of_speech.pxd 2015-01-25 05:31:07 +00:00			`univ_pos_t pos`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00

* Tmp 2015-03-09 05:46:22 +00:00			`cdef struct Entity:`
			`int start`
			`int end`
			`int label`
* NER seems to be working, scoring 69 F. Need to add decision-history features --- currently only use current word, 2 words context. Need refactoring. 2015-03-10 17:00:23 +00:00
* Tmp 2015-03-09 05:46:22 +00:00
* Add ctnt field to State, in preparation for constituency parsing 2015-05-06 14:29:10 +00:00			`cdef struct Constituent:`
* Update Constituent definition 2015-05-20 14:03:26 +00:00			`const TokenC* head`
			`const Constituent* parent`
			`const Constituent* first`
			`const Constituent* last`
* Add ctnt field to State, in preparation for constituency parsing 2015-05-06 14:29:10 +00:00			`int label`
* Update Constituent definition 2015-05-20 14:03:26 +00:00			`int length`
* Add ctnt field to State, in preparation for constituency parsing 2015-05-06 14:29:10 +00:00

* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`cdef struct TokenC:`
* Tmp commit. Refactoring to create a Python Lexeme class. 2015-01-11 23:26:22 +00:00			`const LexemeC* lex`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`Morphology morph`
* Update Constituent definition 2015-05-20 14:03:26 +00:00			`const Constituent* ctnt`
* Move POS tag definitions to parts_of_speech.pxd 2015-01-25 05:31:07 +00:00			`univ_pos_t pos`
* Add TokenC.spacy attr 2015-07-13 17:48:07 +00:00			`bint spacy`
* Work on word vectors, and other stuff 2015-01-17 05:21:17 +00:00			`int tag`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`int idx`
			`int lemma`
			`int sense`
			`int head`
* Work on word vectors, and other stuff 2015-01-17 05:21:17 +00:00			`int dep`
* Rename sent_start to sent_end, to reflect its new usage in the Break transition 2015-06-23 03:39:23 +00:00			`bint sent_start`
* Add l_edge and r_edge props in TokenC for tracking the parse-yield of the token 2015-04-29 17:14:20 +00:00
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00			`uint32_t l_kids`
			`uint32_t r_kids`
* Add l_edge and r_edge props in TokenC for tracking the parse-yield of the token 2015-04-29 17:14:20 +00:00			`uint32_t l_edge`
			`uint32_t r_edge`
* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00
* NER seems to be working, scoring 69 F. Need to add decision-history features --- currently only use current word, 2 words context. Need refactoring. 2015-03-10 17:00:23 +00:00			`int ent_iob`
			`int ent_type`

* Move all struct definitions to structs.pxd, to avoid circular dependencies 2014-12-19 19:51:33 +00:00
			`cdef struct Utf8Str:`
			`id_t i`
			`hash_t key`
			`unsigned char* chars`
			`int length`


			`cdef struct UniStr:`
			`Py_UNICODE* chars`
			`size_t n`
			`hash_t key`