spaCy/spacy/strings.pxd

from libc.stdint cimport int64_t
from libcpp.vector cimport vector
from libcpp.set cimport set
from cymem.cymem cimport Pool
from preshed.maps cimport PreshMap
from murmurhash.mrmr cimport hash64

from .typedefs cimport attr_t, hash_t


cpdef hash_t hash_string(unicode string) except 0
cdef hash_t hash_utf8(char* utf8_string, int length) nogil

cdef unicode decode_Utf8Str(const Utf8Str* string)


ctypedef union Utf8Str:
    unsigned char[8] s
    unsigned char* p


cdef class StringStore:
    cdef Pool mem

    cdef vector[hash_t] keys
    cdef public PreshMap _map

    cdef const Utf8Str* intern_unicode(self, unicode py_string)
    cdef const Utf8Str* _intern_utf8(self, char* utf8_string, int length)
Refactor training, with new spacy.train module. Defaults still a little awkward. 2016-10-09 10:24:24 +00:00			`from libc.stdint cimport int64_t`
Work on changing StringStore to return hashes. 2017-05-28 10:36:27 +00:00			`from libcpp.vector cimport vector`
Try to fix StringStore clean up (see #1506) 2017-11-11 00:11:27 +00:00			`from libcpp.set cimport set`
* Move StringStore class to its own file 2014-12-19 19:42:01 +00:00			`from cymem.cymem cimport Pool`
			`from preshed.maps cimport PreshMap`
* Move murmurhash to .pxd in strings file 2014-12-19 20:41:08 +00:00			`from murmurhash.mrmr cimport hash64`
* Fix type declarations for attr_t. Remove unused id_t. 2015-07-18 20:39:57 +00:00
Refactor training, with new spacy.train module. Defaults still a little awkward. 2016-10-09 10:24:24 +00:00			`from .typedefs cimport attr_t, hash_t`
* Move StringStore class to its own file 2014-12-19 19:42:01 +00:00
* Tmp commit. Refactoring to create a Python Lexeme class. 2015-01-11 23:26:22 +00:00
			`cpdef hash_t hash_string(unicode string) except 0`
Work on changing StringStore to return hashes. 2017-05-28 10:36:27 +00:00			`cdef hash_t hash_utf8(char* utf8_string, int length) nogil`

			`cdef unicode decode_Utf8Str(const Utf8Str* string)`
* Move murmurhash to .pxd in strings file 2014-12-19 20:41:08 +00:00

* Fix header for string store 2015-07-20 10:06:10 +00:00			`ctypedef union Utf8Str:`
			`unsigned char[8] s`
			`unsigned char* p`


* Move StringStore class to its own file 2014-12-19 19:42:01 +00:00			`cdef class StringStore:`
			`cdef Pool mem`

Work on changing StringStore to return hashes. 2017-05-28 10:36:27 +00:00			`cdef vector[hash_t] keys`
* Use unicode in StringStore.intern, instead of unreliably casting to bytes. 2015-11-05 11:32:19 +00:00			`cdef public PreshMap _map`
* Move StringStore class to its own file 2014-12-19 19:42:01 +00:00
Update strings.pxd 2016-10-24 12:00:35 +00:00			`cdef const Utf8Str* intern_unicode(self, unicode py_string)`
			`cdef const Utf8Str* _intern_utf8(self, char* utf8_string, int length)`