Commit Graph

12 Commits

Author SHA1 Message Date
Matthew Honnibal e2133d990e * Move serialization functionality out into a Serializer object 2015-07-16 11:21:44 +02:00
Matthew Honnibal 01fab6bb90 * Improve de/serialize functions 2015-07-16 01:26:35 +02:00
Matthew Honnibal 0e07c1ed2a * draft de/serialization functions in doc.pyx 2015-07-16 01:16:33 +02:00
Matthew Honnibal 9d956b07e9 * Fix import of attrs in doc.pyx, and update the get_token_attr function. 2015-07-16 01:15:34 +02:00
Matthew Honnibal 935ac53ee3 * Extend count_by method 2015-07-14 03:20:09 +02:00
Matthew Honnibal 81aa4e6dcc * Go back to having token reference doc, instead of complicated gymnastics. Rename the attr 'doc', to expose it in the API 2015-07-14 00:10:11 +02:00
Matthew Honnibal 8214b74eec * Restore _py_tokens cache, to handle orphan tokens. 2015-07-13 22:28:10 +02:00
Matthew Honnibal 67641f3b58 * Refactor tokenizer, to set the 'spacy' field on TokenC instead of passing a string 2015-07-13 21:46:02 +02:00
Matthew Honnibal 6eef0bf9ab * Break up tokens.pyx into tokens/doc.pyx, tokens/token.pyx, tokens/spans.pyx 2015-07-13 20:20:58 +02:00
Matthew Honnibal 3ea8756c24 * Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 19:58:26 +02:00
Matthew Honnibal d27899658e * Import classes in spacy.tokens.__init__ 2015-07-13 19:48:55 +02:00
Matthew Honnibal dba6b47d4e * Refactor monster tokens.pyx file, into a tokens/ subpackage. Try to break the cycle between Doc and Token, and remove the need to pass around a unicode string reference 2015-07-13 19:20:48 +02:00