Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,219 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

32 Commits

Author SHA1 Message Date
Matthew Honnibal 25a8774f42 * Fix regression in packer 2015-07-27 21:53:38 +02:00
Matthew Honnibal 1601e488ee * Fix bug in decoding non-ascii characters 2015-07-27 21:43:58 +02:00
Matthew Honnibal 6a95409cd2 * Fix type on bits 2015-07-27 21:16:49 +02:00
Matthew Honnibal 45460f505c * Fix data type on read32 in BitArray 2015-07-27 21:12:13 +02:00
Matthew Honnibal af6ed18f2a * Ensure we don't use orth_encode on OOV words. 2015-07-27 02:12:01 +02:00
Matthew Honnibal d62eb34b76 * More Py 2/3 compatibility in bit strings 2015-07-24 04:52:06 +02:00
Matthew Honnibal c4ff410fdb * Fix bytes problems for Python3 2015-07-24 03:48:23 +02:00
Matthew Honnibal 1ab25e4dad * Fix python3 type error 2015-07-24 02:45:34 +02:00
Matthew Honnibal f35ff173b0 * Fix bits.pyx unicode error 2015-07-23 20:37:57 +02:00
Matthew Honnibal 1406e24327 * Fix unicode error for Python3 2015-07-23 19:36:21 +02:00
Matthew Honnibal 1f31d96bf9 * Fix Packer API, so that it reads and writes bytes strings, instead of BitArray. Docs are always byte aligned anyway. 2015-07-23 01:13:02 +02:00
Matthew Honnibal 815bda201d * Remove UniStr struct 2015-07-22 13:39:17 +02:00
Matthew Honnibal 424854028f * Fix decode_int32 2015-07-21 20:09:59 +00:00
Matthew Honnibal 304d0e2633 * Use decode_int32 in _orth_decode 2015-07-21 20:40:55 +02:00
Matthew Honnibal 9cfa59ec33 * Optimistically try orth encoding, with char as a back-off 2015-07-21 20:22:45 +02:00
Matthew Honnibal c8b89e37a5 * Bug fix to faster huffman decoding 2015-07-21 20:05:53 +02:00
Matthew Honnibal b166d1d2a2 * Use encode32 and decode32 2015-07-21 19:59:06 +02:00
Matthew Honnibal c6cd0ddce8 * Add faster encode_int32 and decode_int32 methods 2015-07-21 19:58:45 +02:00
Matthew Honnibal bb0ba1f0cd * Improve serialization speed 2015-07-20 03:27:59 +02:00
Matthew Honnibal 5a7d060d9c * Switch between the orth and char codecs depending on which is shorter for that message. Mostly orth is shorter, except if there are OOV words. 2015-07-20 01:36:22 +02:00
Matthew Honnibal 5a042ee0d3 * Add function to predict number of bits needed to encode message 2015-07-20 01:35:11 +02:00
Matthew Honnibal b89b489bb4 * Implement both character and orth encoding in Packer, so that we can decide which to use per-text 2015-07-19 22:39:45 +02:00
Matthew Honnibal ae78c9e3ce * Implement character-based codec, so that we can do word/char backoff 2015-07-19 22:03:39 +02:00
Matthew Honnibal cd1d047cb8 * Delete out-dated HuffmanCodec comment 2015-07-19 18:28:14 +02:00
Matthew Honnibal b8086067d5 * Build Huffman codec from unsorted inputs 2015-07-19 17:58:44 +02:00
Matthew Honnibal 317cbbc015 * Serialization round trip now working with decent API, but with rough spots in the organisation and requiring vocabulary to be fixed ahead of time. 2015-07-19 15:18:17 +02:00
Matthew Honnibal 5b4c78bbb2 * Use an AttributeCodec based on orth for words. Still no oov handling mechanism. 2015-07-18 22:43:18 +02:00
Matthew Honnibal cd91914dd8 * Fix hard-coded length 2015-07-18 04:09:56 +02:00
Matthew Honnibal cf0c788892 * Tests passing on round-trip pack/unpack on basic example 2015-07-17 21:20:48 +02:00
Matthew Honnibal 9e3f17051b * Move to ORTH instead of ID for encoding lexemes. Basic tests of the codec wrappers now passing 2015-07-17 16:38:29 +02:00
Matthew Honnibal 234c7e440a * Add spacy/serialize/__init__ files 2015-07-17 01:37:33 +02:00
Matthew Honnibal db9dfd2e23 * Major refactor of serialization. Nearly complete now. 2015-07-17 01:27:54 +02:00