9 Commits

Author	SHA1	Message	Date
Matthew Honnibal	5a7d060d9c	* Switch between the orth and char codecs depending on which is shorter for that message. Mostly orth is shorter, except if there are OOV words.	2015-07-20 01:36:22 +02:00
Matthew Honnibal	b89b489bb4	* Implement both character and orth encoding in Packer, so that we can decide which to use per-text	2015-07-19 22:39:45 +02:00
Matthew Honnibal	ae78c9e3ce	* Implement character-based codec, so that we can do word/char backoff	2015-07-19 22:03:39 +02:00
Matthew Honnibal	317cbbc015	* Serialization round trip now working with decent API, but with rough spots in the organisation and requiring vocabulary to be fixed ahead of time.	2015-07-19 15:18:17 +02:00
Matthew Honnibal	5b4c78bbb2	* Use an AttributeCodec based on orth for words. Still no oov handling mechanism.	2015-07-18 22:43:18 +02:00
Matthew Honnibal	cd91914dd8	* Fix hard-coded length	2015-07-18 04:09:56 +02:00
Matthew Honnibal	cf0c788892	* Tests passing on round-trip pack/unpack on basic example	2015-07-17 21:20:48 +02:00
Matthew Honnibal	9e3f17051b	* Move to ORTH instead of ID for encoding lexemes. Basic tests of the codec wrappers now passing	2015-07-17 16:38:29 +02:00
Matthew Honnibal	db9dfd2e23	* Major refactor of serialization. Nearly complete now.	2015-07-17 01:27:54 +02:00