11 Commits

Author	SHA1	Message	Date
Wolfgang Seeker	03fb498dbe	introduce lang field for LexemeC to hold language id ... put noun_chunk logic into iterators.py for each language separately	2016-03-10 13:01:34 +01:00
Matthew Honnibal	c4017a06d9	* Add placeholders for the new flags in attrs and symbols	2016-02-04 15:49:45 +01:00
Matthew Honnibal	064bd69ad0	* Refactor symbols, so that frequency rank can be derived from the orth id of a word.	2015-10-10 16:03:48 +11:00
Matthew Honnibal	c2d8edd0bd	* Add PROB attribute in attrs.pxd	2015-08-26 19:14:19 +02:00
Matthew Honnibal	9c667b7f15	* Set a value in attrs.pxd on the first flag, to reduce bugs	2015-08-06 16:08:04 +02:00
Matthew Honnibal	8e4c69ee8c	* Add is_oov property, and fix up handling of attributes	2015-07-27 01:50:06 +02:00
Matthew Honnibal	6bb96c122d	* Host IS_ flags in attrs.pxd, and add properties for them on Token and Lexeme objects	2015-07-26 16:37:16 +02:00
Matthew Honnibal	efa80096f1	* Upd attrs id list	2015-07-16 01:26:54 +02:00
Jordan Suchow	3a8d9b37a6	Remove trailing whitespace	2015-04-19 13:01:38 -07:00
Matthew Honnibal	6640386b25	* Fix Issue #43: TAG attr not supported. Also add DEP attr, while I'm at it. Need better way of ensuring future changes don't break in similar way.	2015-04-07 06:00:57 +02:00
Matthew Honnibal	d4c99f7dec	* Add attrs.pxd	2015-01-26 22:22:09 +11:00