Matthew Honnibal
|
c4cd3bc57a
|
* Add prefix and suffix data files
|
2014-09-25 18:24:52 +02:00 |
Matthew Honnibal
|
143e51ec73
|
* Refactor tokenization, splitting it into a clearer life-cycle.
|
2014-09-16 13:16:02 +02:00 |
Matthew Honnibal
|
6fc06bfe2f
|
* Hack a hard-cased unit in to get a test to pass
|
2014-09-15 06:31:35 +02:00 |
Matthew Honnibal
|
3b793cf4f7
|
* Tests passing for new Word object version
|
2014-08-24 18:13:53 +02:00 |
Matthew Honnibal
|
a22101404a
|
* Move en_ptb data
|
2014-08-22 04:28:51 +02:00 |
Matthew Honnibal
|
a2047fa5aa
|
* Add 's suffix to tokenization table
|
2014-08-18 23:21:37 +02:00 |
Matthew Honnibal
|
cc3971ce5c
|
* Fix error in tokenization rules
|
2014-07-07 05:09:34 +02:00 |
Matthew Honnibal
|
997551241f
|
* Upd ptb tokenization rules
|
2014-07-07 05:09:22 +02:00 |
Matthew Honnibal
|
df0458001d
|
* Begin work on full PTB-compatible English tokenization
|
2014-07-07 04:29:24 +02:00 |
Matthew Honnibal
|
d5bef02c72
|
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals
|
2014-07-07 04:21:06 +02:00 |