Matthew Honnibal 25849fc926 * Generalize tokenization rules to capitals		2014-07-07 05:07:21 +02:00
..
my_test.py	…
sun.tokens	* Working tokenization. en doesn't match PTB perfectly. Need to reorganize before adding more schemes.	2014-07-07 01:15:59 +02:00
sun.txt	* Working tokenization. en doesn't match PTB perfectly. Need to reorganize before adding more schemes.	2014-07-07 01:15:59 +02:00
test_contractions.py	* Generalize tokenization rules to capitals	2014-07-07 05:07:21 +02:00
test_post_punct.py	…
test_pre_punct.py	…
test_rules.py	* Tests passing for reorganized version	2014-07-07 04:23:46 +02:00
test_surround_punct.py	…
test_tokenizer.py	* Tests passing for reorganized version	2014-07-07 04:23:46 +02:00
test_vocab.py	…
test_wiki_sun.py	* Tests passing for reorganized version	2014-07-07 04:23:46 +02:00
tokenizer.sed	* Working tokenization. en doesn't match PTB perfectly. Need to reorganize before adding more schemes.	2014-07-07 01:15:59 +02:00