5 Commits

Author	SHA1	Message	Date
Henning Peters	9d8966a2c0	Update test_tokenizer.py	2016-02-10 19:24:37 +01:00
Matthew Honnibal	7f24229f10	* Don't try to pickle the tokenizer	2016-02-06 14:09:05 +01:00
Matthew Honnibal	515493c675	* Add xfail test for Issue #225: tokenization with non-whitespace delimiters	2016-01-19 13:20:14 +01:00
Matthew Honnibal	223d2b3484	* Add test for Issue #154: Additional whitespace introduced when string ends with a whitespace token.	2016-01-16 17:08:07 +01:00
Matthew Honnibal	4e16f9e435	* Move tests underneath spacy/	2015-10-26 00:07:31 +11:00