* Upd tokenization special-cases

This commit is contained in:
Matthew Honnibal 2014-11-11 22:10:15 +11:00
parent 95282d4993
commit da70b6bd60
1 changed files with 8 additions and 0 deletions

View File

@ -104,6 +104,14 @@ U.N. U.N.
Co. Co.
Corp. Corp.
Inc. Inc.
Ltd. Ltd.
CORP. CORP.
Jr. Jr.
St. St.
Dr. Dr.
U.S.A. U.S.A.
S.A. S.A.
A.G. A.G.
Rep. Rep.
Ms. Ms.
Mr. Mr.