From 1ccabc806ef8c4575e12439b05f2d7cdb6d4b7e5 Mon Sep 17 00:00:00 2001 From: Matthew Honnibal Date: Tue, 9 Dec 2014 16:06:18 +1100 Subject: [PATCH] * Work on lemmatization --- data/en/suffix | 1 - data/en/tokenization | 93 -------------------------------------------- 2 files changed, 94 deletions(-) diff --git a/data/en/suffix b/data/en/suffix index 77400d0fd..a6ff32849 100644 --- a/data/en/suffix +++ b/data/en/suffix @@ -16,7 +16,6 @@ $ '' 's 'S -. .. ... .... diff --git a/data/en/tokenization b/data/en/tokenization index e2b78dd28..382b7e383 100644 --- a/data/en/tokenization +++ b/data/en/tokenization @@ -4,99 +4,6 @@ #*---* --- #*'s 's -'s 's -'S 'S -ain't ai n't -aren't are n't -can't ca n't -cannot can not -could've could 've -couldn't could n't -couldn't've could n't 've -didn't did n't -doesn't does n't -don't do n't -hadn't had n't -hadn't've had n't 've -hasn't has n't -haven't have n't -he'd he 'd -he'd've he 'd 've -he'll he 'll -he's he 's -how'd he 'd -how'll he 'll -how's how 's -I'd I 'd -I'd've I 'd 've -I'll I 'll -I'm I 'm -I'ma I 'ma -I've I 've -isn't is n't -it'd it 'd -it'd've it 'd 've -it'll it 'll -it's it 's -let's let 's -mightn't might n't -mightn't've might n't 've -might've might 've -mustn't must n't -must've must 've -needn't need n't -not've not 've -shan't sha n't -she'd she 'd -she'd've she 'd 've -she'll she will -she's she 's -should've should 've -shouldn't should n't -shouldn't've should n't 've -that's that 's -there'd there 'd -there'd've there 'd 've -there's there 's -they'd there 'd -they'd've they 'd 've -they'll they 'll -they're they 're -they've they 've -wasn't was n't -we'd we 'd -we'd've we 'd 've -we'll we 'll -we're we 're -we've we 've -weren't were n't -what'll what 'll -what're what 're -what's what 's -what've what 've -when's when 's -where'd where 'd -where's where 's -where've where 've -who'd who 'd -who'll who 'll -who're who 're -who's who 's -who've who 've -why'll why 'll -why're why 're -why's why 's -won't wo n't -would've would 've -wouldn't would n't -wouldn't've would n't 've -you'd you 'd -you'd've you 'd 've -you'll you 'll -you're you 're -you've you 've -'em 'em -'ol 'ol 10km 10 km U.S. U.S. U.K. U.K.