Commit Graph

1140 Commits

Author SHA1 Message Date
Matthew Honnibal 7a2725bca4 * Read input json in a streaming way 2015-05-27 19:13:11 +02:00
Matthew Honnibal b7fd77779a * Add some tests for reading NER data 2015-05-27 17:37:03 +02:00
Matthew Honnibal 6a1c91675e * Add file to read ENAMEX ner data 2015-05-27 17:36:23 +02:00
Matthew Honnibal ef1333cf89 * Have prepare_treebank read train/dev/test IDs. 2015-05-27 17:35:05 +02:00
Matthew Honnibal e140e03516 * Read in OntoNotes. Doesn't support train/test/dev split yet 2015-05-27 17:04:29 +02:00
Matthew Honnibal 732fa7709a * Edits to align_raw script, for use in prepare_treebank 2015-05-27 04:23:31 +02:00
Matthew Honnibal 4010b9b6d9 * Pass parameter for regularization in parser.pyx 2015-05-27 03:18:50 +02:00
Matthew Honnibal 4c6058baa7 * Fix evaluation of NER in scorer.py 2015-05-27 03:18:16 +02:00
Matthew Honnibal 6016ee83a6 * Fix reading of NER in gold.pyx 2015-05-27 03:17:50 +02:00
Matthew Honnibal 04bda8648d * Pass parameter for regularization to model 2015-05-27 03:16:58 +02:00
Matthew Honnibal 895060e774 * Ensure tagger and NER are trained, even if non-projective problem 2015-05-27 03:16:21 +02:00
Matthew Honnibal f69fe6a635 * Fix heads problem in read_conll 2015-05-27 01:14:54 +02:00
Matthew Honnibal 0eec1d12af * Add comment about zipf reweighting 2015-05-27 01:14:07 +02:00
Matthew Honnibal 4d37b66c55 * Make Zipf regularization a bit more efficient 2015-05-27 01:12:50 +02:00
Matthew Honnibal 7fc24821bc * Experiment with Zipfian corruptions when calculating prediction 2015-05-26 22:17:15 +02:00
Matthew Honnibal 32ae2cdabe * In prepare_treebank, move ner into the token descriptions 2015-05-26 19:52:39 +02:00
Matthew Honnibal 61885aee76 * Work on prepare_treebank script, adding NER to it 2015-05-26 19:28:29 +02:00
Matthew Honnibal 15bbbf4901 * Remove cruft from train.py 2015-05-25 07:54:10 +02:00
Matthew Honnibal eba7b34f66 * Add flag to disable loading of word vectors 2015-05-25 01:02:42 +02:00
Matthew Honnibal 89c3364041 * Update tests, preventing the parser from being loaded if possible 2015-05-25 01:02:03 +02:00
Matthew Honnibal a9c70c9447 * Add tests for ontonotes sgml extraction 2015-05-24 21:52:12 +02:00
Matthew Honnibal f460a8d2b6 * Comment out failing test in test_conjuncts 2015-05-24 21:51:41 +02:00
Matthew Honnibal cc7439a16b * Don't use alignment.pyx file, move functionality to spacy.gold 2015-05-24 21:51:15 +02:00
Matthew Honnibal 3593babd35 * Add functions for Levenshtein distance alignment 2015-05-24 21:50:48 +02:00
Matthew Honnibal 744f06abf5 * Add script to read OntoNotes source documents 2015-05-24 21:49:58 +02:00
Matthew Honnibal 13a8595a4b * Add tests for Levenshtein alignment of training data 2015-05-24 21:46:11 +02:00
Matthew Honnibal fc75210941 * Move spacy.syntax.conll to spacy.gold 2015-05-24 21:35:02 +02:00
Matthew Honnibal 765b61cac4 * Update spacy.scorer, to use P/R/F to support tokenization errors 2015-05-24 20:07:18 +02:00
Matthew Honnibal efe7a7d7d6 * Clean unused functions from spacy.syntax.conll 2015-05-24 20:06:46 +02:00
Matthew Honnibal 78487f3e66 * Update parser oracle for missing heads 2015-05-24 20:05:58 +02:00
Matthew Honnibal 541c62c126 * Remove import of removed read_docparse_file function 2015-05-24 20:05:13 +02:00
Matthew Honnibal 1044a13413 * Begin refactoring scorer to use recall over gold dependencies 2015-05-24 17:40:15 +02:00
Matthew Honnibal acd1245ad4 * Remove cruft from conll.pyx --- unused stuff about evlauation, which now lives in spacy.scorer 2015-05-24 17:35:49 +02:00
Matthew Honnibal bfeb29ebd1 * Tmp commit 2015-05-24 02:50:14 +02:00
Matthew Honnibal 20f1d868a3 * Tmp commit. Working on whole document parsing 2015-05-24 02:49:56 +02:00
Matthew Honnibal 983d954ef4 * Tmp commit, while switch to new format that assumes alignment happens during training 2015-05-23 17:39:04 +02:00
Matthew Honnibal f35503018e * Tmp commit of train, while I move to better alignment in gold standard 2015-05-23 17:21:25 +02:00
Matthew Honnibal bdaddc4103 * Add PTB file read tests 2015-05-20 17:29:27 +02:00
Matthew Honnibal f2ee9c4feb * Comment out constituency parsing stuff, so that code compiles 2015-05-20 16:55:05 +02:00
Matthew Honnibal 8ee7c541f1 * Update Constituent definition 2015-05-20 16:03:26 +02:00
Matthew Honnibal 9dfc9c039c * Work on constituency parsing. 2015-05-20 16:02:51 +02:00
Matthew Honnibal 8e7d5fce90 * Add members to Tokens documentation 2015-05-13 21:53:41 +02:00
Matthew Honnibal 5a5710e711 * Fix Span.subtree property 2015-05-13 21:53:15 +02:00
Matthew Honnibal badf030b6c * Add parse navigation to Span objects 2015-05-13 21:45:19 +02:00
Matthew Honnibal ca320afe86 * Add docstring for ents attribute 2015-05-13 21:20:47 +02:00
Matthew Honnibal 7c8bf0eba5 * Add example JSON-formatted training file 2015-05-12 22:42:37 +02:00
Matthew Honnibal 4230467947 * Update fabfile.py for JSON-formatted training 2015-05-12 22:34:07 +02:00
Matthew Honnibal ba07b925a7 * Fix compile error in conll.pyx 2015-05-12 22:33:47 +02:00
Matthew Honnibal f1e0272b18 * Disable c-parsing transitions 2015-05-12 22:33:25 +02:00
Matthew Honnibal 03a6626545 * Tmp commit 2015-05-12 20:27:56 +02:00