Commit Graph

246 Commits

Author SHA1 Message Date
Matthew Honnibal e167355505 * Use JSON docs for training and evaluation. Currently a bug that is costing 0.6 acc 2015-05-12 20:27:56 +02:00
Matthew Honnibal e0ef6b6992 * Fix alignment in prepare_treebank 2015-05-12 20:27:56 +02:00
Matthew Honnibal 0ad72a77ce * Write JSON files, with both dependency and PSG parses 2015-05-12 20:27:55 +02:00
Matthew Honnibal 5078a32213 * Work on script to format training data as a JSON file. 2015-05-12 20:27:55 +02:00
Matthew Honnibal fb8d50b3d5 Merge branch 'master' of ssh://github.com/honnibal/spaCy 2015-04-30 12:45:15 +02:00
Matthew Honnibal 4489d87550 * Add cluster=0 by default in init_model 2015-04-29 14:23:13 +02:00
Jordan Suchow 3a8d9b37a6 Remove trailing whitespace 2015-04-19 13:01:38 -07:00
Jordan Suchow 5f0f940a1f Remove unused imports 2015-04-19 01:05:22 -07:00
Matthew Honnibal 693c5a1558 * Exclude clusterings for words only seen 1 or 2 times, as their clusters are unreliable 2015-04-17 04:44:52 +02:00
Matthew Honnibal adcad4f353 * Clean up train.py 2015-04-15 06:02:04 +02:00
Matthew Honnibal 1629b33082 * Fix copying of tokenizer data in init_model 2015-04-12 04:45:31 +02:00
Matthew Honnibal a6ac92f077 * Respect the model_dir input parameter to train.py 2015-04-08 22:48:26 +02:00
Matthew Honnibal baff0f8ad8 * Add docstring explaining script a bit, and add handling of word vectors 2015-04-08 08:20:15 +02:00
Matthew Honnibal 156b70ed82 * Add new script to replace make_lexicon, that does full setup of data 2015-04-08 07:46:53 +02:00
Matthew Honnibal e775e05313 * Use merge_mwe=False in evaluation in train.py 2015-04-08 00:35:19 +02:00
Matthew Honnibal f26f381b0e * Add simple ner_tag script 2015-04-03 17:26:58 +02:00
Matthew Honnibal bb27979352 * Add prepare_vecs script 2015-04-02 06:19:39 +02:00
Matthew Honnibal cd054c6c9f * Remove stray print statement 2015-03-27 15:20:42 +01:00
Matthew Honnibal ee385b439a * Ensure StringStore is dumped during training 2015-03-26 16:44:47 +01:00
Matthew Honnibal e854ba0a13 * Remove support for force_gold flag from GreedyParser, since it's not so useful, and it's clutter 2015-03-26 16:44:47 +01:00
Matthew Honnibal 221f43c370 * Ensure better separation between score printing and training in train.py 2015-03-26 16:44:46 +01:00
Matthew Honnibal 6d49f8717b * Move scoring away from training. Does not support scoring on gold preproc. 2015-03-26 16:44:46 +01:00
Matthew Honnibal 52429625f0 * Add write_parses function 2015-03-26 16:44:46 +01:00
Matthew Honnibal 0c91dd9e15 * Re-enable entity training 2015-03-26 16:44:46 +01:00
Matthew Honnibal 9061bbaf61 * Move to fixing up ent_strings and dep_strings passing 2015-03-26 16:44:45 +01:00
Matthew Honnibal 8f7eeb1c2d * Add verbose flag for Scorer, for debugging, and fix ent_strings bug 2015-03-26 16:44:45 +01:00
Matthew Honnibal 8057a95f20 * NER seems to be working, scoring 69 F. Need to add decision-history features --- currently only use current word, 2 words context. Need refactoring. 2015-03-26 16:44:44 +01:00
Matthew Honnibal ae235e07b9 * Refactoring working for parser, but now need to rig up features for NER, and then debug oracle etc. 2015-03-26 16:44:44 +01:00
Matthew Honnibal 4539c70542 * Work on updating train script for named entity recognition 2015-03-26 16:44:44 +01:00
Matthew Honnibal 7a1a333f04 * Allow gold tokenization training, for debugging 2015-03-26 16:44:43 +01:00
Matthew Honnibal f5f15a1ef2 * Tmp commit 2015-03-26 16:44:43 +01:00
Matthew Honnibal 34215de61b * Upd train script, moving lots of functionality to new GoldParse class 2015-03-26 16:44:42 +01:00
Matthew Honnibal 8715101239 * Merge changes from stash 2015-03-26 16:44:42 +01:00
Matthew Honnibal 3a302ae6f2 * Merge train.py 2015-03-26 16:44:41 +01:00
Matthew Honnibal 053814ffc8 * Report LAS in train script 2015-03-03 04:35:11 -05:00
Matthew Honnibal 1019939c7a * Whitespace 2015-02-24 23:03:02 -05:00
Matthew Honnibal ee33be31dd * Fix parser training script 2015-02-09 03:57:56 -05:00
Matthew Honnibal 27986d7f5c * Fix standard conll file reading. Script needs refactoring. 2015-02-02 23:02:48 +11:00
Matthew Honnibal 543fe5a280 * Comment training function back in 2015-01-31 16:37:57 +11:00
Matthew Honnibal df749f257d * Upd train script 2015-01-31 13:44:37 +11:00
Matthew Honnibal ca7577d8a9 * Allow parsers and taggers to be trained on text without gold pre-processing. 2015-01-30 16:36:24 +11:00
Matthew Honnibal b38093237e * More debug prints 2015-01-30 11:15:54 +11:00
Matthew Honnibal 11ed65b93c * Work on alignment, for evaluation with non-gold preprocessing 2015-01-30 10:31:03 +11:00
Matthew Honnibal b4348ce1c3 * Messily use unsegmented sentences to train the parser 2015-01-29 04:21:13 +11:00
Matthew Honnibal 72ff9c5082 * Update parser training script for tweaked parser API 2015-01-25 02:20:49 +11:00
Matthew Honnibal 61904e590f * Add parser training script 2015-01-10 04:53:26 +11:00