Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,366 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

3768 Commits

Author SHA1 Message Date
Matthew Honnibal 3d049af563 Improve vectors to/from disk 2017-08-19 18:42:11 +02:00
Matthew Honnibal d55d6e1cfa Fix comparison of Token from different docs. Closes #1257 2017-08-19 16:39:32 +02:00
Matthew Honnibal 9b6a5df15e Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-19 16:24:57 +02:00
Matthew Honnibal 4fda02c7e6 Add test for new Span.to_array method 2017-08-19 16:24:38 +02:00
Matthew Honnibal dea229c634 Fix Span.to_array method 2017-08-19 16:24:28 +02:00
Matthew Honnibal c606b4a42c Add test for Doc.char_span 2017-08-19 16:18:23 +02:00
Matthew Honnibal 8b7ac77c23 Allow span label to be string in Doc.char_span 2017-08-19 16:18:09 +02:00
Matthew Honnibal 7c47e38c12 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-19 09:03:15 -05:00
Matthew Honnibal ab28f911b4 Fix parser learning rates 2017-08-19 09:02:57 -05:00
ines 1fe5e1a4d1 Add language example sentences (see #1107) 
da, de, en, es, fr, he, it, nb, pl, pt, sv
 2017-08-19 12:22:29 +02:00
Matthew Honnibal 97aabafb5f Document as_tuples keyword arg of Language.pipe 2017-08-19 12:21:33 +02:00
Matthew Honnibal 80236116a6 Add Doc.char_span method, to get a span by character offset 2017-08-19 12:21:09 +02:00
Matthew Honnibal 482bba1722 Add Span.to_array method 2017-08-19 12:20:45 +02:00
Matthew Honnibal 19c495f451 Fix vectors deserialization 2017-08-19 04:33:03 +02:00
Matthew Honnibal 42d47c1e5c Fix tagger serialization 2017-08-19 04:16:32 +02:00
Matthew Honnibal 2da96a0ec7 Fix beam test 2017-08-19 04:15:46 +02:00
Matthew Honnibal a7309a217d Update tagger serialization 2017-08-18 23:12:05 +02:00
Matthew Honnibal bae59bf92f Remove BiLSTM import 2017-08-18 22:46:59 +02:00
Matthew Honnibal c307a0ffb8 Restore patches from nn-beam-parser to spacy/syntax 2017-08-18 22:38:59 +02:00
Matthew Honnibal fe90dfc390 Restore changes from nn-beam-parser to spacy/_ml 2017-08-18 22:38:28 +02:00
Matthew Honnibal de7e8703e3 Restore tests for beam parser 2017-08-18 22:27:42 +02:00
Matthew Honnibal 11c31d285c Restore changes from nn-beam-parser 2017-08-18 22:26:12 +02:00
Matthew Honnibal ce321b0322 Restore changes from nn-beam-parser to spacy/_ml 2017-08-18 22:24:46 +02:00
Matthew Honnibal 5f81d700ff Restore patches from nn-beam-parser to spacy/syntax 2017-08-18 22:23:03 +02:00
Matthew Honnibal ec482580b5 Restore changes to pipeline.pyx from nn-beam-parser branch 2017-08-18 22:02:35 +02:00
Matthew Honnibal 931509d96a Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-18 21:57:15 +02:00
Matthew Honnibal ed95009b5c Fix data loading on Python 2 2017-08-18 21:57:06 +02:00
Matthew Honnibal baf36d0588 Add compat function for importlib.util 2017-08-18 21:56:47 +02:00
Matthew Honnibal 263366729e Don't import BiLSTM 2017-08-18 21:56:31 +02:00
Matthew Honnibal 28162290b3 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-18 14:55:40 -05:00
Matthew Honnibal 85794c1167 Restore state of _ml.py 2017-08-18 14:55:23 -05:00
Matthew Honnibal d456d2efe1 Fix conflicts in nn_parser 2017-08-18 20:55:58 +02:00
Matthew Honnibal 1cec1efca7 Fix merge conflicts in nn_parser from beam stuff 2017-08-18 20:50:49 +02:00
Matthew Honnibal 69bcacdc09 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-18 20:47:13 +02:00
Matthew Honnibal 2993b54fff Load vectors in vocab 2017-08-18 20:46:56 +02:00
Matthew Honnibal a1ec41298c Restore CFile loader 2017-08-18 20:46:16 +02:00
Matthew Honnibal ed4fb991dc Work on vectors loading 2017-08-18 20:45:48 +02:00
Matthew Honnibal 426f84937f Resolve conflicts when merging new beam parsing stuff 2017-08-18 13:38:32 -05:00
Matthew Honnibal 5181e8bedb Fix merge conflict in _ml 2017-08-18 13:35:51 -05:00
Matthew Honnibal f75420ae79 Unhack beam parsing, moving it under options instead of global flags 2017-08-18 13:31:15 -05:00
Jim Geovedi 7ae45bffcf Merge remote-tracking branch 'upstream/develop' into indonesian 2017-08-18 10:14:46 +07:00
Dan O'Huiginn ebf5a3ce59 Allow loading with python < 3.6 
Don't rely on recent python features to load models

Fixes Issue #1271
 2017-08-17 15:15:47 +00:00
Matthew Honnibal 0209a06b4e Update beam parser 2017-08-16 18:25:49 -05:00
Matthew Honnibal 4b1e7bd6d8 Improve tensorizer model 2017-08-16 18:25:20 -05:00
Matthew Honnibal a6d8d7c82e Add is_gold_parse method to transition system 2017-08-16 18:24:09 -05:00
Matthew Honnibal 3533bb61cb Add option of 8 feature parse state 2017-08-16 18:23:27 -05:00
Matthew Honnibal 1cb2f15d65 Clean up unused predict_confidences function 2017-08-16 18:22:26 -05:00
Matthew Honnibal 210f6d5175 Fix efficiency error in batch parse 2017-08-15 03:19:03 -05:00
Matthew Honnibal 23537a011d Tweaks to beam parser 2017-08-15 03:15:28 -05:00
Matthew Honnibal 500e92553d Fix memory error when copying scores in beam 2017-08-15 03:15:04 -05:00
Matthew Honnibal a8e4064dd8 Fix tensor gradient in parser 2017-08-15 03:14:36 -05:00
Matthew Honnibal e420e0366c Remove use of hash function in beam parser 2017-08-15 03:13:57 -05:00
Matthew Honnibal 6259490347 Fix mixture weights in fine_tune 2017-08-14 17:55:18 -05:00
Matthew Honnibal 335fa8b05c Fix gradient in fine_tune 2017-08-14 14:55:47 -05:00
Matthew Honnibal d9f82f6b50 Increment version 2017-08-14 14:55:26 +02:00
ines a29f132ffd Change python -m spacy to spacy 
Reflects latest change to entry point or auto-alias
 2017-08-14 13:04:48 +02:00
ines 65bf80302c Increment version 2017-08-14 13:04:30 +02:00
Matthew Honnibal 52c180ecf5 Revert "Merge branch 'develop' of https://github.com/explosion/spaCy into develop" 
This reverts commit ea8de11ad5, reversing
changes made to 08e443e083.
 2017-08-14 13:00:23 +02:00
Matthew Honnibal dbbfe595a5 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-14 12:09:28 +02:00
Matthew Honnibal ac6c25f762 Check SGD is not None in update 2017-08-14 12:09:18 +02:00
Matthew Honnibal 0ae045256d Fix beam training 2017-08-13 18:02:05 -05:00
Matthew Honnibal 6a42cc16ff Fix beam parser, improve efficiency of non-beam 2017-08-13 12:37:26 +02:00
Matthew Honnibal 4363b4aa4a Fix redundant tokvecs updates during update 2017-08-13 12:36:55 +02:00
Matthew Honnibal 12de263813 Bug fixes to beam parsing. Learns small sample 2017-08-13 09:33:39 +02:00
Matthew Honnibal 4ae0d5e1e6 Set defaults for convert command 2017-08-13 09:03:38 +02:00
Matthew Honnibal 92ebab6073 Update beam-update tests 2017-08-13 08:56:02 +02:00
Matthew Honnibal 17874fe491 Disable beam parsing 2017-08-12 19:35:40 -05:00
Matthew Honnibal 69f21867b5 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-12 19:25:56 -05:00
Matthew Honnibal 3e30712b62 Improve defaults 2017-08-12 19:24:17 -05:00
Matthew Honnibal 28e930aae0 Fixes for beam parsing. Not working 2017-08-12 19:22:52 -05:00
Matthew Honnibal c96d769836 Fix beam parse. Not sure if working 2017-08-12 18:21:54 -05:00
Matthew Honnibal 24b45b45c6 Add test for beam update 2017-08-12 17:15:28 -05:00
Matthew Honnibal 4638f4b869 Fix beam update 2017-08-12 17:15:16 -05:00
Matthew Honnibal d4308d2363 Initialize State offset to 0 2017-08-12 17:14:39 -05:00
Matthew Honnibal b353e4d843 Work on parser beam training 2017-08-12 14:47:45 -05:00
ines d4f2baf7dd Add create_meta option to package command 
Re-create meta.json in model directory, even if it exists. Especially
useful when updating existing spaCy models or training with Prodigy.
Ensures user won't end up with multiple "en_core_web_sm" models, and
offers easy way to change the model's name and settings without having
to edit the meta.json file.
 2017-08-12 21:44:18 +02:00
Matthew Honnibal 4ab0c8c8e9 Try different drop_layer structure in Tok2Vec 2017-08-12 08:56:57 -05:00
Matthew Honnibal cd5ecedf6a Try drop_layer in parser 2017-08-12 08:56:33 -05:00
Matthew Honnibal 8870d491f1 Remove redundant pickling during training 2017-08-12 08:55:53 -05:00
Matthew Honnibal 680043ebca Improve efficiency of tagger.set_annotations for GPU 2017-08-12 08:54:21 -05:00
Matthew Honnibal ebe0f7f641 Pass embed size correctly in tagger, and cache embeddings for efficiency 2017-08-12 05:45:20 -05:00
Matthew Honnibal 1a59db1c86 Fix dropout and learn rate in parser 2017-08-12 05:44:39 -05:00
Matthew Honnibal d01dc3704a Adjust parser model 2017-08-09 20:06:33 -05:00
Matthew Honnibal f37528ef58 Pass embed size for parser fine-tune. Use SELU 2017-08-09 17:52:53 -05:00
Matthew Honnibal f93f2bed58 Revert use of layer normalization in Tok2Vec 2017-08-09 17:47:03 -05:00
Matthew Honnibal 20944dd8aa Fix conflict in parser fine-tuning 2017-08-09 16:43:05 -05:00
Matthew Honnibal ac2de6dced Switch to ReLu layers in Tok2Vec 2017-08-09 16:41:25 -05:00
Matthew Honnibal bbace204be Gate parser fine-tuning behind feature flag 2017-08-09 16:40:42 -05:00
Matthew Honnibal a59a1deac4 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-09 16:23:19 -05:00
Matthew Honnibal bcce6f7de0 Fix parser fine tuning 2017-08-09 16:23:12 -05:00
ines 28e2fec23b Fix autolinking failure on fresh model install (resolves #1138) 
On fresh install via subprocess, pip.get_installed_distributions()
won't show new model, so is_package check in link command fails.
Solution for now is to get model package path explicitly and pass it to
link command.
 2017-08-09 11:52:38 +02:00
Jim Geovedi c62b49b7cc Merge remote-tracking branch 'upstream/develop' into indonesian 2017-08-09 09:17:46 +07:00
Matthew Honnibal dbdd8afc4b Fix parser fine-tune training 2017-08-08 15:46:07 -05:00
Matthew Honnibal 88bf1cf87c Update parser for fine tuning 2017-08-08 15:34:17 -05:00
Matthew Honnibal 5d837c3776 Add mix weights on fine_tune 2017-08-07 06:32:59 -05:00
Matthew Honnibal 42bd26f6f3 Give parser its own tok2vec weights 2017-08-06 18:33:46 +02:00
Matthew Honnibal 3ed203de25 Use LayerNorm and SELU in Tok2Vec 2017-08-06 18:33:18 +02:00
Matthew Honnibal 78498a072d Return Transition for missing actions in lookup_action 2017-08-06 14:16:36 +02:00
Matthew Honnibal 4a5cc89138 Fix tagger 'fine_tune', to keep private CNN weights 2017-08-06 14:15:48 +02:00
Matthew Honnibal 3cb8f06881 Fix NeuralLabeller 2017-08-06 14:15:14 +02:00
Matthew Honnibal 0acce0521b Fix Language.update for pipeline 2017-08-06 14:13:03 +02:00
Matthew Honnibal bfffdeabb2 Fix parser batch-size bug introduced during cleanup 2017-08-06 14:10:48 +02:00
Matthew Honnibal 0eec7c9e9b Fix Language.evaluate 2017-08-06 02:18:31 +02:00
Matthew Honnibal 0a566dc320 Add update_tensors flag to Language.update. Experimental, re #1182 2017-08-06 02:18:12 +02:00
Matthew Honnibal cc19ea0e7c Add update_tensors flag to Language.update. Experimental, re #1182 2017-08-06 02:17:10 +02:00
Matthew Honnibal 4cfb7a54e7 Fix tagger 2017-08-06 01:53:31 +02:00
Matthew Honnibal e9ab800e15 Fix tagging model 2017-08-06 01:50:08 +02:00
Matthew Honnibal 468c138ab3 WIP: Add fine-tuning logic to tagger model, re #1182 2017-08-06 01:13:23 +02:00
Matthew Honnibal 7f876a7a82 Clean up some unused code in parser 2017-08-06 00:00:21 +02:00
Matthew Honnibal ae1ad81069 Increment version 2017-08-05 18:09:32 +02:00
Jim Geovedi cc4772cac2 reworks 2017-08-03 13:08:38 +07:00
Jim Geovedi 37f19f5ed2 added more currencies based on corpus data 2017-08-03 13:03:25 +07:00
Jim Geovedi 30fd068d42 hashtag prefix should be handled somewhere else 2017-08-03 13:03:02 +07:00
Jim Geovedi 4705ae19ba Merge remote-tracking branch 'upstream/develop' into indonesian 2017-08-03 12:40:19 +07:00
Jim Geovedi ba07e23c87 added USD in currency rules 2017-08-02 22:42:47 +07:00
Matthew Honnibal 5c323daa1a Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-08-01 22:10:37 +02:00
Matthew Honnibal 2e00361522 Fix update when 0 docs 2017-08-01 22:10:17 +02:00
Matthew Honnibal 8fce187de4 Fix ArcEager for missing values 2017-08-01 22:10:05 +02:00
ines 78e262140f Add workaround for displaCy server on Python 2/3 (resolves #1227) 
Make sure status and headers are bytes on Python 2 and strings on
Python 3
 2017-08-01 01:11:35 +02:00
Jim Geovedi 2572a9ddf0 Merge remote-tracking branch 'upstream/develop' into indonesian 2017-07-30 21:24:16 +07:00
Jim Geovedi bb08d696f9 added hashtag rule and fixed currency rules 2017-07-30 21:23:28 +07:00
Jim Geovedi e9af79a803 added u-\d+ rules (sports team) 2017-07-30 21:23:01 +07:00
Matthew Honnibal 27abc56e98 Add method to get beam entities 2017-07-29 21:59:02 +02:00
Matthew Honnibal ec63f4fe7b Add option to control how missing entities are handled when getting NER tags 2017-07-29 21:58:37 +02:00
Jim Geovedi e5adc26c72 simplified rules 2017-07-29 18:21:32 +07:00
Jim Geovedi 783f7d8b86 added test set for Indonesian language 2017-07-29 18:21:07 +07:00
Jim Geovedi 4d04898dea updated regexp 2017-07-29 17:44:57 +07:00
Jim Geovedi 7d96d477ea updated like_num 2017-07-29 17:44:46 +07:00
Jim Geovedi 3cca4ed798 added lex attrs rules 2017-07-29 17:22:21 +07:00
Jim Geovedi 8b814c63f1 more exceptions 2017-07-27 19:46:30 +07:00
Jim Geovedi 6c725e8dcf updated lemma 2017-07-27 19:46:21 +07:00
Jim Geovedi c194f7ae26 Merge remote-tracking branch 'upstream/develop' into indonesian 2017-07-27 10:55:34 +07:00
Jim Geovedi 547973b92a wip syntax iterators 2017-07-27 10:51:34 +07:00
Jim Geovedi bbc75da38d enable syntax iterator and lemma lookup 2017-07-27 10:51:15 +07:00
Jim Geovedi 24a8c8bf28 added wip lemma dict 2017-07-26 21:39:54 +07:00
Jim Geovedi 63f14ba46b added hyphen-suffix rules 2017-07-26 19:28:57 +07:00
Jim Geovedi f288964441 removed -el from suffix rules 2017-07-26 19:28:38 +07:00
Jim Geovedi 6eee7a7411 updated tokenizer exceptions 2017-07-26 19:13:47 +07:00
Jim Geovedi edec51b1b1 update punctuation rules 2017-07-26 19:13:36 +07:00
Jim Geovedi 62443d495a enable token match 2017-07-26 19:13:14 +07:00
Jim Geovedi c97f5ae0bb updated tokenizer exceptions 2017-07-26 19:12:52 +07:00
Matthew Honnibal aff325b7e0 Increment version 2017-07-25 19:41:20 +02:00
Matthew Honnibal 6780132821 Fix tagger loading 2017-07-25 19:41:11 +02:00
Matthew Honnibal fd20a4af55 Increment version 2017-07-25 18:58:34 +02:00
Matthew Honnibal 523b0df2c9 Update text classification model 2017-07-25 18:57:59 +02:00
Matthew Honnibal 7c7fac9337 Add spacy.blank() loading function 2017-07-25 18:56:37 +02:00
Jim Geovedi 73f6ac9d9b added hyhen 2017-07-24 15:56:31 +07:00
Jim Geovedi 68454c40bf added missing import 2017-07-24 14:12:34 +07:00
Jim Geovedi eaf9cbd708 cursed of copy & paste 2017-07-24 14:11:51 +07:00
Jim Geovedi 7aad6718bc enable tokenizer exceptions 2017-07-24 14:11:10 +07:00
Jim Geovedi ad56c9179a added tokenizer exceptions list 2017-07-24 14:10:16 +07:00
Jim Geovedi c1f3fe99fe updated punctuation rules 2017-07-24 13:57:21 +07:00
Jim Geovedi 37fa2c8c80 punctution rules 2017-07-24 06:17:18 +07:00
Jim Geovedi 082e94ac1c added inflix rules 2017-07-24 06:17:07 +07:00
Jim Geovedi d0ec484725 reverted 2017-07-24 06:16:29 +07:00
Jim Geovedi 0e590c711f added prefix & suffix rules 2017-07-23 23:46:40 +07:00
Jim Geovedi ba922e30e8 added ampere hour unit 2017-07-23 23:46:18 +07:00
Jim Geovedi 3b17eba27b added frequency units 2017-07-23 23:10:52 +07:00
Jim Geovedi d5fd32a572 added known currencies 2017-07-23 22:56:48 +07:00
Jim Geovedi f6f15678fb added lex_attrs 2017-07-23 22:55:22 +07:00
Jim Geovedi bed8162d00 added tokenizer_exceptions 2017-07-23 22:55:05 +07:00
Jim Geovedi b80c35bc9a added norm_exceptions 2017-07-23 22:54:49 +07:00
Jim Geovedi b5de329ea3 added norm_exceptions 2017-07-23 22:54:19 +07:00
Jim Geovedi 082e9ade46 fixed typo 2017-07-23 21:30:34 +07:00
Jim Geovedi e2efeb186e added stopwords 2017-07-23 20:52:37 +07:00
Jim Geovedi da98676839 use template 2017-07-23 20:51:31 +07:00
Jim Geovedi c2b4dd7809 start working on Indonesian language 2017-07-23 20:50:56 +07:00
Matthew Honnibal 5771bd1ff8 Increment version 2017-07-23 14:18:38 +02:00
Matthew Honnibal c4a81a47a4 Fix deserialization 2017-07-23 14:11:07 +02:00
Matthew Honnibal 2df563ad24 Remove optimization for textcat that caused loading problem 2017-07-23 14:10:51 +02:00
Matthew Honnibal 4fe77bced2 Add cfg attr to pipeline components 2017-07-23 00:52:47 +02:00
Matthew Honnibal d8aa721664 Compute Language.meta with a property 2017-07-23 00:50:18 +02:00
Matthew Honnibal a88a7deffe Five save/load of textcat config 2017-07-23 00:33:43 +02:00
Matthew Honnibal 9bae0ddc50 Fix minibatching 2017-07-22 20:14:49 +02:00
Matthew Honnibal ded0df5e2f Expose hyper-param as keyword arg 2017-07-22 20:14:37 +02:00
Matthew Honnibal f5de8deeec Increment version 2017-07-22 20:04:53 +02:00
Matthew Honnibal b55714d5d1 Make gold_tuples arg optional in begin_training 2017-07-22 20:04:43 +02:00
Matthew Honnibal ed6c85fa3c Fix loading of text categories in GoldParse 2017-07-22 20:04:03 +02:00
Matthew Honnibal 6ffec9dfea Update _ml, for textcat model 2017-07-22 20:03:40 +02:00
Matthew Honnibal d6a5c2c85a Add test for NER 2017-07-22 01:48:58 +02:00
Matthew Honnibal 28244df4da Add test for beam parsing 2017-07-22 01:48:35 +02:00
Matthew Honnibal c86445bdfd Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-07-22 01:14:28 +02:00
Matthew Honnibal b3a749610e Fix name of TextCategorizer 2017-07-22 01:14:07 +02:00
Matthew Honnibal 2424493970 Remove unnecessary import of Mock 2017-07-22 01:13:54 +02:00
Matthew Honnibal baa3d81c35 Add text categorizer to Language 2017-07-22 01:13:36 +02:00
Matthew Honnibal a6a2159969 Add slot for text categories to Doc 2017-07-22 00:34:15 +02:00
Matthew Honnibal 374ab3ecfb Increment alpha version 2017-07-22 00:32:49 +02:00
Matthew Honnibal 289f23df51 Test beam parsing 2017-07-20 15:03:10 +02:00
Matthew Honnibal 3da1063b36 Add beam decoding to parser, to allow NER uncertainties 2017-07-20 15:02:55 +02:00
Matthew Honnibal 0ca5832427 Improve negative example handling in NER oracle 2017-07-20 00:18:49 +02:00
Matthew Honnibal a231b56d40 Add text-classification hook to pipeline 2017-07-20 00:18:15 +02:00
Matthew Honnibal 7ea50182a5 Add support for text-classification labels to GoldParse 2017-07-20 00:17:47 +02:00
Matthew Honnibal 727481377e Add text-classifer thinc models 2017-07-20 00:17:17 +02:00
Matthew Honnibal f014138c11 Fix parser tests 2017-07-20 00:16:52 +02:00
Ines Montani c91642efd5 Port over changes from #1168 2017-07-01 11:43:54 +02:00
Jim Regan d81ceb0cd5 Merge branch 'develop' into polish 2017-06-26 22:42:27 +01:00
Jim O'Regan 2f84c73585 a start 2017-06-26 22:40:04 +01:00
Jim O'Regan 28d7f0a672 reference 2017-06-26 22:38:28 +01:00
Matthew Honnibal 91e52543ef Merge pull request #1118 from Gregory-Howard/patch-2 
Update _tokenizer_exceptions_list (adding cities)
 2017-06-20 11:16:07 +02:00
Matthew Honnibal 8ea785e01a Merge pull request #1119 from oroszgy/patch-3 
Fixed conllu converter
 2017-06-20 11:14:41 +02:00