Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,742 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

338 Commits

Author SHA1 Message Date
Matthew Honnibal 68f479e821 * Rename Doc.data to Doc.c 2015-11-04 00:15:14 +11:00
Matthew Honnibal 329ae57520 * Fix whitespace attachment thing 2015-10-13 09:46:38 +02:00
Matthew Honnibal 37919eac82 * Fix whitespace attachment in simpler way. Leaves problem with setting left/right children. 2015-10-13 18:23:24 +11:00
Matthew Honnibal c70eb776ae * Fix whitespace attachment, so that left/right children are consistent with head. 2015-10-13 15:58:22 +11:00
Matthew Honnibal 20fd36a0f7 * Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-13 13:44:41 +11:00
Matthew Honnibal 9dd2f25c74 * Fix Issue #131: Force whitespace characters to attach syntactically to previous token, and ensure they cannot serve as stand-alone 'sentence' units. 2015-10-10 15:53:30 +11:00
Matthew Honnibal 8b39feefbe * Add dependency post-process rule to ensure spaces are attached to neighbouring tokens, so that they can't be sentence boundaries 2015-10-10 15:32:13 +11:00
Matthew Honnibal 0e24d099a1 * Fix L/R edge bug, by ensuring l_edge and r_edge are preset, and fixing the way the edge update in del_arc. Bugs keep arising here because the edges are absolute positions, where everything else is relative. I'm also not 100% convinced that del_arc is handled correctly. Do we need to update the parents? 2015-09-09 03:40:44 +02:00
Matthew Honnibal 86c888667f * Merge in changes from de branch 2015-09-06 19:49:28 +02:00
Matthew Honnibal 5edac11225 * Wrap self.parse in nogil, and break if an invalid move is predicted. The invalid break is a work-around that papers over likely bugs, but we can't easily break in the nogil block, and otherwise we'll get an infinite loop. Need to set this as an error flag. 2015-09-06 04:15:00 +02:00
Matthew Honnibal a3d5e6c0dd * Reform constructor and save/load workflow in parser model 2015-08-26 19:19:01 +02:00
Matthew Honnibal bf38b3b883 * Hack on l/r reversal bug 2015-08-10 05:58:43 +02:00
Matthew Honnibal 6116413b47 * Fix label prediction in StepwiseState 2015-08-10 05:05:31 +02:00
Matthew Honnibal 2c9753eff2 * Whitespace 2015-08-10 00:09:02 +02:00
Matthew Honnibal 9de98f5a6f * Add Parser.stepthrough method, with context manager 2015-08-10 00:08:46 +02:00
Matthew Honnibal fe43f8cf39 * Whitespace 2015-08-09 02:31:53 +02:00
Matthew Honnibal 9c090945e0 * Add Parser.predict method, and clean up Parser.get_state 2015-08-09 02:29:58 +02:00
Matthew Honnibal 04fccfb984 * Fix get_state for parser prediction 2015-08-09 02:11:22 +02:00
Matthew Honnibal 55fde0e240 * Fix get_state 2015-08-09 01:45:30 +02:00
Matthew Honnibal f0f4fa9838 * Fix Parser.get_state 2015-08-09 01:40:13 +02:00
Matthew Honnibal 18331dca89 * Add continue_for argument to parser 'partial' function, which is now renamed to get_state 2015-08-09 01:31:54 +02:00
Matthew Honnibal 0653288fa5 * Fix stateclass.queue 2015-08-09 00:39:02 +02:00
Matthew Honnibal 9de218b7ba * Fix Parser.partial function 2015-08-08 23:45:18 +02:00
Matthew Honnibal cc9deae960 * Add is_valid method to transition_system 2015-08-08 23:36:18 +02:00
Matthew Honnibal 2a46c77324 * Whitespace 2015-08-08 23:35:59 +02:00
Matthew Honnibal 7bafc789e7 * Add stack and queue properties to stateclass, for python access 2015-08-08 23:32:42 +02:00
Matthew Honnibal 3af938365f * Add function partial to Parser 2015-08-08 23:32:15 +02:00
Matthew Honnibal 76a1f0481a * Whitespace 2015-08-08 23:31:54 +02:00
Matthew Honnibal 59c3bf60a6 * Ensure entity recognizer doesn't over-write preset types 2015-08-06 16:09:08 +02:00
Matthew Honnibal 9c1724ecae * Gazetteer stuff working, now need to wire up to API 2015-08-06 00:35:40 +02:00
Matthew Honnibal a8bbd7312c * Hackishly patch long dependencies problem 2015-07-28 00:14:29 +02:00
Matthew Honnibal bb583f7f09 * Hackishly patch long dependencies problem 2015-07-27 23:14:33 +02:00
Matthew Honnibal 823ef4a00b * Remove profile declarations 2015-07-25 18:13:06 +02:00
Matthew Honnibal aa28e2e01d * Release the GIL around parse function 2015-07-24 04:53:27 +02:00
Matthew Honnibal d5255aad77 * Update freqs for missing tags in ner, for serializer 2015-07-23 01:17:11 +02:00
Matthew Honnibal 12699a1152 * Set initial freqs, to avoid missing values in serializer 2015-07-23 01:16:27 +02:00
Matthew Honnibal 317cbbc015 * Serialization round trip now working with decent API, but with rough spots in the organisation and requiring vocabulary to be fixed ahead of time. 2015-07-19 15:18:17 +02:00
Matthew Honnibal b1d74ce60d * Remove unused joint.pyx and joint.pxd files 2015-07-17 23:31:44 +02:00
Matthew Honnibal fb0a641a2d * Don't release the gil around Parser.parse. Does this indicate thread problems? 2015-07-17 23:07:37 +02:00
Matthew Honnibal e29daea85f * Fix bint/int typing problem in TransitionSystem. In C++ bint* means bool*, but in C it means int*. So, type-casting to bint* is unsafe. 2015-07-17 22:37:24 +02:00
Matthew Honnibal 45ae1ce428 * Remove unused declaration in parser 2015-07-16 01:27:11 +02:00
Matthew Honnibal 9a8db9743c * Remove gil from parser.call 2015-07-14 23:47:33 +02:00
Matthew Honnibal 38ca0c33f5 Merge branch 'neuralnet' into refactor 
Mostly refactors parser, to use new thinc3.2 Example class.
Aim is to remove use of shared memory, so that we can parallelize
over documents easily.

Conflicts:
	setup.py
	spacy/syntax/parser.pxd
	spacy/syntax/parser.pyx
	spacy/syntax/stateclass.pyx
 2015-07-14 14:13:47 +02:00
Matthew Honnibal 6eef0bf9ab * Break up tokens.pyx into tokens/doc.pyx, tokens/token.pyx, tokens/spans.pyx 2015-07-13 20:20:58 +02:00
Matthew Honnibal 55f1042443 * Improve efficiency of L and R features, correcting the non-linear-in-length problem. 2015-07-09 12:17:26 +02:00
Matthew Honnibal 70d2acb579 * Fix edge features 2015-07-09 12:15:01 +02:00
Matthew Honnibal adb868bdad * Add warning for models not found in parser 2015-07-08 20:04:55 +02:00
Matthew Honnibal 05b28ec9eb * Add warning for models not found in parser 2015-07-08 20:02:13 +02:00
Matthew Honnibal ef700401a6 * Add warning for models not found in parser 2015-07-08 20:00:46 +02:00
Matthew Honnibal 6218d8b389 * Add warning for models not found in parser 2015-07-08 19:59:16 +02:00
Matthew Honnibal f6a6c39ce8 * Add warning for models not found in parser 2015-07-08 19:52:30 +02:00
Matthew Honnibal 0ceb1f71c2 * Update parse features 2015-07-08 19:11:36 +02:00
Matthew Honnibal bb522496dd * Rename Tokens to Doc 2015-07-08 18:53:00 +02:00
Matthew Honnibal ff885e8511 * Add ParserFactory convenience function 2015-07-08 12:35:46 +02:00
Matthew Honnibal 52fd80c6c6 * Add experimental supersense features for parsing, based on lookup into wordnet. 2015-07-01 20:12:44 +02:00
Matthew Honnibal e20106fdff * Begin reorganizing neuralnet work 2015-06-30 14:26:32 +02:00
Matthew Honnibal 3bb5876c5a * Inline methods in StateClass 2015-06-29 01:10:14 +02:00
Matthew Honnibal 313a7f87b3 * Inline methods in StateClass 2015-06-29 01:06:28 +02:00
Matthew Honnibal a02fd3af5d * Check valency in L and R feature methods, to make feaure calculation faster 2015-06-29 00:27:56 +02:00
Matthew Honnibal 5d870720bc * Check valency in L and R feature methods, to make feaure calculation faster 2015-06-29 00:17:29 +02:00
Matthew Honnibal f4986d5d3c * Use new Example class 2015-06-28 22:36:03 +02:00
Matthew Honnibal 735f1af91f * Fix neural net stuff 2015-06-28 11:44:58 +02:00
Matthew Honnibal e7003f1cf3 * Remove hard-coding of vector lengths 2015-06-28 11:37:17 +02:00
Matthew Honnibal 897dd0dd0b * Merge changes, and adjust Example to use memoryview 2015-06-28 11:36:11 +02:00
Matthew Honnibal 9282a8e72c * Prepare for new models to be plugged in by using Example class 2015-06-28 11:02:35 +02:00
Matthew Honnibal 75aeccc064 * Rejig parser interface to use new thinc.api.Example class, in prep of theano model. Comment out beam search 2015-06-28 11:02:34 +02:00
Matthew Honnibal bbef71f213 * Fix min function in fill_context 2015-06-28 10:46:39 +02:00
Matthew Honnibal 142b6f9510 * Revert last changes 2015-06-28 10:44:28 +02:00
Matthew Honnibal b06962f18b * Pad buffers in state 2015-06-28 10:36:14 +02:00
Matthew Honnibal 53be72387c * Hack at fill_context to investigate performance loss 2015-06-28 10:34:28 +02:00
Matthew Honnibal 71a4e876a9 * Fix parse features 2015-06-28 09:27:33 +02:00
Matthew Honnibal 5af500909c * Remove unused directve from parser.pyx 2015-06-28 06:20:21 +02:00
Matthew Honnibal d5b4090705 * Add profile directive 2015-06-28 06:19:33 +02:00
Matthew Honnibal 2b5421e60c * Add profile directive 2015-06-28 06:07:04 +02:00
Matthew Honnibal 8b5de4a411 * Add word / tag / label sets, for use in neural net 2015-06-28 05:46:53 +02:00
Matthew Honnibal ed40a8380e * Remove hard-coding of vector lengths 2015-06-27 04:18:47 +02:00
Matthew Honnibal ebe630cc8d * Enable more features for NN 2015-06-27 04:17:29 +02:00
Matthew Honnibal f8bb43475e * Bridge to Theano working. Very disorganised. Using thinc adb60aba966ed2 2015-06-27 02:39:18 +02:00
Matthew Honnibal 2fe98b8a9a * Prepare for new models to be plugged in by using Example class 2015-06-26 13:51:39 +02:00
Matthew Honnibal 6896455884 * Rejig parser interface to use new thinc.api.Example class, in prep of theano model. Comment out beam search 2015-06-26 06:25:36 +02:00
Matthew Honnibal 02b171ee67 * Bug fixes to edge calculation 2015-06-24 04:28:02 +02:00
Matthew Honnibal 7f9384f53c * Remove deprecated _state module 2015-06-23 17:28:24 +02:00
Matthew Honnibal 6dbe182491 * Fix merge conflicts 2015-06-23 17:28:00 +02:00
Matthew Honnibal 579735a095 * Remove import of _state module 2015-06-23 17:25:08 +02:00
Matthew Honnibal 88f55d136b * Remove deprecated _state module 2015-06-23 17:19:51 +02:00
Matthew Honnibal 9ab9dd2bf7 * Clean up unused orig_arc_eager and tree_arc_eager modules, which were only added for EMNLP experiments 2015-06-23 17:17:33 +02:00
Matthew Honnibal 7ebfe4b983 * Fixes to edge features 2015-06-23 16:32:54 +02:00
Matthew Honnibal 7b125f5a86 * Fixes to edge features 2015-06-23 16:31:01 +02:00
Matthew Honnibal 35c290bee4 * Fix edge features 2015-06-23 15:50:56 +02:00
Matthew Honnibal 221e2e485f * Assign 'ROOT' as label, not 'root' 2015-06-23 15:09:54 +02:00
Matthew Honnibal a7bf7b0626 * Rename sent_start to sent_end, to reflect its new usage in the Break transition 2015-06-23 05:39:43 +02:00
Matthew Honnibal ee3e56f27b * Fix bounds checking on entities 2015-06-23 04:35:08 +02:00
Matthew Honnibal 43ef5ddea5 * Ensure root albel is spelled ROOT, for backwards compatibility 2015-06-23 04:14:03 +02:00
Matthew Honnibal 065c2e1d2d * Add some bounds checking around state arrays 2015-06-23 04:13:09 +02:00
Matthew Honnibal f01b3d043e * Add padding to arrays in stateclass. May be papering over a deeper bug. 2015-06-23 03:03:41 +02:00
Matthew Honnibal 69507bc729 * Re-enable Break transition in arc_eager.pyx 2015-06-23 00:03:30 +02:00
Matthew Honnibal ab110be125 * Remove debugging in parser.pyx 2015-06-16 23:37:25 +02:00
Matthew Honnibal 9b13d11ab3 * Fix handling of entities in StateClass 2015-06-16 23:35:21 +02:00
Matthew Honnibal c40a2c661c * Add tree_arc_eager 2015-06-15 08:23:24 +02:00
Matthew Honnibal 5da5cf7084 * Add some more features for S1/S0 2015-06-15 04:07:13 +02:00