Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,352 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

8561 Commits

Author SHA1 Message Date
Adriane Boyd 0ee2ae86bf Update trf quickstart recommendations 
Add/update trf recommendations for Bengali, Hindi, Sinhala, and Tamil
based on #7044.
 2021-02-12 15:55:17 +01:00
svlandeg 03b4ec7d7f fix typo 2021-02-12 14:30:16 +01:00
Adriane Boyd 5e47a54d29 Include noun chunks method when pickling Vocab 2021-02-12 13:27:46 +01:00
svlandeg aa3ad8825d loop instead of any 2021-02-12 13:14:30 +01:00
svlandeg 278e9eaa14 remove ner 2021-02-11 21:08:04 +01:00
svlandeg ebeedfc70b regression test for 7029 2021-02-11 20:56:48 +01:00
svlandeg a52d466bfc any instead of all 2021-02-11 20:50:55 +01:00
Shumi 4e514f1ea8
Update stop_words.py 
I have deleted line 1 to 5 and the statement print(STOP_WORDS)
 2021-02-11 21:30:34 +02:00
Shumi 0d57e84b7b
Update lex_attrs.py 
I have removed line 1 to 4
 2021-02-11 21:28:23 +02:00
Shumi 37ec67f868
Update examples.py 
I have removed two lines:
# coding: utf8
from __future__ import unicode_literals

And updated: >>> from spacy.lang.tn.examples import sentences
 2021-02-11 21:25:58 +02:00
Shumi 39eeba6760
Update __init__.py 
Added infixes = TOKENIZER_INFIXES
 2021-02-11 21:20:46 +02:00
Ines Montani 26bf642afd
Fix issue #7019: Handle None scores in evaluate printer (#7026) 2021-02-11 16:45:23 +11:00
Ines Montani 6b9026a219
Merge pull request #7000 from explosion/feature/project-yml-overrides 
Support env vars and CLI overrides for project.yml
 2021-02-11 12:31:45 +11:00
Ines Montani ad9ce3c8f6 Fix issue #6950: allow pickling Tok2Vec with listeners 2021-02-11 11:37:39 +11:00
Shumi ed3397727e
Delete tag_map.py 
Tag map file is deleted. I will add it later because it was failing validations
 2021-02-10 20:41:18 +02:00
Shumi 7c8721b1bd
Update tag_map.py 
Updated tag_map
 2021-02-10 20:21:22 +02:00
Shumi f6be28cfb2
Added files to Setswana Language 
Add South African Setswana Language
 2021-02-10 20:15:13 +02:00
Shumi 24046fef17
South African Setswana language 
Please accept the additional of Setswana language
 2021-02-10 20:12:33 +02:00
Peter Baumann 61b04a70d5
Run PhraseMatcher on Spans (#6918) 
* Add regression test

* Run PhraseMatcher on Spans

* Add test for PhraseMatcher on Spans and Docs

* Add SCA

* Add test with 3 matches in Doc, 1 match in Span

* Update docs

* Use doc.length for find_matches in tokenizer

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2021-02-10 23:43:32 +11:00
Ines Montani 21176c69b0 Update and add test 2021-02-10 14:12:00 +11:00
Ines Montani c08b3f294c Support env vars and CLI overrides for project.yml 2021-02-10 13:45:27 +11:00
Koichi Yasuoka 8ed788660b
Several callable objects do not have __qualname__ 2021-02-09 14:43:02 +09:00
Adriane Boyd 6108dabdc8 Rephrase error related to sample data initialization 
Now that the initialize step is fully implemented, the source of E923 is
typically missing or improperly converted/formatted data rather than a
bug in spaCy, so rephrase the error and message and remove the prompt to
open an issue.
 2021-02-08 09:21:36 +01:00
Sofie Van Landeghem 6ed423c16c
reduce memory load when reading all vectors from file (#6945) 
* reduce memory load when reading all vectors from file

* one more small typo fix
 2021-02-07 08:05:43 +08:00
Sofie Van Landeghem a323ef90df
ensure the loss value is cast as float (#6928) 2021-02-07 07:51:56 +08:00
melonwater211 a7977b5143
The test `spacy/tests/vocab_vectors/test_lexeme.py::test_vocab_lexeme_add_flag_auto_id` seems to fail occasionally when the test suite is run in a random order. (#6956) 
```python
    def test_vocab_lexeme_add_flag_auto_id(en_vocab):
        is_len4 = en_vocab.add_flag(lambda string: len(string) == 4)
        assert en_vocab["1999"].check_flag(is_len4) is True
        assert en_vocab["1999"].check_flag(IS_DIGIT) is True
        assert en_vocab["199"].check_flag(is_len4) is False
>       assert en_vocab["199"].check_flag(IS_DIGIT) is True
E       assert False is True
E        +  where False = <built-in method check_flag of spacy.lexeme.Lexeme object at 0x7fa155c36840>(3)
E        +    where <built-in method check_flag of spacy.lexeme.Lexeme object at 0x7fa155c36840> = <spacy.lexeme.Lexeme object at 0x7fa155c36840>.check_flag

spacy/tests/vocab_vectors/test_lexeme.py:49: AssertionError
```

>  `pytest==6.1.1`
>
>  `numpy==1.19.2`
>
> `Python version: 3.8.3`

To reproduce the error, run `pytest --random-order-bucket=global --random-order-seed=170158 -v spacy/tests`

If `test_vocab_lexeme_add_flag_auto_id` is run after `test_vocab_lexeme_add_flag_provided_id`, it fails.
It seems like `test_vocab_lexeme_add_flag_provided_id` uses the `IS_DIGIT` bit for testing purposes but does not reset the bit.

This solution seems to work but, if anyone has a better fix, please let me know and I will integrate it.
 2021-02-07 07:51:34 +08:00
René Octavio Queiroz Dias 59271e887a
fix: TransformerListener with TextCatEnsemble (#6951) 
* bug: Regression test
Issue #6946

* fix: Fix issue #6946

* chore: Remove regression test
 2021-02-06 13:44:51 +01:00
René Octavio Queiroz Dias 999ff03b19
fix: Fix textcat labels to expect a Optional[Iterable[str]] instead of Optional[Dict] (#6911) 
* docs: Add agreement

* bug: Regression test

Issue #6908

* fix: Changed from Dict to Iterable[str]

Fix #6908

* Update test to use make_tempdir

* fix: Fix WindowsPath error

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2021-02-04 23:37:13 +01:00
Adriane Boyd b903de3fcb
Pass on vocab arg in spacy.blank() (#6924) 2021-02-04 15:09:01 +01:00
svlandeg f852af2acf add capture arg 2021-02-02 19:47:12 +01:00
Matthew Honnibal b6a198481b Set version to v3.0.0 2021-02-02 20:26:17 +11:00
Sofie Van Landeghem f319d2765f
Add capture argument to project_run (#6878) 
* add capture argument to project_run and run_commands

* git bump to 3.0.1

* Set version to 3.0.1.dev0

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2021-02-02 10:11:15 +08:00
Sofie Van Landeghem f638306598
remove link_components flag again (#6883) 2021-02-02 10:08:40 +08:00
Ines Montani a59f3fcf5d Make wheel the default format and update docs [ci skip] 2021-02-01 23:18:43 +11:00
Ines Montani b9573e9e22 Fix pip args 2021-02-01 23:15:00 +11:00
Ines Montani b46073234a Fix default clone branch and error handling [ci skip] 2021-02-01 22:29:04 +11:00
Sofie Van Landeghem acabb284dd
Fix linking resumed components (#6859) 
* link components across enabled, resumed and frozen

* revert renaming

* revert renaming, the sequel
 2021-02-01 22:19:58 +11:00
Adriane Boyd 35a863cd27 Remove nlp.tokenizer from quickstart template 
Remove `nlp.tokenizer` from quickstart template so that the default
language-specific tokenizer settings are filled instead.
 2021-02-01 11:20:12 +01:00
svlandeg 91e72c031e reformatting 2021-01-30 17:29:33 +01:00
svlandeg a8d84188f0 add stop words 
Co-authored-by: tewodrosm <tedmaam2006@gmail.com>
 2021-01-30 17:26:49 +01:00
Ines Montani f058cbd751 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2021-01-30 21:03:25 +11:00
Ines Montani 14f631f52c Update parent package and version [ci skip] 2021-01-30 20:12:42 +11:00
Ines Montani 3435b894df Remove nightly reference from auto docs [ci skip] 2021-01-30 20:12:08 +11:00
Ines Montani d0c3775712 Replace links to nightly docs [ci skip] 2021-01-30 20:09:38 +11:00
Ines Montani b26a3daa9a
Merge pull request #6860 from explosion/feature/package-wheel 2021-01-30 14:17:01 +11:00
Ines Montani 2332c4280b Update and use unified --build option 2021-01-30 13:11:36 +11:00
Ines Montani e6accb3a9e Tidy up and auto-format 2021-01-30 12:52:33 +11:00
Ines Montani 817b0db521 Fix escape sequence 2021-01-30 12:39:58 +11:00
Ines Montani 526b416118 Tidy up comments 2021-01-30 12:34:09 +11:00
Ines Montani 30765674d0 Merge branch 'master' into develop 2021-01-30 12:20:28 +11:00
Ines Montani 2609ba4e89 Support building wheel in spacy package 2021-01-30 11:54:02 +11:00
Pamphile ROY 41ee75ac6d
Remove --no-cache-dir when downloading models 
When `--no-cache-dir` is present, it prevents caching to properly function.

If the user still wants to do this, there is the possibility to pass options with `user_pip_args`.
But you should not enforce options like these. In my case this is preventing some docker build (using buildkit caching) to have proper caching of models.
 2021-01-29 15:37:44 +01:00
Ines Montani bbf080dfe5
Merge pull request #6645 from bittlingmayer/patch-3 2021-01-30 01:26:28 +11:00
Adriane Boyd bced6309e5
Add full exceptions with spaces 2021-01-29 14:27:22 +01:00
Ines Montani 7886d59c56 Add check for remove_listener method 2021-01-29 23:47:30 +11:00
Ines Montani 7694f76dd1 Update warning and mention replace_listeners 2021-01-29 23:46:01 +11:00
Ines Montani 94232aea08 Improve E889 2021-01-29 23:39:23 +11:00
Ines Montani 924396c20c Merge branch 'feature/replace-listeners' of https://github.com/explosion/spaCy into feature/replace-listeners 2021-01-29 21:43:10 +11:00
Ines Montani 2102082478 Make Tok2Vec.remove_listener return bool 
Whether listener was removed
 2021-01-29 21:41:38 +11:00
Ines Montani e766e8c56d
Apply suggestions from code review 
Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
 2021-01-29 21:41:17 +11:00
Ines Montani bc089b693c Update tests 2021-01-29 19:38:09 +11:00
Ines Montani 325f47500d Move replacement logic to Language.from_config 2021-01-29 19:37:04 +11:00
Ines Montani 0f3e3eedc2 Add Tok2vec.remove_listener 2021-01-29 19:36:38 +11:00
Ines Montani 99842387cb Remove default value 2021-01-29 18:45:37 +11:00
Ines Montani 44b5542d14 Change method order 2021-01-29 18:42:41 +11:00
Ines Montani 8c15d1daec Update and validate config first and exit early if paths don't exist 2021-01-29 18:24:47 +11:00
Ines Montani bbb94b37c6 Update error handling and docstring 2021-01-29 16:27:49 +11:00
Ines Montani 01ecfbcc45 Merge branch 'develop' into feature/replace-listeners 2021-01-29 15:57:32 +11:00
Ines Montani 911dfcccfc Add option to replace listeners for sourced components 2021-01-29 15:57:04 +11:00
Adriane Boyd fcce3600ed
Forbid OP matching 2+ tokens in DependencyMatcher (#6824) 
Instead of silently using only the first token in each matched span:

* Forbid `OP: ?/*/+` through `DependencyMatcher` validation
* As a fail-safe, add warning if a token match that's not exactly one
token long is found by a token pattern.
 2021-01-29 08:52:01 +08:00
Sofie Van Landeghem 24a697abb8
avoid empty aliases and improve UX and docs (#6840) 2021-01-29 08:51:40 +08:00
Sofie Van Landeghem 837a4f53c2
Error handling in nlp.pipe (#6817) 
* add error handler for pipe methods

* add unit tests

* remove pipe method that are the same as their base class

* have Language keep track of a default error handler

* cleanup

* formatting

* small refactor

* add documentation
 2021-01-29 08:51:21 +08:00
Ines Montani cc18f3f23c
Improve Example error handling for NER data (#6835) 
* Improve Example error handling for NER data

* Fix conditional
 2021-01-28 13:11:20 +11:00
Ines Montani 78d6ff4dd4 Update quickstart recommendations 2021-01-28 11:14:49 +11:00
Ines Montani ec5f55aa5b
Update config generation defaults and transformers (#6832) 2021-01-27 23:56:33 +11:00
Adriane Boyd 4096a79de7
Add alignment mode error and fix Doc.char_span docs (#6820) 
* Raise an error on an unrecognized alignment mode rather than
defaulting to `strict`
* Fix the `Doc.char_span` API doc alignment mode details
 2021-01-27 23:40:42 +11:00
Sofie Van Landeghem 6b68ad027b
Fix beam NER resizing (#6834) 
* move label check to sub methods

* add tests
 2021-01-27 23:39:14 +11:00
Ines Montani 5ed51c9dd2
Merge pull request #6828 from explosion/master-tmp 2021-01-27 23:05:46 +11:00
Adriane Boyd d17afb4826
Add Spanish rule-based lemmatizer (#6833) 
* Initial Spanish lemmatizer

* Handle merged verb+pron(s) multi-word tokens

* Use VERB for AUX rule lookup

* Add morph to lemma cache key

* Fix aux lookups, minor refactoring

* Improve verb+pron handling

* Move verb+pron handling into its own method
* Check for exceptions (primarily for se)
* Collect pronouns in the same (not reversed) order

* Only add modified possible lemmas
 2021-01-27 19:21:35 +08:00
Ines Montani 615dba9d99 Fix tokenizer exceptions 2021-01-27 22:11:42 +11:00
Ines Montani abb24fdc0f
Merge pull request #6827 from explosion/feature/add-labels-implicitly 2021-01-27 21:34:58 +11:00
Ines Montani 80ba9eaf7d Fix test 2021-01-27 21:29:02 +11:00
Ines Montani e3f8be9a94 Update language data 2021-01-27 13:29:22 +11:00
Ines Montani 230e651ad6 Merge branch 'develop' into master-tmp 2021-01-27 13:26:29 +11:00
Matthew Honnibal 05050210f3 Dont add labels implicitly for parser 2021-01-27 13:04:47 +11:00
Matthew Honnibal 1d20e21f3e Add labels implicitly for parser and ner 2021-01-27 12:54:47 +11:00
Matthew Honnibal 68b1c2984d Test labels are added implicitly 2021-01-27 12:52:29 +11:00
Ines Montani fabd3a3394 Tidy up code comments [ci skip] 2021-01-27 12:40:03 +11:00
Dhruv Naik e7db07a0b9
Fix Span.char_span bug (#6816) 
* Create dhruvrnaik.md

* add test for issue #6815

* bugfix for issue #6815

* update dhruvrnaik.md

* add span.vector test for #6815
 2021-01-26 15:50:37 +08:00
Matthew Honnibal e8674c5c42 Set version to v3.0.0rc5 2021-01-26 14:55:41 +11:00
Adriane Boyd 71a6350744
Implement overwrite param for all custom lemmatizers (#6794) 2021-01-26 14:53:43 +11:00
Adriane Boyd 2263bc7b28
Update develop from master for v3.0.0rc5 (#6811) 
* Fix `spacy.util.minibatch` when the size iterator is finished (#6745)

* Skip 0-length matches (#6759)

Add hack to prevent matcher from returning 0-length matches.

* support IS_SENT_START in PhraseMatcher (#6771)

* support IS_SENT_START in PhraseMatcher

* add unit test and friendlier error

* use IDS.get instead

* ensure span.text works for an empty span (#6772)

* Remove unicode_literals

Co-authored-by: Santiago Castro <bryant@montevideo.com.uy>
Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
 2021-01-26 14:52:45 +11:00
Ines Montani c0926c9088
WIP: Various small training changes (#6818) 
* Allow output_path to be None during training

* Fix cat scoring (?)

* Improve error message for weighted None score

* Improve messages

So we can call this in other places etc.

* FIx output path check

* Use latest wasabi

* Revert "Improve error message for weighted None score"

This reverts commit 7059926763.

* Exclude None scores from final score by default

It's otherwise very difficult to keep track of the score weights if we modify a config programmatically, source components etc.

* Update warnings and use logger.warning
 2021-01-26 14:51:52 +11:00
Matthew Honnibal f049df1715
Revert "Set annotations in update" (#6810) 
* Revert "Set annotations in update (#6767)"

This reverts commit e680efc7cc.

* Fix version

* Update spacy/pipeline/entity_linker.py

* Update spacy/pipeline/entity_linker.py

* Update spacy/pipeline/tagger.pyx

* Update spacy/pipeline/tok2vec.py

* Update spacy/pipeline/tok2vec.py

* Update spacy/pipeline/transition_parser.pyx

* Update spacy/pipeline/transition_parser.pyx

* Update website/docs/api/multilabel_textcategorizer.md

* Update website/docs/api/tok2vec.md

* Update website/docs/usage/layers-architectures.md

* Update website/docs/usage/layers-architectures.md

* Update website/docs/api/transformer.md

* Update website/docs/api/textcategorizer.md

* Update website/docs/api/tagger.md

* Update spacy/pipeline/entity_linker.py

* Update website/docs/api/sentencerecognizer.md

* Update website/docs/api/pipe.md

* Update website/docs/api/morphologizer.md

* Update website/docs/api/entityrecognizer.md

* Update spacy/pipeline/entity_linker.py

* Update spacy/pipeline/multitask.pyx

* Update spacy/pipeline/tagger.pyx

* Update spacy/pipeline/tagger.pyx

* Update spacy/pipeline/textcat.py

* Update spacy/pipeline/textcat.py

* Update spacy/pipeline/textcat.py

* Update spacy/pipeline/tok2vec.py

* Update spacy/pipeline/trainable_pipe.pyx

* Update spacy/pipeline/trainable_pipe.pyx

* Update spacy/pipeline/transition_parser.pyx

* Update spacy/pipeline/transition_parser.pyx

* Update website/docs/api/entitylinker.md

* Update website/docs/api/dependencyparser.md

* Update spacy/pipeline/trainable_pipe.pyx
 2021-01-25 22:18:45 +08:00
Matthew Honnibal 42b117e561
Fix Doc.copy bugs (#6809) 
* Dont let the Doc own LexemeC, to fix Doc.copy

* Copy doc.spans

* Copy doc.spans
 2021-01-25 21:40:18 +08:00
Adriane Boyd 0f2de39efb
Fix types for exclude args in info CLI (#6808) 2021-01-25 20:00:22 +08:00
muratjumashev 2b19ebad59 Remove Kyrgyz chars fr. char_classes since Tatar ones already cover 2021-01-25 00:46:45 +06:00
muratjumashev 87168eb81f Add tests 2021-01-24 20:56:16 +06:00
muratjumashev 53abf759ad Fix punctuation 2021-01-24 20:54:22 +06:00
Matthew Honnibal ffc371350a
Avoid assuming encode.get_dim('nO') is set in tok2vec (#6800) 2021-01-24 14:37:33 +11:00