Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
13,260 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

2184 Commits

Author SHA1 Message Date
Ines Montani 23b7d9cfa3 Prefix span getters 2020-09-03 17:37:06 +02:00
Ines Montani 5afe6447cd registry.assets -> registry.misc 2020-09-03 17:31:14 +02:00
Ines Montani c063e55eb7 Add prefix to batchers 2020-09-03 17:30:41 +02:00
Ines Montani 804f120361 Don't use registered function version in title 2020-09-03 17:29:47 +02:00
Ines Montani c53b1433b9 Adjust more arguments [ci skip] 2020-09-03 17:12:24 +02:00
Ines Montani 121809dd1e Fix anchor [ci skip] 2020-09-03 16:49:56 +02:00
Ines Montani 25a595dc10 Fix typos and wording [ci skip] 2020-09-03 16:37:45 +02:00
Ines Montani b5a0657fd6 "model" terminology consistency in docs 2020-09-03 13:13:03 +02:00
Ines Montani b02ad8045b Update docs [ci skip] 2020-09-03 10:10:13 +02:00
Ines Montani 1815c613c9 Update docs [ci skip] 2020-09-03 10:07:45 +02:00
Adriane Boyd 960d9cfadc Officially support DependencyMatcher 
Add official support for the `DependencyMatcher`. Redesign the pattern
specification. Fix and extend operator implementations. Update API docs
and add usage docs.

Patterns
--------

Refactor pattern structure to:

```
{
  "LEFT_ID": str,
  "REL_OP": str,
  "RIGHT_ID": str,
  "RIGHT_ATTRS": dict,
}
```

The first node contains only `RIGHT_ID` and `RIGHT_ATTRS` and all
subsequent nodes contain all four keys.

New operators
-------------

Because of the way patterns are constructed from left to right, it's
helpful to have `follows` operators along with `precedes` operators. Add
operators for simple precedes / follows alongside immediate precedes /
follows.

* `.*`: precedes
* `;`: immediately follows
* `;*`: follows

Operator fixes
--------------

* `<` and `<<` do not include the node itself
* Fix reversed order for all operators involving linear precedence (`.`,
  all sibling operators)
* Linear precedence operators do not match nodes outside the same parse

Additional fixes
----------------

* Use v3 Matcher API
* Support `get` and `remove`
* Support pickling
 2020-09-02 17:45:29 +02:00
svlandeg 19298de352 small fix 2020-09-02 17:43:11 +02:00
svlandeg bbaea530f6 sublayers paragraph 2020-09-02 17:36:22 +02:00
svlandeg 1be7ff02a6 swapping section 2020-09-02 15:26:07 +02:00
svlandeg 57e432ba2a editor tip as Accordion instead of Infobox 2020-09-02 14:26:57 +02:00
svlandeg d19ec6c67b small rewrites in types paragraph 2020-09-02 14:25:18 +02:00
svlandeg 821b2d4e63 update examples 2020-09-02 14:15:50 +02:00
svlandeg e29a33449d rewrite intro, simpel Model example 2020-09-02 13:41:18 +02:00
svlandeg 422df9c2e2 Merge remote-tracking branch 'upstream/develop' into feature/docs-layers 
# Conflicts:
#	website/docs/usage/layers-architectures.md
 2020-09-02 13:17:11 +02:00
Ines Montani 70238543c8 Update layers/arch docs structure [ci skip] 2020-09-02 13:04:35 +02:00
svlandeg 6fd7f140ec custom-architectures section 2020-09-02 11:14:06 +02:00
svlandeg 3d9ae9286f small fixes 2020-09-02 10:46:38 +02:00
Ines Montani 690bd77669 Add todos [ci skip] 2020-09-01 14:04:36 +02:00
Ines Montani 70b226f69d Support ignore marker in project document [ci skip] 2020-09-01 12:49:04 +02:00
Ines Montani 9af82f3f11
Merge pull request #6003 from explosion/feature/matcher-as-spans 2020-08-31 17:50:56 +02:00
Sofie Van Landeghem 3ac620f09d
fix config example [ci skip] 2020-08-31 17:40:04 +02:00
Ines Montani 3929431af1 Update docs [ci skip] 2020-08-31 17:06:33 +02:00
Ines Montani add9de5487 Deprecate (Phrase)Matcher.pipe 2020-08-31 17:01:24 +02:00
svlandeg 2c3b64a567 console logging example 2020-08-31 16:56:13 +02:00
Ines Montani bca6bf8dda Update docs [ci skip] 2020-08-31 16:39:53 +02:00
Ines Montani db9f8896f5 Add docs [ci skip] 2020-08-31 16:10:41 +02:00
svlandeg fe6c08218e fixes 2020-08-31 14:51:49 +02:00
svlandeg 0e0abb0378 fix 2020-08-31 14:50:29 +02:00
svlandeg 56ba691ecd small fixes 2020-08-31 14:46:00 +02:00
svlandeg e47ea88aeb revert annotations refactor 2020-08-31 14:40:55 +02:00
svlandeg 13ee742fb4 example of custom logger 2020-08-31 14:24:41 +02:00
svlandeg 2c90a06fee some more information about the loggers 2020-08-31 13:43:17 +02:00
svlandeg c18eb63483 Merge remote-tracking branch 'upstream/develop' into feature/vectors-docs 
# Conflicts:
#	website/docs/usage/embeddings-transformers.md
 2020-08-31 13:21:36 +02:00
Juan Gutiérrez 9002bea29f
Update suffixes example (#5989) 
* Update suffixes example

The current example will throw `TypeError: can only concatenate list (not "tuple") to list`

* Signing Contributor Agreement
 2020-08-31 12:44:56 +02:00
Sofie Van Landeghem ec14744ee4
Rename Transformer listener (#6001) 
* rename to spacy-transformers.TransformerListener

* add some more tok2vec tests

* use select_pipes

* fix docs - annotation setter was not changed in the end
 2020-08-31 12:41:39 +02:00
Adriane Boyd 216efaf5f5 Restrict tokenizer exceptions to ORTH and NORM 2020-08-31 09:55:01 +02:00
Ines Montani 9b86312bab Update docs [ci skip] 2020-08-29 18:43:19 +02:00
Adriane Boyd 870774f475
Merge branch 'develop' into docs/morph-usage-v3 2020-08-29 16:00:50 +02:00
Ines Montani 45f46a5c85
Merge pull request #5993 from explosion/feature/disabled-components 2020-08-29 15:58:41 +02:00
Adriane Boyd f9ed31a757 Update usage docs for lemmatization and morphology 2020-08-29 15:56:50 +02:00
Ines Montani bc0730be3f Update docs [ci skip] 2020-08-29 12:53:14 +02:00
Ines Montani 450bf806b0
Merge pull request #5991 from adrianeboyd/docs/sent-usage-v3 
Update sentence segmentation usage docs
 2020-08-29 12:40:06 +02:00
Ines Montani 66d76f5126 Update docs 2020-08-29 12:36:05 +02:00
svlandeg 9f00a20ce4 proofreading and custom examples 2020-08-28 21:50:42 +02:00
svlandeg 5230529de2 add loggers registry & logger docs sections 2020-08-28 21:44:04 +02:00
Adriane Boyd 48df50533d Update sentence segmentation usage docs 
Update sentence segmentation usage docs to incorporate `senter`.
 2020-08-28 10:58:16 +02:00
svlandeg 72a87095d9 add loggers registry 2020-08-27 20:26:28 +02:00
svlandeg aa9e0c9c39 small fix 2020-08-27 19:56:52 +02:00
svlandeg 8cde6ccb7d Merge remote-tracking branch 'upstream/develop' into feature/vectors-docs 2020-08-27 19:56:09 +02:00
svlandeg 556e975a30 various fixes 2020-08-27 19:24:44 +02:00
Ines Montani ff4175e839 Add more info to debug config 2020-08-27 18:17:58 +02:00
svlandeg 329e490560 small import fixes 2020-08-27 14:50:43 +02:00
svlandeg 28e4ba7270 fix references to TransformerListener 2020-08-27 14:33:28 +02:00
svlandeg 4d37ac3f33 configure_custom_sent_spans example 2020-08-27 14:14:16 +02:00
svlandeg c68169f83f fix link 2020-08-27 10:19:43 +02:00
svlandeg acc794c975 example of writing to other custom attribute 2020-08-27 10:10:10 +02:00
svlandeg 559b65f2e0 adjust references to null_annotation_setter to trfdata_setter 2020-08-27 09:43:32 +02:00
Ines Montani 696f167478 Add diff example to docs [ci skip] 2020-08-26 15:57:54 +02:00
Adriane Boyd 90d88729e0
Add AttributeRuler.score (#5963) 
* Add AttributeRuler.score

Add scoring for TAG / POS / MORPH / LEMMA if these are present in the
assigned token attributes.

Add default score weights (that don't really make a lot of sense) so
that the scores are in the default config in some form.

* Update docs
 2020-08-26 15:39:30 +02:00
svlandeg ec069627fe rename to TransformerListener 2020-08-26 13:31:01 +02:00
Ines Montani 627617a079 Tidy up and add docs [ci skip] 2020-08-26 13:24:55 +02:00
svlandeg 15902c5aa2 fix link 2020-08-26 11:51:57 +02:00
svlandeg feb86d5206 clarify default 2020-08-26 11:21:30 +02:00
Ines Montani f31c4462ca Update docs [ci skip] 2020-08-25 13:27:59 +02:00
Ines Montani 8ac5ef1284 Update docs 2020-08-25 11:54:37 +02:00
Matthew Honnibal 8038b87f04
Various small tweaks to project CLI (#5965) 
* Fix up/download of http and local paths

* Support git_sparse_checkout for assets

* Fix scorer

* Handle already-present directories for git assets

* Improve convert command

* Fix support for existant files in git assets

* Support branches in git sparse checkout

* Format

* Fix git assets

* Document git block in assets

* Fix test

* Fix test

* Revert "Fix test"

This reverts commit cf3097260f.

* Revert "Fix test"

This reverts commit 964d636e27.

* Dont multiply p/r/f by 100

* Display scores * 100 during training
 2020-08-25 00:30:52 +02:00
Ines Montani 967d69ec50 Fix website deployment [ci skip] 2020-08-24 14:28:24 +02:00
Ines Montani 26405710e0 Add icon credit [ci skip] 2020-08-24 10:28:15 +02:00
Matthew Honnibal e559867605
Allow spacy project to push and pull to/from remote storage (#5949) 
* Add utils for working with remote storage

* WIP add remote_cache for project

* WIP add push and pull commands

* Use pathy in remote_cache

* Updarte util

* Update remote_cache

* Update util

* Update project assets

* Update pull script

* Update push script

* Fix type annotation in util

* Work on remote storage

* Remove site and env hash

* Fix imports

* Fix type annotation

* Require pathy

* Require pathy

* Fix import

* Add a util to handle project variable substitution

* Import push and pull commands

* Fix pull command

* Fix push command

* Fix tarfile in remote_storage

* Improve printing

* Fiddle with status messages

* Set version to v3.0.0a9

* Draft docs for spacy project remote storages

* Update docs [ci skip]

* Use Thinc config to simplify and unify template variables

* Auto-format

* Don't import Pathy globally for now

Causes slow and annoying Google Cloud warning

* Tidy up test

* Tidy up and update tests

* Update to latest Thinc

* Update docs

* variables -> vars

* Update docs [ci skip]

* Update docs [ci skip]

Co-authored-by: Ines Montani <ines@ines.io>
 2020-08-23 18:32:09 +02:00
Ines Montani f27aecac14 Update formatting [ci skip] 2020-08-23 11:57:56 +02:00
Ines Montani 98a9e063b6 Update docs [ci skip] 2020-08-22 17:15:05 +02:00
Matthew Honnibal 8dfc4cbfe7 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-22 17:12:09 +02:00
Matthew Honnibal 048de64d4c Suggest edits 2020-08-22 17:11:28 +02:00
Ines Montani adcf790b96 Update docs[ci skip] 2020-08-22 17:04:16 +02:00
Ines Montani 37ebff6997 Update docs [ci skip] 2020-08-22 16:47:03 +02:00
Matthew Honnibal 8685229891 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-22 16:06:59 +02:00
Matthew Honnibal d97695d09d Update embeddings-transformers.md 2020-08-22 15:41:35 +02:00
Ines Montani c7c9b0451f Update docs [ci skip] 2020-08-22 13:52:52 +02:00
Ines Montani 9740f1712b Re-add font with lowercase name [ci skip] 2020-08-22 12:27:02 +02:00
Ines Montani ff5ba14d06 Remove font [ci skip] 2020-08-22 12:26:50 +02:00
Ines Montani 71aeae89c5
Merge pull request #5948 from svlandeg/feature/docs-docs-docs [ci skip] 2020-08-22 12:18:47 +02:00
Ines Montani 27f81109d6 Update docs [ci skip] 2020-08-21 20:02:18 +02:00
Ines Montani f102164a1f Update docs [ci skip] 2020-08-21 19:34:06 +02:00
svlandeg 1b7cfa7347 Merge remote-tracking branch 'upstream/develop' into feature/docs-docs-docs 2020-08-21 18:36:18 +02:00
svlandeg 942adf0f4d comma 2020-08-21 18:36:02 +02:00
svlandeg 262552010d context manager with space (for consistency) 2020-08-21 18:34:02 +02:00
svlandeg da48c6a2a2 several small updates 2020-08-21 18:25:26 +02:00
svlandeg ad2332d4b7 alphabetize registries 2020-08-21 18:10:31 +02:00
svlandeg dc98f69b57 alphabetize registries 2020-08-21 18:10:21 +02:00
svlandeg c6659e37d8 small fixes 2020-08-21 18:02:20 +02:00
svlandeg 518a1f97f3 remove outdated TODO's 2020-08-21 17:55:15 +02:00
svlandeg e92bd6e1c1 alphabetize training lists 2020-08-21 17:42:19 +02:00
Ines Montani 2cc4640385 Update docs [ci skip] 2020-08-21 16:21:55 +02:00
Ines Montani 74cb6d39d0 Update docs [ci skip] 2020-08-21 16:11:38 +02:00
Matthew Honnibal f5bcc10268 Update architectures 2020-08-21 15:34:54 +02:00
Matthew Honnibal 7ed8f4504b Update API docs for architectures 2020-08-21 15:22:19 +02:00
svlandeg dcc21e44cb delete empty file 2020-08-21 15:17:20 +02:00
svlandeg 3060e4ae65 Merge remote-tracking branch 'upstream/develop' into feature/docs-docs-docs 
# Conflicts:
#	website/src/widgets/quickstart-training-generator.js
 2020-08-21 15:16:30 +02:00
svlandeg cc926267f8 small fixes 2020-08-21 15:05:40 +02:00
Ines Montani aa6a7cd6e7 Update docs and consistency [ci skip] 2020-08-21 13:49:18 +02:00
Ines Montani 52bd3a8b48 Update docs [ci skip] 2020-08-21 13:22:59 +02:00
Ines Montani e60442d83a Adjust label casing in displaCy NER visualizer (resolves #4866) 
- Accept any case for label names in ents and colors option, even if actual predicted label uses different casing
- Don't text-transform: uppercase visually, if it's important to users that the label is represented as-is in the UI
 2020-08-21 11:51:31 +02:00
Ines Montani 04e4d59235 Update docs [ci skip] 2020-08-20 16:17:25 +02:00
Ines Montani 7f2e4244df
Merge pull request #5941 from svlandeg/feature/update-more-docs 2020-08-20 11:21:24 +02:00
Ines Montani 6ad59d59fe Merge branch 'develop' of https://github.com/explosion/spaCy into develop [ci skip] 2020-08-20 11:20:58 +02:00
Ines Montani fb51b55eb9 Add comment [ci skip] 2020-08-20 11:20:43 +02:00
Sofie Van Landeghem 410b54e10e
Update website/docs/api/data-formats.md 
Co-authored-by: Ines Montani <ines@ines.io>
 2020-08-20 11:15:34 +02:00
svlandeg ae719b354f fix typos 2020-08-20 10:20:40 +02:00
svlandeg f728c00cbb Merge remote-tracking branch 'upstream/develop' into feature/update-more-docs 
# Conflicts:
#	website/docs/api/data-formats.md
 2020-08-20 10:02:13 +02:00
svlandeg 229033831a add explanation of raw_text 2020-08-20 10:00:45 +02:00
Ines Montani 2253d26b82 Update vectors and similarity docs [ci skip] 2020-08-19 21:18:26 +02:00
Ines Montani ea6640ea72
Merge pull request #5939 from explosion/feature/thinc-v8.0.0a28 
Update Thinc and config variables
 2020-08-19 21:14:36 +02:00
Ines Montani 15e6feed01 Update docs [ci skip] 2020-08-19 20:37:54 +02:00
svlandeg 09f3cfc985 add version 2020-08-19 19:58:45 +02:00
svlandeg 7d9f00bdbf waltzing schedule 2020-08-19 19:53:00 +02:00
Ines Montani 3dd390b1a1 Update Thinc and config variables 2020-08-19 19:46:12 +02:00
svlandeg 85b39639e1 small fix 2020-08-19 19:17:36 +02:00
svlandeg d8f6abdc23 add linking TODO back in 2020-08-19 18:00:35 +02:00
svlandeg 169b5bcda0 Merge remote-tracking branch 'upstream/develop' into feature/update-docs 
# Conflicts:
#	website/docs/usage/training.md
 2020-08-19 17:58:25 +02:00
svlandeg 7119295a8a badgers intro 2020-08-19 17:53:22 +02:00
svlandeg 4906a2ae6c custom functions intro 2020-08-19 17:32:35 +02:00
svlandeg 7a2e6a96f5 fix typo 2020-08-19 16:54:16 +02:00
svlandeg 648499157a rename "custom models" to "custom functions" 2020-08-19 16:53:51 +02:00
Ines Montani 63921161c8 Update docs [ci skip] 2020-08-19 16:04:21 +02:00
svlandeg d3a8321172 fix typos 2020-08-19 15:12:12 +02:00
svlandeg 60fedb8518 fix 2 more API lines 2020-08-19 14:55:32 +02:00
svlandeg 2dfd919585 add kb_loader and get_candidates back to EL API 2020-08-19 14:52:49 +02:00
Ines Montani e2f2ef3a5a Update init config and recommendations 
- As much as I dislike YAML, it seemed like a better format here because it allows us to add comments if we want to explain the different recommendations
- Don't include the generated JS in the repo by default and build it on the fly when running or deploying the site. This ensures it's always up to date.
- Simplify jinja_to_js script and use fewer dependencies
 2020-08-19 13:33:15 +02:00
Ines Montani 225f8866a1 Fix consistency 2020-08-19 12:47:57 +02:00
Ines Montani 9c25656ccc Update docs [ci skip] 2020-08-19 12:14:41 +02:00
Ines Montani 2285e59765
Merge pull request #5933 from svlandeg/feature/more-v3-docs [ci skip] 2020-08-19 11:29:02 +02:00
Ines Montani 13291e97ba Update docs [ci skip] 2020-08-19 00:28:37 +02:00
svlandeg 6ed67d495a format 2020-08-18 19:43:20 +02:00
svlandeg f9fe5eb323 clean up example 2020-08-18 19:35:23 +02:00
svlandeg a8acedd4ba example of custom reader and batcher 2020-08-18 19:15:16 +02:00
svlandeg 0d55b6ebb4 formatting 2020-08-18 18:55:56 +02:00
svlandeg abba639565 Merge remote-tracking branch 'upstream/develop' into feature/more-v3-docs 2020-08-18 18:55:12 +02:00
Sofie Van Landeghem 358cbb21e3
Define candidate generator in EL config (#5876) 
* candidate generator as separate part of EL config

* update comment

* ent instead of str as input for candidate generation

* Span instead of str: correct type indication

* fix types

* unit test to create new candidate generator

* fix replace_pipe argument passing

* move error message, general cleanup

* add vocab back to KB constructor

* provide KB as callable from Vocab arg

* rename to kb_loader, fix KB serialization as part of the EL pipe

* fix typo

* reformatting

* cleanup

* fix comment

* fix wrongly duplicated code from merge conflict

* rename dump to to_disk

* from_disk instead of load_bulk

* update test after recent removal of set_morphology in tagger

* remove old doc
 2020-08-18 16:10:36 +02:00
Ines Montani 82f0e20318 Update docs and consistency [ci skip] 2020-08-18 14:39:40 +02:00
Matthew Honnibal b72bd1767f Remove todo 2020-08-18 13:52:22 +02:00
Matthew Honnibal 574fd53289 Add precision/recall description 2020-08-18 13:51:08 +02:00
Matthew Honnibal 96a9c65f97 Add model architectures intro 2020-08-18 13:50:55 +02:00
svlandeg 705e1cb06c typo in link 2020-08-18 12:04:05 +02:00
svlandeg f7b76d2d83 Merge remote-tracking branch 'upstream/develop' into feature/more-v3-docs 2020-08-18 11:57:52 +02:00
svlandeg 8dcda351ec typo's and quick note on default values 2020-08-18 10:23:27 +02:00
Ines Montani ef6cf3b276 Update docs [ci skip] 2020-08-18 01:29:34 +02:00
Ines Montani 1c3bcfb488 Update docs and util consistency 2020-08-18 01:22:59 +02:00
Ines Montani 728fec0194 Update docs [ci skip] 2020-08-18 00:49:19 +02:00
Ines Montani 9299166c75
Merge pull request #5925 from explosion/docs/vectors [ci skip] 
Update the 'vectors' docs page
 2020-08-17 21:45:09 +02:00
Ines Montani 990c6b4c32 Update docs and CLI [ci skip] 2020-08-17 21:38:20 +02:00
svlandeg 4fe4bab1c9 typo fixes 2020-08-17 17:10:15 +02:00
svlandeg da80c18660 merge develop into branch 2020-08-17 16:57:18 +02:00
Ines Montani 3ae5e02f4f Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
Matthew Honnibal 052d82aa4e Suggest vectors changes 2020-08-17 15:32:30 +02:00
svlandeg 961e818be6 p/r definitions 2020-08-17 15:02:39 +02:00
svlandeg 6b6f7f3e73 fix windows compat 2020-08-17 14:48:58 +02:00
svlandeg 319692aa53 fix typos 2020-08-17 14:05:48 +02:00
Matthew Honnibal 61dfdd9fbd Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-16 20:30:01 +02:00
Matthew Honnibal be07567ac6 Update transformers page 2020-08-16 20:29:50 +02:00
Matthew Honnibal 8e5f99ee25 Update transformer docs intro. Also write system requirements 2020-08-16 20:13:24 +02:00
Ines Montani 2ac4b0ef3e Finish Transformer docs [ci skip] 2020-08-16 15:56:32 +02:00
Ines Montani 6ae83bde0c Fix CLI consistency [ci skip] 2020-08-16 15:46:29 +02:00
Ines Montani a570c304df Update quickstart, template and docs 2020-08-15 14:50:29 +02:00
Ines Montani 8736bfc052 Add comment about auto-generated file [ci skip] 2020-08-13 23:27:25 +02:00
Ines Montani 88b0a96801 Update for new Thinc and adjust config 2020-08-13 17:38:30 +02:00
Matthew Honnibal 965805f372 Add draft transformer template 2020-08-13 15:21:42 +02:00
Matthew Honnibal efcf15bddf Fix quickstart cpu template 2020-08-13 15:21:26 +02:00
Ines Montani 7d526d0d40 Update docs and quickstart widget [ci skip] 2020-08-13 01:17:40 +02:00
Ines Montani 950832f087
Tidy up pipes (#5906) 
* Tidy up pipes

* Fix init, defaults and raise custom errors

* Update docs

* Update docs [ci skip]

* Apply suggestions from code review

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>

* Tidy up error handling and validation, fix consistency

* Simplify get_examples check

* Remove unused import [ci skip]

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-08-11 23:29:31 +02:00
Ines Montani b7ec06e331 Update docs [ci skip] 2020-08-11 20:57:23 +02:00
Ines Montani 10f42e3a39 Update docs [ci skip] 2020-08-11 00:09:49 +02:00
Ines Montani 2778d04377 Update docs [ci skip] 2020-08-10 23:41:09 +02:00
Ines Montani adf2b1c8a9 Update graphic [ci skip] 2020-08-10 17:20:04 +02:00
Ines Montani 023ba7ae26 Update docs 2020-08-10 17:13:11 +02:00
Ines Montani c099f6eece Add Token.lex 2020-08-10 16:43:52 +02:00
Ines Montani 64f2f84098 Update docstrings and docs [ci skip] 2020-08-10 13:45:22 +02:00
Ines Montani 12052bd8f6 Update docs [ci skip] 2020-08-10 01:20:10 +02:00
Ines Montani 0832cdd443 Fix formatting [ci skip] 2020-08-10 00:46:32 +02:00
Ines Montani d611cbef43 Update docs [ci skip] 2020-08-10 00:42:26 +02:00
Ines Montani c044460823 Update docs [ci skip] 2020-08-10 00:01:38 +02:00
Ines Montani 05dcab10aa Fix typo 2020-08-09 22:34:03 +02:00
Ines Montani d5c78c7a34 Update docs and fix consistency 2020-08-09 22:31:52 +02:00
Ines Montani a15c5fb191 Update docstrings and docs 2020-08-09 16:10:48 +02:00
Ines Montani 8d2baa153d Update tokenizer docs and add test 2020-08-09 15:24:01 +02:00
Ines Montani 46bc513a4e Update docs [ci skip] 2020-08-07 20:14:31 +02:00
Ines Montani fe29ceec9e Merge branch 'develop' into docs/model-docstrings 2020-08-07 18:42:01 +02:00
Ines Montani 470b6f8073 Update docs 2020-08-07 18:41:15 +02:00
Ines Montani 3901b088ff Update graphics and 101 [ci skip] 2020-08-07 17:14:13 +02:00
Ines Montani 5e1421e5a6 Update docs [ci skip] 2020-08-07 16:23:12 +02:00
Ines Montani b7e34c1451 Update docs [ci skip] 2020-08-07 16:13:13 +02:00
Ines Montani 6f3649923c
Merge pull request #5893 from explosion/feature/validate-arg 2020-08-07 15:47:20 +02:00
Ines Montani e829d3bf14 Update docs [ci skip] 2020-08-07 15:46:20 +02:00
Adriane Boyd e962784531
Add Lemmatizer and simplify related components (#5848) 
* Add Lemmatizer and simplify related components

* Add `Lemmatizer` pipe with `lookup` and `rule` modes using the
`Lookups` tables.
* Reduce `Tagger` to a simple tagger that sets `Token.tag` (no pos or lemma)
* Reduce `Morphology` to only keep track of morph tags (no tag map, lemmatizer,
or morph rules)
* Remove lemmatizer from `Vocab`
* Adjust many many tests

Differences:

* No default lookup lemmas
* No special treatment of TAG in `from_array` and similar required
* Easier to modify labels in a `Tagger`
* No extra strings added from morphology / tag map

* Fix test

* Initial fix for Lemmatizer config/serialization

* Adjust init test to be more generic

* Adjust init test to force empty Lookups

* Add simple cache to rule-based lemmatizer

* Convert language-specific lemmatizers

Convert language-specific lemmatizers to component lemmatizers. Remove
previous lemmatizer class.

* Fix French and Polish lemmatizers

* Remove outdated UPOS conversions

* Update Russian lemmatizer init in tests

* Add minimal init/run tests for custom lemmatizers

* Add option to overwrite existing lemmas

* Update mode setting, lookup loading, and caching

* Make `mode` an immutable property
* Only enforce strict `load_lookups` for known supported modes
* Move caching into individual `_lemmatize` methods

* Implement strict when lang is not found in lookups

* Fix tables/lookups in make_lemmatizer

* Reallow provided lookups and allow for stricter checks

* Add lookups asset to all Lemmatizer pipe tests

* Rename lookups in lemmatizer init test

* Clean up merge

* Refactor lookup table loading

* Add helper from `load_lemmatizer_lookups` that loads required and
optional lookups tables based on settings provided by a config.

Additional slight refactor of lookups:

* Add `Lookups.set_table` to set a table from a provided `Table`
* Reorder class definitions to be able to specify type as `Table`

* Move registry assets into test methods

* Refactor lookups tables config

Use class methods within `Lemmatizer` to provide the config for
particular modes and to load the lookups from a config.

* Add pipe and score to lemmatizer

* Simplify Tagger.score

* Add missing import

* Clean up imports and auto-format

* Remove unused kwarg

* Tidy up and auto-format

* Update docstrings for Lemmatizer

Update docstrings for Lemmatizer.

Additionally modify `is_base_form` API to take `Token` instead of
individual features.

* Update docstrings

* Remove tag map values from Tagger.add_label

* Update API docs

* Fix relative link in Lemmatizer API docs
 2020-08-07 15:27:13 +02:00
Adriane Boyd 4aecccf153 Update API docs for AttributeRuler.__init__ 2020-08-07 15:17:25 +02:00
Ines Montani a8404c3517 validation -> validate 2020-08-07 14:43:47 +02:00