Commit Graph

457 Commits

Author SHA1 Message Date
Ines Montani df19e2bff6
💫 Allow setting of custom attributes during retokenization (closes #3314) (#3324)
<!--- Provide a general summary of your changes in the title. -->

## Description

This PR adds the abilility to override custom extension attributes during merging. This will only work for attributes that are writable, i.e. attributes registered with a default value like `default=False` or attribute that have both a getter *and* a setter implemented.

```python
Token.set_extension('is_musician', default=False)

doc = nlp("I like David Bowie.")
with doc.retokenize() as retokenizer:
    attrs = {"LEMMA": "David Bowie", "_": {"is_musician": True}}
    retokenizer.merge(doc[2:4], attrs=attrs)

assert doc[2].text == "David Bowie"
assert doc[2].lemma_ == "David Bowie"
assert doc[2]._.is_musician
```

### Types of change
enhancement

## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-24 18:38:47 +01:00
Ines Montani 403b9cd58b Add docs on adding to existing tokenizer rules [ci skip] 2019-02-24 18:35:19 +01:00
Ines Montani 1ea1bc98e7 Document regex utilities [ci skip] 2019-02-24 18:34:10 +01:00
Ines Montani 46ec5cdccc Update TextCategorizer docs 2019-02-24 13:11:57 +01:00
Ines Montani c03cb1cc63 Improve built-in component API docs 2019-02-24 13:11:49 +01:00
Ines Montani 383e2e1f12 Update Python versions [ci skip] 2019-02-24 11:49:45 +01:00
Ines Montani b624cb4b89 Update v2-1.md 2019-02-24 11:49:27 +01:00
Ines Montani 250e88ef55 Fix docs example (see #2728) 2019-02-21 14:22:06 +01:00
Ines Montani 0fc908d7a5 Add note on merging speed in v2.1 (see #3300) [ci skip] 2019-02-21 12:34:18 +01:00
Ines Montani 236aa94ded Update v2-1.md 2019-02-21 12:33:56 +01:00
Sofie 9a478b6db8 Clean up of char classes, few tokenizer fixes and faster default French tokenizer (#3293)
* splitting up latin unicode interval

* removing hyphen as infix for French

* adding failing test for issue 1235

* test for issue #3002 which now works

* partial fix for issue #2070

* keep the hyphen as infix for French (as it was)

* restore french expressions with hyphen as infix (as it was)

* added succeeding unit test for Issue #2656

* Fix issue #2822 with custom Italian exception

* Fix issue #2926 by allowing numbers right before infix /

* splitting up latin unicode interval

* removing hyphen as infix for French

* adding failing test for issue 1235

* test for issue #3002 which now works

* partial fix for issue #2070

* keep the hyphen as infix for French (as it was)

* restore french expressions with hyphen as infix (as it was)

* added succeeding unit test for Issue #2656

* Fix issue #2822 with custom Italian exception

* Fix issue #2926 by allowing numbers right before infix /

* remove duplicate

* remove xfail for Issue #2179 fixed by Matt

* adjust documentation and remove reference to regex lib
2019-02-20 22:10:13 +01:00
Ines Montani 57ae71ea95 Add docs on serializing the pipeline (see #3289) [ci skip] 2019-02-18 14:13:29 +01:00
Ines Montani 38e4422c0d Improve matcher example (resolves #3287) 2019-02-18 13:26:37 +01:00
Ines Montani 660cfe44c5 Fix formatting 2019-02-18 13:26:22 +01:00
Ines Montani 212ff359ef Fix links [ci skip] 2019-02-17 22:25:50 +01:00
Ines Montani 04b4df0ec9 Remove n_threads 2019-02-17 22:25:42 +01:00
Ines Montani e597110d31
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->

## Description

The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.

This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.


### Types of change
enhancement

## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 19:31:19 +01:00
ines 808f7ee417 Update API documentation 2017-10-03 14:27:22 +02:00
ines 3f4fd2c5d5 Update usage documentation 2017-10-03 14:26:20 +02:00
Reza Gharibi 0461b82158 Fix typos 2017-09-27 03:56:20 +03:30
Reza Gharibi fa1844b132 Fix typo 2017-09-27 03:55:54 +03:30
Reza Gharibi b5dd7e7cc4 Fix typo 2017-09-27 03:55:28 +03:30
Ines Montani b8e81daccf Fix typo (closes #1312) 2017-09-14 12:49:59 +02:00
ines d15775c3ad Fix typos and commands in alpha docs 2017-08-21 13:40:11 +02:00
ines 3c33003078 Port over typo corrections from #1245 2017-08-20 12:00:17 +02:00
ines 1261b01e46 Update Doc.char_span docs 2017-08-19 16:34:32 +02:00
ines 5cb0200e63 Document new Span.to_array() method 2017-08-19 12:45:28 +02:00
ines 471eed4126 Add example to Span.merge() 2017-08-19 12:45:16 +02:00
ines 404d3067b8 Document new Doc.char_span() method 2017-08-19 12:45:00 +02:00
ines d53cbf369f Document as_tuples kwarg on Language.pipe() 2017-08-19 12:44:50 +02:00
ines 6a37c93311 Update argument type 2017-08-19 12:44:33 +02:00
ines 4731d50220 Add break utility for long nowrap items (e.g. code) 2017-08-19 12:44:23 +02:00
ines 0aba11b64b Update package command docs 2017-08-14 16:45:44 +02:00
ines a29f132ffd Change python -m spacy to spacy
Reflects latest change to entry point or auto-alias
2017-08-14 13:04:48 +02:00
Nikolai Kruglikov 08e443e083 Fix small typo in documentation 2017-08-14 12:19:04 +02:00
ines ab8ffbaab7 Add text classification to v2 overview 2017-07-22 17:56:51 +02:00
ines f085b88f9d Add TextCategorizer API docs stub 2017-07-22 17:56:33 +02:00
ines ab1a4e8b3c Add Tensorizer API docs stub 2017-07-22 17:56:25 +02:00
ines 0fb89dd204 Add text classification usage guide template 2017-07-22 17:56:07 +02:00
ines d05ab1b3a0 Add text classification to 101 overview and change order 2017-07-22 17:55:53 +02:00
ines d2a7e5b8e5 Add GoldParse.cats attribute 2017-07-22 17:55:35 +02:00
ines 23d976ed00 Add Doc.cats attribute and missing v2 tag 2017-07-22 17:55:14 +02:00
Ines Montani 1ddbeddca2 Fix typo 2017-07-22 15:00:58 +02:00
Jarle Mathiesen f20533ec0c fix small typo 2017-06-24 12:31:33 +02:00
Savva Kolbachev 800a8faff4 Changed the capital of Lithuania to Vilnius
Hi,
There is a typo about the capital of Lithuania.

Vilnius is the capital of Lithuania https://en.wikipedia.org/wiki/Vilnius
Ljubljana is the capital of Slovenia https://en.wikipedia.org/wiki/Ljubljana
2017-06-12 23:27:00 +03:00
Ines Montani 57f64b9e1c Merge pull request #1124 from v3t3a/patch-3
docs - Fix url error for Displacy Ent visualizer
2017-06-12 21:20:32 +02:00
Ines Montani b2a28028cf Merge pull request #1115 from v3t3a/patch-2
docs - Add read() method when opening file (Lightning tour)
2017-06-12 21:19:25 +02:00
Ines Montani fe8d136ae0 Merge pull request #1114 from v3t3a/patch-1
docs - Update doc.jade (Just remove a duplicate 'doc =')
2017-06-12 21:19:02 +02:00
Vetea eae1f7b19c Fix url error for Displacy Ent visualizer 2017-06-12 14:30:02 +02:00
ines 49026a1346 Fix typos in example (see #1105) 2017-06-08 19:15:50 +02:00