Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,543 Commits 70 Branches 177 Tags 298 MiB
Commit Graph

13 Commits

Author SHA1 Message Date
Ines Montani 24f72c669c Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
Ines Montani d8f3190c0a Tidy up and auto-format 2020-05-21 14:14:01 +02:00
adrianeboyd a6e521cd79
Add is_sent_end token property (#5375) 
Reconstruction of the original PR #4697 by @MiniLau.

Removes unused `SENT_END` symbol and `IS_SENT_END` from `Matcher` schema
because the Matcher is only going to be able to support `IS_SENT_START`.
 2020-04-29 12:53:16 +02:00
Ines Montani de11ea753a Merge branch 'master' into develop 2020-02-18 14:47:23 +01:00
adrianeboyd a938566b62 Fix Sentencizer.pipe() for empty doc (#4940) 2020-01-28 11:36:49 +01:00
adrianeboyd 199d89943e Add as_example to Sentencizer pipe() (#4933) 2020-01-22 15:40:31 +01:00
Ines Montani db55577c45
Drop Python 2.7 and 3.5 (#4828) 
* Remove unicode declarations

* Remove Python 3.5 and 2.7 from CI

* Don't require pathlib

* Replace compat helpers

* Remove OrderedDict

* Use f-strings

* Set Cython compiler language level

* Fix typo

* Re-add OrderedDict for Table

* Update setup.cfg

* Revert CONTRIBUTING.md

* Revert lookups.md

* Revert top-level.md

* Small adjustments and docs [ci skip]
 2019-12-22 01:53:56 +01:00
adrianeboyd 48ea2e8d0f Restructure Sentencizer to follow Pipe API (#4721) 
* Restructure Sentencizer to follow Pipe API

Restructure Sentencizer to follow Pipe API so that it can be scored with
`nlp.evaluate()`.

* Add Sentencizer pipe() test
 2019-11-27 16:33:34 +01:00
adrianeboyd a3509f67d4 Extend unicode character block for Sinhala (#4378) 
* Extend unicode character block for Sinhala

* Add sentencizer tests for more languages
 2019-10-07 13:17:03 +02:00
Ines Montani 3d8fd4b461 Revert #4334 2019-09-29 17:32:12 +02:00
Ines Montani c9cd516d96 Move tests out of package (#4334) 
* Move tests out of package

* Fix typo
 2019-09-28 18:05:00 +02:00
adrianeboyd 6942a6a69b Extend default punct for sentencizer (#4290) 
Most of these characters are for languages / writing systems that aren't
supported by spacy, but I don't think it causes problems to include
them. In the UD evals, Hindi and Urdu improve a lot as expected (from
0-10% to 70-80%) and Persian improves a little (90% to 96%). Tamil
improves in combination with #4288.

The punctuation list is converted to a set internally because of its
increased length.

Sentence final punctuation generated with:

```
unichars -gas '[\p{Sentence_Break=STerm}\p{Sentence_Break=ATerm}]' '\p{Terminal_Punctuation}'
```

See: https://stackoverflow.com/a/9508766/461847

Fixes #4269.
 2019-09-14 15:25:48 +02:00
Ines Montani 06bf130890 💫 Add better and serializable sentencizer (#3471) 
* Add better serializable sentencizer component

* Replace default factory

* Add tests

* Tidy up

* Pass test

* Update docs
 2019-03-23 15:45:02 +01:00