Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add Left and Right Pointing Angle Brackets as punctuation to ancient Greek (#12829) 

* Update universe.json

* Update universe.json

add some missing commas in the greCy's description.

* Update punctuation.py

Add mathematical left and right angle brackets as punctuation for ancient Greek for better tokenization.
This commit is contained in:
Jacobo Myerston 2023-07-20 02:16:01 -07:00 committed by GitHub
parent ea54d1775a
commit 4f8daa4f00
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
spacy/lang/grc/punctuation.py
View File

@ -15,6 +15,7 @@ _prefixes = (
[ [
"†", "†",
"⸏", "⸏",
"〈",
] ]
+ LIST_PUNCT + LIST_PUNCT
+ LIST_ELLIPSES + LIST_ELLIPSES
@ -31,6 +32,7 @@ _suffixes = (
+ [ + [
"†", "†",
"⸎", "⸎",
"〉",
r"(?<=[\u1F00-\u1FFF\u0370-\u03FF])[\-\.⸏]", r"(?<=[\u1F00-\u1FFF\u0370-\u03FF])[\-\.⸏]",
] ]
) )