Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
spaCy/spacy/lang/uk/tokenizer_exceptions.py

37 lines
1.4 KiB
Python
Raw Normal View History

Configure isort to use the Black profile, recursively isort the `spacy` module (#12721) * Use isort with Black profile * isort all the things * Fix import cycles as a result of import sorting * Add DOCBIN_ALL_ATTRS type definition * Add isort to requirements * Remove isort from build dependencies check * Typo
2023-06-14 15:48:41 +00:00
from ...symbols import NORM, ORTH
Tidy up and move noun_chunks, token_match, url_match
2020-07-22 20:18:46 +00:00
from ...util import update_exc
Configure isort to use the Black profile, recursively isort the `spacy` module (#12721) * Use isort with Black profile * isort all the things * Fix import cycles as a result of import sorting * Add DOCBIN_ALL_ATTRS type definition * Add isort to requirements * Remove isort from build dependencies check * Typo
2023-06-14 15:48:41 +00:00
from ..tokenizer_exceptions import BASE_EXCEPTIONS
Ukrainian language added. Small fixes in Russian (#3241) * Classes for Ukrainian; small fix in Russian. * Contributor agreement
2019-02-07 20:05:11 +00:00
_exc = {}
for exc_data in [
Update Ukrainian tokenizer_exceptions
2022-02-01 11:24:00 +00:00
{ORTH: "обл.", NORM: "область"},
{ORTH: "р-н.", NORM: "район"},
{ORTH: "р-н", NORM: "район"},
{ORTH: "м.", NORM: "місто"},
Remove POS, TAG and LEMMA from tokenizer exceptions
2020-07-22 21:09:01 +00:00
{ORTH: "вул.", NORM: "вулиця"},
{ORTH: "просп.", NORM: "проспект"},
Update Ukrainian tokenizer_exceptions
2022-02-01 11:24:00 +00:00
{ORTH: "пр-кт", NORM: "проспект"},
Remove POS, TAG and LEMMA from tokenizer exceptions
2020-07-22 21:09:01 +00:00
{ORTH: "бул.", NORM: "бульвар"},
{ORTH: "пров.", NORM: "провулок"},
{ORTH: "пл.", NORM: "площа"},
Update Ukrainian tokenizer_exceptions
2022-02-01 11:24:00 +00:00
{ORTH: "майд.", NORM: "майдан"},
{ORTH: "мкр.", NORM: "мікрорайон"},
{ORTH: "ст.", NORM: "станція"},
{ORTH: "ж/м", NORM: "житловий масив"},
{ORTH: "наб.", NORM: "набережна"},
{ORTH: "в/ч", NORM: "військова частина"},
{ORTH: "в/м", NORM: "військове містечко"},
{ORTH: "оз.", NORM: "озеро"},
{ORTH: "ім.", NORM: "імені"},
Remove POS, TAG and LEMMA from tokenizer exceptions
2020-07-22 21:09:01 +00:00
{ORTH: "г.", NORM: "гора"},
{ORTH: "п.", NORM: "пан"},
{ORTH: "проф.", NORM: "професор"},
{ORTH: "акад.", NORM: "академік"},
{ORTH: "доц.", NORM: "доцент"},
Tidy up and fix small bugs and typos
2019-02-08 13:14:49 +00:00
]:
Ukrainian language added. Small fixes in Russian (#3241) * Classes for Ukrainian; small fix in Russian. * Contributor agreement
2019-02-07 20:05:11 +00:00
_exc[exc_data[ORTH]] = [exc_data]
Tidy up and move noun_chunks, token_match, url_match
2020-07-22 20:18:46 +00:00
TOKENIZER_EXCEPTIONS = update_exc(BASE_EXCEPTIONS, _exc)