Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
spaCy/spacy/lang/tl/__init__.py

19 lines
407 B
Python
Raw Normal View History

Added alpha support for Tagalog language (#3062) I have added alpha support for the Tagalog language from the Philippines. It is the basis for the country's national language Filipino. I have heavily based the format to the EN and ES languages. I have provided several words in the lemmatizer lookup table, added stop words from a source, translated numeric words to its Tagalog counterpart, added some tokenizer exceptions, and kept the tag map the same as the English language. While the alpha language passed the preliminary testing that you provided, I think it needs more data to be useful for most cases. * Added alpha support for Tagalog language * Edited contributor template * Included SCA; Reverted templates * Fixed SCA template * Fixed changes in SCA template
2018-12-18 12:08:38 +00:00
from .tokenizer_exceptions import TOKENIZER_EXCEPTIONS
from .stop_words import STOP_WORDS
from .lex_attrs import LEX_ATTRS
from ...language import Language
class TagalogDefaults(Language.Defaults):
Tidy up and move noun_chunks, token_match, url_match
2020-07-22 20:18:46 +00:00
tokenizer_exceptions = TOKENIZER_EXCEPTIONS
Simplify language data and revert detailed configs
2020-07-24 12:50:26 +00:00
lex_attr_getters = LEX_ATTRS
stop_words = STOP_WORDS
Added alpha support for Tagalog language (#3062) I have added alpha support for the Tagalog language from the Philippines. It is the basis for the country's national language Filipino. I have heavily based the format to the EN and ES languages. I have provided several words in the lemmatizer lookup table, added stop words from a source, translated numeric words to its Tagalog counterpart, added some tokenizer exceptions, and kept the tag map the same as the English language. While the alpha language passed the preliminary testing that you provided, I think it needs more data to be useful for most cases. * Added alpha support for Tagalog language * Edited contributor template * Included SCA; Reverted templates * Fixed SCA template * Fixed changes in SCA template
2018-12-18 12:08:38 +00:00
class Tagalog(Language):
Tidy up and fix small bugs and typos
2019-02-08 13:14:49 +00:00
lang = "tl"
Defaults = TagalogDefaults
Added alpha support for Tagalog language (#3062) I have added alpha support for the Tagalog language from the Philippines. It is the basis for the country's national language Filipino. I have heavily based the format to the EN and ES languages. I have provided several words in the lemmatizer lookup table, added stop words from a source, translated numeric words to its Tagalog counterpart, added some tokenizer exceptions, and kept the tag map the same as the English language. While the alpha language passed the preliminary testing that you provided, I think it needs more data to be useful for most cases. * Added alpha support for Tagalog language * Edited contributor template * Included SCA; Reverted templates * Fixed SCA template * Fixed changes in SCA template
2018-12-18 12:08:38 +00:00
Tidy up and fix small bugs and typos
2019-02-08 13:14:49 +00:00
__all__ = ["Tagalog"]