spaCy/spacy/lang/hi/stop_words.py

# coding: utf8
from __future__ import unicode_literals


# Source: https://github.com/taranjeet/hindi-tokenizer/blob/master/stopwords.txt, https://data.mendeley.com/datasets/bsr3frvvjc/1#file-a21d5092-99d7-45d8-b044-3ae9edd391c6

STOP_WORDS = set(
    """
अंदर
अत
अदि
अप
अपना
अपनि
अपनी
अपने
अभि
अभी
अंदर
आदि
आप
अगर
इंहिं
इंहें
इंहों
इतयादि
इत्यादि
इन
इनका
इन्हीं
इन्हें
इन्हों
इस
इसका
इसकि
इसकी
इसके
इसमें
इसि
इसी
इसे
उंहिं
उंहें
उंहों
उन
उनका
उनकि
उनकी
उनके
उनको
उन्हीं
उन्हें
उन्हों
उस
उसके
उसि
उसी
उसे
एक
एवं
एस
एसे
ऐसे
ओर
और
कइ
कई
कर
करता
करते
करना
करने
करें
कहते
कहा
का
काफि
काफ़ी
कि
किंहें
किंहों
कितना
किन्हें
किन्हों
किया
किर
किस
किसि
किसी
किसे
की
कुछ
कुल
के
को
कोइ
कोई
कोन
कोनसा
कौन
कौनसा
गया
घर
जब
जहाँ
जहां
जा
जिंहें
जिंहों
जितना
जिधर
जिन
जिन्हें
जिन्हों
जिस
जिसे
जीधर
जेसा
जेसे
जैसा
जैसे
जो
तक
तब
तरह
तिंहें
तिंहों
तिन
तिन्हें
तिन्हों
तिस
तिसे
तो
था
थि
थी
थे
दबारा
दवारा
दिया
दुसरा
दुसरे
दूसरे
दो
द्वारा
न
नहिं
नहीं
ना
निचे
निहायत
नीचे
ने
पर
पहले
पुरा
पूरा
पे
फिर
बनि
बनी
बहि
बही
बहुत
बाद
बाला
बिलकुल
भि
भितर
भी
भीतर
मगर
मानो
मे
में
मैं
मुझको
मेरा
यदि
यह
यहाँ
यहां
यहि
यही
या
यिह
ये
रखें
रवासा
रहा
रहे
ऱ्वासा
लिए
लिये
लेकिन
व
वगेरह
वग़ैरह
वरग
वर्ग
वह
वहाँ
वहां
वहिं
वहीं
वाले
वुह
वे
वग़ैरह
संग
सकता
सकते
सबसे
सभि
सभी
साथ
साबुत
साभ
सारा
से
सो
संग
हि
ही
हुअ
हुआ
हुइ
हुई
हुए
हे
हें
है
हैं
हो
हूँ
होता
होति
होती
होते
होना
होने
""".split()
)
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`# coding: utf8`
			`from __future__ import unicode_literals`


Update stop_words.py and add name in contributors (#4325) * Update stop_words.py and add name in contributors * add jaydeepborkar.md in contributors directory * Reset template [ci skip] Co-authored-by: Ines Montani <ines@ines.io> 2019-09-27 09:57:27 +00:00			`# Source: https://github.com/taranjeet/hindi-tokenizer/blob/master/stopwords.txt, https://data.mendeley.com/datasets/bsr3frvvjc/1#file-a21d5092-99d7-45d8-b044-3ae9edd391c6`

💫 Tidy up and auto-format .py files (#2983) <!--- Provide a general summary of your changes in the title. --> ## Description - [x] Use [`black`](https://github.com/ambv/black) to auto-format all `.py` files. - [x] Update flake8 config to exclude very large files (lemmatization tables etc.) - [x] Update code to be compatible with flake8 rules - [x] Fix various small bugs, inconsistencies and messy stuff in the language data - [x] Update docs to explain new code style (`black`, `flake8`, when to use `# fmt: off` and `# fmt: on` and what `# noqa` means) Once #2932 is merged, which auto-formats and tidies up the CLI, we'll be able to run `flake8 spacy` actually get meaningful results. At the moment, the code style and linting isn't applied automatically, but I'm hoping that the new [GitHub Actions](https://github.com/features/actions) will let us auto-format pull requests and post comments with relevant linting information. ### Types of change enhancement, code style ## Checklist <!--- Before you submit the PR, go over this checklist and make sure you can tick off all the boxes. [] -> [x] --> - [x] I have submitted the spaCy Contributor Agreement. - [x] I ran the tests, and all new and existing tests passed. - [x] My changes don't require a change to the documentation, or if they do, I've added all required information. 2018-11-30 16:03:03 +00:00			`STOP_WORDS = set(`
			`"""`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`अंदर`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`अत`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`अदि`
			`अप`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`अपना`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`अपनि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`अपनी`
			`अपने`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`अभि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`अभी`
			`अंदर`
			`आदि`
			`आप`
Update stop_words.py and add name in contributors (#4325) * Update stop_words.py and add name in contributors * add jaydeepborkar.md in contributors directory * Reset template [ci skip] Co-authored-by: Ines Montani <ines@ines.io> 2019-09-27 09:57:27 +00:00			`अगर`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`इंहिं`
			`इंहें`
			`इंहों`
			`इतयादि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`इत्यादि`
			`इन`
			`इनका`
			`इन्हीं`
			`इन्हें`
			`इन्हों`
			`इस`
			`इसका`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`इसकि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`इसकी`
			`इसके`
			`इसमें`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`इसि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`इसी`
			`इसे`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`उंहिं`
			`उंहें`
			`उंहों`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`उन`
			`उनका`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`उनकि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`उनकी`
			`उनके`
			`उनको`
			`उन्हीं`
			`उन्हें`
			`उन्हों`
			`उस`
			`उसके`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`उसि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`उसी`
			`उसे`
			`एक`
			`एवं`
			`एस`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`एसे`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`ऐसे`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`ओर`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`और`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`कइ`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`कई`
			`कर`
			`करता`
			`करते`
			`करना`
			`करने`
			`करें`
			`कहते`
			`कहा`
			`का`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`काफि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`काफ़ी`
			`कि`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`किंहें`
			`किंहों`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`कितना`
			`किन्हें`
			`किन्हों`
			`किया`
			`किर`
			`किस`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`किसि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`किसी`
			`किसे`
			`की`
			`कुछ`
			`कुल`
			`के`
			`को`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`कोइ`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`कोई`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`कोन`
			`कोनसा`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`कौन`
			`कौनसा`
			`गया`
			`घर`
			`जब`
			`जहाँ`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`जहां`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`जा`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`जिंहें`
			`जिंहों`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`जितना`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`जिधर`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`जिन`
			`जिन्हें`
			`जिन्हों`
			`जिस`
			`जिसे`
			`जीधर`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`जेसा`
			`जेसे`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`जैसा`
			`जैसे`
			`जो`
			`तक`
			`तब`
			`तरह`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`तिंहें`
			`तिंहों`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`तिन`
			`तिन्हें`
			`तिन्हों`
			`तिस`
			`तिसे`
			`तो`
			`था`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`थि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`थी`
			`थे`
			`दबारा`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`दवारा`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`दिया`
			`दुसरा`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`दुसरे`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`दूसरे`
			`दो`
			`द्वारा`
			`न`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`नहिं`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`नहीं`
			`ना`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`निचे`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`निहायत`
			`नीचे`
			`ने`
			`पर`
			`पहले`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`पुरा`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`पूरा`
			`पे`
			`फिर`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`बनि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`बनी`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`बहि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`बही`
			`बहुत`
			`बाद`
			`बाला`
			`बिलकुल`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`भि`
			`भितर`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`भी`
			`भीतर`
			`मगर`
			`मानो`
			`मे`
			`में`
Update stop_words.py and add name in contributors (#4325) * Update stop_words.py and add name in contributors * add jaydeepborkar.md in contributors directory * Reset template [ci skip] Co-authored-by: Ines Montani <ines@ines.io> 2019-09-27 09:57:27 +00:00			`मैं`
			`मुझको`
			`मेरा`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`यदि`
			`यह`
			`यहाँ`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`यहां`
			`यहि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`यही`
			`या`
			`यिह`
			`ये`
			`रखें`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`रवासा`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`रहा`
			`रहे`
			`ऱ्वासा`
			`लिए`
			`लिये`
			`लेकिन`
			`व`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`वगेरह`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`वग़ैरह`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`वरग`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`वर्ग`
			`वह`
			`वहाँ`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`वहां`
			`वहिं`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`वहीं`
			`वाले`
			`वुह`
			`वे`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`वग़ैरह`
			`संग`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`सकता`
			`सकते`
			`सबसे`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`सभि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`सभी`
			`साथ`
			`साबुत`
			`साभ`
			`सारा`
			`से`
			`सो`
			`संग`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`हि`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`ही`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`हुअ`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`हुआ`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`हुइ`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`हुई`
			`हुए`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`हे`
			`हें`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`है`
			`हैं`
			`हो`
Update stop_words.py and add name in contributors (#4325) * Update stop_words.py and add name in contributors * add jaydeepborkar.md in contributors directory * Reset template [ci skip] Co-authored-by: Ines Montani <ines@ines.io> 2019-09-27 09:57:27 +00:00			`हूँ`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`होता`
improved upon the list of included stop_words 2017-11-13 11:43:49 +00:00			`होति`
Add Language class, stop words and basic stemmer that sets NORM 2017-10-14 12:59:52 +00:00			`होती`
			`होते`
			`होना`
			`होने`
💫 Tidy up and auto-format .py files (#2983) <!--- Provide a general summary of your changes in the title. --> ## Description - [x] Use [`black`](https://github.com/ambv/black) to auto-format all `.py` files. - [x] Update flake8 config to exclude very large files (lemmatization tables etc.) - [x] Update code to be compatible with flake8 rules - [x] Fix various small bugs, inconsistencies and messy stuff in the language data - [x] Update docs to explain new code style (`black`, `flake8`, when to use `# fmt: off` and `# fmt: on` and what `# noqa` means) Once #2932 is merged, which auto-formats and tidies up the CLI, we'll be able to run `flake8 spacy` actually get meaningful results. At the moment, the code style and linting isn't applied automatically, but I'm hoping that the new [GitHub Actions](https://github.com/features/actions) will let us auto-format pull requests and post comments with relevant linting information. ### Types of change enhancement, code style ## Checklist <!--- Before you submit the PR, go over this checklist and make sure you can tick off all the boxes. [] -> [x] --> - [x] I have submitted the spaCy Contributor Agreement. - [x] I ran the tests, and all new and existing tests passed. - [x] My changes don't require a change to the documentation, or if they do, I've added all required information. 2018-11-30 16:03:03 +00:00			`""".split()`
			`)`