spaCy/spacy/__init__.py

# coding: utf8
from __future__ import unicode_literals

from . import util
from .deprecated import resolve_model_name
from .cli.info import info

from . import en, de, zh, es, it, hu, fr, pt, nl, sv, fi, bn, he, nb, ja


_languages = (en.English, de.German, es.Spanish, pt.Portuguese, fr.French,
             it.Italian, hu.Hungarian, zh.Chinese, nl.Dutch, sv.Swedish,
             fi.Finnish, bn.Bengali, he.Hebrew, nb.Norwegian, ja.Japanese)


for _lang in _languages:
    util.set_lang_class(_lang.lang, _lang)


def load(name, **overrides):
    if overrides.get('path') in (None, False, True):
        data_path = util.get_data_path()
        model_name = resolve_model_name(name)
        model_path = data_path / model_name
        if not model_path.exists():
            lang_name = util.get_lang_class(name).lang
            model_path = None
            util.print_msg(
                "Only loading the '{}' tokenizer.".format(lang_name),
                title="Warning: no model found for '{}'".format(name))
    else:
        model_path = util.ensure_path(overrides['path'])
        data_path = model_path.parent
        model_name = ''
    meta = util.parse_package_meta(data_path, model_name, require=False)
    lang = meta['lang'] if meta and 'lang' in meta else name
    cls = util.get_lang_class(lang)
    overrides['meta'] = meta
    overrides['path'] = model_path
    return cls(**overrides)
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.) 2017-03-16 16:13:08 +00:00			`# coding: utf8`
Clean up imports, unused code, whitespace, docstrings 2017-04-15 10:05:47 +00:00			`from __future__ import unicode_literals`
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.) 2017-03-16 16:13:08 +00:00
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`from . import util`
Add function to resolve model names and link them 2017-03-17 17:47:05 +00:00			`from .deprecated import resolve_model_name`
more norwegian 2017-03-28 12:10:20 +00:00			`from .cli.info import info`
refactored data_dir->via, add zip_safe, add spacy.load() 2016-01-15 17:01:02 +00:00
Add basic japanese support 2017-05-03 04:56:21 +00:00			`from . import en, de, zh, es, it, hu, fr, pt, nl, sv, fi, bn, he, nb, ja`
Import metadata into __init__ 2017-03-18 18:30:03 +00:00
Fix formatting and remove unused imports 2017-03-15 16:33:57 +00:00
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`_languages = (en.English, de.German, es.Spanish, pt.Portuguese, fr.French,`
			`it.Italian, hu.Hungarian, zh.Chinese, nl.Dutch, sv.Swedish,`
Add basic japanese support 2017-05-03 04:56:21 +00:00			`fi.Finnish, bn.Bengali, he.Hebrew, nb.Norwegian, ja.Japanese)`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00

			`for _lang in _languages:`
			`util.set_lang_class(_lang.lang, _lang)`
Add preliminary support for Finnish 2017-01-31 22:27:29 +00:00
add lang registration facility 2016-03-25 17:54:45 +00:00
Fix spacy.load 2016-10-18 17:23:31 +00:00			`def load(name, **overrides):`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`if overrides.get('path') in (None, False, True):`
			`data_path = util.get_data_path()`
			`model_name = resolve_model_name(name)`
			`model_path = data_path / model_name`
			`if not model_path.exists():`
Get language name first if no model path exists Makes sure spaCy fails early if no tokenizer exists, and allows printing better error message. 2017-04-16 20:16:47 +00:00			`lang_name = util.get_lang_class(name).lang`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`model_path = None`
			`util.print_msg(`
Get language name first if no model path exists Makes sure spaCy fails early if no tokenizer exists, and allows printing better error message. 2017-04-16 20:16:47 +00:00			`"Only loading the '{}' tokenizer.".format(lang_name),`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`title="Warning: no model found for '{}'".format(name))`
			`else:`
			`model_path = util.ensure_path(overrides['path'])`
			`data_path = model_path.parent`
Set model name to empty string if path override exists Required for parse_package_meta, which composes path of data_path and model_name (needs to be fixed in the future) 2017-04-16 20:15:51 +00:00			`model_name = ''`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`meta = util.parse_package_meta(data_path, model_name, require=False)`
Fix loading when no package found 2017-03-16 23:30:02 +00:00			`lang = meta['lang'] if meta and 'lang' in meta else name`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`cls = util.get_lang_class(lang)`
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.) 2017-03-16 16:13:08 +00:00			`overrides['meta'] = meta`
Update spacy.load() and fix path checks 2017-04-16 18:35:00 +00:00			`overrides['path'] = model_path`
Add newline 2017-04-27 09:14:26 +00:00			`return cls(**overrides)`