Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
spaCy/spacy/__init__.py

36 lines
1.2 KiB
Python
Raw Normal View History

Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.)
2017-03-16 16:13:08 +00:00
# coding: utf8
Clean up imports, unused code, whitespace, docstrings
2017-04-15 10:05:47 +00:00
from __future__ import unicode_literals
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.)
2017-03-16 16:13:08 +00:00
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
from . import util
Add function to resolve model names and link them
2017-03-17 17:47:05 +00:00
from .deprecated import resolve_model_name
Lazy imports language
2017-05-03 09:01:42 +00:00
import importlib
more norwegian
2017-03-28 12:10:20 +00:00
from .cli.info import info
refactored data_dir->via, add zip_safe, add spacy.load()
2016-01-15 17:01:02 +00:00
Import metadata into __init__
2017-03-18 18:30:03 +00:00
Lazy imports language
2017-05-03 09:01:42 +00:00
_languages_name = set(["en", "de", "es", "pt", "fr",
"it", "hu", "zh", "nl", "sv",
"fi", "bn", "he", "nb"])
Add preliminary support for Finnish
2017-01-31 22:27:29 +00:00
add lang registration facility
2016-03-25 17:54:45 +00:00
Fix spacy.load
2016-10-18 17:23:31 +00:00
def load(name, **overrides):
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
if overrides.get('path') in (None, False, True):
data_path = util.get_data_path()
model_name = resolve_model_name(name)
model_path = data_path / model_name
if not model_path.exists():
Get language name first if no model path exists Makes sure spaCy fails early if no tokenizer exists, and allows printing better error message.
2017-04-16 20:16:47 +00:00
lang_name = util.get_lang_class(name).lang
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
model_path = None
util.print_msg(
Get language name first if no model path exists Makes sure spaCy fails early if no tokenizer exists, and allows printing better error message.
2017-04-16 20:16:47 +00:00
"Only loading the '{}' tokenizer.".format(lang_name),
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
title="Warning: no model found for '{}'".format(name))
else:
model_path = util.ensure_path(overrides['path'])
data_path = model_path.parent
Set model name to empty string if path override exists Required for parse_package_meta, which composes path of data_path and model_name (needs to be fixed in the future)
2017-04-16 20:15:51 +00:00
model_name = ''
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
meta = util.parse_package_meta(data_path, model_name, require=False)
Fix loading when no package found
2017-03-16 23:30:02 +00:00
lang = meta['lang'] if meta and 'lang' in meta else name
Lazy imports language
2017-05-03 09:01:42 +00:00
cls = importlib.import_module("."+lang, "spacy")
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.)
2017-03-16 16:13:08 +00:00
overrides['meta'] = meta
Update spacy.load() and fix path checks
2017-04-16 18:35:00 +00:00
overrides['path'] = model_path
Lazy imports language
2017-05-03 09:01:42 +00:00
return cls.EXPORT(**overrides)