Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
spaCy/spacy/__init__.py

49 lines
1.3 KiB
Python
Raw Normal View History

Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.)
2017-03-16 16:13:08 +00:00
# coding: utf8
from __future__ import unicode_literals, print_function
import json
from pathlib import Path
from .util import set_lang_class, get_lang_class, parse_package_meta
refactored data_dir->via, add zip_safe, add spacy.load()
2016-01-15 17:01:02 +00:00
avoid polluting spacy namespace with lang classes
2016-04-12 14:31:16 +00:00
from . import en
from . import de
* Register Chinese language in spacy/__init__.py
2016-04-24 16:45:16 +00:00
from . import zh
Link languages in __init__.py
2016-11-02 19:05:14 +00:00
from . import es
from . import it
Hungarian module is exposed in spacy.
2016-12-21 19:45:36 +00:00
from . import hu
Add import fr
2016-11-02 19:48:29 +00:00
from . import fr
Link languages in __init__.py
2016-11-02 19:05:14 +00:00
from . import pt
Added nl module for dutch
2016-11-24 15:39:49 +00:00
from . import nl
Initial commit for Swedish
2016-12-20 10:05:06 +00:00
from . import sv
Add preliminary support for Finnish
2017-01-31 22:27:29 +00:00
from . import fi
add basic Bengali support
2017-02-28 01:48:37 +00:00
from . import bn
first idea for supporting multiple langs in download script
2016-03-24 10:19:43 +00:00
Fix formatting and remove unused imports
2017-03-15 16:33:57 +00:00
avoid polluting spacy namespace with lang classes
2016-04-12 14:31:16 +00:00
set_lang_class(en.English.lang, en.English)
set_lang_class(de.German.lang, de.German)
Link languages in __init__.py
2016-11-02 19:05:14 +00:00
set_lang_class(es.Spanish.lang, es.Spanish)
set_lang_class(pt.Portuguese.lang, pt.Portuguese)
set_lang_class(fr.French.lang, fr.French)
set_lang_class(it.Italian.lang, it.Italian)
First steps towards the Hungarian tokenizer code.
2016-12-07 22:07:43 +00:00
set_lang_class(hu.Hungarian.lang, hu.Hungarian)
* Register Chinese language in spacy/__init__.py
2016-04-24 16:45:16 +00:00
set_lang_class(zh.Chinese.lang, zh.Chinese)
Added language Dutch to init file
2016-11-29 15:42:05 +00:00
set_lang_class(nl.Dutch.lang, nl.Dutch)
Initial commit for Swedish
2016-12-20 10:05:06 +00:00
set_lang_class(sv.Swedish.lang, sv.Swedish)
Add preliminary support for Finnish
2017-01-31 22:27:29 +00:00
set_lang_class(fi.Finnish.lang, fi.Finnish)
add basic Bengali support
2017-02-28 01:48:37 +00:00
set_lang_class(bn.Bengali.lang, bn.Bengali)
Add preliminary support for Finnish
2017-01-31 22:27:29 +00:00
add lang registration facility
2016-03-25 17:54:45 +00:00
Fix spacy.load
2016-10-18 17:23:31 +00:00
def load(name, **overrides):
Fix loading of GloVe vectors, to address Issue #541
2016-10-20 16:27:48 +00:00
data_path = overrides.get('path', util.get_data_path())
Update model loading to support links Remove match_best_version check, fetch model language from meta instead of directory name, and don't make too many assumptions – if model is downloaded via downloader, version should match anyway. (Otherwise, users should be free to add and load whichever models they want.)
2017-03-16 16:13:08 +00:00
meta = parse_package_meta(data_path, name)
lang = meta['lang'] if meta and 'lang' in meta else 'en'
cls = get_lang_class(lang)
overrides['meta'] = meta
overrides['path'] = Path(data_path / name)
Fix path-override bug in spacy.load
2016-12-18 21:15:29 +00:00
return cls(**overrides)
Add spacy.info(model_name) to show model meta Allows "previewing" model before loading and making sure it's linked correctly.
2017-03-16 16:13:40 +00:00
def info(name):
meta = parse_package_meta(util.get_data_path(), name)
print(json.dumps(meta, indent=2))