From a260ea3bbf7aad3a427a4a1dc77134479aba467e Mon Sep 17 00:00:00 2001 From: Roman Yurchak Date: Sat, 31 Oct 2020 10:02:23 +0100 Subject: [PATCH] DOC Refactor instructions for loading packages (#781) --- docs/changelog.md | 6 +-- docs/index.rst | 2 +- docs/{pypi.md => loading_packages.md} | 56 ++++++++++++++++++++++++--- docs/requirements-doc.txt | 2 +- docs/using_pyodide_from_javascript.md | 38 ++---------------- docs/using_pyodide_from_webworker.md | 13 +------ 6 files changed, 60 insertions(+), 57 deletions(-) rename docs/{pypi.md => loading_packages.md} (60%) diff --git a/docs/changelog.md b/docs/changelog.md index 67647e5c4..384be0836 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -6,11 +6,11 @@ - Pyodide includes CPython 3.8.2 [#712](https://github.com/iodide-project/pyodide/pull/712) -- Pyodide files are distributed by [JsDelivr CDN](https://www.jsdelivr.com/), +- Pyodide files are distributed by [JsDelivr](https://www.jsdelivr.com/), `https://cdn.jsdelivr.net/pyodide/v0.16.0/full/pyodide.js` The previous CDN `pyodide-cdn2.iodide.io` still works and there are no plans for deprecating it. However please use - JsDelivr CDN as a more sustainable solution. + JsDelivr as a more sustainable solution. - FIX Only call `Py_INCREF()` once when proxied by PyProxy [#708](https://github.com/iodide-project/pyodide/pull/708) - Updated docker image to Debian buster @@ -68,7 +68,7 @@ **User improvements:** - Packages with pure Python wheels can now be loaded directly from PyPI. See - {ref}`pypi` for more information. + {ref}`micropip` for more information. - Thanks to PEP 562, you can now `import js` from Python and use it to access anything in the global Javascript namespace. diff --git a/docs/index.rst b/docs/index.rst index efb8b7f34..ca9ff319f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -36,8 +36,8 @@ to be used with Pyodide. using_pyodide_from_javascript.md using_pyodide_from_webworker.md serving_pyodide_packages.md + loading_packages.md type_conversions.md - pypi.md api_reference.md faq.md diff --git a/docs/pypi.md b/docs/loading_packages.md similarity index 60% rename from docs/pypi.md rename to docs/loading_packages.md index d65ff1305..d68becc2b 100644 --- a/docs/pypi.md +++ b/docs/loading_packages.md @@ -1,7 +1,52 @@ -(pypi)= -# Installing packages from PyPI +(loading_packages)= +# Loading Python packages -Pyodide has experimental support for installing pure Python wheels from PyPI. +Only the Python standard library and six are available after importing Pyodide. To use other libraries, you’ll need to load their package using either, + - `pyodide.loadPackage` for packages built with pyodide. + - `micropip.install` for pure Python packages with wheels available on PyPi or on other URLs. + +```{note} +Note that `micropip` can also be used to load packages built in pyodide (in which case it relies on `pyodide.loadPackage`). +``` + +Alternatively you can run Python code without manually pre-loading packages. You can do this with {ref}`pyodide.runPythonAsync `) function, which will automatically download all packages that the code snippet imports. It only supports packages included in Pyodide (not on PyPi) at present. + +## Loading packages with pyodide.loadPackage + +Packages can be loaded by name, for those included in the official pyodide +repository using, +```js +pyodide.loadPackage('numpy') +``` +It is also possible to load packages from custom URLs, +```js +pyodide.loadPackage('https://foo/bar/numpy.js') +``` +in which case the URL must end with `.js`. + +When you request a package from the official repository, all of that package's +dependencies are also loaded. Dependency resolution is not yet implemented +when loading packages from custom URLs. + +Multiple packages can also be loaded in a single call, +```js +pyodide.loadPackage(['cycler', 'pytz']) +``` + +`pyodide.loadPackage` returns a `Promise`. + +```javascript +pyodide.loadPackage('matplotlib').then(() => { + // matplotlib is now available +}); +``` + +(micropip)= +## Micropip + +### Installing packages from PyPI + +Pyodide supports installing pure Python wheels from PyPI. For use in Iodide: @@ -37,7 +82,8 @@ Micropip implements file integrity validation by checking the hash of the downloaded wheel against pre-recorded hash digests from the PyPi JSON API. (micropip-installing-from-arbitrary-urls)= -## Installing wheels from arbitrary URLs + +### Installing wheels from arbitrary URLs Pure python wheels can also be installed from any URL with micropip, ```py @@ -62,7 +108,7 @@ since we are not able to check the file integrity, unlike with installs from PyPi. -## Complete example +## Example Adapting the setup from the section on {ref}`using_from_javascript` a complete example would be, diff --git a/docs/requirements-doc.txt b/docs/requirements-doc.txt index 28c8172f4..fda4068e0 100644 --- a/docs/requirements-doc.txt +++ b/docs/requirements-doc.txt @@ -1,3 +1,3 @@ -sphinx +# sphinx sphinx_rtd_theme myst-parser diff --git a/docs/using_pyodide_from_javascript.md b/docs/using_pyodide_from_javascript.md index f6207938a..d64e27ff1 100644 --- a/docs/using_pyodide_from_javascript.md +++ b/docs/using_pyodide_from_javascript.md @@ -42,6 +42,9 @@ sys.version `); ``` +After importing pyodide, only packages from the standard library are available. +See {ref}`loading_packages` documentation to load additional packages. + ## Complete example Create and save a test `index.html` page with the following contents: @@ -71,41 +74,6 @@ Create and save a test `index.html` page with the following contents: ``` -## Loading packages - -Only the Python standard library and `six` are available after importing -Pyodide. To use other libraries, you'll need to load their package using -`pyodide.loadPackage`. This downloads the file data over the network (as a -`.data` and `.js` index file) and installs the files in the virtual filesystem. - -Packages can be loaded by name, for those included in the official pyodide -repository (e.g. `pyodide.loadPackage('numpy')`). It is also possible to load -packages from custom URLs (e.g. -`pyodide.loadPackage('https://foo/bar/numpy.js')`), in which case the URL must -end with `.js`. - -When you request a package from the official repository, all of that package's -dependencies are also loaded. Dependency resolution is not yet implemented -when loading packages from custom URLs. - -Multiple packages can also be loaded in a single call, -```js -pyodide.loadPackage(['cycler', 'pytz']) -``` - -`pyodide.loadPackage` returns a `Promise`. - -```javascript -pyodide.loadPackage('matplotlib').then(() => { - // matplotlib is now available -}); -``` - -## Alternative way to load packages and run Python code - -Alternatively you can run Python code without manually pre-loading packages. You can do this with {ref}`pyodide.runPythonAsync `) function, which will automatically download all packages that the code snippet imports. - -Note: although the function is called Async, it still blocks the main thread. To run Python code asynchronously see {ref}`using_from_webworker`. ## Alternative Example diff --git a/docs/using_pyodide_from_webworker.md b/docs/using_pyodide_from_webworker.md index 913304985..19a1c735d 100644 --- a/docs/using_pyodide_from_webworker.md +++ b/docs/using_pyodide_from_webworker.md @@ -53,18 +53,7 @@ pyodideWorker.postMessage(data) ``` -## Loading packages - -Packages referenced from your python script will be automatically downloaded -the first time they are encountered. Please note that some of the larger -packages such as `numpy` or `pandas` may take several seconds to load. -Subsequent uses of these packages will not incur the download overhead of the -first run, but there is still some time required for the `import` in python -itself. - -If you would like to pre-load some packages, or the automatic package loading -does not work for you for some reason, you may modify the `webworker.js` source -to load some specific packages as described in {ref}`using_from_javascript`. +See {ref}`loading_packages` for instructions on how to load packages. For example, to always load packages `numpy` and `pytz`, you would insert the line `self.pyodide.loadPackage(['numpy', 'pytz']).then(() => {` as shown below: