DOC Improve README and documentation (#665)

This commit is contained in:
Roman Yurchak 2020-05-19 18:14:56 +02:00 committed by GitHub
parent 00ef8956f8
commit f2c4d63c89
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 243 additions and 239 deletions

View File

@ -1,65 +1,3 @@
## Unreleased
# Release notes
- The built-in `sqlite` and `bz2` modules of Python are now enabled.
- New package: `nltk`
- `micropip` can now be used from web workers.
## Version 0.13.0
- Tagged versions of Pyodide are now deployed to Netlify.
## Version 0.12.0
**User improvements:**
- Packages with pure Python wheels can now be loaded directly from PyPI. See
`docs/pypi.md` for more information.
- Thanks to PEP 562, you can now `import js` from Python and use it to access
anything in the global Javascript namespace.
- Passing a Python object to Javascript always creates the same object in
Javascript. This makes APIs like `removeEventListener` usable.
- Calling `dir()` in Python on a JavaScript proxy now works.
- Passing an `ArrayBuffer` from Javascript to Python now correctly creates
a `memoryview` object.
- Pyodide now works on Safari.
## Version 0.11.0
**User improvements:**
- Support for built-in modules:
- `sqlite`, `crypt`
- New packages: `mne`
**Developer improvements:**
- The `mkpkg` command will now select an appropriate archive to use, rather than
just using the first.
- The included version of emscripten has been upgraded to 1.38.30 (plus a
bugfix).
- New packages: `jinja2`, `MarkupSafe`
## Version 0.10.0
**User improvements:**
- New packages: `html5lib`, `pygments`, `beautifulsoup4`, `soupsieve`,
`docutils`, `bleach`, `mne`
**Developer improvements:**
- `console.html` provides a simple text-only interactive console to test local
changes to Pyodide. The existing notebooks based on legacy versions of Iodide
have been removed.
- The `run_docker` script can now be configured with environment variables.
See https://pyodide.readthedocs.io/en/latest/changelog.html

178
README.md
View File

@ -1,9 +1,3 @@
**[What is Pyodide?](#what-is-pyodide)** |
**[Try Pyodide](#try-pyodide-no-installation-needed)** |
**[Getting Started](#getting-started)** |
**[Contributing](#contributing)** |
**[License](#license)**
# [Pyodide](https://github.com/iodide-project/pyodide)
@ -12,8 +6,6 @@
The Python scientific stack, compiled to WebAssembly.
[**Try Pyodide and Iodide in your browser**](https://alpha.iodide.io/notebooks/300/)
## What is Pyodide?
**Pyodide** brings the Python runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The [`packages` directory](https://github.com/iodide-project/pyodide/tree/master/packages) lists over 35 packages which are currently available.
@ -27,7 +19,9 @@ browser**.
## Try Pyodide (no installation needed)
Try the [iodide demo notebook](https://alpha.iodide.io/notebooks/300/) or fire up a [Python REPL](https://pyodide.cdn.iodide.io/console.html) directly in your browser.
Try the [iodide demo notebook](https://alpha.iodide.io/notebooks/300/) or fire
up a [Python REPL](https://pyodide.cdn.iodide.io/console.html) directly in your
browser.
For further information, look through the [documentation](https://pyodide.readthedocs.io/).
@ -36,161 +30,27 @@ For further information, look through the [documentation](https://pyodide.readth
Pyodide offers three different ways to get started depending on your needs and technical resources.
These include:
- [Download a pre-built version](#download-a-pre-built-version) (the quickest way to get started)
- [Build Pyodide from source](#building-from-source) (this method requires installing prerequistes and using `make`. Primarily for Linux users who want to experiment or contribute back to the project.)
- [Use a Docker image](#using-docker) (recommended for Windows and macOS users and for Linux users who prefer a Debian-based Docker image on Docker Hub with the dependencies
already installed)
### Download a pre-built version
Pre-built versions of Pyodide may be downloaded from
this repository's [releases page](https://github.com/iodide-project/pyodide/releases/).
### Building from source
Building is easiest on Linux and relatively straightforward on Mac. For
Windows, we currently recommend using the Docker image (described below) to
build Pyodide.
Make sure the prerequisites for [emsdk](https://github.com/emscripten-core/emsdk) are
installed. Pyodide will build a custom, patched version of emsdk, so there is no
need to build it yourself prior.
Additional build prerequisites are:
- A working native compiler toolchain, enough to build CPython.
- A native Python 3.7 to run the build scripts.
- PyYAML
- [lessc](http://lesscss.org/) to compile less to css.
- [uglifyjs](https://github.com/mishoo/UglifyJS) to minify Javascript builds.
- gfortran (GNU Fortran 95 compiler)
- [f2c](http://www.netlib.org/f2c/)
- [ccache](https://ccache.samba.org) (optional) *highly* recommended for much faster rebuilds.
On Mac, you will also need:
- [Homebrew](https://brew.sh/) for installing dependencies
- System libraries in the root directory (`sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /` should do it, see https://github.com/pyenv/pyenv/issues/1219#issuecomment-428305417)
- coreutils for md5sum and other essential Unix utilities (`brew install coreutils`)
- cmake (`brew install cmake`)
- pkg-config (`brew install pkg-config`)
- openssl (`brew install openssl`)
- gfortran (`brew cask install gfortran`)
- f2c: Install wget (`brew install wget`), and then run the buildf2c script from the root directory (`sudo ./tools/buildf2c`)
#### Build using `make`
After installing the build prerequisites, run from the command line:
```bash
make
```
### Using Docker
We provide a Debian-based Docker image on Docker Hub with the dependencies
already installed to make it easier to build Pyodide. Note that building from
the Docker image is *very* slow on Mac, building on the host machine is
preferred if at all possible.
1. Install Docker
2. From a git checkout of Pyodide, run `./run_docker`
3. Run `make` to build.
If running ``make`` deterministically stops at one point in each subsequent try, increasing
the maximum RAM usage available to the docker container might help [This is different
from the physical RAM capacity inside the system]. Ideally, at least 3 GB of RAM
should be available to the docker container to build `pyodide` smoothly. These settings can
be changed via Docker Preferences [See [here](https://stackoverflow.com/questions/44533319/how-to-assign-more-memory-to-docker-container)].
You can edit the files in your source checkout on your host machine, and then
repeatedly run `make` inside the Docker environment to test your changes.
- Use hosted distribution of pyodide: see [using pyodide from
Javascript](https://pyodide.readthedocs.io/en/latest/using_pyodide_from_javascript.html)
documentation.
- Download a pre-built version from this
repository's [releases
page](https://github.com/iodide-project/pyodide/releases/) and serve its contents with
a web server.
- [Build Pyodide from source](https://pyodide.readthedocs.io/en/latest/building_from_sources.html)
- Build natively with `make`: primarily for Linux users who want to
experiment or contribute back to the project.
- [Use a Docker image](https://pyodide.readthedocs.io/en/latest/building_from_sources.html#using-docker):
recommended for Windows and macOS users and for Linux users who prefer a
Debian-based Docker image with the dependencies already installed.
## Contributing
Please view the [CONTRIBUTING](CONTRIBUTING.md) document for tips on filing
issues, making changes, and submitting pull requests. The following sections
describe how to run tests, run Pyodide benchmarks, and lint the source code.
### Testing
Install the following dependencies into the default Python installation:
```bash
pip install pytest selenium pytest-instafail
```
Install [geckodriver](https://github.com/mozilla/geckodriver/releases) and
[chromedriver](https://sites.google.com/a/chromium.org/chromedriver/downloads)
and check that they are in your `PATH`.
### Automated testing
To run the pytest suite of tests, type on the command line:
```bash
pytest test/ packages/
```
### Manual interactive testing
To run manual interactive tests, a docker environment and a webserver will be
used.
1. Bind port 8000 for testing. To automatically bind port 8000 of the docker
environment and the host system, run: `./run_docker`
2. Now, this can be used to test the `pyodide` builds running within the
docker environment using external browser programs on the host system. To do
this, run: `./bin/pyodide serve`
3. This serves the ``build`` directory of the ``pyodide`` project on port 8000.
* To serve a different directory, use the ``--build_dir`` argument followed
by the path of the directory.
* To serve on a different port, use the ``--port`` argument followed by the
desired port number. Make sure that the port passed in ``--port`` argument
is same as the one defined as ``DOCKER_PORT`` in the ``run_docker`` script.
4. Once the webserver is running, simple interactive testing can be run by
visiting this URL:
[http://localhost:8000/console.html](http://localhost:8000/console.html)
### Benchmarking
To run common benchmarks to understand Pyodide's performance, begin by
installing the same prerequisites as for testing. Then run:
```bash
make benchmark
```
### Linting
Python is linted with `flake8`. C and Javascript are linted with
`clang-format`.
To lint the code, run:
```bash
make lint
```
Please view the
[contributing guide](https://pyodide.readthedocs.io/en/latest/rootdir.html#how-to-contribute)
for tips on filing issues, making changes, and submitting pull requests.
## License
Pyodide uses the Mozilla Public License Version 2.0. See the
[LICENSE file](LICENSE) for more details.
---
**[What is Pyodide?](#what-is-pyodide)** |
**[Try Pyodide](#try-pyodide-no-installation-needed)** |
**[Getting Started](#getting-started)** |
**[Contributing](#contributing)** |
**[License](#license)** |
**[Back to top](#pyodide)**

18
docs/about.md Normal file
View File

@ -0,0 +1,18 @@
# About the Project
The Python scientific stack, compiled to WebAssembly.
Pyodide brings the Python runtime to the browser via WebAssembly, along with
the Python scientific stack including NumPy, Pandas, Matplotlib, parts of
SciPy, and NetworkX. The [packages
directory](https://github.com/iodide-project/pyodide/tree/master/packages)
lists over 35 packages which are currently available.
Pyodide provides transparent conversion of objects between Javascript and
Python. When used inside a browser, Python has full access to the Web APIs.
While closely related to the [iodide project](https://iodide.io), a tool for
*literate scientific computing and communication for the web*, Pyodide goes
beyond running in a notebook environment. To maximize the flexibility of the
modern web, **Pyodide** may be used standalone in any context where you want to
**run Python inside a web browser**.

View File

@ -1,8 +1,64 @@
# Building from sources
To install Pyodide from sources follow the steps in the
[readme](./rootdir.html#building-from-source).
Building is easiest on Linux and relatively straightforward on Mac. For
Windows, we currently recommend using the Docker image (described below) to
build Pyodide.
Make sure the prerequisites for [emsdk](https://github.com/emscripten-core/emsdk) are
installed. Pyodide will build a custom, patched version of emsdk, so there is no
need to build it yourself prior.
Additional build prerequisites are:
- A working native compiler toolchain, enough to build CPython.
- A native Python 3.7 to run the build scripts.
- PyYAML
- [lessc](http://lesscss.org/) to compile less to css.
- [uglifyjs](https://github.com/mishoo/UglifyJS) to minify Javascript builds.
- gfortran (GNU Fortran 95 compiler)
- [f2c](http://www.netlib.org/f2c/)
- [ccache](https://ccache.samba.org) (optional) *highly* recommended for much faster rebuilds.
On Mac, you will also need:
- [Homebrew](https://brew.sh/) for installing dependencies
- System libraries in the root directory (`sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /` should do it, see https://github.com/pyenv/pyenv/issues/1219#issuecomment-428305417)
- coreutils for md5sum and other essential Unix utilities (`brew install coreutils`)
- cmake (`brew install cmake`)
- pkg-config (`brew install pkg-config`)
- openssl (`brew install openssl`)
- gfortran (`brew cask install gfortran`)
- f2c: Install wget (`brew install wget`), and then run the buildf2c script from the root directory (`sudo ./tools/buildf2c`)
#### Build using `make`
After installing the build prerequisites, run from the command line:
```bash
make
```
### Using Docker
We provide a Debian-based Docker image on Docker Hub with the dependencies
already installed to make it easier to build Pyodide. Note that building from
the Docker image is *very* slow on Mac, building on the host machine is
preferred if at all possible.
1. Install Docker
2. From a git checkout of Pyodide, run `./run_docker`
3. Run `make` to build.
If running ``make`` deterministically stops at one point in each subsequent try, increasing
the maximum RAM usage available to the docker container might help [This is different
from the physical RAM capacity inside the system]. Ideally, at least 3 GB of RAM
should be available to the docker container to build `pyodide` smoothly. These settings can
be changed via Docker Preferences [See [here](https://stackoverflow.com/questions/44533319/how-to-assign-more-memory-to-docker-container)].
You can edit the files in your source checkout on your host machine, and then
repeatedly run `make` inside the Docker environment to test your changes.
## Partial builds

67
docs/changelog.md Normal file
View File

@ -0,0 +1,67 @@
# Release notes
## Unreleased
- The built-in `sqlite` and `bz2` modules of Python are now enabled.
- New package: `nltk`
- `micropip` can now be used from web workers.
## Version 0.13.0
- Tagged versions of Pyodide are now deployed to Netlify.
## Version 0.12.0
**User improvements:**
- Packages with pure Python wheels can now be loaded directly from PyPI. See
`docs/pypi.md` for more information.
- Thanks to PEP 562, you can now `import js` from Python and use it to access
anything in the global Javascript namespace.
- Passing a Python object to Javascript always creates the same object in
Javascript. This makes APIs like `removeEventListener` usable.
- Calling `dir()` in Python on a JavaScript proxy now works.
- Passing an `ArrayBuffer` from Javascript to Python now correctly creates
a `memoryview` object.
- Pyodide now works on Safari.
## Version 0.11.0
**User improvements:**
- Support for built-in modules:
- `sqlite`, `crypt`
- New packages: `mne`
**Developer improvements:**
- The `mkpkg` command will now select an appropriate archive to use, rather than
just using the first.
- The included version of emscripten has been upgraded to 1.38.30 (plus a
bugfix).
- New packages: `jinja2`, `MarkupSafe`
## Version 0.10.0
**User improvements:**
- New packages: `html5lib`, `pygments`, `beautifulsoup4`, `soupsieve`,
`docutils`, `bleach`, `mne`
**Developer improvements:**
- `console.html` provides a simple text-only interactive console to test local
changes to Pyodide. The existing notebooks based on legacy versions of Iodide
have been removed.
- The `run_docker` script can now be configured with environment variables.

View File

@ -1,11 +1,4 @@
.. This rST file provides a way to easily add Markdown files
which are located in the root directory of the repo.
About the Project
=================
.. mdinclude:: ../README.md
.. mdinclude:: ../CONTRIBUTING.md
.. mdinclude:: ../CODE-OF-CONDUCT.md

View File

@ -2,9 +2,13 @@
Thank you for your interest in contributing to PYODIDE! There are many ways to contribute, and we appreciate all of them. Here are some guidelines & pointers for diving into it.
## Development Workflow
See [building from sources](./building_from_sources.html) and [testing](./testing.html) documentation.
## Code of Conduct
PYODIDE has adopted a [Code of Conduct](CODE-OF-CONDUCT.md) that we expect all contributors and core members to adhere to.
PYODIDE has adopted a [Code of Conduct](code-of-conduct.html) that we expect all contributors and core members to adhere to.
## Development
@ -14,7 +18,7 @@ Well do our best to keep `master` in a non-breaking state, ideally with tests
PYODIDE follows semantic versioning (http://semver.org/) - major versions for breaking changes (x.0.0), minor versions for new features (0.x.0), and patches for bug fixes (0.0.x).
We keep a file, [CHANGELOG.md](CHANGELOG.md), outlining changes to PYODIDE in each release. We like to think of the audience for changelogs as non-developers who primarily run the latest stable. So the change log will primarily outline user-visible changes such as new features and deprecations, and will exclude things that might otherwise be inconsequential to the end user experience, such as infrastructure or refactoring.
We keep a file, [doc/changelog.md](./changelog.html), outlining changes to PYODIDE in each release. We like to think of the audience for changelogs as non-developers who primarily run the latest stable. So the change log will primarily outline user-visible changes such as new features and deprecations, and will exclude things that might otherwise be inconsequential to the end user experience, such as infrastructure or refactoring.
## Bugs & Issues
@ -36,9 +40,6 @@ We use [py.test](https://pytest.org), driving [Selenium](https://www.seleniumhq.
All code submissions should pass `make lint`. Python is checked with the default settings of `flake8`. C and Javascript are checked against the Mozilla style in `clang-format`.
## Development Workflow
See `README.md`.
## License

View File

@ -60,12 +60,16 @@ information about the project's organization.
building_from_sources.md
new_packages.md
type_conversions.md
contributing.md
testing.md
.. toctree::
:titlesonly:
:caption: Project
rootdir.rst
about.md
code-of-conduct
changelog.md
Indices and tables
==================

67
docs/testing.md Normal file
View File

@ -0,0 +1,67 @@
# Testing and benchmarking
## Testing
### Requirements
Install the following dependencies into the default Python installation:
```bash
pip install pytest selenium pytest-instafail
```
Install [geckodriver](https://github.com/mozilla/geckodriver/releases) and
[chromedriver](https://sites.google.com/a/chromium.org/chromedriver/downloads)
and check that they are in your `PATH`.
### Running the test suite
To run the pytest suite of tests, type on the command line:
```bash
pytest test/ packages/
```
### Manual interactive testing
To run manual interactive tests, a docker environment and a webserver will be
used.
1. Bind port 8000 for testing. To automatically bind port 8000 of the docker
environment and the host system, run: `./run_docker`
2. Now, this can be used to test the `pyodide` builds running within the
docker environment using external browser programs on the host system. To do
this, run: `./bin/pyodide serve`
3. This serves the ``build`` directory of the ``pyodide`` project on port 8000.
* To serve a different directory, use the ``--build_dir`` argument followed
by the path of the directory.
* To serve on a different port, use the ``--port`` argument followed by the
desired port number. Make sure that the port passed in ``--port`` argument
is same as the one defined as ``DOCKER_PORT`` in the ``run_docker`` script.
4. Once the webserver is running, simple interactive testing can be run by
visiting this URL:
[http://localhost:8000/console.html](http://localhost:8000/console.html)
## Benchmarking
To run common benchmarks to understand Pyodide's performance, begin by
installing the same prerequisites as for testing. Then run:
```bash
make benchmark
```
## Linting
Python is linted with `flake8`. C and Javascript are linted with
`clang-format`.
To lint the code, run:
```bash
make lint
```