From 79aa563f2c199fb69f962d3d066abbdedfaeb9bc Mon Sep 17 00:00:00 2001 From: Roman Yurchak Date: Thu, 9 Mar 2023 13:37:25 +0100 Subject: [PATCH] DOC using conda for building from sources (Linux & Mac OS) (#3644) Co-authored-by: Jo Bovy --- .github/workflows/main.yml | 5 +- docs/development/building-from-sources.md | 75 ++++++++++++++++------- environment.yml | 19 ++++++ 3 files changed, 74 insertions(+), 25 deletions(-) create mode 100644 environment.yml diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 7056b64cd..672f83a2f 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -97,7 +97,7 @@ jobs: - uses: conda-incubator/setup-miniconda@v2 with: activate-environment: pyodide-env - python-version: "3.11" + environment-file: environment.yml channels: conda-forge - name: Check Python versions @@ -118,10 +118,9 @@ jobs: run: | brew install coreutils - - name: Install dependencies (common) + - name: Install dependencies (Python) shell: bash -l {0} run: | - conda install -y nodejs ccache f2c pkg-config swig make pkg-config texinfo autoconf automake libtool patch sed pip install -r requirements.txt - name: Build emsdk diff --git a/docs/development/building-from-sources.md b/docs/development/building-from-sources.md index 15da0bfb0..63ee988c3 100644 --- a/docs/development/building-from-sources.md +++ b/docs/development/building-from-sources.md @@ -9,19 +9,24 @@ version of the documentation at [pyodide.org/en/latest/](https://pyodide.org/en/latest/development/building-from-sources.html) ``` -Building Pyodide is easiest using the Pyodide Docker image. This approach works -with any native operating system as long as Docker is installed. You can also -build on your native Linux OS if the correct build prerequisites are installed. -Building on MacOS is possible, but there are known issues as of version 0.18 -that you will need to work around. It is not possible to build on Windows, but -you can use [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10) -to create a Linux build environment. +Pyodide can be built from sources on different platforms, + +- on **Linux** it is easiest using the Pyodide Docker image. This approach + works with any native operating system as long as Docker is installed. You + can also build on your native Linux OS if the correct build prerequisites + are installed. +- on **MacOS** it is recommended to install dependencies via conda-forge or + using Homebrew, particularly with the M1 ARM CPU. Building with Docker is + possible but very slow. +- It is not possible to build on **Windows**, but you can use [Windows Subsystem + for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10) to + create a Linux build environment. ## Build instructions ### Using Docker -We provide a Debian-based Docker image +We provide a Debian-based x86_64 Docker image ([`pyodide/pyodide-env`](https://hub.docker.com/r/pyodide/pyodide-env)) on Docker Hub with the dependencies already installed to make it easier to build Pyodide. @@ -62,9 +67,10 @@ Make sure the prerequisites for build a custom, patched version of emsdk, so there is no need to build it yourself prior. -You need Python 3.10.2 to run the build scripts. To make sure that the correct +You need Python 3.11.2 to run the build scripts. To make sure that the correct Python is used during the build it is recommended to use a [virtual -environment](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment), +environment](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment) +or a conda environment. ````{tab-set} @@ -78,21 +84,46 @@ To build on Linux, you need: ``` -```{tab-item} MacOS +```{tab-item} Linux with conda -To build on MacOS, you need: -- A working native compiler toolchain, enough to build - [CPython](https://devguide.python.org/getting-started/setup-building/index.html#macos-and-os-x). +You would need a working native compiler toolchain, enough to build + [CPython](https://devguide.python.org/getting-started/setup-building/index.html#linux), for example, +- `apt install build-essential` on Debian based systems. +- Conda which can be installed from [MiniForge](https://github.com/conda-forge/miniforge) + +Then install the required Python version and other build dependencies in a separate conda environment, + + conda env create -f environment.yml + conda activate conda-forge + +``` +```{tab-item} MacOS with conda + +You would need, +- System libraries in the root directory: + `xcode-select --install` +- Conda which can be installed using [Miniforge](https://github.com/conda-forge/miniforge) (both for Intel and M1 CPU) + + +Then install the required Python version and other build dependencies in a separate conda environment, + + conda env create -f environment.yml + conda activate conda-forge + +``` + +```{tab-item} MacOS with Homebrew + +To build on MacOS with Homebrew, you need: + +- System command line tools + `xcode-select --install` - [Homebrew](https://brew.sh/) for installing dependencies -- System libraries in the root directory ( - `sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /` - should do it, see https://github.com/pyenv/pyenv/issues/1219#issuecomment-428305417) -- coreutils for and other essential Unix utilities (`brew install coreutils`). -- cmake (`brew install cmake`) -- autoconf, automaker & libtool (`brew install autoconf automaker libtool`) -- It is also recommended installing the GNU patch (`brew install gpatch`), and - GNU sed (`brew install gnu-sed`) and [re-defining them temporarily as `patch` and +- `brew install coreutils cmake autoconf automaker libtool` +- It is also recommended installing the GNU patch and + GNU sed (`brew install gpatch gnu-sed`) + and [re-defining them temporarily as `patch` and `sed`](https://formulae.brew.sh/formula/gnu-sed). ``` ```` diff --git a/environment.yml b/environment.yml new file mode 100644 index 000000000..4ae7d52c5 --- /dev/null +++ b/environment.yml @@ -0,0 +1,19 @@ +name: pyodide-env +channels: + - conda-forge +dependencies: + - python=3.11 + - nodejs + - ccache + - f2c + - pkg-config + - swig + - make + - cmake + - pkg-config + - texinfo + - autoconf + - automake + - libtool + - patch # only needed on MacOS + - sed # only needed on MacOS