If wasm exception handling is available, use dynamically generated modules
with wasm exception handling for the invoke stubs instead of a JS trampoline.
This is useful because JS trampolines interact poorly with JS Promise Integration.
In the future, we should switch exception handling ABIs to the compiler-provided
wasm exception handling ABI and we can remove this code. Currently we are
blocked on using compiler-provided wasm eh by Rust support.
This is work towards unvendoring the Pyodide foreign function interface.
Prior to this point, we included a large amount of critical functionality with `--pre-js`.
So we could create an archive called `libpyodide.a` with the object files but to use it
you would have to pass `--pre-js _pyodide.out.js` at link time. This embeds all of this
stuff in an object file called `pyodide_pre.o` which goes in our archive so you get all
the needed js runtime by linking it.
Of course someone trying to use this still has to get the Python code onto the import
path, either using `--preload-file`, using Python to unpack it as a zip archive as we now
do, with zipimporter, or otherwise. They also will have to link `libpython.a` (is CPython
going to start distributing an Emscripten libpython?) and probably various other things.
We have to use a hack to inject the JavaScript code into the object files. The normal
`EM_JS` macro cannot handle arbitrary JavaScript code -- for example it fails with many
regex. Instead we manually generate write a C source file that does what we need using
`xxd`. The generated C code is similar to what `EM_JS` generates, but it uses an array
initializer rather than a string initializer for the characters avoiding the C preprocessor /
compiler's strange opinions about strings.
This adds pyodide_build command create_xbuildenv which creates a
crossbuild environment (from a copy of Pyodide where scipy has been
build) and install_xbuildenv which installs the cross build environment
into a fresh copy of Pyodide.
I successfully installed the xbuild environment into a fresh checkout of
Pyodide then built statsmodels and scikit-learn in isolation, without
building the Python interpreter, numpy, or scipy. I dumped the generated
wheels into a copy of Pyodide downloaded from CI, and was able to import
and use them as normal.
The size of the xbuild environment is 1.5 megabytes, of which 1.2 megabytes
is Python headers.
In a subsequent PR, we can update the CI to automatically upload these
to aws s3 and then install the environment from there.
The installation is very fast, and this avoids having two copies of node
around. In particular, this avoids the need to monkey-patch uglify-js to
use the system node.
This can be further streamlined when #792 is merged
Hood Chatham