Buster isn't available for anymore for Python 312 so we have to update.
This also updates to Python 3.11.7, which turns out to be relatively invasive. I broadly
xfailed failing tests in `python_tests.yaml` since we'll upgrade to Python 3.12 soon anyways.
If wasm exception handling is available, use dynamically generated modules
with wasm exception handling for the invoke stubs instead of a JS trampoline.
This is useful because JS trampolines interact poorly with JS Promise Integration.
In the future, we should switch exception handling ABIs to the compiler-provided
wasm exception handling ABI and we can remove this code. Currently we are
blocked on using compiler-provided wasm eh by Rust support.
It seems that there is a bug in table instructions in Safari 16.3 which is used in the safari tests
but that Safari 16.4 has fixed it. xcode 15.0.0 uses Safari 16.6 which does not have the bug.
If wasm exception handling is available, use dynamically generated modules
with wasm exception handling for the invoke stubs instead of a JS trampoline.
This is useful because JS trampolines interact poorly with JS Promise Integration.
In the future, we should switch exception handling ABIs to the compiler-provided
wasm exception handling ABI and we can remove this code. Currently we are
blocked on using compiler-provided wasm eh by Rust support.
This PR makes build variables only accessible through the API, instead of polluting os.environ.
I also changed the API as below, as a preparation to move away from makefiles.
- `get_make_environment_vars` => `get_build_environment_vars`
- `get_make_flag` => `get_build_flag`
Closes https://github.com/pyodide/pyodide/issues/3687
I forgot to make the changes for the dev deployment in
https://github.com/pyodide/pyodide/pull/3667
Overall those are mostly identical lines repeated several times, however
because the cost of error is pretty high I prefer not to factorize it
for now. Instead, we should rethink this system, maybe moving dev
deployment to a separate bucket and making release deployment immutable.
Set the MIME type for wheels and zip files to application/wasm to enable CDN compression. Also optionally disable compression in zip files and wheels to make the brotli compression by the CDN more efficient.
Co-authored-by: Gyeongjae Choi <def6488@gmail.com>
This adds a short helper script which shows a gzip and brotli compressed size of a file,
and uses it in CI to check compressed size of pyodide.asm.* after build in addition to
the original file size.
This is work towards unvendoring the Pyodide foreign function interface.
Prior to this point, we included a large amount of critical functionality with `--pre-js`.
So we could create an archive called `libpyodide.a` with the object files but to use it
you would have to pass `--pre-js _pyodide.out.js` at link time. This embeds all of this
stuff in an object file called `pyodide_pre.o` which goes in our archive so you get all
the needed js runtime by linking it.
Of course someone trying to use this still has to get the Python code onto the import
path, either using `--preload-file`, using Python to unpack it as a zip archive as we now
do, with zipimporter, or otherwise. They also will have to link `libpython.a` (is CPython
going to start distributing an Emscripten libpython?) and probably various other things.
We have to use a hack to inject the JavaScript code into the object files. The normal
`EM_JS` macro cannot handle arbitrary JavaScript code -- for example it fails with many
regex. Instead we manually generate write a C source file that does what we need using
`xxd`. The generated C code is similar to what `EM_JS` generates, but it uses an array
initializer rather than a string initializer for the characters avoiding the C preprocessor /
compiler's strange opinions about strings.
This adds rust toolchain into our docker image so we don't have to install it every time.
make rust command still exists, but I removed it from the docs. So now a user (who wants to build a package that requires rust) is responsible for installing rust toolchain in their build setup, which is reasonable I think.
Co-authored-by: josephrocca <1167575+josephrocca@users.noreply.github.com>
Co-authored-by: Roman Yurchak <rth.yurchak@gmail.com>
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Currently `pip` can't invoke the Pyodide build backend, but it should still be
able to build pure Python packages from source. To allow this, we need to
restore `_PYTHON_SYSCONFIGDATA_NAME` so that it isn't inherited by
subprocesses. Otherwise, they will fail to import `sysconfig` because the
subprocess doesn't have our altered `sys.path`. Annoyingly, when pip invokes
the subprocess it seems to pass `_PYTHON_SYSCONFIGDATA_NAME` down
but not `PYTHONPATH` so just setting the `PYTHONPATH` isn't good enough.
This uses sed to insert /* webpackIgnore: true */ comments into pyodide.js.
This resolves#3087. I also enabled a check that a simple webpack config
builds without warnings. In the future it would be good to add a test that it
also runs without error.
Hood Chatham