This PR adds `package_type` field to repodata.json and use it to create a list of
unvendored standard libraries. After this we don't need to manage a hard-coded
list of unvendored stdlib lists in pyodide-py.
This adds a short helper script which shows a gzip and brotli compressed size of a file,
and uses it in CI to check compressed size of pyodide.asm.* after build in addition to
the original file size.
This leads to more consistent rendering (functions and methods get parens after
them) and reduces chances of warnings about getting the wrong link. It is also
possible to use `~fully.quallified.name` to just show `name` if we use a specific
reference type, but it doesn't work with `any` for some reason.
Removes / unvendors some python modules:
- Remove `_aix_support.py`, which is for supporting IBM AIX OS.
- Unvendor `_pydecimal.py`.
- _pydecimal is a pure Python implementation of `decimal` module. [Importing `decimal` fallbacks](https://github.com/python/cpython/blob/main/Lib/decimal.py) to `_pydecimal` if the C-implementation `_decimal` is not available. In our case, _decimal is available, so _pydecimal will not be normally used.
- Unvendor `pydoc_data`.
- pydoc_data contains [a large (~700KB) dictionary](https://github.com/python/cpython/blob/main/Lib/pydoc_data/topics.py) for explaining python builtins. This is mostly used when `help("...")` is called.
This is work towards unvendoring the Pyodide foreign function interface.
Prior to this point, we included a large amount of critical functionality with `--pre-js`.
So we could create an archive called `libpyodide.a` with the object files but to use it
you would have to pass `--pre-js _pyodide.out.js` at link time. This embeds all of this
stuff in an object file called `pyodide_pre.o` which goes in our archive so you get all
the needed js runtime by linking it.
Of course someone trying to use this still has to get the Python code onto the import
path, either using `--preload-file`, using Python to unpack it as a zip archive as we now
do, with zipimporter, or otherwise. They also will have to link `libpython.a` (is CPython
going to start distributing an Emscripten libpython?) and probably various other things.
We have to use a hack to inject the JavaScript code into the object files. The normal
`EM_JS` macro cannot handle arbitrary JavaScript code -- for example it fails with many
regex. Instead we manually generate write a C source file that does what we need using
`xxd`. The generated C code is similar to what `EM_JS` generates, but it uses an array
initializer rather than a string initializer for the characters avoiding the C preprocessor /
compiler's strange opinions about strings.
Instead of putting stuff behind `IN_SPHINX`, define functions and call them from the `setup` function.
In these functions, if we want to expose variables as part of the config we have to assign to `app.config.some_var` which is more explicit.
We still have to make the path change at top level. To improve this, in the future we should:
1. rename the sphinx_pyodide folder to sphinx-pyodide
2. add a `pyproject.toml` and `setup.py` so we can `pip install -e` it
3. instead of modifying the path, source the virtual environment
Up to this point, we've used this dynamic subclassing method for
producing JsProxies for everything but errors. For errors, we make
a wrapper which is not a JsProxy that inherits from Exception and
give the wrapper a "js_error" attribute that points to an actual
error. We also make python2js know about this wrapper so it can
unwrap it. But the raw js_error object is a bit weird. There isn't
anything terrible about this situation but it is mildly unsatisfying.
This changes it so that errors subclass both JsProxy and Exception.
To do this we need to:
1. ensure that they have compatible memory layouts, and
2. convince Python that they have compatible memory layouts
I switched to using a union for the different subtypes of JsProxy
that need extra space: JsCallable, JsBuffer, and JsError. We need
js to be at the end so it won't get in the way of the BaseException
memory layout. I added _Static_asserts to double check that the
memory layouts do in fact agree.
To convince Python that they have compatible memory layouts, we have
to temporarily tell it that JsProxy is a subtype of BaseException.
To do this, we just set JsProxy.__mro__ = (BaseException,) before
creating the type and then restore it afterward.
This prints a better set of error messages in case someone calls os._exit() or in C code exit() is used.
In the future we might like to do something better here, but for now at least we can print a clear
error message.
Also remove almost everything from `_core.py` and moves it to `ffi/__init__.py`.
Micropip imports `IN_BROWSER` from `pyodide._core` so we leave that in there.
Resolves https://github.com/pyodide/pyodide/issues/3432
I added a new decorator called `@docs_argspec` to override the argument specification
of a function used in the docs by setting `func.__wraps__` to a fake function. This only
happens when building the docs, normally it is a no-op.
`@docs_argspec` is needed when using `@overload` because mypy requires the argspec
of the main function to be at least as general as the argspecs of all the overloads, which
causes suboptimal rendering in the api docs.
Gyeongjae Choi