This uses pyodide/pytest-pyodide#117 to run doctests in
Pyodide. I also turned on and fixed various doctests that were not working for
unrelated reasons
This makes IDEs, documentation generation tools, and linters handle our
generated files better. I set the project root to be `src` instead of `src/js`
so that it is allowed to import files directly from `src/core`. This way we
don't have to copy `error_handling.ts` we can just import
`../core/error_handling`.
I made a new folder called `src/js/generated` to place generated files into and
added typescript resolution rules so that when we import a file called
"generated/blah" we first look for `blah` in `src/js/generated` and then fall
back to a file called `blah` in `src/core`.
This also allows us to move around fewer files when building the docs
and in the makefile.
This fixes a number problems with the old stream handling:
1. Not possible to set a custom errno (necessary for proper interrupt
handling and possibly for other things)
2. Inefficient: in a lot of cases we have data in one buffer and we need
it placed into a different buffer, but we have to implement a function
that gets one byte out of the source buffer and then call it repeatedly
to move one byte at a time to the target buffer.
3. Ease of implementation: in many cases we already have perfectly good
buffer manipulation APIs, so if we have direct access to the true source
or target buffer we can just use these. See: the node IO code, which got
much simpler.
This is backwards compatible, so you can still use the old input mechanism
or use buffered or raw output. But it adds a new method of directly implementing
read/write. For simplicity, we insure that the source/destination buffers are
always `Uint8Array` views that point to exactly the region that is meant to be
read/written.
The old mechanisms are faster than before and can correctly support keyboard
interrupts. Other than that I think the original behavior is unchanged. I added a
lot more test coverage to ensure backwards compatibility since there was pretty
anemic coverage before.
I think the read/write APIs are mostly pretty simple to use, with the exception
that someone might forget to return the number of bytes read. JavaScript's ordinary
behavior coerces the `undefined` to a 0, which leads to an infinite loop where the
filesystem repeatedly asks to read/write the same data since it sees no progress.
I added a check that writes an error message to the console and sets EIO when undefined
is returned so the infinite loop is prevented and the problem is explained.
Work towards making a pyproxy of a list a drop-in replacement for an array.
This can't be done entirely because lists don't allow weird things like holes,
writing to the length field, writing out of bounds entries, etc. But we
implement the main Array APIs.
This creates a new `pyodide.ffi` submodule and adds a bunch of new subclasses of
`PyProxy` to it.
There are three stages in which we are concerned with the behavior of the
objects we define:
1. at time of static typechecks
2. at execution time
3. when generating docs
Prior to this PR, the subtypes of PyProxy only work well for static type checks,
they work acceptably at runtime (just don't let the user access them), and the
docs don't look that great. This PR is primarily intended to improve the docs
for PyProxy, but they also make execution time checks work better: you can now
say `obj instanceof pyodide.ffi.PyCallable` instead of `obj.isCallable()` which
I is easier to understand and to cross reference against the documentation. I am
marking `isCallable` as deprecated.
I also made a bunch of edits and improvements to the docs.
I have deprecated `PyProxyCallable` in favor of `pyodide.ffi.PyCallable` and
`PyProxy.isCallable` in favor of `obj instanceof pyodide.ffi.PyCallable`.
`PyBuffer` has been renamed to `pyodide.ffi.PyBufferView` and a new `PyBuffer`
has been created which is a subtype of `PyProxy`.
This leads to more consistent rendering (functions and methods get parens after
them) and reduces chances of warnings about getting the wrong link. It is also
possible to use `~fully.quallified.name` to just show `name` if we use a specific
reference type, but it doesn't work with `any` for some reason.
The `pyodide.ffi` stuff is defined in `_pyodide._core_docs`. We don't want `_pyodide._core_docs` to appear in the documentation because this isn't where you should import things from so we override the `__name__` of `_pyodide._core_docs` to be `pyodide.ffi`. But then Sphinx fails to locate the source for the stuff defined in `_pyodide._core_docs`. This patches `ModuleAnalyzer` to tell it to look for the source of things from `pyodide.ffi` in `_pyodide._core_docs`.
Remove the property prefix from properties, add a link for ast.Module.
Previously this included more significant changes but they have been upstreamed into sphinx-autodoc-typehints.
I opened a PR for one of the Napoleon changes:
https://github.com/sphinx-doc/sphinx/pull/11131
But we use Sphinx v5.3 so even if we upstream Napoleon fixes into Sphinx v6.x,
we won't get to use them for a while.
* chore: add some incomplete types
* chore: modernize pyproject.toml
Adding more incomplete types. About 2/3 of the way through being
able to turn on the strictness flag for it.
Hood Chatham