Stack switching was leaking datastack chunks. This seems to fix it. It is very confusing,
but the leaked chunks are allocated in slabs of size 2^14 so it doesn't take long to leak
a substantial amount of memory.
Code inspired by greenlet:
937f150e07/src/greenlet/TPythonState.cpp (L291)
Resolves#4569. It still doesn't make sure `str(jsproxy)` never throws. It will throw if:
1. accessing `obj.toString` succeeds and returns a function, but calling the function throws
2. accessing `obj.toString` fails or returns not a function, and `Object.prototype.toString.call` fails.
Before we moved the exception state into the saved ExceptionState as part of
saving the Python state but we didn't clear the exception state. This logic
was copied from greenlet; I can't find where they clear the exception state
either. However, in our case it is definitely wrong. If we stack switch inside
an except block and then enter and stack switch inside a second except block,
after exiting from everything we end up witht the exception state set instead of
cleared (as if we were still in an except block) and worse the exception state
is set to an already freed exception. I'm not actually sure why the bug
manifests in this particular way, but this change fixes it.
This adds the altair packages and closes#4579. I wasn't sure what tests would be appropriate, it seems like some package like mpl, pandas, etc tests very specific things whereas others such as shapely, bokeh, statsmodels, etc keep it more minimal. I started minimal for now but happy to add more if there are issues.
Consider the code:
```js
pyodide.runPython(`
def a():
raise Exception("hi")
def b():
return 7;
`);
const a = pyodide.globals.get("a");
const b = pyodide.globals.get("b");
const p = a.callSyncifying();
assert(b() === 7);
await p;
```
This used to misbehave because because a()'s error status got stolen by b().
This happened because the promising function is a separate task from the js code
in callPyObjectSuspending, so the sequence of events goes:
- enter main task,
- enter callPyObjectSuspending(a)
- enter promisingApply(a)
- sets error flag and returns NULL
- queue continue callPyObjectSuspending(a) in event loop
now looks like [main task, continue callPyObjectSuspending(a)]
- enter b()
- enter Python
- returns 7 with error state still set
- rejects with "SystemError: <function b at 0x1140f20> returned a result with an exception set"
- queue continue main() in event loop
- continue callPyObjectSuspending(a)
- pythonexc2js called attempting to read error flag set by promisingApply(a), fails with
PythonError: TypeError: Pyodide internal error: no exception type or value
The solution: at the end of `_pyproxy_apply_promising` we move the error
flag into errStatus argument. In callPyObjectSuspending when we're ready we
move the error back from the errorStatus variable into the error flag before
calling `pythonexc2js()`
The only behavior change here should be that we are setting `sys.last_exc`
(added in Python3.12). Since we haven't released with Python 3.12 yet, this
doesn't need a changelog.
Python 3.12 added a bunch of modern APIs that handle a single exception object
rather than the (type, val, tb) triples. They also repaired various quirks of
the old error handling APIs. They are much easier to work with. Also, now that
we have `capture_stderr` and `restore_stderr` we can use `PyErr_Print()` to
format the traceback. This allows us to cut out a fair amount of code in
error_handling.c.
This now uses `sys.excepthook` to format exceptions. We set `sys.excepthook` to
`traceback.print_exception` because the default excepthook does not respect the
linecache which we use to implement `pyodide.runPython("...", {file:
"some_file.py"});`
Alexey Ignatiev