Commit Graph

3 Commits

Author SHA1 Message Date
Hood Chatham e19621d483
ENH Rework streams handling (#4035)
This fixes a number problems with the old stream handling:

1. Not possible to set a custom errno (necessary for proper interrupt
   handling and possibly for other things)

2. Inefficient: in a lot of cases we have data in one buffer and we need
   it placed into a different buffer, but we have to implement a function
   that gets one byte out of the source buffer and then call it repeatedly
   to move one byte at a time to the target buffer.

3. Ease of implementation: in many cases we already have perfectly good
   buffer manipulation APIs, so if we have direct access to the true source
   or target buffer we can just use these. See: the node IO code, which got
   much simpler.

This is backwards compatible, so you can still use the old input mechanism
or use buffered or raw output. But it adds a new method of directly implementing
read/write. For simplicity, we insure that the source/destination buffers are
always `Uint8Array` views that point to exactly the region that is meant to be
read/written.

The old mechanisms are faster than before and can correctly support keyboard
interrupts. Other than that I think the original behavior is unchanged. I added a
lot more test coverage to ensure backwards compatibility since there was pretty
anemic coverage before.

I think the read/write APIs are mostly pretty simple to use, with the exception
that someone might forget to return the number of bytes read. JavaScript's ordinary
behavior coerces the `undefined` to a 0, which leads to an infinite loop where the
filesystem repeatedly asks to read/write the same data since it sees no progress.
I added a check that writes an error message to the console and sets EIO when undefined
is returned so the infinite loop is prevented and the problem is explained.
2023-08-21 08:41:44 +02:00
Hood Chatham 5780584407
Docs: Minor improvements (#3546) 2023-02-06 08:45:12 +01:00
Hood Chatham 7422ab370d
Implement more detailed streams support (#3268)
Resolves https://github.com/pyodide/pyodide/issues/3112
This adds a carefully designed API for controlling stdin, stdout, and stderr. It changes
the default behavior to be a bit more useful, though in doing so introduces some mild
backwards incompatibility. In particular:

1. By default, stdin reads directly from `process.stdin` in node (as before) and raises an
error if in browser (not as before).
2. By default, stdout writes directly to `process.stdout` in node (before it called console.log)
and calls console.log in browser (as before).
3. By default, stderr writes directly to `process.stderr` in node (before it called console.warn)
and calls console.warn in browser (as before).
4. In all three cases, by default isatty(stdin/stdout/stderr) is true in node and false in browser
(in the browser it used to be true).
5. As before, if you pass `stdin`, `stdout`, or `stderr` as arguments to `loadPyodide`, `isatty` of
the corresponding stream is set to false.

The stdin function is now more flexible: we now correctly handle the case where it returns an
ArrayBuffer or ArrayBufferView.

I also added 3 new functions to set streams after Pyodide is loaded which offer additional
control:
* `setStdin({stdin?, error?, isatty = false})` -- Sets the stdin function. The stdin function takes no
arguments and should return null, undefined, a string, or a buffer. Sets and `isatty(stdin)` to 
`isatty` (by default `false`). If error is true, set stdin to always raise an EIO error when it is read.
* `setStdout({raw?, batched?, isatty = false})` -- If neither raw nor batched is passed, restore 
default stdout behavior. If rwa is passed, the raw stdout function receives a byte which it should
interpret as a utf8 character code. Sets `isatty(stdout)` to isatty (by default `false`). If batched is
passed but not raw, it sets a batched stdout function. The stdout function receives a string and
should do something with it. In this case it ignores isatty and sets isatty(stdout) to false.
* `setStderr({raw?, batched?, isatty = false})` -- same but with stderr.
2022-12-18 15:55:52 -08:00