Commit Graph

94 Commits

Author SHA1 Message Date
David Wilson 175abee2c9 Fix inverted logic. 2018-03-19 21:35:36 +05:45
David Wilson 2fed3613e9 Syntax error. 2018-03-19 21:35:36 +05:45
David Wilson 8054fa867a Only import compiler when necessary, it's not available in 3.x. 2018-03-19 21:35:36 +05:45
Alex Willmer aa817f1b83 Handle failure to import the ast module 2018-03-19 21:35:36 +05:45
Alex Willmer dd8271f061 Fix stale references to m.master.connect & m.ssh.connect
I'm fairly sure these updates are correct, but I'm not able to run the
test suite to confirm.
2018-03-19 21:35:36 +05:45
Alex Willmer d706b7d6b8 Update lingering references to with_broker & run_with_broker
A previous commit renamed run_with_broker() and with_broker() to
run_with_router() and with_router() respctively. Some references were
missed.
2018-03-19 21:35:36 +05:45
Alex Willmer dc26460a28 Add missing LOG import
Fixes an undefined variable error found by pylint
2018-03-19 21:35:36 +05:45
Alex Willmer 3b24314a8b Use fully qualified Context
Fixes an undefined variable error found by pylint
2018-03-19 21:35:36 +05:45
Alex Willmer 57a0aa26bf Add missing import of AnsibleError
Found with pylint
2018-03-19 21:35:36 +05:45
Alex Willmer f2ea38a963 Add missing auth_incorrect_msg used to in exception message
Found using pylint
2018-03-19 21:35:36 +05:45
Alex Willmer 1f955fc655 Declare self-less method as static
Found using pylint
2018-03-19 21:35:36 +05:45
Alex Willmer 36754254b2 Add missing imports: ast, compiler
Found using pylint
2018-03-19 21:35:36 +05:45
David Wilson 01729b18a5 core: use an output deque rather than string to improve worst case perf
This probably worsens performance in the common case, but it prevents
runaway producers (see e.g. issue #36) from spending all their CPU
copying around huge strings.

It's also a small step towards a solution to issue #6, which will
replace the output buffer with some sort of fancier queue anyway.

This reduces a particular 40 second run of rsync to 1.5 seconds.
2018-03-19 21:35:36 +05:45
David Wilson eb060e1a0e Document Router.profiling. 2018-03-19 21:35:36 +05:45
David Wilson effe4117e1 Treat EPIPE as disconnect too; needed for fakessh. 2018-03-19 21:35:36 +05:45
David Wilson 9c4bf37cfc Remove final vestiges of context.key. 2018-03-19 21:35:36 +05:45
David Wilson 0ff65c9780 docs: Use sphinx-programoutput plugin for preamble_size.py 2018-03-19 21:35:36 +05:45
David Wilson 2af1830323 Update preamble_size output. 2018-03-19 21:35:36 +05:45
David Wilson 05cc74d142 core: Support profiling 2018-03-19 21:35:36 +05:45
David Wilson b827ee1bc7 Delete mitogen.utils.log_to_tmp() 2018-03-19 21:35:36 +05:45
David Wilson 400c971f4a fakessh: SSH server uses user's shell to parse command line. 2018-03-19 21:35:36 +05:45
David Wilson b12433695c fakessh: add to preamble_size.py 2018-03-19 21:35:35 +05:45
David Wilson 04ea5306e9 Document a bunch of mitogen.master and move more docstrings into Sphinx. 2018-03-19 21:35:35 +05:45
David Wilson 1510b72a25 fakessh: Add integration test for inter-child rsync. 2018-03-19 21:35:35 +05:45
David Wilson b2f13f1fa4 master: propagate routes for IDs allocated via ALLOCATE_ID
needed for inter-child rsync.
2018-03-19 21:35:35 +05:45
David Wilson f01e457d70 Allow running without py.test. 2018-03-19 21:35:35 +05:45
David Wilson 750e9fab24 Make log_to_file accept string level rather than logging package constant
Saves an import, pain in the ass to type all the time.
2018-03-19 21:35:35 +05:45
David Wilson a9387b0504 core: remove pointless eval() of ARGV0 environment variable. 2018-03-19 21:35:35 +05:45
David Wilson 1411790f56 master: make write_all() handle O_NONBLOCK and deadlines. 2018-03-19 21:35:35 +05:45
David Wilson 2ee7309378 master: send_await() always unpickles now. 2018-03-19 21:35:35 +05:45
David Wilson 769ece2514 fakessh: improve logging slightly.
Makes it easier to match up reads/writes across log files by using
message byte count
2018-03-19 21:35:35 +05:45
David Wilson fb9ce1054c core: set O_NONBLOCK on every side. Closes #33
The last time I tested set_nonblock() as a fix for the rsync hang, I
used F_SETFD rather than F_SETFL, which resulted in no error, but also
did not set O_NONBLOCK. Turns out missing O_NONBLOCK was the problem.

The rsync hang was due to every context blocking in os.write() waiting
for either a parent or child buffer to empty, which was exacerbated by
rsync's own pipelining, that allows writes from both sides to proceed
even while reads aren't progressing. The hang was due to os.write() on a
blocking fd blocking until buffer space is available to complete the
write. Partial writes are only supported when O_NONBLOCK is enabled.
2018-03-19 21:35:35 +05:45
David Wilson db225638f0 core: Make iter_read() handle deadline (and non-blocking IO) properly 2018-03-19 21:35:35 +05:45
David Wilson 092bbef0b3 Reorder table of contents. 2018-03-19 21:35:35 +05:45
David Wilson caa1fde4fb Add serialization rules section. 2018-03-19 21:35:35 +05:45
David Wilson 3c8c1f71a6 Update test. 2018-03-19 21:35:35 +05:45
David Wilson 7a60b20dc6 core: Generalize/duplicate the call/send_await code using Receiver. 2018-03-19 21:35:35 +05:45
David Wilson 76d35df889 master: use decorators rather than call_with_deadline(with_context=..) 2018-03-19 21:35:35 +05:45
David Wilson 5dff221842 fakessh: cleanup master test. 2018-03-19 21:35:35 +05:45
David Wilson 76d1e66790 fakessh: use decorators rather than call_with_deadline(with_context=..) 2018-03-19 21:35:35 +05:45
David Wilson e4c832685d core: synchronize Stream._output_buf by deferring send()
Previously _output_buf was racy. This may or may not be cheaper than
simply using a lock, but it requires much less code, so I prefer it for
now.
2018-03-19 21:35:35 +05:45
David Wilson ead67de883 core: make Side.write() return None rather than crash if side already closed. 2018-03-19 21:35:35 +05:45
David Wilson 74b31bbe47 core: better Message.__repr__. 2018-03-19 21:35:35 +05:45
David Wilson 2a365aa9b0 Replace `with_context` parameter with mitogen.core.takes_econtext decorator 2018-03-19 21:35:35 +05:45
David Wilson 8d4005c264 fakessh: fix non-master context ID allocation. 2018-03-19 21:35:35 +05:45
David Wilson bd8001d481 fakessh: Fix NameError. 2018-03-19 21:35:35 +05:45
David Wilson e7c70127bf fakessh: return child exit status 2018-03-19 21:35:35 +05:45
David Wilson 77b8d28c68 fakessh: Use local process's sys.executable. 2018-03-19 21:35:35 +05:45
David Wilson 175fe7a4e1 fakessh: IoPump() doesn't need Process reference. 2018-03-19 21:35:35 +05:45
David Wilson 8dad396e27 Make subsystem requests die with a useful error. 2018-03-19 21:35:35 +05:45