Commit Graph

323 Commits

Author SHA1 Message Date
David Wilson be2264f4c1 master: send_await() always unpickles now. 2017-09-21 15:57:02 +05:30
David Wilson a6c97aad84 fakessh: improve logging slightly.
Makes it easier to match up reads/writes across log files by using
message byte count
2017-09-21 15:56:29 +05:30
David Wilson 68f3918072 core: set O_NONBLOCK on every side. Closes #33
The last time I tested set_nonblock() as a fix for the rsync hang, I
used F_SETFD rather than F_SETFL, which resulted in no error, but also
did not set O_NONBLOCK. Turns out missing O_NONBLOCK was the problem.

The rsync hang was due to every context blocking in os.write() waiting
for either a parent or child buffer to empty, which was exacerbated by
rsync's own pipelining, that allows writes from both sides to proceed
even while reads aren't progressing. The hang was due to os.write() on a
blocking fd blocking until buffer space is available to complete the
write. Partial writes are only supported when O_NONBLOCK is enabled.
2017-09-21 15:29:21 +05:30
David Wilson 16950a1620 core: Make iter_read() handle deadline (and non-blocking IO) properly 2017-09-21 15:28:02 +05:30
David Wilson 25adb7416a Reorder table of contents. 2017-09-21 14:38:11 +05:30
David Wilson 1de875b4aa Add serialization rules section. 2017-09-21 14:36:36 +05:30
David Wilson aa5bc53f6d Update test. 2017-09-21 13:41:47 +05:30
David Wilson 91d766ec81 core: Generalize/duplicate the call/send_await code using Receiver. 2017-09-21 13:41:12 +05:30
David Wilson 4ff1d4e523 master: use decorators rather than call_with_deadline(with_context=..) 2017-09-21 13:40:24 +05:30
David Wilson d6d83b3fac fakessh: cleanup master test. 2017-09-21 13:06:59 +05:30
David Wilson 5d0dcd051c fakessh: use decorators rather than call_with_deadline(with_context=..) 2017-09-21 03:26:14 +05:30
David Wilson 456a40ce9c core: synchronize Stream._output_buf by deferring send()
Previously _output_buf was racy. This may or may not be cheaper than
simply using a lock, but it requires much less code, so I prefer it for
now.
2017-09-21 03:21:16 +05:30
David Wilson 05055ebfc0 core: make Side.write() return None rather than crash if side already closed. 2017-09-21 03:20:52 +05:30
David Wilson 9ce71a546a core: better Message.__repr__. 2017-09-21 02:37:27 +05:30
David Wilson b7f5bd82bc Replace `with_context` parameter with mitogen.core.takes_econtext decorator 2017-09-21 01:37:07 +05:30
David Wilson 22df965a46 fakessh: fix non-master context ID allocation. 2017-09-20 20:00:29 +05:30
David Wilson 68515bc9e1 fakessh: Fix NameError. 2017-09-20 18:15:16 +05:30
David Wilson 3bb02161dc fakessh: return child exit status 2017-09-20 18:14:01 +05:30
David Wilson 9c306001f0 fakessh: Use local process's sys.executable. 2017-09-20 18:09:51 +05:30
David Wilson 023c100494 fakessh: IoPump() doesn't need Process reference. 2017-09-20 15:10:06 +05:30
David Wilson 29e4487d6a Make subsystem requests die with a useful error. 2017-09-20 15:05:39 +05:30
David Wilson 69dd35784d Reduce CHUNK_SIZE to paper over a hang with rsync 2017-09-19 01:04:46 +05:30
David Wilson d84b4bf72a Keep allocate_id() in the enhanced router class. 2017-09-18 17:03:30 +05:30
David Wilson f3cdb4e050 Fix format string. 2017-09-18 15:56:33 +05:30
David Wilson 491b5ab575 core: add ALLOCATE_ID message for fakessh. 2017-09-18 15:37:21 +05:30
David Wilson e00c2f2e39 core: allow sending 0-byte messages. 2017-09-18 15:36:30 +05:30
David Wilson 18a0529887 Import unused images. 2017-09-18 15:16:42 +05:30
David Wilson 6cfcd6c2c8 Fix proxy connection: pickling changes broke it. 2017-09-18 15:16:08 +05:30
David Wilson 535ffc25b7 Move more docstrings out of core.py. 2017-09-18 14:50:41 +05:30
David Wilson e7673e72f6 Add TODO 2017-09-18 14:44:17 +05:30
David Wilson 3e020ddd09 Fix Channel constructor and add simple test; closes #32 2017-09-18 14:34:05 +05:30
David Wilson 05a0feb2f9 core: support takes_econtext and takes_router decorators. 2017-09-17 22:52:11 +05:30
David Wilson d9bf091e6f Simple working fakessh_test. 2017-09-17 21:19:06 +05:30
David Wilson 21fb601872 Tidy up logging of command lines for easier cutpaste. 2017-09-17 21:09:52 +05:30
David Wilson bd2c613b9c Fix up a few more ssh_tests, stop sending 400 modules over network.
Defining functions in the current module was causing the entirety of
py.test and all dependencies to be sucked in.
2017-09-17 21:09:03 +05:30
David Wilson cffaa92fc4 Add rsync and git to build_docker_image too. 2017-09-17 21:07:32 +05:30
David Wilson a89c20e54e Fixup some more tests. 2017-09-17 20:33:43 +05:30
David Wilson 0f89756c1e Initial version of ModuleFinder, but it's slooooow. 2017-09-17 19:52:29 +05:30
David Wilson 6f25fac430 Log forwarded events just as the docs suggest they're logged. 2017-09-17 18:35:52 +05:30
David Wilson 60dc0f6058 Typo. 2017-09-17 18:30:06 +05:30
David Wilson 04e3336c32 issue #31: move error log to ticket.
Original commit:

commit 3535e45245
Author: David Wilson <dw@botanicus.net>
Date:   Fri Aug 26 19:35:09 2016 +0100

    Broken SSH unit test.

A  todo/six-module-import-error.py
2017-09-17 18:28:24 +05:30
David Wilson a1525b8bb8 Add subheading to describe each import hack. 2017-09-17 18:27:58 +05:30
David Wilson 38602d5eb3 Link to six bug. 2017-09-17 18:24:57 +05:30
David Wilson 700bfe31a4 Pad out shame.rst. 2017-09-17 18:22:15 +05:30
David Wilson 4110d6de60 Add instructions for running tests. 2017-09-17 18:16:23 +05:30
David Wilson cf506dfdf2 First handful of functional tests for SSH against Docker. 2017-09-17 18:11:48 +05:30
David Wilson a882497b62 ssh: Learn to type passwords and supply pubkeys.
Now ssh requires a tty allocation. This presents a scalability problem,
a future version could selectively allocate a tty only if typing
passwords is desired.

Sudo's tty handling is now moved into mitogen.master.
2017-09-17 18:11:20 +05:30
David Wilson db793003e0 Tidy. 2017-09-17 18:00:12 +05:30
David Wilson 5fd3337ed2 Add more dependencies to dev_requirements.txt. 2017-09-17 17:56:57 +05:30
David Wilson f60e8bc47d Add hacks to allow Mock to be imported. 2017-09-17 17:56:51 +05:30