Commit Graph

2653 Commits

Author SHA1 Message Date
David Wilson 93342ba60c Normalize docstring formatting 2019-07-28 10:40:59 +00:00
David Wilson 4e6aadc40a [stream-refactor] fix LogHandler.uncork() race
During early initialization under hackbench, it is possible for Broker
to be in LogHandler._send() while the main thread has already destroyed
_buffer. So we must synchronize them, but only while the handler is
corked.
2019-07-28 10:40:59 +00:00
David Wilson 90c989ee59 [stream-refactor] BufferedWriter must disconenct Stream, not Protocol
Fix a race where if Stream.on_receive() detects disconnect, it calls
Stream.on_disconnect(), which fires Stream 'disconnect' event, whereas
if BufferedWriter.on_transmit() detects disconnect, it called
Protocol.on_disconnect(), which did not fire the Stream 'disconnect'
event.

Since mitogen.parent listens on Stream's 'disconnect' event to reap
children, this was causing a very difficult to trigger test failure.

Triggered after <1000 runs on a Xeon E5530 with hyperthreading using
hackbench running at the same priority:

    $ hackbench -s 1048576 -l 100000000000 -g 4
2019-07-28 10:40:59 +00:00
David Wilson 5f7ab220cb [stream-refactor] statically link doas binary using musl
So it can run on CentOS 5
2019-07-28 10:40:59 +00:00
David Wilson 462c4ff59f [stream-refactor] stop writing to /tmp/foo in fd_check.py. 2019-07-28 10:40:59 +00:00
David Wilson cd0a557602 [stream-refactor] yet another 2.4 issue in create_child_test 2019-07-28 10:40:59 +00:00
David Wilson 65e31f63fe [stream-refactor] fix Py2.4 failure by implementing missing Timer method 2019-07-28 10:40:59 +00:00
David Wilson 54987100b2 [stream-refactor] allow up to 30 seconds to connect in unix_test
It reliably fails when running on a (intentionally) heavily loaded
machine
2019-07-28 10:40:59 +00:00
David Wilson db9066fbfb [stream-refactor] mark setns module as requiring Python >2.4 2019-07-28 10:40:59 +00:00
David Wilson 856dfcebcd [stream-refactor] another 2.4 fix for create_child_test 2019-07-28 10:40:59 +00:00
David Wilson 054643783c .travis.yml: Add reverse shell spawn for Travis too 2019-07-28 10:40:59 +00:00
David Wilson 11ae6f3873 core: better Side attribute docstrings 2019-07-28 10:40:59 +00:00
David Wilson bed5931194 [stream-refactor] remove one more getuser() usage 2019-07-28 10:40:59 +00:00
David Wilson 2f950b3bda [stream-refactor] allow doas_test to succeed on CentOS
Unlike on Debian, some environment variables that tickle
getpass.getuser() are being inherited. So use getuid() instead.

Also install the doas binary on CentOS. CI was changed (I believe) to
shrink the configuration matrix, and now these tests run on CentOS too.
2019-07-28 10:40:59 +00:00
David Wilson 2ba3973bc5 Pin idna==2.7 when running on Python<2.7. 2019-07-28 10:40:59 +00:00
David Wilson 07f3179e58 [stream-refactor] Py2.4 compat fix for iter_split_test. 2019-07-28 10:40:59 +00:00
David Wilson f0065d76d8 [stream-refactor] add descriptive task names to _container_prep 2019-07-28 10:40:59 +00:00
David Wilson fdf3484a2a [stream-refactor] 3.x socket.send() requires bytes 2019-07-28 10:40:30 +00:00
David Wilson c09bbdc2f9 [stream-refactor] fix 2.4 syntax error. 2019-07-23 18:33:35 +01:00
David Wilson bd80d4b0af [stream-refactor] avoid os.wait3() for Py2.4. 2019-07-23 17:30:46 +01:00
David Wilson 321dac3046 Allow specifying -vvv to debops_tests. 2019-07-23 17:25:53 +01:00
David Wilson b1379e6f45 [stream-refactor] send MITO002 earlier
Prevents 2.4 bootstrap from attempting to fetch os_fork too early.

Connection(None).connect(): pid:25098 stdin:81 stdout:81 stderr:79
ssh.localhost:2201: (partial): mitogen__has_sudo_nopw@localhost's password:
ssh.localhost:2201: (password prompt): mitogen__has_sudo_nopw@localhost's password:
ssh.localhost:2201: (unrecognized): mitogen__has_sudo_nopw@localhost's password:
BootstrapProtocol(ssh.localhost:2201): first stage started succcessfully
BootstrapProtocol(ssh.localhost:2201): first stage received bootstrap
ssh.localhost:2201: (partial): MIdmitogen.os_fork
ssh.localhost:2201: (unrecognized partial): MIdmitogen.os_fork
ssh.localhost:2201: failing connection due to TimeoutError(u'Failed to setup connection after 10.00 seconds',)
2019-07-23 17:24:59 +01:00
David Wilson 402dba4197 module_finder: pass raw file to compile()
Newer Ansibles have e.g. UTF-8 present in apt.py.
2019-07-23 16:04:44 +01:00
David Wilson 4eecc08047 [stream-refactor] merge stdout+stderr when reporting EofError
Fixes sudo regression
2019-07-23 15:40:12 +01:00
David Wilson 1d2bfc28da [stream-refactor] fix crash in detach() / during async/multiple_items_loop.yml 2019-07-23 14:17:03 +01:00
David Wilson 93abbcaf7a [stream-refactor] fix crash in runner/forking_active.yml 2019-07-23 14:13:59 +01:00
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 2019-07-23 14:07:00 +01:00
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
2019-07-23 14:04:22 +01:00
David Wilson 6e33de7cd2 unix: ensure mitogen.context_id is reset when client disconnects
To ensure a test process can successfully recreate an Ansible
MuxProcess, reset fork-inherited globals during disconnection.

There is basically no good place for this. Per the comments on #91, it
would be far better if the context's identity was tied to its router,
rather than some global variable.
2019-07-23 14:01:57 +01:00
David Wilson 7c4621a010 [stream-refactor] make syntax 2.4 compatible 2019-07-22 21:35:44 +01:00
David Wilson 2ce3383a01 [stream-refactor] make trusty our Travis dist.
They updated to xenial recently, and it no longer supports Py2.6.
2019-07-22 21:32:31 +01:00
David Wilson 0ff5fb8fc4 [stream-refactor] fix su_test failure (issue #363) 2019-07-22 21:06:53 +01:00
David Wilson 8769c3ce24 [stream-refactor] more readable log string format 2019-07-22 21:06:35 +01:00
David Wilson d411003b64 [stream-refactor] dont doubly log last partial line 2019-07-22 21:06:21 +01:00
David Wilson 869e04af10 [stream-refactor] import fd_check.py used by create_child_test 2019-07-22 20:38:14 +01:00
David Wilson 1069ca43d6 [stream-refactor] port mitogen.buildah, added to master since work began 2019-07-22 19:34:08 +01:00
David Wilson 26b6333787 [stream-refactor] fix unix.Listener construction 2019-07-22 18:30:51 +01:00
David Wilson 1fb3852fa6 [stream-refactor] fix crash when no stderr present. 2019-07-22 18:30:51 +01:00
David Wilson 4b0870aa6e [stream-refactor] fix Process constructor invocation 2019-07-22 18:30:51 +01:00
David Wilson 8e9f47a2e9 Add tests/ansible/.*.pid to gitignore (for ansible_mitogen/process.py) 2019-07-22 18:30:51 +01:00
David Wilson 2e371c96b4 Add extra/ to gitignore 2019-07-22 18:30:51 +01:00
David Wilson 29c63f56ae import release-notes script. 2019-07-22 18:30:51 +01:00
David Wilson f43e24e970 [stream-refactor] repaired rest of create_child_test. 2019-07-22 18:30:51 +01:00
David Wilson f039c81bb0 [stream-refactor] rename Process attrs, fix up more create_child_test 2019-07-22 18:30:51 +01:00
David Wilson cfe337b3c0 [stream-refactor] import incomplete create_child_test 2019-07-22 18:30:51 +01:00
David Wilson 9c38093aa7 issue #482: tests: check for zombie process after test. 2019-07-22 18:30:51 +01:00
David Wilson 3a1125a7bd issue #363: add test. 2019-07-22 18:30:51 +01:00
David Wilson baafc746fe tests: clean up old-style SSH exception catch 2019-07-22 18:30:51 +01:00
David Wilson 4524f03a48 issue #271: add mitogen__permdenied user to Docker image. 2019-07-22 18:30:51 +01:00
David Wilson acade4ce88 ssh: fix issue #271 regression due to refactor, add test. 2019-07-22 18:30:51 +01:00