Commit Graph

1123 Commits

Author SHA1 Message Date
David Wilson 7a592d1c34 core: better Poller.__repr__ 2018-05-24 23:30:56 +01:00
David Wilson b0ce6eecd7 fork: support on_start= argument. 2018-05-23 16:25:17 +01:00
David Wilson 00edf0d66d core: have ExternalContext accept a config dict rather than kwargs.
The parameter lists had gotten out of control.
2018-05-23 16:24:38 +01:00
David Wilson eac4cc7afe tests: speed things up 2018-05-18 10:58:25 +01:00
David Wilson 863a95e860 docs: update contributors 2018-05-18 10:58:25 +01:00
David Wilson b0aa413173 tests: import benchmark reproduction setup playbook
Incomplete, also needs to mess around with Gcloud routing and handle box
setup/teardown, because that's another "well engineered" waste of time.
2018-05-18 10:58:25 +01:00
David Wilson a99c7a12f9 tests: output split groups in gcloud.py 2018-05-15 23:41:02 +00:00
David Wilson 1a0e630a80 tests: add debops to requirements 2018-05-15 22:15:49 +01:00
David Wilson dbcee4041a tests: refactor gcloud.py to be dynamic inventory. 2018-05-15 22:10:36 +01:00
David Wilson a3995f8e5f ansible: remove hard-coded dw username. 2018-05-15 21:34:13 +01:00
David Wilson 1eb5c20f57 ansible: add dummy init.pys so setup.py find_packages() DTRT. 2018-05-15 21:29:00 +01:00
David Wilson 2c58591129 Bump Ansible version in dev_requirements.txt 2018-05-15 21:17:48 +01:00
David Wilson 7d0209d8de issue #249: have upgrade_router() upgrade the poller too.
Now when a child becomes a parent, it gets a new poller suitable for
many more children than was possible using select().
2018-05-15 09:28:03 +01:00
David Wilson 55fff54774 core: make try/catch logic a little clearer in Latch.get() 2018-05-15 09:22:35 +01:00
David Wilson 05a5f2b6e5 core: if Poller.poll() fails, TimeoutError would be raised.
We must check whether poller threw an exception both in the case that we
weren't woken and the case that we were.
2018-05-15 04:18:55 +01:00
David Wilson 5bdc1719c5 issue #249: epoll() raises IOError for EINTR, not select.error. 2018-05-15 04:15:32 +01:00
David Wilson 70376d861a issue #217: docs: remove limitation 2018-05-15 01:57:09 +01:00
David Wilson 6d18ce81d8 issue #249: restore duplex behaviour for epoll
With epoll() there is only one kernel-side object per file descriptor,
which is why _control() is such a pain. Since we merge receive/transmit
watching into that single object, we must always test the mask for both
conditions when reading results.

Kqueue isn't/doesn't appear to be like this. The identity of a Kqueue
event is keyed on (fd, filter), and we register a separate event for
both transmit and receive, so the 'elif' in KqueuePoller.poll() does not
appear to need to change.

Previously, a FD marked for read+write would not indicate writeability
until it was no longer readable.
2018-05-15 01:49:03 +01:00
David Wilson 6b98818046 issue #249: epoll distinguishes between hangup and disconnect
..typical Linux, for certain descriptor types only. So our receive mask
must match both, and normalize it into a read event like every other
poller.
2018-05-15 01:32:46 +01:00
David Wilson 07056b0dd1 issue #249: fix ordering bug masked by previous implementation 2018-05-15 00:27:41 +01:00
David Wilson 4cd9e09130 issue #249: docs: remove limitation 2018-05-15 00:22:02 +01:00
David Wilson 36a1024861 issue #249: port Latch to poller too.
This is probably going to suck for perf :/
2018-05-15 00:21:02 +01:00
David Wilson 1070dfae72 issue #249: fix iter_read() regression. 2018-05-15 00:17:38 +01:00
David Wilson dcf0aa351e issue #249: whoops, fix new poller timeouts. 2018-05-15 00:15:32 +01:00
David Wilson aa8f786413 issue #249: fix Poller.from_existing() for v2 API 2018-05-15 00:11:08 +01:00
David Wilson 5645629e5d issue #249: the new pollers must handle syscall restarts too. 2018-05-15 00:06:31 +01:00
David Wilson 4df020827d issue #249: explicitly close pollers when done. 2018-05-15 00:01:01 +01:00
David Wilson 9905f6d8b4 issue #249: make write_all() and iter_read() use PREFERRED_POLLER. 2018-05-14 23:56:11 +01:00
David Wilson b6124f8396 issue #249: EpollPoller v2. 2018-05-14 22:50:35 +00:00
David Wilson 9abcf63155 issue #249: Poller API v2 (BSD only).
Now it's BasicStream/Side-agnostic, so it can be reused for Latch and
iter_read().
2018-05-14 23:32:26 +01:00
David Wilson 11c2e4ab3e core: set _v and _vv to True in enable_debug_logging().
router.enable_debug() has been broken for ages.
2018-05-14 22:58:56 +01:00
David Wilson 7320c542df issue #249: EpollPoller() for Linux. 2018-05-14 17:52:08 +00:00
David Wilson bc7be1879d issue #249: initial poller implementation (BSD only) 2018-05-14 18:31:38 +01:00
David Wilson b7ab473343 issue #186: split handle list up so it makes sense 2018-05-13 17:05:09 +01:00
David Wilson d1a22cb5d4 issue #186: parent: implement FORWARD_MODULE.
To support detach, we must be able to preload the target with every
module it will need prior to detachment. This implements the
intermediary part of the process (i.e. the Ansible fork parent) --
receiving LOAD_MODULE/FORWARD_MODULE pairs and ensuring they reach the
child.
2018-05-13 16:58:47 +01:00
David Wilson 8a089e975d docs: Document Router.unidirectional. 2018-05-13 16:13:50 +01:00
David Wilson bb61745a1a issue #217: pass through non-custom module utils to regular importer.
This may come back to bite later, but in the meantime it avoids shipping
up to 12KiB of junk metadata for every single task invocation.

For detachment (aka. async), we must ensure the target has two types of
preloads completed (modules and module_utils files) before detaching.
2018-05-13 04:40:07 +01:00
David Wilson 30034877a5 issue #217: ansible: working, if extremely inefficient implementation 2018-05-13 01:47:11 +01:00
David Wilson 81b62d9a1a issue #217: ansible: beginnings of ModuleDepService. 2018-05-12 15:12:39 +01:00
David Wilson 0f8190eff6 issue #217: ansible: working module_finder.py 2018-05-12 15:12:35 +01:00
David Wilson 6a17042020 issue #217: ansible: merge duplicate ReplacerPlanner. 2018-05-12 15:12:29 +01:00
David Wilson 78d375e0c5 core: CallError should handle any exception type.
Previously SystemExit would be pickled.
2018-05-12 00:51:50 +01:00
David Wilson faaac43a78 Disable SSH variables test on vanilla Ansible. 2018-05-12 00:51:50 +01:00
David Wilson 68d6e6bf34 ansible: tests for all SSH variables.
Closes #184.
2018-05-12 00:51:50 +01:00
dw f01f237e6d
Merge pull request #247 from dw/issue179
issue #179: don't reap first stage until core_src_fd is drained.
2018-05-11 22:18:31 +01:00
David Wilson def22c226f issue #179: don't reap first stage until core_src_fd is drained.
Bootstrap would hang if (as of writing) a pipe sufficient to hold 42,006
bytes was not handed out by the kernel to the first stage. It was luck
that this didn't manifest before, as first stage could write the full
source and exit completely before reading begun.

It is not clear under which circumstances this could previously occur,
but at least since Linux 4.5, it can be triggered if
/proc/sys/fs/pipe-max-size is reduced from the default of 1MiB, which
can have the effect of capping the default pipe buffer size of 64KiB to
something lower.

Suspicion is that buffer pipe size could also be reduced under memory
pressure, as reference to busy machines appeared a few times in the bug
report.
2018-05-11 21:58:38 +01:00
dw 3ff37113e2
Merge pull request #246 from dw/issue179
issue #179: ec0_receive() and connect_bootstrap() should use deadline.
2018-05-11 18:33:13 +01:00
David Wilson d65e047b12 issue #179: ec0_receive() and connect_bootstrap() should use deadline.
Now there is a single global deadline derived from ansible.cfg timeout=
value.
2018-05-11 18:19:27 +01:00
dw 2c309a378a
Merge pull request #245 from dw/dmw
Log masking, unlinked contributors page, resolv.conf fix, unidirectional routing
2018-05-08 16:24:06 +01:00
David Wilson bd2cc0830c Enable unidirectional routing in Ansible; closes #132. 2018-05-08 16:10:44 +01:00