Commit Graph

460 Commits

Author SHA1 Message Date
David Wilson 395b03a77d issue #549: fix setrlimit() crash and hard-wire OS X default
OS X advertised unlimited, but really it means kern.maxfilesperproc.
2019-08-02 22:30:58 +01:00
David Wilson 33bceb6eb4 issue #602: recover task_vars for synchronize and meta: reset_connection 2019-08-02 04:05:34 +01:00
David Wilson 6b4bcf4fe0 ansible: remove cutpasted docstring 2019-08-02 04:05:28 +01:00
David Wilson 619f4dee07 [linear2] merge fallout: restore optimization from #491 / 7b129e857 2019-08-02 04:05:19 +01:00
David Wilson e4321f81a0 issue #600: /etc/environment may be non-ASCII in an unknown encoding 2019-08-01 12:12:18 +01:00
David Wilson 75d179e4b9 remove unused imports flagged by lgtm 2019-07-31 11:46:23 +01:00
David Wilson c80fddd487 [linear2]: merge fallout flaggged by LGTM 2019-07-31 11:41:29 +01:00
David Wilson eeb7150f24 issue #549: increase open file limit automatically if possible
While catching every possible case where "open file limit exceeded" is
not possible, we can at least increase the soft limit to the available
hard limit without any user effort.

Do this in Ansible top-level process, even though we probably only need
it in the MuxProcess. It seems there is no reason this could hurt
2019-07-31 04:20:04 +01:00
David Wilson acab26d796 ansible: improve process.py docs 2019-07-31 04:09:48 +01:00
David Wilson 4dfbe82e76 tests: hide ugly error during Ansible tests 2019-07-31 01:50:37 +01:00
David Wilson 108015aa22 ansible: gracefully handle failure to connect to MuxProcess
It's possible to hit an ugly exception during early CTRL+C
2019-07-31 01:50:37 +01:00
David Wilson bf1f3682aa ansible: pin per-CPU muxes to their corresponding CPU
This slightly breaks the old scheme, in that CPU 1 may now end up with a
mux and the top-level process pinned to it.
2019-07-31 01:50:37 +01:00
David Wilson dc9f4e89e6 ansible: reap mux processes on shut down
Previously we exitted without calling waitpid(), which meant the
top-level process struct rusage did not reflect the resource usage
consumed by the multiplexer processes.

Existing benchmarks are made using perf so this never created a problem,
but it could be confusing to others using the "time" command, and also
allows logging the final exit status of the process.
2019-07-31 01:50:37 +01:00
David Wilson 136dee1fb4 [linear2] more merge fallout, fix Connection._mitogen_reset(mode=) 2019-07-29 17:52:44 +01:00
David Wilson a9755d4ad0 [linear2] update mitogen_get_stack for new _build_stack() return value 2019-07-29 16:30:01 +01:00
David Wilson 1fca0b7a94 [linear2] fix MuxProcess test fixture and some merge fallout 2019-07-29 16:10:36 +01:00
David Wilson 0f63ca4c68 Make setting affinity optional. 2019-07-29 13:52:30 +01:00
David Wilson 9035884c77 ansible: abstract worker process model.
Move all details of broker/router setup out of connection.py, instead
deferring it to a WorkerModel class exported by process.py via
get_worker_model(). The running strategy can override the configured
worker model via _get_worker_model().

ClassicWorkerModel is installed by default, which implements the
extension's existing process model.

Add optional support for the third party setproctitle module, so
children have pretty names in ps output.

Add optional support for per-CPU multiplexers to classic runs.
2019-07-29 13:52:30 +01:00
David Wilson 402dba4197 module_finder: pass raw file to compile()
Newer Ansibles have e.g. UTF-8 present in apt.py.
2019-07-23 16:04:44 +01:00
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 2019-07-23 14:07:00 +01:00
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
2019-07-23 14:04:22 +01:00
David Wilson 26b6333787 [stream-refactor] fix unix.Listener construction 2019-07-22 18:30:51 +01:00
Jordan Webb 1a02a86331
Add buildah transport 2019-06-08 18:15:58 -05:00
David Wilson 7ae926b325 ansible: prevent tempfile.mkstemp() leaks.
This avoids a leak present in Ansible 2.7.0..current HEAD, and all
similar leaks.

See ansible/ansible#57327.
2019-06-03 23:43:15 +01:00
David Wilson 3620fce071 issue #593: expose configurables for SSH keepalive and increase the default 2019-06-03 17:40:58 +01:00
David Wilson 0b7fd3f290 issue #591: ansible: restore CWD prior to AnsibleModule initialization. 2019-06-03 02:34:40 +01:00
David Wilson 4f23f0bec1 issue #590: update comment to indicate the hack is permanent 2019-06-02 17:52:58 +01:00
David Wilson 1a92995a24 issue #590: include nasty workaround for sys.modules junk 2019-05-30 06:00:54 +01:00
David Wilson 92b4724010 issue #587: consistent become_exe() behaviour for older Ansibles. 2019-05-27 14:59:57 +01:00
David Wilson f35194fe0f issue #587: mitogen_doas should not become_exe for doas_path
Looks like this has always been wrong - when used as a connection
method, PlayContext.become_method/become_exe may hold totally unrelated
data.
2019-05-27 14:59:57 +01:00
David Wilson c1c8d5c31e issue #587: 2.8 PlayContext lacks sudo_flags attribute.
This is a huge bodge.
2019-05-26 16:46:38 +01:00
David Wilson e11b251c75 issue #587: 2.8 PluginLoader.get() introduced new collection_list kwarg 2019-05-26 12:07:55 +01:00
David Wilson 46dde95962 issue #587: 2.8 PlayContext.connection no longer contains connection name
Not clear what the intention is here. Either need to ferret it out of
some other location, or just stop preloading the connection class in the
top-level process.
2019-05-26 12:06:53 +01:00
David Wilson 4a614c3950 issue #587: bump max Ansible version 2019-05-26 12:06:14 +01:00
David Wilson f105a81e20 ansible: descriptive version check during startup. 2019-05-19 13:13:24 +01:00
David Wilson f30a4c05c8 issue #581: expose mitogen_mask_remote_name variable. 2019-04-16 22:46:12 +01:00
David Wilson 65deb3feac issue #575: fix exception text rendering 2019-04-02 14:06:41 +01:00
David Wilson 34fb9da1be issue #570: add firewalld to always-fork list for now. 2019-03-22 19:46:08 +00:00
David Wilson 3ff6123483 issue #557: support correct cpu_set_t size 2019-03-06 17:55:31 +00:00
David Wilson 2bd0bbd4df issue #555: ansible: workaround ancient reload(sys) hack.
This is the most minimal change for what might be relatively minimal
edge case. Alternative is replacing reload(), but let's not do that yet.

Closes #555
2019-03-06 14:05:29 +00:00
David Wilson 6309774be2 issue #554: fix Ansible 2.4 compatibility 2019-03-06 12:06:55 +00:00
David Wilson 7743e57ff3 issue #554: track and remove multiple make_tmp_path() calls. 2019-03-06 10:55:07 +00:00
David Wilson 7dacb68eeb issue #552: include process identity in log messages. 2019-02-25 17:25:09 +00:00
David Wilson 26e6194d0a issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.
2019-02-19 16:26:47 +00:00
David Wilson 458a4faa97 ansible: create stub __init__.py for sdist.
This went into 0.2.5 sdist tarball but it's not checked in.
2019-02-14 01:28:14 +00:00
David Wilson 8f9c67daf1 ansible: refactor affinity class and add abstract tests. 2019-02-14 00:35:16 +00:00
David Wilson 0f30808234 ansible: quiesce boto logger; closes #541. 2019-02-13 17:28:01 +00:00
David Wilson 7fd0d34910 tests/ansible: Spec.port() test & mitogen_via= fix.
ansible_ssh_port was not respected.
2019-02-13 16:32:11 +00:00
David Wilson 1f77d24bec Update copyright year everywhere. 2019-02-13 16:16:49 +00:00
David Wilson b5b23e8f3d tests/ansible: Spec.become_pass() test. 2019-02-13 16:14:15 +00:00