Commit Graph

448 Commits

Author SHA1 Message Date
David Wilson dc9f4e89e6 ansible: reap mux processes on shut down
Previously we exitted without calling waitpid(), which meant the
top-level process struct rusage did not reflect the resource usage
consumed by the multiplexer processes.

Existing benchmarks are made using perf so this never created a problem,
but it could be confusing to others using the "time" command, and also
allows logging the final exit status of the process.
2019-07-31 01:50:37 +01:00
David Wilson 136dee1fb4 [linear2] more merge fallout, fix Connection._mitogen_reset(mode=) 2019-07-29 17:52:44 +01:00
David Wilson a9755d4ad0 [linear2] update mitogen_get_stack for new _build_stack() return value 2019-07-29 16:30:01 +01:00
David Wilson 1fca0b7a94 [linear2] fix MuxProcess test fixture and some merge fallout 2019-07-29 16:10:36 +01:00
David Wilson 0f63ca4c68 Make setting affinity optional. 2019-07-29 13:52:30 +01:00
David Wilson 9035884c77 ansible: abstract worker process model.
Move all details of broker/router setup out of connection.py, instead
deferring it to a WorkerModel class exported by process.py via
get_worker_model(). The running strategy can override the configured
worker model via _get_worker_model().

ClassicWorkerModel is installed by default, which implements the
extension's existing process model.

Add optional support for the third party setproctitle module, so
children have pretty names in ps output.

Add optional support for per-CPU multiplexers to classic runs.
2019-07-29 13:52:30 +01:00
David Wilson 402dba4197 module_finder: pass raw file to compile()
Newer Ansibles have e.g. UTF-8 present in apt.py.
2019-07-23 16:04:44 +01:00
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 2019-07-23 14:07:00 +01:00
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
2019-07-23 14:04:22 +01:00
David Wilson 26b6333787 [stream-refactor] fix unix.Listener construction 2019-07-22 18:30:51 +01:00
Jordan Webb 1a02a86331
Add buildah transport 2019-06-08 18:15:58 -05:00
David Wilson 7ae926b325 ansible: prevent tempfile.mkstemp() leaks.
This avoids a leak present in Ansible 2.7.0..current HEAD, and all
similar leaks.

See ansible/ansible#57327.
2019-06-03 23:43:15 +01:00
David Wilson 3620fce071 issue #593: expose configurables for SSH keepalive and increase the default 2019-06-03 17:40:58 +01:00
David Wilson 0b7fd3f290 issue #591: ansible: restore CWD prior to AnsibleModule initialization. 2019-06-03 02:34:40 +01:00
David Wilson 4f23f0bec1 issue #590: update comment to indicate the hack is permanent 2019-06-02 17:52:58 +01:00
David Wilson 1a92995a24 issue #590: include nasty workaround for sys.modules junk 2019-05-30 06:00:54 +01:00
David Wilson 92b4724010 issue #587: consistent become_exe() behaviour for older Ansibles. 2019-05-27 14:59:57 +01:00
David Wilson f35194fe0f issue #587: mitogen_doas should not become_exe for doas_path
Looks like this has always been wrong - when used as a connection
method, PlayContext.become_method/become_exe may hold totally unrelated
data.
2019-05-27 14:59:57 +01:00
David Wilson c1c8d5c31e issue #587: 2.8 PlayContext lacks sudo_flags attribute.
This is a huge bodge.
2019-05-26 16:46:38 +01:00
David Wilson e11b251c75 issue #587: 2.8 PluginLoader.get() introduced new collection_list kwarg 2019-05-26 12:07:55 +01:00
David Wilson 46dde95962 issue #587: 2.8 PlayContext.connection no longer contains connection name
Not clear what the intention is here. Either need to ferret it out of
some other location, or just stop preloading the connection class in the
top-level process.
2019-05-26 12:06:53 +01:00
David Wilson 4a614c3950 issue #587: bump max Ansible version 2019-05-26 12:06:14 +01:00
David Wilson f105a81e20 ansible: descriptive version check during startup. 2019-05-19 13:13:24 +01:00
David Wilson f30a4c05c8 issue #581: expose mitogen_mask_remote_name variable. 2019-04-16 22:46:12 +01:00
David Wilson 65deb3feac issue #575: fix exception text rendering 2019-04-02 14:06:41 +01:00
David Wilson 34fb9da1be issue #570: add firewalld to always-fork list for now. 2019-03-22 19:46:08 +00:00
David Wilson 3ff6123483 issue #557: support correct cpu_set_t size 2019-03-06 17:55:31 +00:00
David Wilson 2bd0bbd4df issue #555: ansible: workaround ancient reload(sys) hack.
This is the most minimal change for what might be relatively minimal
edge case. Alternative is replacing reload(), but let's not do that yet.

Closes #555
2019-03-06 14:05:29 +00:00
David Wilson 6309774be2 issue #554: fix Ansible 2.4 compatibility 2019-03-06 12:06:55 +00:00
David Wilson 7743e57ff3 issue #554: track and remove multiple make_tmp_path() calls. 2019-03-06 10:55:07 +00:00
David Wilson 7dacb68eeb issue #552: include process identity in log messages. 2019-02-25 17:25:09 +00:00
David Wilson 26e6194d0a issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.
2019-02-19 16:26:47 +00:00
David Wilson 458a4faa97 ansible: create stub __init__.py for sdist.
This went into 0.2.5 sdist tarball but it's not checked in.
2019-02-14 01:28:14 +00:00
David Wilson 8f9c67daf1 ansible: refactor affinity class and add abstract tests. 2019-02-14 00:35:16 +00:00
David Wilson 0f30808234 ansible: quiesce boto logger; closes #541. 2019-02-13 17:28:01 +00:00
David Wilson 7fd0d34910 tests/ansible: Spec.port() test & mitogen_via= fix.
ansible_ssh_port was not respected.
2019-02-13 16:32:11 +00:00
David Wilson 1f77d24bec Update copyright year everywhere. 2019-02-13 16:16:49 +00:00
David Wilson b5b23e8f3d tests/ansible: Spec.become_pass() test. 2019-02-13 16:14:15 +00:00
David Wilson ae5a471e31 issue #539: disable logger propagation. 2019-02-13 03:48:56 +00:00
David Wilson 1c955a9876 ansible: capture stderr stream of async tasks. Closes #540. 2019-02-13 03:36:39 +00:00
David Wilson 7ff4e6694c issue #536: rework how 2.3-compatible simplejson is served
Regardless of the version of simplejson loaded in the master, load up
the ModuleResponder cache with our 2.4-compatible version.

To cope with simplejson being loaded due to modules like ec2_group that
try to import it before importing 'json', also update target.py to
remove it from the whitelist if a local 'json' module import succeeds.
2019-02-13 02:16:49 +00:00
David Wilson 8ae6ca1d5b tests/ansible: Spec.become_method() test & mitogen_via= fix.
ansible_become_method hostvar was not taken into account.
2019-02-12 16:32:11 +00:00
David Wilson d1cadf8ac8 tests/ansible: Spec.password() test, document interactive pw limitation. 2019-02-12 16:04:05 +00:00
David Wilson 21ad299d7b tests/ansible: Spec.remote_user() test & mitogen_via= fix.
ansible_ssh_user precedence was incorrect.
2019-02-12 14:52:36 +00:00
David Wilson 748f5f675d tests/ansible: Spec.remote_addr() test & mitogen_via= fix.
ansible_ssh_host was not respected.
2019-02-12 14:45:46 +00:00
David Wilson e1df98168c issue #536: add mitogen_via= tests too. 2019-02-12 02:12:39 +00:00
David Wilson 604b418412 ansible: fix a crash on 2.3 when mitogen_via= host is missing. 2019-02-12 02:10:32 +00:00
David Wilson 001e3fee86 issue #536: restore correct Python interpreter selection behaviour. 2019-02-12 00:12:28 +00:00
David Wilson 05b1ccb658 ansible: stash PID files in CWD if requested for debugging. 2019-02-09 22:20:47 +00:00
David Wilson eb67fbe9d2 ansible: double the default pool size.
Tempted to push this up to 64, but let's do it incrementally just in
case.
2019-02-09 22:20:40 +00:00