92b4724010
issue #587 : consistent become_exe() behaviour for older Ansibles.
2019-05-27 14:59:57 +01:00
f35194fe0f
issue #587 : mitogen_doas should not become_exe for doas_path
...
Looks like this has always been wrong - when used as a connection
method, PlayContext.become_method/become_exe may hold totally unrelated
data.
2019-05-27 14:59:57 +01:00
c1c8d5c31e
issue #587 : 2.8 PlayContext lacks sudo_flags attribute.
...
This is a huge bodge.
2019-05-26 16:46:38 +01:00
e11b251c75
issue #587 : 2.8 PluginLoader.get() introduced new collection_list kwarg
2019-05-26 12:07:55 +01:00
46dde95962
issue #587 : 2.8 PlayContext.connection no longer contains connection name
...
Not clear what the intention is here. Either need to ferret it out of
some other location, or just stop preloading the connection class in the
top-level process.
2019-05-26 12:06:53 +01:00
4a614c3950
issue #587 : bump max Ansible version
2019-05-26 12:06:14 +01:00
f105a81e20
ansible: descriptive version check during startup.
2019-05-19 13:13:24 +01:00
f30a4c05c8
issue #581 : expose mitogen_mask_remote_name variable.
2019-04-16 22:46:12 +01:00
65deb3feac
issue #575 : fix exception text rendering
2019-04-02 14:06:41 +01:00
34fb9da1be
issue #570 : add firewalld to always-fork list for now.
2019-03-22 19:46:08 +00:00
3ff6123483
issue #557 : support correct cpu_set_t size
2019-03-06 17:55:31 +00:00
2bd0bbd4df
issue #555 : ansible: workaround ancient reload(sys) hack.
...
This is the most minimal change for what might be relatively minimal
edge case. Alternative is replacing reload(), but let's not do that yet.
Closes #555
2019-03-06 14:05:29 +00:00
6309774be2
issue #554 : fix Ansible 2.4 compatibility
2019-03-06 12:06:55 +00:00
7743e57ff3
issue #554 : track and remove multiple make_tmp_path() calls.
2019-03-06 10:55:07 +00:00
7dacb68eeb
issue #552 : include process identity in log messages.
2019-02-25 17:25:09 +00:00
26e6194d0a
issue #548 : always treat transport=smart as 'ssh' for mitogen_via=.
...
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.
2019-02-19 16:26:47 +00:00
458a4faa97
ansible: create stub __init__.py for sdist.
...
This went into 0.2.5 sdist tarball but it's not checked in.
2019-02-14 01:28:14 +00:00
8f9c67daf1
ansible: refactor affinity class and add abstract tests.
2019-02-14 00:35:16 +00:00
0f30808234
ansible: quiesce boto logger; closes #541 .
2019-02-13 17:28:01 +00:00
7fd0d34910
tests/ansible: Spec.port() test & mitogen_via= fix.
...
ansible_ssh_port was not respected.
2019-02-13 16:32:11 +00:00
1f77d24bec
Update copyright year everywhere.
2019-02-13 16:16:49 +00:00
b5b23e8f3d
tests/ansible: Spec.become_pass() test.
2019-02-13 16:14:15 +00:00
ae5a471e31
issue #539 : disable logger propagation.
2019-02-13 03:48:56 +00:00
1c955a9876
ansible: capture stderr stream of async tasks. Closes #540 .
2019-02-13 03:36:39 +00:00
7ff4e6694c
issue #536 : rework how 2.3-compatible simplejson is served
...
Regardless of the version of simplejson loaded in the master, load up
the ModuleResponder cache with our 2.4-compatible version.
To cope with simplejson being loaded due to modules like ec2_group that
try to import it before importing 'json', also update target.py to
remove it from the whitelist if a local 'json' module import succeeds.
2019-02-13 02:16:49 +00:00
8ae6ca1d5b
tests/ansible: Spec.become_method() test & mitogen_via= fix.
...
ansible_become_method hostvar was not taken into account.
2019-02-12 16:32:11 +00:00
d1cadf8ac8
tests/ansible: Spec.password() test, document interactive pw limitation.
2019-02-12 16:04:05 +00:00
21ad299d7b
tests/ansible: Spec.remote_user() test & mitogen_via= fix.
...
ansible_ssh_user precedence was incorrect.
2019-02-12 14:52:36 +00:00
748f5f675d
tests/ansible: Spec.remote_addr() test & mitogen_via= fix.
...
ansible_ssh_host was not respected.
2019-02-12 14:45:46 +00:00
e1df98168c
issue #536 : add mitogen_via= tests too.
2019-02-12 02:12:39 +00:00
604b418412
ansible: fix a crash on 2.3 when mitogen_via= host is missing.
2019-02-12 02:10:32 +00:00
001e3fee86
issue #536 : restore correct Python interpreter selection behaviour.
2019-02-12 00:12:28 +00:00
05b1ccb658
ansible: stash PID files in CWD if requested for debugging.
2019-02-09 22:20:47 +00:00
eb67fbe9d2
ansible: double the default pool size.
...
Tempted to push this up to 64, but let's do it incrementally just in
case.
2019-02-09 22:20:40 +00:00
b89e53fd70
ansible: raise error with correct exception type.
2019-02-09 22:20:40 +00:00
0e193c223c
issue #508 : master: minify all Mitogen/ansible_mitogen sources.
...
Minify-safe files are marked with a magical "# !mitogen: minify_safe"
comment anywhere in the file, which activates the minifier. The result
is naturally cached by ModuleResponder, therefore lru_cache is gone too.
Given:
import os, mitogen
@mitogen.main()
def main(router):
c = router.ssh(hostname='k3')
c.call(os.getpid)
router.sudo(via=c)
SSH footprint drops from 56.2 KiB to 42.75 KiB (-23.9%)
Ansible "shell: hostname" drops 149.26 KiB to 117.42 KiB (-21.3%)
2019-02-09 22:20:34 +00:00
7badb4a25b
ansible: hacky parser to alow bools to be specified on command line
2019-02-09 22:04:19 +00:00
b499fbe29b
ansible: add mitogen_ssh_compression variable.
2019-02-09 22:04:19 +00:00
a2ae4ed696
SyntaxError.
2019-02-09 22:04:19 +00:00
a9d48a8fdc
ansible: don't pin controller if <4 cores.
2019-02-09 22:04:19 +00:00
4531338b12
ansible: document and make affinity stuff portable to non-Linux
...
Portable as in does nothing for the time at least for now.
2019-02-09 22:04:18 +00:00
de5c050707
ansible: fix affinity.py test failure on 2 cores.
2019-02-09 22:04:18 +00:00
00ae90b2b2
ansible: preheat PluginLoader caches before fork.
...
This has been broken for some time, but somehow it has become noticeable
on recent Ansible.
loop-100-tasks.yml before:
15.532724001 seconds time elapsed
8.453850000 seconds user
5.808627000 seconds sys
loop-100-tasks.yml after:
8.991635735 seconds time elapsed
5.059232000 seconds user
2.578842000 seconds sys
2019-02-09 22:04:18 +00:00
7b129e8576
ansible: use Poller for WorkerProcess; closes #491 .
2019-02-09 22:04:18 +00:00
c6d5aa29ba
ansible: new multiplexer/workers configuration
...
Following on from 152effc26c9a5918cb7ead7a97fe7fa7f81b6764,
* Pin mux to CPU 0
* Pin top-level CPU 1
* Pin workers sequentially to CPU 2..n
Nets 19.5% improvement on issue_140__thread_pileup.yml when targetting
64 Docker containers on the same 8 core/16 thread machine.
Before (prior to last scheme, no affinity at all):
2294528.731458 task-clock (msec) # 6.443 CPUs utilized
10,429,745 context-switches # 0.005 M/sec
2,049,618 cpu-migrations # 0.893 K/sec
8,258,952 page-faults # 0.004 M/sec
5,532,719,253,824 cycles # 2.411 GHz (83.35%)
3,267,471,616,230 instructions # 0.59 insn per cycle
# 1.22 stalled cycles per insn (83.35%)
662,006,455,943 branches # 288.515 M/sec (83.33%)
39,453,895,977 branch-misses # 5.96% of all branches (83.37%)
356.148064576 seconds time elapsed
After:
2226463.958975 task-clock (msec) # 7.784 CPUs utilized
9,831,466 context-switches # 0.004 M/sec
180,065 cpu-migrations # 0.081 K/sec
5,082,278 page-faults # 0.002 M/sec
5,592,548,587,259 cycles # 2.512 GHz (83.35%)
3,135,038,855,414 instructions # 0.56 insn per cycle
# 1.32 stalled cycles per insn (83.32%)
636,397,509,232 branches # 285.833 M/sec (83.30%)
39,135,441,790 branch-misses # 6.15% of all branches (83.35%)
286.036681644 seconds time elapsed
2019-02-09 22:04:18 +00:00
1b909e8697
ansible: pin connection multiplexer to a single core
...
Nets a reliable 8% improvement in issue_140__thread_pileup.yml when
targetting 64 Docker containers on the same 8 core/16 thread machine.
Before:
2294528.731458 task-clock (msec) # 6.443 CPUs utilized
10,429,745 context-switches # 0.005 M/sec
2,049,618 cpu-migrations # 0.893 K/sec
8,258,952 page-faults # 0.004 M/sec
5,532,719,253,824 cycles # 2.411 GHz (83.35%)
4,001,276,805,120 stalled-cycles-frontend # 72.32% frontend cycles idle (83.30%)
2,024,159,442,463 stalled-cycles-backend # 36.59% backend cycles idle (66.65%)
3,267,471,616,230 instructions # 0.59 insn per cycle
# 1.22 stalled cycles per insn (83.35%)
662,006,455,943 branches # 288.515 M/sec (83.33%)
39,453,895,977 branch-misses # 5.96% of all branches (83.37%)
356.148064576 seconds time elapsed
After:
2208247.938562 task-clock (msec) # 6.735 CPUs utilized
8,489,840 context-switches # 0.004 M/sec
1,432,967 cpu-migrations # 0.649 K/sec
7,508,957 page-faults # 0.003 M/sec
5,477,293,750,357 cycles # 2.480 GHz (83.31%)
3,984,360,350,811 stalled-cycles-frontend # 72.74% frontend cycles idle (83.32%)
1,976,646,418,711 stalled-cycles-backend # 36.09% backend cycles idle (66.64%)
3,196,197,480,792 instructions # 0.58 insn per cycle
# 1.25 stalled cycles per insn (83.36%)
648,247,332,967 branches # 293.557 M/sec (83.35%)
39,004,881,070 branch-misses # 6.02% of all branches (83.37%)
327.876903668 seconds time elapsed
2019-02-09 22:04:18 +00:00
e587396e70
ansible: hook strategy and worker processes into profiler
2019-02-09 22:04:18 +00:00
84944a9a61
ansible: ensure MuxProcess MITOGEN_PROFILING results reach disk.
...
This has been broken for quite some time.
2019-02-09 22:04:18 +00:00
954f874085
issue #527 : catch new-style module tracebacks like vanilla.
2019-02-09 19:43:35 +00:00
a1121c5a84
issue #499 : respect C.BECOME_ALLOW_SAME_USER.
2019-01-29 02:53:21 +00:00
David Wilson