Commit Graph

238 Commits

Author SHA1 Message Date
David Wilson 633585524f tests: don't need separate module for id_allocation_test any more
This used to be because everything would explode while importing
__main__ under py.test, but that was fixed months ago.
2018-05-28 04:58:45 +01:00
David Wilson ddf28987a0 master: split Select() into new module to reduce wire size.
service.py currently imports master.py(+parent.py) just to get Select().
2018-05-24 23:59:10 +01:00
David Wilson b0ce6eecd7 fork: support on_start= argument. 2018-05-23 16:25:17 +01:00
David Wilson 00edf0d66d core: have ExternalContext accept a config dict rather than kwargs.
The parameter lists had gotten out of control.
2018-05-23 16:24:38 +01:00
David Wilson eac4cc7afe tests: speed things up 2018-05-18 10:58:25 +01:00
David Wilson b0aa413173 tests: import benchmark reproduction setup playbook
Incomplete, also needs to mess around with Gcloud routing and handle box
setup/teardown, because that's another "well engineered" waste of time.
2018-05-18 10:58:25 +01:00
David Wilson a99c7a12f9 tests: output split groups in gcloud.py 2018-05-15 23:41:02 +00:00
David Wilson dbcee4041a tests: refactor gcloud.py to be dynamic inventory. 2018-05-15 22:10:36 +01:00
David Wilson a3995f8e5f ansible: remove hard-coded dw username. 2018-05-15 21:34:13 +01:00
David Wilson 2c58591129 Bump Ansible version in dev_requirements.txt 2018-05-15 21:17:48 +01:00
David Wilson 78d375e0c5 core: CallError should handle any exception type.
Previously SystemExit would be pickled.
2018-05-12 00:51:50 +01:00
David Wilson faaac43a78 Disable SSH variables test on vanilla Ansible. 2018-05-12 00:51:50 +01:00
David Wilson 68d6e6bf34 ansible: tests for all SSH variables.
Closes #184.
2018-05-12 00:51:50 +01:00
David Wilson 356647bef4 issue #132: initial unidirectional routing mode. 2018-05-08 16:06:08 +01:00
David Wilson 2a4157723a tests: fix breakage from moving is_stdlib_name() 2018-05-08 16:00:59 +01:00
David Wilson 7d9b7eec0c tests: must wrap log_to_file() in is_master
Without it, end up with a billion loggers attached in some tests
2018-05-08 14:32:32 +01:00
David Wilson 2ad0d0521d ansible: reload /etc/resolv.conf for each task.
The OpenShift installer modifies /etc/resolv.conf then tests the new
resolver configuration, however, there was no mechanism to reload
resolv.conf in our reuseable interpreter.

https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/openshift_web_console/tasks/install.yml#L137

This inserts an explicit call to res_init() for every new style
invocation, with an approximate cost of ~1usec on Linux since glibc
verifies resolv.conf has changed before reloading it.

There is little to be done for users of the thread-safe resolver APIs,
their state is hidden from us. If bugs like that manifest, whack-a-mole
style 'del sys.modules[thatmod]' patches may suffice.
2018-05-07 23:14:27 +01:00
David Wilson 60781ecab6 tests: fix test for vanilla Ansible. 2018-05-06 09:03:14 +01:00
David Wilson eb56a8f98d tests: add debug 2018-05-06 08:51:13 +01:00
David Wilson 267f787d20 issue #217: tests: import (unused) module_utils tests. 2018-05-05 04:00:21 +01:00
David Wilson ee741da2b3 tests: reenable remaining Ansible tests. 2018-05-04 23:51:01 +01:00
David Wilson cecef992b0 issue #218: core: add Secret and Blob types. 2018-05-04 21:34:11 +01:00
David Wilson be5c03c152 tests: many test fixes. Travis broken for >1 week. 2018-05-04 20:28:05 +01:00
David Wilson 69f58875c8 tests: more ansible fixes from recent brakage. 2018-05-04 17:55:23 +01:00
David Wilson b20174729d issue #199: fix readonly_homedir test. 2018-05-04 17:43:06 +01:00
David Wilson f98279fc95 tests: fix LRU test after splitting up Connection class.
The module the connection class is now loaded as is
"ansible.plugins.connection.mitogen_ssh", etc., which breaks the test.
Instead, check if the connection is an instance of the base Connection
class.
2018-05-04 17:38:49 +01:00
David Wilson f9e1905ec6 issue #199: ansible: stop writing temp files for new style modules
While adding support for non-new style module types, NewStyleRunner
began writing modules to a temporary file, and sys.argv was patched to
actually include the script filename. The argv change was never required
to fix any particular bug, and a search of the standard modules reveals
no argv users. Update argv[0] to be '', like an interactive interpreter
would have.

While fixing #210, new style runner began setting __file__ to the
temporary file path in order to allow apt.py to discover the Ansiballz
temporary directory. 5 out of 1,516 standard modules follow this
pattern, but in each case, none actually attempt to access __file__,
they just call dirname on it. Therefore do not write the contents of
file, simply set it to the path as it would exist, within a real
temporary directory.

Finally move temporary directory creation out of runner and into target.
Now a single directory exists for the duration of a run, and is emptied
by runner.py as necessary after each task invocation.

This could be further extended to stop rewriting non-new-style modules
in a with_items loop, but that's another step.

Finally the last bullet point in the documentation almost isn't a lie
again.
2018-05-04 06:16:25 +01:00
David Wilson 6109de51a0 tests: Ansible SSH timeout test
Also change Docker image to new 'mitogen' organization.
2018-05-03 18:19:40 +01:00
David Wilson d4d4f46006 tests: update check_host_keys parameter style in testlib. 2018-05-02 17:59:23 +01:00
David Wilson b7fe0e09f4 tests: su integration test. 2018-05-02 06:28:10 +01:00
David Wilson 3203846708 issue #239: ansible: ignore remote_tmp in new style runner. 2018-05-01 14:06:09 +01:00
David Wilson baa4e75526 tests: tidy up thread_pileup for use as a toy benchmark. 2018-04-29 02:19:32 +01:00
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
2018-04-29 00:38:53 +01:00
David Wilson 1be03eb458 parent: change create_child interface.
To allow for additional arguments.
2018-04-27 03:50:03 +01:00
David Wilson fa4746f656 ansible: add connection delegation test targets.
For command line use only, no integration tests yet.
2018-04-27 01:32:21 +01:00
David Wilson 962ba862e9 tests: use test-targets group, not all group 2018-04-22 19:40:58 +01:00
David Wilson 86c9978e09 Ensure mod.__file__ is set; closes #210. 2018-04-22 13:29:50 +01:00
David Wilson e63ae4768e core: support Receiver.get(thread_dead=False)
For tests.
2018-04-22 05:08:37 +01:00
David Wilson 6541779dd6 tests: import Ansible file transfer benchmark 2018-04-22 03:21:59 +01:00
David Wilson 7c88e4d013 Move _DEAD into header, autogenerate dead messages
This change blocks off 2 common scenarios where a race condition is
upgraded to a hang, when the library could internally do better.

* Since we don't know whether the receiver of a `reply_to` is expecting
  a raw or pickled message, and since in the case of a raw reply, there
  is no way to signal "dead" to the receiver, override the reply_to
  field to explicitly mark a message as dead using a special handle.

  This replaces the serialized _DEAD sentinel value with a slightly
  neater interface, in the form of the reserved IS_DEAD handle, and
  enables an important subsequent change: when a context cannot route a
  message, it can send a generic 'dead' reply back towards the message
  source, ensuring any sleeping thread is woken with ChannelError.

  The use of this field could potentially be extended later on if
  additional flags are needed, but for now this seems to suffice.

* Teach Router._invoke() to reply with a dead message when it receives a
  message for an invalid local handle.

* Teach Router._async_route() to reply with a dead message when it
  receives an unroutable message.
2018-04-22 00:17:27 +01:00
David Wilson 8c3b1fcf15 ansible: disable script interpreter processing for new-style
Re: #199
2018-04-21 20:59:36 +01:00
David Wilson 2f02b5c0b1 ansible: prevent Unicode strings leaking into sys.argv
The module name comes from YAML via Jinja2.. it's always Unicode. Mixing
it into a temporary directory name produces a Unicode tempdir name,
which ends up in sys.argv via TemporaryArgv.
2018-04-21 20:26:53 +01:00
David Wilson e56608ab91 parent: don't wait for SIGTERM to complete. 2018-04-21 19:39:12 +01:00
David Wilson dc4433ace6 issue #202: ansible: forget all dependent contexts on Stream disconnect
This is a partial fix, there are still at least 2 cases needing covered:

- In-progress connections must have CallError or similar sent to any
  waiters
- Once connection delegation exists, it is possible for other worker
  processes to be active (and in any step in the process), trying to
  communicate with a context that we know can no longer be communicated
  with. The solution to that isn't clear yet.

Additionally ensure root has /bin/bash shell in both Docker images.
2018-04-21 19:04:19 +01:00
David Wilson 9fe14e841c parent: reap the child process if connection fails
For example if no response is received in :attr:`connect_timeout`
seconds, the child would be left running.
2018-04-21 15:01:23 +00:00
David Wilson e7831a801f issue #195: handle non-ASCII scripts in runner.py. 2018-04-20 20:47:07 +01:00
David Wilson 85e1f5f515 ansible: remove JobResultService, more compatible async jobs; closes #191.
And by "compatible" I mean "terrible". This does not implement async job
timeouts, but I'm not going to bother, upstream async implementation is
so buggy and inconsistent it resists even having its behaviour captured
in tests.
2018-04-20 17:02:48 +01:00
David Wilson 6ad18b6719 issue #191: move async tests to their own category 2018-04-19 22:12:18 +01:00
David Wilson e43c6c531b Mostly implement hybrid TTY/socket mode for sudo and SSH.
Presently there is still no mechanism to add :attr:`tty_stream` to the
multiplexer after connection is successful, but for now it's not
expected that anything will be logged to it anyway.

Closes #148.
2018-04-18 21:40:24 +01:00
David Wilson fca22efe90 tests: enable profile_tasks callback plugin 2018-04-18 19:13:03 +01:00