mitogen/tests
David Wilson 3613162bc0 ansible: enable forking when requested and for async jobs.
Closes #105.
References #155.

mitogen/service.py:
    Refactor services to support individually exposed methods with
    different security policies for each method.

    - @mitogen.service.expose() to expose a method and set its policy
    - @mitogen.service.arg_spec() to validate input.
    - Require basic service message format to be a tuple of
      `(method, kwargs)`, where kwargs is always a dict.
    - Update DeduplicatingService to match the new scheme.

ansible_mitogen/connection.py:
    - Rename 'method' to 'method_name' to disambiguate it from the
      service.call()'s method= argument.

ansible_mitogen/planner.py:
    - Generate an ID for every job, sync or not, and fetch job results
      from JobResultService rather than via the initiating function
      call's return value.
    - Planner subclasses now get to select whether their Runner should
      run in a forked process. The base implementation requests this if
      the 'mitogen_isolation_mode=fork' task variable is present.

ansible_mitogen/runner.py:
    Teach runners to deliver their result via JobResultService executing
    in their indirect parent mux process.

ansible_mitogen/plugins/actions/mitogen_async_status.py:
    Split the implementation up into methods, and more compatibly
    emulate Ansible's existing output.

ansible_mitogen/process.py:
    Mux processes now host JobResultService.

ansible_mitogen/services.py:
    Update existing services to the new mitogen.service scheme, and
    implement JobResultService:

    * listen() method for synchronous jobs. planner.invoke() registers a
      Sender with the service prior to invoking the job, then sleeps
      waiting for the service to write the job result to the
      corresponding Receiver.

    * Non-blocking get() method for implementing mitogen_async_status
      action.

    * Child-accessible push() method for delivering task results.

ansible_mitogen/target.py:
    New helpers for spawning a virginal subprocess on startup, from
    which asynchronous and mitogen_task_isolation=fork jobs are forked.
    Necessary to avoid a task inheriting potentially
    polluted/monkey-patched parent environment, since remaining jobs
    continue to run in the original child process.

docs/ansible.rst:
    Add/merge/remove some behaviours/risks.

tests/ansible/integration:
    New tests for forking/async.
2018-04-09 00:03:09 +01:00
..
ansible ansible: enable forking when requested and for async jobs. 2018-04-09 00:03:09 +01:00
bench tests/bench: import "slightly more reliable time" script 2018-04-02 07:30:06 +00:00
data tests: Fix no such option -o running FakeSsh.test_okay() 2018-03-19 21:58:34 +05:45
soak tests: Import Latch soak test 2018-03-29 14:31:57 +05:45
README.md tests: Switch to unit2 test runner, with coverage 2018-04-01 01:43:45 +01:00
__init__.py tests: Make the tests directory an importable package 2018-04-01 01:32:13 +01:00
build_docker_image.py issue #164: teach debops_tests.sh to use SSH 2018-04-05 22:52:02 +01:00
call_error_test.py core: support CallError(str) for service.py. 2018-03-29 11:53:02 +05:45
call_function_test.py call_function_test: Fix assumption that we run as a script 2018-04-01 01:23:13 +01:00
channel_test.py Import and use unittest2 without aliasing it 2018-03-19 21:58:28 +05:45
fakessh_test.py fakessh_test: Apply timeout decorators to rsync tests 2018-04-01 01:17:29 +01:00
first_stage_test.py tests: first_stage_test regression due to 1ff27ada49 2018-03-29 21:33:18 +05:45
fork_test.py issue #155: fix double-fork behaviour and test it this time. 2018-03-29 23:57:24 +05:45
id_allocation_test.py issue #155: allocate child IDs in batches of 1000. 2018-03-22 03:41:04 +05:45
importer_test.py tests: just call log_to_file() from testlib 2018-03-29 07:25:11 +00:00
io_op_test.py issue #174: test all io_op() logic. 2018-04-01 16:39:10 +01:00
latch_test.py core: support throwing LatchError in every sleeping thread 2018-03-19 21:58:35 +05:45
local_test.py Import and use unittest2 without aliasing it 2018-03-19 21:58:28 +05:45
master_test.py master_test: Don't assume __file__ points to source code 2018-04-01 01:21:42 +01:00
module_finder_test.py tests: Fix typo in ResolveRelPathTest 2018-03-19 21:58:30 +05:45
nested_test.py Import and use unittest2 without aliasing it 2018-03-19 21:58:28 +05:45
parent_test.py parent_test: Add explanation. 2018-03-29 23:49:45 +05:45
receiver_test.py core: Rename Sender.put() to Sender.send(). 2018-03-29 15:43:48 +05:45
responder_test.py tests: mark some tests as skipped 2018-03-19 21:58:35 +05:45
router_test.py Introduce handler policy functions; closes #138. 2018-03-29 21:40:33 +05:45
select_test.py select: clean up API. 2018-03-19 21:58:28 +05:45
show_docker_hostname.py issue #164: utility to print Docker hostname for use from shell scripts. 2018-04-05 20:24:11 +01:00
ssh_test.py tests: test_stream_name: fix non-localhost Docker 2018-03-22 13:25:56 +05:45
testlib.py issue #164: utility to print Docker hostname for use from shell scripts. 2018-04-05 20:24:11 +01:00
utils_test.py Import and use unittest2 without aliasing it 2018-03-19 21:58:28 +05:45

README.md

Warning

This directory is full of disorganized crap, including random hacks I checked in that I'd like to turn into tests. The effort to write tests only really started in September 2017. Pull requests in this area are very welcome!

Running The Tests

Build Status

Your computer should have an Internet connection, and the docker command line tool should be able to connect to a working Docker daemon (localhost or elsewhere for OS X etc.) that can run new images.

The IP address of the Docker daemon must allow exposing ports from running containers, e.g. it should not be firewalled or port forwarded.

If in doubt, just install Docker on a Linux box in the default configuration and run the tests there.

Steps To Prepare Development Environment

  1. Get the code git clone https://github.com/dw/mitogen.git
  2. Go into the working directory cd mitogen
  3. Establish the docker image ./tests/build_docker_image.py
  4. Build the virtual environment virtualenv ../venv
  5. Enable the virtual environment we just built source ../venv/bin/activate
  6. Install Mitogen in pip editable mode pip install -e .
  7. Run test