Commit Graph

2846 Commits

Author SHA1 Message Date
David Wilson edde251d58 issue #549: ansible: reduce risk by capping RLIM_INFINITY 2019-08-03 21:40:57 +01:00
David Wilson 7ca2d00fca Merge remote-tracking branch 'origin/osx-ci-fixes'
* origin/osx-ci-fixes:
  issue #573: guard against a forked top-level Ansible process
  [linear2] simplify ClassicWorkerModel and fix repeat initialization
  issue #549 / [stream-refactor]: fix close/poller deregister crash on OSX
  issue #549: skip Docker tests if Docker is unavailable
  issue #549: remove Linux-specific assumptions from create_child_test
  issue #549: fix setrlimit() crash and hard-wire OS X default
2019-08-03 18:58:53 +01:00
David Wilson d408caccf5 issue #573: guard against a forked top-level Ansible process
See comment.
2019-08-03 18:43:18 +01:00
David Wilson 3ceac2c9ed [linear2] simplify ClassicWorkerModel and fix repeat initialization
"self.initialized = False" slipped in a few days ago, on second thoughts
that flag is not needed at all, by simply rearranging ClassicWorkerModel
to have a regular constructor.

This hierarchy is still squishy, it needs more love. Remaining
MuxProcess class attributes should eliminated.
2019-08-03 18:24:52 +01:00
David Wilson cebccf6f41 issue #549 / [stream-refactor]: fix close/poller deregister crash on OSX
See source comment.
2019-08-03 07:13:12 +01:00
David Wilson 19b259a45f issue #549: skip Docker tests if Docker is unavailable 2019-08-03 05:48:47 +01:00
David Wilson dcfd733e6f issue #549: remove Linux-specific assumptions from create_child_test
Some stat fields are implementation-specific, little value even testing
them on Linux
2019-08-03 05:12:33 +01:00
David Wilson 395b03a77d issue #549: fix setrlimit() crash and hard-wire OS X default
OS X advertised unlimited, but really it means kern.maxfilesperproc.
2019-08-02 22:30:58 +01:00
David Wilson c9d890b865 Merge remote-tracking branch 'origin/602-recover-taskvars'
* origin/602-recover-taskvars:
  issue #602: update Changelog
  issue #602: recover task_vars for synchronize and meta: reset_connection
  ansible: remove cutpasted docstring
  [linear2] merge fallout: restore optimization from #491 / 7b129e857
2019-08-02 04:39:26 +01:00
David Wilson 7629ff9e6d issue #602: update Changelog 2019-08-02 04:10:34 +01:00
David Wilson 33bceb6eb4 issue #602: recover task_vars for synchronize and meta: reset_connection 2019-08-02 04:05:34 +01:00
David Wilson 6b4bcf4fe0 ansible: remove cutpasted docstring 2019-08-02 04:05:28 +01:00
David Wilson 619f4dee07 [linear2] merge fallout: restore optimization from #491 / 7b129e857 2019-08-02 04:05:19 +01:00
David Wilson 5c39f58edf Merge remote-tracking branch 'origin/unienv'
* origin/unienv:
  docs: update Changelog.
  issue #600: /etc/environment may be non-ASCII in an unknown encoding
  docs: finished Changelog locking note
  Fix for sample in doc
  docs: break out install_app.py and fix API use.
2019-08-01 12:33:21 +01:00
David Wilson cf2b8f1c24 docs: update Changelog. 2019-08-01 12:13:58 +01:00
David Wilson e4321f81a0 issue #600: /etc/environment may be non-ASCII in an unknown encoding 2019-08-01 12:12:18 +01:00
David Wilson e87e41e69e docs: finished Changelog locking note 2019-08-01 11:01:14 +01:00
Stefane Fermigier aa15975ad4 Fix for sample in doc
`log_to_file()` expects a filename.
2019-08-01 11:01:14 +01:00
David Wilson 9bb3dac450 docs: break out install_app.py and fix API use. 2019-08-01 11:01:14 +01:00
David Wilson 86337c4f0d Merge remote-tracking branch 'origin/549-open-files'
* origin/549-open-files:
  issue #603: Revert "ci: update to Ansible 2.8.3"
  Fix unit_Test.ClientTest following 108015aa22
  service: clean up log messages, especially at shutdown
  remove unused imports flagged by lgtm
  [linear2]: merge fallout flaggged by LGTM
  issue #549: docs: update Changelog
  issue #549: increase open file limit automatically if possible
  ansible: improve process.py docs
  docs: remove old list link.
  docs: migrate email list
  docs: changelog tweaks
  parent: decode logged stdout as UTF-8.
  scripts: import affin.sh
  ci: update to Ansible 2.8.3
  tests: terraform tweaks
  unix: include more IO in the try/except for connection failure
  tests: update gcloud.py to match terraform config
  tests: hide ugly error during Ansible tests
  tests/ansible/gcloud: terraform conf for load testing
  ansible: gracefully handle failure to connect to MuxProcess
  ansible: fix affinity tests for 5ae45f6612390bbc888b65964fb5c218feed1679
  ansible: pin per-CPU muxes to their corresponding CPU
  ansible: reap mux processes on shut down
2019-07-31 14:05:11 +01:00
David Wilson cd2689af0a issue #603: Revert "ci: update to Ansible 2.8.3"
This reverts commit 558ebfa914.
2019-07-31 13:44:34 +01:00
David Wilson b75198396d Fix unit_Test.ClientTest following 108015aa22
Closes #604
2019-07-31 13:18:47 +01:00
David Wilson 2fede49078 service: clean up log messages, especially at shutdown 2019-07-31 11:49:53 +01:00
David Wilson 75d179e4b9 remove unused imports flagged by lgtm 2019-07-31 11:46:23 +01:00
David Wilson c80fddd487 [linear2]: merge fallout flaggged by LGTM 2019-07-31 11:41:29 +01:00
David Wilson d0aee1ef3c issue #549: docs: update Changelog 2019-07-31 04:28:27 +01:00
David Wilson eeb7150f24 issue #549: increase open file limit automatically if possible
While catching every possible case where "open file limit exceeded" is
not possible, we can at least increase the soft limit to the available
hard limit without any user effort.

Do this in Ansible top-level process, even though we probably only need
it in the MuxProcess. It seems there is no reason this could hurt
2019-07-31 04:20:04 +01:00
David Wilson acab26d796 ansible: improve process.py docs 2019-07-31 04:09:48 +01:00
David Wilson edeaa3c6ee docs: remove old list link. 2019-07-31 02:03:54 +01:00
David Wilson c4bcfa4c49 docs: migrate email list 2019-07-31 02:03:54 +01:00
David Wilson 7accc092de docs: changelog tweaks 2019-07-31 02:03:54 +01:00
David Wilson 45a3014fd4 parent: decode logged stdout as UTF-8. 2019-07-31 02:01:11 +01:00
David Wilson 6f7941d616 scripts: import affin.sh 2019-07-31 01:50:37 +01:00
David Wilson 558ebfa914 ci: update to Ansible 2.8.3 2019-07-31 01:50:37 +01:00
David Wilson 28b4d63e49 tests: terraform tweaks 2019-07-31 01:50:37 +01:00
David Wilson 3b000c7d15 unix: include more IO in the try/except for connection failure 2019-07-31 01:50:37 +01:00
David Wilson 5ed0b93669 tests: update gcloud.py to match terraform config 2019-07-31 01:50:37 +01:00
David Wilson 4dfbe82e76 tests: hide ugly error during Ansible tests 2019-07-31 01:50:37 +01:00
David Wilson de2e1ec184 tests/ansible/gcloud: terraform conf for load testing 2019-07-31 01:50:37 +01:00
David Wilson 108015aa22 ansible: gracefully handle failure to connect to MuxProcess
It's possible to hit an ugly exception during early CTRL+C
2019-07-31 01:50:37 +01:00
David Wilson a9d3fdf6b7 ansible: fix affinity tests for 5ae45f6612390bbc888b65964fb5c218feed1679 2019-07-31 01:50:37 +01:00
David Wilson bf1f3682aa ansible: pin per-CPU muxes to their corresponding CPU
This slightly breaks the old scheme, in that CPU 1 may now end up with a
mux and the top-level process pinned to it.
2019-07-31 01:50:37 +01:00
David Wilson dc9f4e89e6 ansible: reap mux processes on shut down
Previously we exitted without calling waitpid(), which meant the
top-level process struct rusage did not reflect the resource usage
consumed by the multiplexer processes.

Existing benchmarks are made using perf so this never created a problem,
but it could be confusing to others using the "time" command, and also
allows logging the final exit status of the process.
2019-07-31 01:50:37 +01:00
David Wilson adbad76925 Merge remote-tracking branch 'origin/543-darwin-ansible-ci'
* origin/543-darwin-ansible-ci:
  issue #543: install virtualenv for Azure
  issue #543: dumb fix for file vs. stat :(
  issue #543: disable host key checking
  issue #543: create ~/.ssh if it doesn't exist
  issue #543: Hide Mitogen test users from gdm
  issue #543: skip test that's hard to do on Mac
  issue #543: use key from Git, newer ssh-keygen unsupported by Paramiko
  image_prep: ensure Mac users can SSH without manual intervention
  issue #543: make localhost_ansible_tests run locally
  issue #543: add Ansible job to Azure matrix
  issue #543: localhost_ansible scripts.
2019-07-31 01:38:41 +01:00
David Wilson 57db3a36e1 issue #543: install virtualenv for Azure 2019-07-31 01:14:34 +01:00
David Wilson ebb4a7ca6a issue #543: dumb fix for file vs. stat :( 2019-07-31 01:14:34 +01:00
David Wilson f3915b5f40 issue #543: disable host key checking 2019-07-31 01:14:34 +01:00
David Wilson edb745f434 issue #543: create ~/.ssh if it doesn't exist 2019-07-31 01:14:34 +01:00
David Wilson 0741876392 issue #543: Hide Mitogen test users from gdm 2019-07-31 01:14:34 +01:00
David Wilson 17d0e1b315 issue #543: skip test that's hard to do on Mac 2019-07-31 00:15:05 +01:00