Commit Graph

1123 Commits

Author SHA1 Message Date
dw 8702b7521f
Merge pull request #240 from dw/dmw
FileService optimizations, more compatible temp dir handling
2018-05-01 14:15:45 +01:00
David Wilson 3203846708 issue #239: ansible: ignore remote_tmp in new style runner. 2018-05-01 14:06:09 +01:00
David Wilson 94e048a2e5 ansible: ensure FileService uses exact CHUNK_SIZE multiple
9.8% throughput increase with sudo.
2018-05-01 01:57:17 +01:00
David Wilson 187e3a3fc1 ansible: support 2.3 too. 2018-04-30 19:15:51 +01:00
David Wilson dafe12b315 ansible: fix AnsibleUnicode crash when processing "~username". 2018-04-30 15:21:26 +01:00
David Wilson e1a3cea2f9 ansible: FileService: don't send empty last chunk 2018-04-30 01:24:59 +01:00
David Wilson 2a56c672ca ansible: FileService docstring updates. 2018-04-29 21:38:43 +01:00
dw 76102927c8
Merge pull request #237 from dw/dmw
connection delegation fixes & issue #212
2018-04-29 19:58:36 +01:00
David Wilson f5d22a3ca1 core: support deleting handlers, make Receiver.close() unregister 2018-04-29 19:41:23 +01:00
David Wilson 58eb9828b0 docs: remove limitation. 2018-04-29 19:35:43 +01:00
David Wilson 69e5902e61 issue #212: support explicit acknowledgements in FileService. 2018-04-29 19:30:57 +01:00
David Wilson 5e6e56f0c5 issue #212: service: make call_async kwargs optional. 2018-04-29 19:16:25 +01:00
David Wilson afe983d6c9 issue #212: service: support no_reply decorator. 2018-04-29 19:16:15 +01:00
David Wilson bf6c2fa97c issue #212: service: more concise repr 2018-04-29 19:16:03 +01:00
David Wilson ff7fb00569 parent: return latch to wait() caller to allow graceful timeout 2018-04-29 10:11:13 +01:00
David Wilson b0309b539c ansible: disable interpreter recycling for connections.
Must explicitly specify enable_lru=True in ContextService.get() to
trigger recycling.
2018-04-29 09:53:16 +01:00
dw be2954db6f
Merge pull request #235 from dw/dmw
docs: more ansible updates
2018-04-29 09:38:15 +01:00
David Wilson ec22fee7a7 docs: more ansible updates 2018-04-29 09:37:49 +01:00
dw bd9d9e3600
Merge pull request #234 from dw/dmw
Fix one Latch race, one huge leak around fork
2018-04-29 07:41:23 +01:00
David Wilson c0ced6d04a core: fix monster fork FD leak
_sockets only refers to the idle sockets list, it doesn't refer to every
socket currently in use by a Latch, for example, the 2*16 used by e.g.
Ansible's sleeping service pool.
2018-04-29 06:15:23 +01:00
David Wilson 7316c08237 core: fix _tls_init() race.
The GIL could be lost between the check for an empty list and popping a
socket off the list. Previously _tls_init (per its name) used per-thread
storage, hence the bug.
2018-04-29 06:03:40 +01:00
David Wilson 79fd35092b docs: more links 2018-04-29 04:00:11 +01:00
David Wilson fdac4ecce8 setns: typos 2018-04-29 02:55:28 +01:00
dw 617f84432d
Merge pull request #233 from dw/dmw
Dmw
2018-04-29 02:51:16 +01:00
David Wilson 65e6a44fe7 docs: add links. 2018-04-29 02:31:32 +01:00
David Wilson e93ac2f3a7 debug: implement some basic helpers to debugger. 2018-04-29 02:22:22 +01:00
David Wilson baa4e75526 tests: tidy up thread_pileup for use as a toy benchmark. 2018-04-29 02:19:32 +01:00
David Wilson 90f7b4baea ansible: stub plugin documentation. 2018-04-29 02:18:26 +01:00
David Wilson c85a5b6446 ansible: make call timing more readable 2018-04-29 02:17:10 +01:00
David Wilson 29f6c46e97 docs: add file transfer safety section. 2018-04-29 02:14:29 +01:00
David Wilson bba2a42e44 ansible: add mitogen_sudo method, split out connection subclasses.
Slowly moving towards real implementations in those files.
2018-04-29 01:45:52 +01:00
David Wilson 0a9126c510 ansible: connection plugins missing from previous commit. 2018-04-29 01:22:01 +01:00
David Wilson b61c291faf examples: import mitogen-fuse.py. 2018-04-29 01:20:26 +01:00
dw 0d9d3cfe7b
Merge pull request #232 from dw/dmw
Better error messages, support username in setns
2018-04-29 00:48:06 +01:00
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
2018-04-29 00:38:53 +01:00
David Wilson 947d35649c parent: note exception machine's hostname.
For dumb situations where user (i.e. me) is trying to fix a problem in
the wrong place.
2018-04-28 23:39:33 +01:00
dw b99b4eb048
Merge pull request #231 from dw/dmw
Many file transfer improvements, docs improvements.
2018-04-28 22:38:35 +01:00
David Wilson 003f30b5a9 issue #226: test fixes. 2018-04-28 22:10:41 +01:00
David Wilson 6edb3f165d ansible: avoid a race during shutdown. 2018-04-28 22:08:35 +01:00
David Wilson 780b63520f issue #226: don't attempt to fchmod() a pathname 2018-04-28 21:56:23 +01:00
David Wilson 58d8f60f57 docs: better connection type docs 2018-04-28 21:36:51 +01:00
David Wilson 219a202a82 issue #226: ansible: file transfer improvements
* put_data() supports setting mode and times.
* put_file() refuses to copy non-regular files (sockets, FIFOs).
* put_file() saves one RTT for <32KiB files by using put_data() and
  embedding file content in argument list.
* FileService returns dict with size/mode/owner/group/mtime/atime.
* FileService refuses to copy non-regular files.
* transfer_file() preserves file mode.
* transfer_file() preserves atime/mtime.
* transfer_file() optionally preserves ownership.
* transfer_file() optionally calls fsync().
* transfer_file() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* transfer_file() ensures temporary file is deleted on any error.
* write_path() writes to a temporary file and deletes it on failure.
* write_path() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* write_path() supports setting symbolic owner/group.
* write_path() optionally calls fsync().
* write_path() supports setting symbolic mode/mtime/atime.

Closes #226, #227, #229
2018-04-28 21:33:43 +01:00
dw 8beb6d339b
Merge pull request #225 from willmerae/setup-import-os
setup: Add missing os import
2018-04-28 08:26:52 +01:00
dw 956e39fd66
Merge pull request #228 from dw/dmw
setns method
2018-04-28 07:02:22 +01:00
David Wilson e8b4c4e683 issue #223: implement setns connection type
machinectl does not support any sensible form of pipe to the child
process, so it is necessary to bypass it when talking to a systemd
container (see systemd/systemd#8850).

This can also form the basis for issue #223, where the post-fork
namespace switching dance required to connect to the Pythonless
container will be the same.
2018-04-28 06:41:34 +01:00
Alex Willmer fbcb789f3f setup: Add missing os import 2018-04-27 16:47:03 +01:00
David Wilson b5be0fd65b ansible: log _get_file() timings. 2018-04-27 10:58:33 +01:00
David Wilson 95039eea11 ansible: make key_from_kwargs() 10x faster
It was half the cost of the service call
2018-04-27 10:44:06 +01:00
dw 0ee4ddcec1
Merge pull request #224 from dw/dmw
docs: links to Ansible docs
2018-04-27 07:38:11 +01:00
David Wilson 79c2d6c289 docs: links to Ansible docs 2018-04-27 06:50:03 +01:00