Rooba
/
mitogen
mirror of https://github.com/mitogen-hq/mitogen.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
mitogen/docs/changelog.rst

795 lines
35 KiB
ReStructuredText
Raw Blame History

.. _changelog:
Release Notes
=============
.. raw:: html
<style>
div#release-notes h2 {
border-bottom: 1px dotted #c0c0c0;
margin-top: 40px;
}
</style>
.. _known_issues:
Known Issues
------------
Mitogen For Ansible
~~~~~~~~~~~~~~~~~~~
* The Ansible 2.7 `reboot
<https://docs.ansible.com/ansible/latest/modules/reboot_module.html>`_ module
may require a ``pre_reboot_delay`` on systemd hosts, as insufficient time
exists for the reboot command's exit status to be reported before necessary
processes are torn down.
* On OS X when a SSH password is specified and the default connection type of
``smart`` is used, Ansible may select the Paramiko plug-in rather than
Mitogen. If you specify a password on OS X, ensure ``connection: ssh``
appears in your playbook, ``ansible.cfg``, or as ``-c ssh`` on the
command-line.
* The ``raw`` action executes as a regular Mitogen connection, which requires
Python on the target, precluding its use for installing Python. This will be
addressed in a future 0.2 release. For now, simply mix Mitogen and vanilla
Ansible strategies in your playbook:
.. code-block:: yaml
- hosts: web-servers
strategy: linear
tasks:
- name: Install Python if necessary.
raw: test -e /usr/bin/python || apt install -y python-minimal
- hosts: web-servers
strategy: mitogen_linear
roles:
- nginx
- initech_app
- y2k_fix
.. * When running with ``-vvv``, log messages will be printed to the console
*after* the Ansible run completes, as connection multiplexer shutdown only
begins after Ansible exits. This is due to a lack of suitable shutdown hook
in Ansible, and is fairly harmless, albeit cosmetically annoying. A future
release may include a solution.
.. * Configurations will break that rely on the `hashbang argument splitting
behaviour <https://github.com/ansible/ansible/issues/15635>`_ of the
``ansible_python_interpreter`` setting, contrary to the Ansible
documentation. This will be addressed in a future 0.2 release.
* Performance does not scale linearly with target count. This requires
significant additional work, as major bottlenecks exist in the surrounding
Ansible code. Performance-related bug reports for any scenario remain
welcome with open arms.
* Performance on Python 3 is significantly worse than on Python 2. While this
has not yet been investigated, at least some of the regression appears to be
part of the core library, and should therefore be straightforward to fix as
part of 0.2.x.
* *Module Replacer* style Ansible modules are not supported.
* Actions are single-threaded for each `(host, user account)` combination,
including actions that execute on the local machine. Playbooks may experience
slowdown compared to vanilla Ansible if they employ long-running
``local_action`` or ``delegate_to`` tasks delegating many target hosts to a
single machine and user account.
* Connection Delegation remains in preview and has bugs around how it infers
connections. Connection establishment will remain single-threaded for the 0.2
series, however connection inference bugs will be addressed in a future 0.2
release.
* Connection Delegation does not support automatic tunnelling of SSH-dependent
actions, such as the ``synchronize`` module. This will be addressed in the
0.3 series.
Core Library
~~~~~~~~~~~~
* Serialization is still based on :mod:`pickle`. While there is high confidence
remote code execution is impossible in Mitogen's configuration, an untrusted
context may at least trigger disproportionately high memory usage injecting
small messages (*"billion laughs attack"*). Replacement is an important
future priority, but not critical for an initial release.
* Child processes are not reliably reaped, leading to a pileup of zombie
processes when a program makes many short-lived connections in a single
invocation. This does not impact Mitogen for Ansible, however it limits the
usefulness of the core library. A future 0.2 release will address it.
* Some races remain around :class:`mitogen.core.Broker <Broker>` destruction,
disconnection and corresponding file descriptor closure. These are only
problematic in situations where child process reaping is also problematic.
* The `fakessh` component does not shut down correctly and requires flow
control added to the design. While minimal fixes are possible, due to the
absence of flow control the original design is functionally incomplete.
* The multi-threaded :ref:`service` remains in a state of design flux and
should be considered obsolete, despite heavy use in Mitogen for Ansible. A
future replacement may be integrated more tightly with, or entirely replace
the RPC dispatcher on the main thread.
* Documentation is in a state of disrepair. This will be improved over the 0.2
series.
v0.2.4 (2018-??-??)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
Enhancements
^^^^^^^^^^^^
* `#76 <https://github.com/dw/mitogen/issues/76>`_,
`#351 <https://github.com/dw/mitogen/issues/351>`_,
`#352 <https://github.com/dw/mitogen/issues/352>`_: disconnect propagation
has improved, allowing Ansible to cancel waits for responses from abruptly
disconnected targets. This ensures a task will gracefully fail rather than
hang, for example on network failure or EC2 instance maintenance.
* `#369 <https://github.com/dw/mitogen/issues/369>`_: :meth:`Connection.reset`
is implemented, allowing `meta: reset_connection
<https://docs.ansible.com/ansible/latest/modules/meta_module.html>`_ to shut
down the remote interpreter as documented, and improving support for the
`reboot
<https://docs.ansible.com/ansible/latest/modules/reboot_module.html>`_
module.
* `09aa27a6 <https://github.com/dw/mitogen/commit/09aa27a6>`_: the
``mitogen_host_pinned`` strategy wraps the ``host_pinned`` strategy
introduced in Ansible 2.7.
* `#415 <https://github.com/dw/mitogen/issues/415>`_: the interface employed for
in-process queues was changed from Kqueue/epoll() to poll(), which requires
no setup or teardown, yielding a 30% latency reduction for inter-thread
communication. This may manifest as a runtime improvement in many-host runs.
Fixes
^^^^^
* `#323 <https://github.com/dw/mitogen/issues/323>`_,
`#333 <https://github.com/dw/mitogen/issues/333>`_: work around a Windows
Subsystem for Linux bug that caused tracebacks to appear during shutdown.
* `#334 <https://github.com/dw/mitogen/issues/334>`_: the SSH method
tilde-expands private key paths using Ansible's logic. Previously the path
was passed unmodified to SSH, which expanded it using :func:`pwd.getpwnam`.
This differs from :func:`os.path.expanduser`, which uses the ``HOME``
environment variable if it is set, causing behaviour to diverge when Ansible
was invoked across user accounts via ``sudo``.
* `#364 <https://github.com/dw/mitogen/issues/364>`_: file transfers from
controllers running Python 2.7.2 or earlier could be interrupted due to a
forking bug in the :mod:`tempfile` module.
* `#370 <https://github.com/dw/mitogen/issues/370>`_: the Ansible
`reboot <https://docs.ansible.com/ansible/latest/modules/reboot_module.html>`_
module is supported.
* `#373 <https://github.com/dw/mitogen/issues/373>`_: the LXC and LXD methods
print a useful hint on failure, as no useful error is normally logged to the
console by these tools.
* `#374 <https://github.com/dw/mitogen/issues/374>`_,
`#391 <https://github.com/dw/mitogen/issues/391>`_: file transfer and module
execution from 2.x controllers to 3.x targets was broken due to a regression
caused by refactoring, and compounded by `#426
<https://github.com/dw/mitogen/issues/426>`_.
* `#400 <https://github.com/dw/mitogen/issues/400>`_: work around a threading
bug in the AWX display callback when running with high verbosity setting.
* `#409 <https://github.com/dw/mitogen/issues/409>`_: the setns method was
silently broken due to missing tests. Basic coverage was added to prevent a
recurrence.
* `#409 <https://github.com/dw/mitogen/issues/409>`_: the LXC and LXD methods
support ``mitogen_lxc_path`` and ``mitogen_lxc_attach_path`` variables to
control the location of third pary utilities.
* `#410 <https://github.com/dw/mitogen/issues/410>`_: the sudo method supports
the SELinux ``--type`` and ``--role`` options.
* `#420 <https://github.com/dw/mitogen/issues/420>`_: if a :class:`Connection`
was constructed in the Ansible top-level process, for example while executing
``meta: reset_connection``, resources could become undesirably shared in
subsequent children.
* `#426 <https://github.com/dw/mitogen/issues/426>`_: an oversight while
porting to Python 3 meant no automated 2->3 tests were running. A significant
number of 2->3 bugs were fixed, mostly in the form of Unicode/bytes
mismatches.
* `#362 <https://github.com/dw/mitogen/issues/362>`_,
`#435 <https://github.com/dw/mitogen/issues/435>`_: the previous fix for slow
Python 2.x subprocess creation on Red Hat caused newly spawned children to
have a reduced open files limit. A more intrusive fix has been added to
directly address the problem without modifying the subprocess environment.
* `#397 <https://github.com/dw/mitogen/issues/397>`_,
`#454 <https://github.com/dw/mitogen/issues/454>`_: the previous approach to
handling modern Ansible temporary file cleanup was too aggressive, and could
trigger early finalization of Cython-based extension modules, leading to
segmentation faults.
Core Library
~~~~~~~~~~~~
* `#76 <https://github.com/dw/mitogen/issues/76>`_: routing records the
destination context IDs ever received on each stream, and when disconnection
occurs, propagates :data:`mitogen.core.DEL_ROUTE` messages towards every
stream that ever communicated with the disappearing peer, rather than simply
towards parents. Conversations between nodes anywhere in the tree receive
:data:`mitogen.core.DEL_ROUTE` when either participant disconnects, allowing
receivers to wake with :class:`mitogen.core.ChannelError`, even when one
participant is not a parent of the other.
* `#109 <https://github.com/dw/mitogen/issues/109>`_,
`57504ba6 <https://github.com/dw/mitogen/commit/57504ba6>`_: newer Python 3
releases explicitly populate :data:`sys.meta_path` with importer internals,
causing Mitogen to install itself at the end of the importer chain rather
than the front.
* `#387 <https://github.com/dw/mitogen/issues/387>`_,
`#413 <https://github.com/dw/mitogen/issues/413>`_: dead messages include an
optional reason in their body. This is used to cause
:class:`mitogen.core.ChannelError` to report far more useful diagnostics at
the point the error occurs that previously would have been buried in debug
log output from an unrelated context.
* `#408 <https://github.com/dw/mitogen/issues/408>`_: a variety of fixes were
made to restore Python 2.4 compatibility.
* `#399 <https://github.com/dw/mitogen/issues/399>`_,
`#437 <https://github.com/dw/mitogen/issues/437>`_: ignore a
:class:`DeprecationWarning` to avoid failure of the ``su`` method on Python
3.7.
* `#405 <https://github.com/dw/mitogen/issues/405>`_: if an oversized message
is rejected, and it has a ``reply_to`` set, a dead message is returned to the
sender. This ensures function calls exceeding the configured maximum size
crash rather than hang.
* `#406 <https://github.com/dw/mitogen/issues/406>`_:
:class:`mitogen.core.Broker` did not call :meth:`mitogen.core.Poller.close`
during shutdown, leaking the underlying poller FD in masters and parents.
* `#406 <https://github.com/dw/mitogen/issues/406>`_: connections could leak
FDs when a child process failed to start.
* `#288 <https://github.com/dw/mitogen/issues/288>`_,
`#406 <https://github.com/dw/mitogen/issues/406>`_,
`#417 <https://github.com/dw/mitogen/issues/417>`_: connections could leave
FD wrapper objects that had not been closed lying around to be closed during
garbage collection, causing reused FD numbers to be closed at random moments.
* `#411 <https://github.com/dw/mitogen/issues/411>`_: the SSH method typed
"``y``" rather than the requisite "``yes``" when `check_host_keys="accept"`
was configured. This would lead to connection timeouts due to the hung
response.
* `#416 <https://github.com/dw/mitogen/issues/416>`_: around 1.4KiB of memory
was leaked on every RPC, due to a list of strong references keeping alive any
handler ever registered for disconnect notification.
* `#418 <https://github.com/dw/mitogen/issues/418>`_: the
:func:`mitogen.parent.iter_read` helper would leak poller FDs, because
execution of its :keyword:`finally` block was delayed on Python 3. Now
callers explicitly close the generator when finished.
* `#422 <https://github.com/dw/mitogen/issues/422>`_: the fork method could
fail to start if :data:`sys.stdout` was opened in block buffered mode, and
buffered data was pending in the parent prior to fork.
* `#438 <https://github.com/dw/mitogen/issues/438>`_: a descriptive error is
logged when stream corruption is detected.
* `#439 <https://github.com/dw/mitogen/issues/439>`_: descriptive errors are
raised when attempting to invoke unsupported function types.
* `#444 <https://github.com/dw/mitogen/issues/444>`_: messages regarding
unforwardable extension module are no longer logged as errors.
* `#445 <https://github.com/dw/mitogen/issues/445>`_: service pools unregister
the :data:`mitogen.core.CALL_SERVICE` handle at shutdown, ensuring any
outstanding messages are either processed by the pool as it shuts down, or
have dead messages sent in reply to them, preventing peer contexts from
hanging due to a forgotten buffered message.
* `#446 <https://github.com/dw/mitogen/issues/446>`_: given thread A calling
:meth:`mitogen.core.Receiver.close`, and thread B, C, and D sleeping in
:meth:`mitogen.core.Receiver.get`, previously only one sleeping thread would
be woken with :class:`mitogen.core.ChannelError` when the receiver was
closed. Now all threads are woken per the docstring.
* `#447 <https://github.com/dw/mitogen/issues/447>`_: duplicate attempts to
invoke :meth:`mitogen.core.Router.add_handler` cause an error to be raised,
ensuring accidental re-registration of service pools are reported correctly.
* `#448 <https://github.com/dw/mitogen/issues/448>`_: the import hook
implementation now raises :class:`ModuleNotFoundError` instead of
:class:`ImportError` in Python 3.6 and above, to cope with an uncoming
version of the :class:`subprocess` module requiring this new subclass in the
middle of a minor Python release series.
* `#453 <https://github.com/dw/mitogen/issues/453>`_: the loggers used in
children for standard IO redirection have propagation disabled, preventing
accidental reconfiguration of the :mod:`logging` package in a child from
setting up a feedback loop.
* `#456 <https://github.com/dw/mitogen/issues/456>`_: a descriptive error is
logged when :meth:`mitogen.core.Broker.defer` is called after the broker has
shut down, preventing new messages being enqueued that will never be sent,
and subsequently producing a program hang.
* `#459 <https://github.com/dw/mitogen/issues/459>`_: the beginnings of a
:meth:`mitogen.master.Router.get_stats` call has been added. The initial
statistics cover the module loader only.
* `#462 <https://github.com/dw/mitogen/issues/462>`_: Mitogen could fail to
open a PTY on broken Linux systems due to a bad interaction between the glibc
:func:`grantpt` function and an incorrectly mounted ``/dev/pts`` filesystem.
Since correct group ownership is not required in most scenarios, when this
problem is detected, the PTY is allocated and opened directly by the library.
* `16ca111e <https://github.com/dw/mitogen/commit/16ca111e>`_: handle OpenSSH
7.5 permission denied prompts when ``~/.ssh/config`` rewrites are present.
* `9ec360c2 <https://github.com/dw/mitogen/commit/9ec360c2>`_: a new
:meth:`mitogen.core.Broker.defer_sync` utility function is provided.
* `f20e0bba <https://github.com/dw/mitogen/commit/f20e0bba>`_:
:meth:`mitogen.service.FileService.register_prefix` permits granting
unprivileged access to whole filesystem subtrees, rather than single files at
a time.
* `8f85ee03 <https://github.com/dw/mitogen/commit/8f85ee03>`_:
:meth:`mitogen.core.Router.myself` returns a :class:`mitogen.core.Context`
referring to the current process.
* `824c7931 <https://github.com/dw/mitogen/commit/824c7931>`_: exceptions
raised by the import hook were updated to include probable reasons for
a failure.
* `57b652ed <https://github.com/dw/mitogen/commit/57b652ed>`_: a stray import
meant an extra roundtrip and ~20KiB of data was wasted for any context that
imported :mod:`mitogen.parent`.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Andreas Krüger <https://github.com/woopstar>`_,
`Berend De Schouwer <https://github.com/berenddeschouwer>`_,
`Brian Candler <https://github.com/candlerb>`_,
`Duane Zamrok <https://github.com/dewthefifth>`_,
`Eric Chang <https://github.com/changchichung>`_,
`Guy Knights <https://github.com/knightsg>`_,
`Jiří Vávra <https://github.com/Houbovo>`_,
`Jonathan Rosser <https://github.com/jrosser>`_,
`Johan Beisser <https://github.com/jbeisser>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Mehdi <https://github.com/mehdisat7>`_,
`Michael DeHaan <https://github.com/mpdehaan>`_,
`Mohammed Naser <https://github.com/mnaser/>`_,
`Stéphane <https://github.com/sboisson/>`_,
`@whky <https://github.com/whky/>`_,
`@syntonym <https://github.com/syntonym/>`_,
`@yodatak <https://github.com/yodatak/>`_, and
`Younès HAFRI <https://github.com/yhafri>`_.
v0.2.3 (2018-10-23)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
Enhancements
^^^^^^^^^^^^
* `#315 <https://github.com/dw/mitogen/pull/315>`_,
`#392 <https://github.com/dw/mitogen/issues/392>`_: Ansible 2.6 and 2.7 are
supported.
* `#321 <https://github.com/dw/mitogen/issues/321>`_,
`#336 <https://github.com/dw/mitogen/issues/336>`_: temporary file handling
was simplified, undoing earlier damage caused by compatibility fixes,
improving 2.6 compatibility, and avoiding two network roundtrips for every
related action
(`assemble <http://docs.ansible.com/ansible/latest/modules/assemble_module.html>`_,
`aws_s3 <http://docs.ansible.com/ansible/latest/modules/aws_s3_module.html>`_,
`copy <http://docs.ansible.com/ansible/latest/modules/copy_module.html>`_,
`patch <http://docs.ansible.com/ansible/latest/modules/patch_module.html>`_,
`script <http://docs.ansible.com/ansible/latest/modules/script_module.html>`_,
`template <http://docs.ansible.com/ansible/latest/modules/template_module.html>`_,
`unarchive <http://docs.ansible.com/ansible/latest/modules/unarchive_module.html>`_,
`uri <http://docs.ansible.com/ansible/latest/modules/uri_module.html>`_). See
:ref:`ansible_tempfiles` for a complete description.
* `#376 <https://github.com/dw/mitogen/pull/376>`_,
`#377 <https://github.com/dw/mitogen/pull/377>`_: the ``kubectl`` connection
type is now supported. Contributed by Yannig Perré.
* `084c0ac0 <https://github.com/dw/mitogen/commit/084c0ac0>`_: avoid a
roundtrip in
`copy <http://docs.ansible.com/ansible/latest/modules/copy_module.html>`_ and
`template <http://docs.ansible.com/ansible/latest/modules/template_module.html>`_
due to an unfortunate default.
* `7458dfae <https://github.com/dw/mitogen/commit/7458dfae>`_: avoid a
roundtrip when transferring files smaller than 124KiB. Copy and template
actions are now 2-RTT, reducing runtime for a 20-iteration template loop over
a 250 ms link from 30 seconds to 10 seconds compared to v0.2.2, down from 120
seconds compared to vanilla.
* `#337 <https://github.com/dw/mitogen/issues/337>`_: To avoid a scaling
limitation, a PTY is no longer allocated for an SSH connection unless the
configuration specifies a password.
* `d62e6e2a <https://github.com/dw/mitogen/commit/d62e6e2a>`_: many-target
runs executed the dependency scanner redundantly due to missing
synchronization, wasting significant runtime in the connection multiplexer.
In one case work was reduced by 95%, which may manifest as faster runs.
* `5189408e <https://github.com/dw/mitogen/commit/5189408e>`_: threads are
cooperatively scheduled, minimizing `GIL
<https://en.wikipedia.org/wiki/Global_interpreter_lock>`_ contention, and
reducing context switching by around 90%. This manifests as an overall
improvement, but is easily noticeable on short many-target runs, where
startup overhead dominates runtime.
* The `faulthandler <https://faulthandler.readthedocs.io/>`_ module is
automatically activated if it is installed, simplifying debugging of hangs.
See :ref:`diagnosing-hangs` for details.
* The ``MITOGEN_DUMP_THREAD_STACKS`` environment variable's value now indicates
the number of seconds between stack dumps. See :ref:`diagnosing-hangs` for
details.
Fixes
^^^^^
* `#251 <https://github.com/dw/mitogen/issues/251>`_,
`#340 <https://github.com/dw/mitogen/issues/340>`_: Connection Delegation
could establish connections to the wrong target when ``delegate_to:`` is
present.
* `#291 <https://github.com/dw/mitogen/issues/291>`_: when Mitogen had
previously been installed using ``pip`` or ``setuptools``, the globally
installed version could conflict with a newer version bundled with an
extension that had been installed using the documented steps. Now the bundled
library always overrides over any system-installed copy.
* `#324 <https://github.com/dw/mitogen/issues/324>`_: plays with a
`custom module_utils <https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-module-utils-path>`_
would fail due to fallout from the Python 3 port and related tests being
disabled.
* `#331 <https://github.com/dw/mitogen/issues/331>`_: the connection
multiplexer subprocess always exits before the main Ansible process, ensuring
logs generated by it do not overwrite the user's prompt when ``-vvv`` is
enabled.
* `#332 <https://github.com/dw/mitogen/issues/332>`_: support a new
:func:`sys.excepthook`-based module exit mechanism added in Ansible 2.6.
* `#338 <https://github.com/dw/mitogen/issues/338>`_: compatibility: changes to
``/etc/environment`` and ``~/.pam_environment`` made by a task are reflected
in the runtime environment of subsequent tasks. See
:ref:`ansible_process_env` for a complete description.
* `#343 <https://github.com/dw/mitogen/issues/343>`_: the sudo ``--login``
option is supported.
* `#344 <https://github.com/dw/mitogen/issues/344>`_: connections no longer
fail when the controller's login username contains slashes.
* `#345 <https://github.com/dw/mitogen/issues/345>`_: the ``IdentitiesOnly
yes`` option is no longer supplied to OpenSSH by default, better matching
Ansible's behaviour.
* `#355 <https://github.com/dw/mitogen/issues/355>`_: tasks configured to run
in an isolated forked subprocess were forked from the wrong parent context.
This meant built-in modules overridden via a custom ``module_utils`` search
path may not have had any effect.
* `#362 <https://github.com/dw/mitogen/issues/362>`_: to work around a slow
algorithm in the :mod:`subprocess` module, the maximum number of open files
in processes running on the target is capped to 512, reducing the work
required to start a subprocess by >2000x in default CentOS configurations.
* `#397 <https://github.com/dw/mitogen/issues/397>`_: recent Mitogen master
versions could fail to clean up temporary directories in a number of
circumstances, and newer Ansibles moved to using :mod:`atexit` to effect
temporary directory cleanup in some circumstances.
* `b9112a9c <https://github.com/dw/mitogen/commit/b9112a9c>`_,
`2c287801 <https://github.com/dw/mitogen/commit/2c287801>`_: OpenSSH 7.5
permission denied prompts are now recognized. Contributed by Alex Willmer.
* A missing check caused an exception traceback to appear when using the
``ansible`` command-line tool with a missing or misspelled module name.
* Ansible since >=2.7 began importing :mod:`__main__` from
:mod:`ansible.module_utils.basic`, causing an error during execution, due to
the controller being configured to refuse network imports outside the
``ansible.*`` namespace. Update the target implementation to construct a stub
:mod:`__main__` module to satisfy the otherwise seemingly vestigial import.
Core Library
~~~~~~~~~~~~
* A new :class:`mitogen.parent.CallChain` class abstracts safe pipelining of
related function calls to a target context, cancelling the chain if an
exception occurs.
* `#305 <https://github.com/dw/mitogen/issues/305>`_: fix a long-standing minor
race relating to the logging framework, where *no route for Message..*
would frequently appear during startup.
* `#313 <https://github.com/dw/mitogen/issues/313>`_:
:meth:`mitogen.parent.Context.call` was documented as capable of accepting
static methods. While possible on Python 2.x the result is ugly, and in every
case it should be trivial to replace with a classmethod. The documentation
was fixed.
* `#337 <https://github.com/dw/mitogen/issues/337>`_: to avoid a scaling
limitation, a PTY is no longer allocated for each OpenSSH client if it can be
avoided. PTYs are only allocated if a password is supplied, or when
`host_key_checking=accept`. This is since Linux has a default of 4096 PTYs
(``kernel.pty.max``), while OS X has a default of 127 and an absolute maximum
of 999 (``kern.tty.ptmx_max``).
* `#339 <https://github.com/dw/mitogen/issues/339>`_: the LXD connection method
was erroneously executing LXC Classic commands.
* `#345 <https://github.com/dw/mitogen/issues/345>`_: the SSH connection method
allows optionally disabling ``IdentitiesOnly yes``.
* `#356 <https://github.com/dw/mitogen/issues/356>`_: if the master Python
process does not have :data:`sys.executable` set, the default Python
interpreter used for new children on the local machine defaults to
``"/usr/bin/python"``.
* `#366 <https://github.com/dw/mitogen/issues/366>`_,
`#380 <https://github.com/dw/mitogen/issues/380>`_: attempts by children to
import :mod:`__main__` where the main program module lacks an execution guard
are refused, and an error is logged. This prevents a common and highly
confusing error when prototyping new scripts.
* `#371 <https://github.com/dw/mitogen/pull/371>`_: the LXC connection method
uses a more compatible method to establish an non-interactive session.
Contributed by Brian Candler.
* `af2ded66 <https://github.com/dw/mitogen/commit/af2ded66>`_: add
:func:`mitogen.fork.on_fork` to allow non-Mitogen managed process forks to
clean up Mitogen resources in the child.
* `d6784242 <https://github.com/dw/mitogen/commit/d6784242>`_: the setns method
always resets ``HOME``, ``SHELL``, ``LOGNAME`` and ``USER`` environment
variables to an account in the target container, defaulting to ``root``.
* `830966bf <https://github.com/dw/mitogen/commit/830966bf>`_: the UNIX
listener no longer crashes if the peer process disappears in the middle of
connection setup.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Alex Russu <https://github.com/alexrussu>`_,
`Alex Willmer <https://github.com/moreati>`_,
`atoom <https://github.com/atoom>`_,
`Berend De Schouwer <https://github.com/berenddeschouwer>`_,
`Brian Candler <https://github.com/candlerb>`_,
`Dan Quackenbush <https://github.com/danquack>`_,
`dsgnr <https://github.com/dsgnr>`_,
`Jesse London <https://github.com/jesteria>`_,
`John McGrath <https://github.com/jmcgrath207>`_,
`Jonathan Rosser <https://github.com/jrosser>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Luca Nunzi <https://github.com/0xlc>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`Peter V. Saveliev <https://github.com/svinota>`_,
`Pierre-Henry Muller <https://github.com/pierrehenrymuller>`_,
`Pierre-Louis Bonicoli <https://github.com/jesteria>`_,
`Prateek Jain <https://github.com/prateekj201>`_,
`RedheatWei <https://github.com/RedheatWei>`_,
`Rick Box <https://github.com/boxrick>`_,
`nikitakazantsev12 <https://github.com/nikitakazantsev12>`_,
`Tawana Musewe <https://github.com/tbtmuse>`_,
`Timo Beckers <https://github.com/ti-mo>`_, and
`Yannig Perré <https://github.com/yannig>`_.
v0.2.2 (2018-07-26)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
* `#291 <https://github.com/dw/mitogen/issues/291>`_: ``ansible_*_interpreter``
variables are parsed using a restrictive shell-like syntax, supporting a
common idiom where ``ansible_python_interpreter`` is set to ``/usr/bin/env
python``.
* `#299 <https://github.com/dw/mitogen/issues/299>`_: fix the ``network_cli``
connection type when the Mitogen strategy is active. Mitogen cannot help
network device connections, however it should still be possible to use device
connections while Mitogen is active.
* `#301 <https://github.com/dw/mitogen/pull/301>`_: variables like ``$HOME`` in
the ``remote_tmp`` setting are evaluated correctly.
* `#303 <https://github.com/dw/mitogen/pull/303>`_: the :ref:`doas` become method
is supported. Contributed by `Mike Walker
<https://github.com/napkindrawing>`_.
* `#309 <https://github.com/dw/mitogen/issues/309>`_: fix a regression to
process environment cleanup, caused by the change in v0.2.1 to run local
tasks with the correct environment.
* `#317 <https://github.com/dw/mitogen/issues/317>`_: respect the verbosity
setting when writing to Ansible's ``log_path``, if it is enabled. Child log
filtering was also incorrect, causing the master to needlessly wake many
times. This nets a 3.5% runtime improvement running against the local
machine.
* The ``mitogen_ssh_debug_level`` variable is supported, permitting SSH debug
output to be included in Mitogen's ``-vvv`` output when both are specified.
Core Library
~~~~~~~~~~~~
* `#291 <https://github.com/dw/mitogen/issues/291>`_: the ``python_path``
parameter may specify an argument vector prefix rather than a string program
path.
* `#300 <https://github.com/dw/mitogen/issues/300>`_: the broker could crash on
OS X during shutdown due to scheduled `kqueue
<https://www.freebsd.org/cgi/man.cgi?query=kevent>`_ filter changes for
descriptors that were closed before the IO loop resumes. As a temporary
workaround, kqueue's bulk change feature is not used.
* `#303 <https://github.com/dw/mitogen/pull/303>`_: the :ref:`doas` become method
is now supported. Contributed by `Mike Walker
<https://github.com/napkindrawing>`_.
* `#307 <https://github.com/dw/mitogen/issues/307>`_: SSH login banner output
containing the word 'password' is no longer confused for a password prompt.
* `#319 <https://github.com/dw/mitogen/issues/319>`_: SSH connections would
fail immediately on Windows Subsystem for Linux, due to use of `TCSAFLUSH`
with :func:`termios.tcsetattr`. The flag is omitted if WSL is detected.
* `#320 <https://github.com/dw/mitogen/issues/320>`_: The OS X poller
could spuriously wake up due to ignoring an error bit set on events returned
by the kernel, manifesting as a failure to read from an unrelated descriptor.
* `#342 <https://github.com/dw/mitogen/issues/342>`_: The ``network_cli``
connection type would fail due to a missing internal SSH plugin method.
* Standard IO forwarding accidentally configured the replacement ``stdout`` and
``stderr`` write descriptors as non-blocking, causing subprocesses that
generate more output than kernel buffer space existed to throw errors. The
write ends are now configured as blocking.
* When :func:`mitogen.core.enable_profiling` is active, :mod:`mitogen.service`
threads are profiled just like other threads.
* The ``ssh_debug_level`` parameter is supported, permitting SSH debug output
to be redirected to a Mitogen logger when specified.
* Debug logs containing command lines are printed with the minimal quoting and
escaping required.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
the bug reports and pull requests in this release contributed by
`Alex Russu <https://github.com/alexrussu>`_,
`Andy Freeland <https://github.com/rouge8>`_,
`Ayaz Ahmed Khan <https://github.com/ayaz>`_,
`Colin McCarthy <https://github.com/colin-mccarthy>`_,
`Dan Quackenbush <https://github.com/danquack>`_,
`Duane Zamrok <https://github.com/dewthefifth>`_,
`Gonzalo Servat <https://github.com/gservat>`_,
`Guy Knights <https://github.com/knightsg>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Mark Janssen <https://github.com/sigio>`_,
`Mike Walker <https://github.com/napkindrawing>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`falbanese <https://github.com/falbanese>`_,
`Tawana Musewe <https://github.com/tbtmuse>`_, and
`Zach Swanson <https://github.com/zswanson>`_.
v0.2.1 (2018-07-10)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
* `#297 <https://github.com/dw/mitogen/issues/297>`_: compatibility: local
actions set their working directory to that of their defining playbook, and
inherit a process environment as if they were executed as a subprocess of the
forked task worker.
v0.2.0 (2018-07-09)
-------------------
Mitogen 0.2.x is the inaugural feature-frozen branch eligible for fixes only,
except for problem areas listed as in-scope below. While stable from a
development perspective, it should still be considered "beta" at least for the
initial releases.
**In Scope**
* Python 3.x performance improvements
* Subprocess reaping improvements
* Major documentation improvements
* PyPI/packaging improvements
* Test suite improvements
* Replacement CI system to handle every supported OS
* Minor deviations from vanilla Ansible behaviour
* Ansible ``raw`` action support
The goal is a *tick/tock* model where even-numbered series are a maturation of
the previous unstable series, and unstable series are released on PyPI with
``--pre`` enabled. The API and user visible behaviour should remain unchanged
within a stable series.
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
* Support for Ansible 2.3 - 2.7.x and any mixture of Python 2.6, 2.7 or 3.6 on
controller and target nodes.
* Drop-in support for many Ansible connection types.
* Preview of Connection Delegation feature.
* Built-in file transfer compatible with connection delegation.
Core Library
~~~~~~~~~~~~
* Synchronous connection establishment via OpenSSH, sudo, su, Docker, LXC and
FreeBSD Jails, local subprocesses and :func:`os.fork`. Parallel connection
setup is possible using multiple threads. Connections may be used from one or
many threads after establishment.
* UNIX masters and children, with Linux, MacOS, FreeBSD, NetBSD, OpenBSD and
Windows Subsystem for Linux explicitly supported.
* Automatic tests covering Python 2.6, 2.7 and 3.6 on Linux only.
Reference in New Issue View Git Blame Copy Permalink