diff --git a/docs/caresresolver.rst b/docs/caresresolver.rst new file mode 100644 index 00000000..b5d6ddd1 --- /dev/null +++ b/docs/caresresolver.rst @@ -0,0 +1,20 @@ +``tornado.platform.caresresolver`` --- Asynchronous DNS Resolver using C-Ares +============================================================================= + +.. module:: tornado.platform.caresresolver + +This module contains a DNS resolver using the c-ares library (and its +wrapper ``pycares``). + +.. py:class:: CaresResolver + + Name resolver based on the c-ares library. + + This is a non-blocking and non-threaded resolver. It may not produce + the same results as the system resolver, but can be used for non-blocking + resolution when threads cannot be used. + + c-ares fails to resolve some names when ``family`` is ``AF_UNSPEC``, + so it is only recommended for use in ``AF_INET`` (i.e. IPv4). This is + the default for ``tornado.simple_httpclient``, but other libraries + may default to ``AF_UNSPEC``. diff --git a/docs/integration.rst b/docs/integration.rst index 8bdcadae..6435912d 100644 --- a/docs/integration.rst +++ b/docs/integration.rst @@ -4,6 +4,7 @@ Integration with other services .. toctree:: auth + caresresolver twisted websocket wsgi diff --git a/docs/releases/next.rst b/docs/releases/next.rst index e9f07839..1874a91d 100644 --- a/docs/releases/next.rst +++ b/docs/releases/next.rst @@ -25,9 +25,9 @@ Highlights to the root logger, allowing for finer-grained configuration. * New class `tornado.process.Subprocess` wraps `subprocess.Popen` with `.PipeIOStream` access to the child's file descriptors. -* `.IOLoop` now has a static ``configure`` method like the one on - `.AsyncHTTPClient`, which can be used to select an IOLoop implementation - other than the default. +* `.IOLoop` now has a static `configure <.Configurable.configure>` + method like the one on `.AsyncHTTPClient`, which can be used to + select an `.IOLoop` implementation other than the default. * `.IOLoop` can now optionally use a monotonic clock if available (see below for more details). @@ -191,9 +191,9 @@ Multiple modules specific thread's (usually the main thread's) IOLoop). * New method `.IOLoop.add_future` to run a callback on the IOLoop when an asynchronous ``Future`` finishes. -* `.IOLoop` now has a static ``configure`` method like the one on - `.AsyncHTTPClient`, which can be used to select an IOLoop implementation - other than the default. +* `.IOLoop` now has a static `configure <.Configurable.configure>` + method like the one on `.AsyncHTTPClient`, which can be used to + select an `.IOLoop` implementation other than the default. * The `.IOLoop` poller implementations (``select``, ``epoll``, ``kqueue``) are now available as distinct subclasses of `.IOLoop`. Instantiating `.IOLoop` will continue to automatically choose the best available @@ -262,8 +262,8 @@ Multiple modules blocking, but non-blocking implementations are available using one of three optional dependencies: `~tornado.netutil.ThreadedResolver` using the `concurrent.futures` thread pool, - ``tornado.platform.caresresolver.CaresResolver`` using the ``pycares`` - library, or ``tornado.platform.twisted.TwistedResolver`` using ``twisted`` + `tornado.platform.caresresolver.CaresResolver` using the ``pycares`` + library, or `tornado.platform.twisted.TwistedResolver` using ``twisted`` * New function `tornado.netutil.is_valid_ip` returns true if a given string is a valid IP (v4 or v6) address. * `tornado.netutil.bind_sockets` has a new ``flags`` argument that can @@ -301,8 +301,8 @@ Multiple modules * Function ``tornado.options.enable_pretty_logging`` has been moved to the `tornado.log` module. -```tornado.platform.caresresolver`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +`tornado.platform.caresresolver` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * New module containing an asynchronous implementation of the `.Resolver` interface, using the ``pycares`` library. @@ -310,10 +310,10 @@ Multiple modules `tornado.platform.twisted` ~~~~~~~~~~~~~~~~~~~~~~~~~~ -* New class ``tornado.platform.twisted.TwistedIOLoop`` allows Tornado +* New class `tornado.platform.twisted.TwistedIOLoop` allows Tornado code to be run on the Twisted reactor (as opposed to the existing - ``TornadoReactor``, which bridges the gap in the other direction). -* New class ``tornado.platform.twisted.TwistedResolver`` is an asynchronous + `.TornadoReactor`, which bridges the gap in the other direction). +* New class `tornado.platform.twisted.TwistedResolver` is an asynchronous implementation of the `.Resolver` interface. `tornado.process` @@ -325,9 +325,10 @@ Multiple modules ``tornado.simple_httpclient`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* ``SimpleAsyncHTTPClient`` now takes a ``resolver`` keyword argument (which - may be passed to either the constructor or ``configure``), to allow it to - use the new non-blocking `tornado.netutil.Resolver`. +* ``SimpleAsyncHTTPClient`` now takes a ``resolver`` keyword argument + (which may be passed to either the constructor or `configure + <.Configurable.configure>`), to allow it to use the new non-blocking + `tornado.netutil.Resolver`. * When following redirects, ``SimpleAsyncHTTPClient`` now treats a 302 response code the same as a 303. This is contrary to the HTTP spec but consistent with all browsers and other major HTTP clients diff --git a/docs/twisted.rst b/docs/twisted.rst index d7e2cd84..7709e769 100644 --- a/docs/twisted.rst +++ b/docs/twisted.rst @@ -7,43 +7,63 @@ This module lets you run applications and libraries written for Twisted in a Tornado application. It can be used in two modes, depending on which library's underlying event loop you want to use. +This module has been tested with Twisted versions 11.0.0 and newer. + Twisted on Tornado ------------------ -``TornadoReactor`` implements the Twisted reactor interface on top of -the Tornado IOLoop. To use it, simply call ``install`` at the beginning -of the application:: +.. py:class:: TornadoReactor - import tornado.platform.twisted - tornado.platform.twisted.install() - from twisted.internet import reactor + ``TornadoReactor`` implements the Twisted reactor interface on top of + the Tornado IOLoop. To use it, simply call ``install`` at the beginning + of the application:: -When the app is ready to start, call ``IOLoop.instance().start()`` -instead of ``reactor.run()``. + import tornado.platform.twisted + tornado.platform.twisted.install() + from twisted.internet import reactor -It is also possible to create a non-global reactor by calling -``tornado.platform.twisted.TornadoReactor(io_loop)``. However, if -the `.IOLoop` and reactor are to be short-lived (such as those used in -unit tests), additional cleanup may be required. Specifically, it is -recommended to call:: + When the app is ready to start, call ``IOLoop.instance().start()`` + instead of ``reactor.run()``. - reactor.fireSystemEvent('shutdown') - reactor.disconnectAll() + It is also possible to create a non-global reactor by calling + ``tornado.platform.twisted.TornadoReactor(io_loop)``. However, if + the `.IOLoop` and reactor are to be short-lived (such as those used in + unit tests), additional cleanup may be required. Specifically, it is + recommended to call:: -before closing the `.IOLoop`. + reactor.fireSystemEvent('shutdown') + reactor.disconnectAll() + + before closing the `.IOLoop`. Tornado on Twisted ------------------ -``TwistedIOLoop`` implements the Tornado IOLoop interface on top of the Twisted -reactor. Recommended usage:: +.. py:class:: TwistedIOLoop - from tornado.platform.twisted import TwistedIOLoop - from twisted.internet import reactor - TwistedIOLoop().install() - # Set up your tornado application as usual using `IOLoop.instance` - reactor.run() + ``TwistedIOLoop`` implements the Tornado IOLoop interface on top + of the Twisted reactor. Recommended usage:: -``TwistedIOLoop`` always uses the global Twisted reactor. + from tornado.platform.twisted import TwistedIOLoop + from twisted.internet import reactor + TwistedIOLoop().install() + # Set up your tornado application as usual using `IOLoop.instance` + reactor.run() -This module has been tested with Twisted versions 11.0.0 and newer. + ``TwistedIOLoop`` always uses the global Twisted reactor. + +Twisted DNS resolver +-------------------- + +.. py:class:: TwistedResolver + + This is a non-blocking and non-threaded resolver. It is + recommended only when threads cannot be used, since it has + limitations compared to the standard ``getaddrinfo``-based + `~tornado.netutil.Resolver` and + `~tornado.netutil.ThreadedResolver`. Specifically, it returns at + most one result, and arguments other than ``host`` and ``family`` + are ignored. It may fail to resolve when ``family`` is not + ``socket.AF_UNSPEC``. + + Requires Twisted 12.1 or newer. diff --git a/docs/util.rst b/docs/util.rst new file mode 100644 index 00000000..162aa974 --- /dev/null +++ b/docs/util.rst @@ -0,0 +1,5 @@ +``tornado.util`` --- General-purpose utilities +============================================== + +.. automodule:: tornado.util + :members: diff --git a/docs/utilities.rst b/docs/utilities.rst index 239c3c0b..70e8a6b7 100644 --- a/docs/utilities.rst +++ b/docs/utilities.rst @@ -12,3 +12,4 @@ Utilities process stack_context testing + util diff --git a/tornado/netutil.py b/tornado/netutil.py index 190bba9b..112dcd06 100644 --- a/tornado/netutil.py +++ b/tornado/netutil.py @@ -164,7 +164,8 @@ class Resolver(Configurable): By default, a blocking implementation is used (which simply calls `socket.getaddrinfo`). An alternative implementation can be - chosen with the ``Resolver.configure`` class method:: + chosen with the `Resolver.configure <.Configurable.configure>` + class method:: Resolver.configure('tornado.netutil.ThreadedResolver') @@ -173,8 +174,8 @@ class Resolver(Configurable): * `tornado.netutil.BlockingResolver` * `tornado.netutil.ThreadedResolver` * `tornado.netutil.OverrideResolver` - * ``tornado.platform.twisted.TwistedResolver`` - * ``tornado.platform.caresresolver.CaresResolver`` + * `tornado.platform.twisted.TwistedResolver` + * `tornado.platform.caresresolver.CaresResolver` """ @classmethod def configurable_base(cls): diff --git a/tornado/platform/caresresolver.py b/tornado/platform/caresresolver.py index f0052f15..7c16705d 100644 --- a/tornado/platform/caresresolver.py +++ b/tornado/platform/caresresolver.py @@ -15,7 +15,7 @@ class CaresResolver(Resolver): c-ares fails to resolve some names when ``family`` is ``AF_UNSPEC``, so it is only recommended for use in ``AF_INET`` (i.e. IPv4). This is - the default for `tornado.simple_httpclient`, but other libraries + the default for ``tornado.simple_httpclient``, but other libraries may default to ``AF_UNSPEC``. """ def initialize(self, io_loop=None): diff --git a/tornado/platform/twisted.py b/tornado/platform/twisted.py index 3cd7b4b5..910e46af 100644 --- a/tornado/platform/twisted.py +++ b/tornado/platform/twisted.py @@ -22,6 +22,8 @@ This module lets you run applications and libraries written for Twisted in a Tornado application. It can be used in two modes, depending on which library's underlying event loop you want to use. +This module has been tested with Twisted versions 11.0.0 and newer. + Twisted on Tornado ------------------ @@ -60,8 +62,6 @@ reactor. Recommended usage:: reactor.run() `TwistedIOLoop` always uses the global Twisted reactor. - -This module has been tested with Twisted versions 11.0.0 and newer. """ from __future__ import absolute_import, division, print_function, with_statement diff --git a/tornado/util.py b/tornado/util.py index 69de2c8e..939ee6ab 100644 --- a/tornado/util.py +++ b/tornado/util.py @@ -1,4 +1,14 @@ -"""Miscellaneous utility functions.""" +"""Miscellaneous utility functions and classes. + +This module is used internally by Tornado. It is not necessarily expected +that the functions and classes defined here will be useful to other +applications, but they are documented here in case they are. + +The one public-facing part of this module is the `Configurable` class +and its `~Configurable.configure` method, which becomes a part of the +interface of its subclasses, including `.AsyncHTTPClient`, `.IOLoop`, +and `.Resolver`. +""" from __future__ import absolute_import, division, print_function, with_statement @@ -8,7 +18,8 @@ import zlib class ObjectDict(dict): - """Makes a dictionary behave like an object.""" + """Makes a dictionary behave like an object, with attribute-style access. + """ def __getattr__(self, name): try: return self[name] @@ -115,16 +126,17 @@ class Configurable(object): The implementation subclass as well as optional keyword arguments to its initializer can be set globally at runtime with `configure`. - By using the constructor as the factory method, the interface looks like - a normal class, ``isinstance()`` works as usual, etc. This pattern - is most useful when the choice of implementation is likely to be a - global decision (e.g. when epoll is available, always use it instead of - select), or when a previously-monolithic class has been split into - specialized subclasses. + By using the constructor as the factory method, the interface + looks like a normal class, `isinstance` works as usual, etc. This + pattern is most useful when the choice of implementation is likely + to be a global decision (e.g. when `~select.epoll` is available, + always use it instead of `~select.select`), or when a + previously-monolithic class has been split into specialized + subclasses. Configurable subclasses must define the class methods `configurable_base` and `configurable_default`, and use the instance - method `initialize` instead of `__init__`. + method `initialize` instead of ``__init__``. """ __impl_class = None __impl_kwargs = None @@ -163,7 +175,7 @@ class Configurable(object): def initialize(self): """Initialize a `Configurable` subclass instance. - Configurable classes should use `initialize` instead of `__init__`. + Configurable classes should use `initialize` instead of ``__init__``. """ @classmethod @@ -210,8 +222,6 @@ class ArgReplacer(object): and similar wrappers. """ def __init__(self, func, name): - """Create an ArgReplacer for the named argument to the given function. - """ self.name = name try: self.arg_pos = inspect.getargspec(func).args.index(self.name)