tornado/docs/releases/next.rst

What's new in the next release of Tornado
=========================================

In progress
-----------

Highlights
^^^^^^^^^^

* The ``callback`` argument to many asynchronous methods is now optional,
  and these methods return a `Future`.  The `tornado.gen` module now
  understands `Futures`, and these methods can be used directly without
  a `gen.Task` wrapper.
* New function `IOLoop.current` returns the `IOLoop` that is running
  on the current thread (as opposed to `IOLoop.instance`, which
  returns a specific thread's (usually the main thread's) IOLoop.
* New class `tornado.netutil.Resolver` provides an asynchronous
  interface to DNS resolution.  The default implementation is still
  blocking, but non-blocking implementations are available using one
  of three optional dependencies: `~tornado.netutil.ThreadedResolver`
  using the `concurrent.futures` thread pool,
  `~tornado.platform.caresresolver.CaresResolver` using the `pycares`
  library, or `~tornado.platform.twisted.TwistedResolver` using `twisted`
* Tornado's logging is now less noisy, and it no longer goes directly
  to the root logger, allowing for finer-grained configuration.
* New class `tornado.process.Subprocess` wraps `subprocess.Popen` with
  `PipeIOStream` access to the child's file descriptors.
* `IOLoop` now has a static ``configure`` method like the one on
  `AsyncHTTPClient`, which can be used to select an IOLoop implementation
  other than the default.
* `IOLoop` can now optionally use a monotonic clock if available
  (see below for more details).


Backwards-incompatible changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* Python 2.5 is no longer supported.  Python 3 is now supported in a single
  codebase instead of using ``2to3``
* The ``tornado.database`` module has been removed.  It is now available
  as a separate package, `torndb <https://github.com/bdarnell/torndb>`_
* Functions that take an ``io_loop`` parameter now default to
  `IOLoop.current()` instead of `IOLoop.instance()`.
* Empty HTTP request arguments are no longer ignored.  This applies to
  ``HTTPRequest.arguments`` and ``RequestHandler.get_argument[s]``
  in WSGI and non-WSGI modes.
* On Python 3, `tornado.escape.json_encode` no longer accepts byte strings.
* On Python 3, the ``get_authenticated_user`` methods in `tornado.auth`
  now return character strings instead of byte strings.
* `tornado.netutil.TCPServer` has moved to its own module, `tornado.tcpserver`.
* The Tornado test suite now requires ``unittest2`` when run on Python 2.6.


Detailed changes by module
^^^^^^^^^^^^^^^^^^^^^^^^^^

Multiple modules
~~~~~~~~~~~~~~~~

* Tornado no longer logs to the root logger.  Details on the new logging
  scheme can be found under the `tornado.log` module.  Note that in some
  cases this will require that you add an explicit logging configuration
  in order to see any output (perhaps just calling ``logging.basicConfig()``),
  although both `IOLoop.start()` and `tornado.options.parse_command_line`
  will do this for you.
* On python 3.2+, methods that take an ``ssl_options`` argument (on
  `SSLIOStream`, `TCPServer`, and `HTTPServer`) now accept either a
  dictionary of options or an `ssl.SSLContext` object.
* New optional dependency on `concurrent.futures` to provide better support
  for working with threads.  `concurrent.futures` is in the standard library
  for Python 3.2+, and can be installed on older versions with
  ``pip install futures``.

`tornado.autoreload`
~~~~~~~~~~~~~~~~~~~~

* `tornado.autoreload` is now more reliable when there are errors at import
  time.
* Calling `tornado.autoreload.start` (or creating an `Application` with
  ``debug=True``) twice on the same `IOLoop` now does nothing (instead of
  creating multiple periodic callbacks).  Starting autoreload on
  more than one `IOLoop` in the same process now logs a warning.
* Scripts run by autoreload no longer inherit ``__future__`` imports
  used by Tornado.

`tornado.auth`
~~~~~~~~~~~~~~

* On Python 3, the ``get_authenticated_user`` method family now returns
  character strings instead of byte strings.
* Asynchronous methods defined in `tornado.auth` now return a `Future`,
  and their ``callback`` argument is optional.  The `Future` interface is
  preferred as it offers better error handling (the previous interface
  just logged a warning and returned None).
* The `tornado.auth` mixin classes now define a method
  ``get_auth_http_client``, which can be overridden to use a non-default
  `AsyncHTTPClient` instance (e.g. to use a different `IOLoop`)
* Subclasses of `OAuthMixin` are encouraged to override
  `_oauth_get_user_future` instead of `_oauth_get_user`, although both
  methods are still supported.

`tornado.concurrent`
~~~~~~~~~~~~~~~~~~~~

* New module `tornado.concurrent` contains code to support working with
  `concurrent.futures`, or to emulate future-based interface when that module
  is not available.

`tornado.curl_httpclient`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Preliminary support for `tornado.curl_httpclient` on Python 3.  The latest
  official release of pycurl only supports Python 2, but Ubuntu has a
  port available in 12.10 (``apt-get install python3-pycurl``).  This port
  currently has bugs that prevent it from handling arbitrary binary data
  but it should work for textual (utf8) resources.
* Fix a crash with libcurl 7.29.0 if a curl object is created and closed
  without being used.

`tornado.escape`
~~~~~~~~~~~~~~~~

* On Python 3, `~tornado.escape.json_encode` no longer accepts byte strings.
  This mirrors the behavior of the underlying json module.  Python 2 behavior
  is unchanged but should be faster.

`tornado.gen`
~~~~~~~~~~~~~

* New decorator ``@gen.coroutine`` is available as an alternative to
  ``@gen.engine``.  It automatically returns a `Future`, and within the
  function instead of calling a callback you return a value with
  ``raise gen.Return(value)`` (or simply ``return value`` in Python 3.3).
* Generators may now yield ``Future`` objects.
* Callbacks produced by `gen.Callback` and `gen.Task` are now automatically
  stack-context-wrapped, to minimize the risk of context leaks when used
  with asynchronous functions that don't do their own wrapping.
* Fixed a memory leak involving generators, `RequestHandler.flush`,
  and clients closing connections while output is being written.
* Yielding a large list no longer has quadratic performance.


`tornado.httpclient`
~~~~~~~~~~~~~~~~~~~~

* `AsyncHTTPClient.fetch` now returns a ``Future`` and its callback argument
  is optional.  When the future interface is used, any error will be raised
  automatically, as if `HTTPResponse.rethrow` was called.
* `AsyncHTTPClient.configure` and all `AsyncHTTPClient` constructors
  now take a ``defaults`` keyword argument.  This argument should be a
  dictionary, and its values will be used in place of corresponding
  attributes of `HTTPRequest` that are not set.
* All unset attributes of `tornado.httpclient.HTTPRequest` are now ``None``.
  The default values of some attributes (``connect_timeout``,
  ``request_timeout``, ``follow_redirects``, ``max_redirects``,
  ``use_gzip``, ``proxy_password``, ``allow_nonstandard_methods``,
  and ``validate_cert`` have been moved from `HTTPRequest` to the
  client implementations.
* The ``max_clients`` argument to `AsyncHTTPClient` is now a keyword-only
  argument.
* Keyword arguments to `AsyncHTTPClient.configure` are no longer used
  when instantiating an implementation subclass directly.
* Secondary `AsyncHTTPClient` callbacks (``streaming_callback``,
  ``header_callback``, and ``prepare_curl_callback``) now respect
  `StackContext`.

`tornado.httpserver`
~~~~~~~~~~~~~~~~~~~~

* `HTTPServer` no longer logs an error when it is unable to read a second
  request from an HTTP 1.1 keep-alive connection.
* `HTTPServer` now takes a ``protocol`` keyword argument which can be set
  to ``https`` if the server is behind an SSL-decoding proxy that does not
  set any supported X-headers.
* `tornado.httpserver.HTTPConnection` now has a `set_close_callback`
  method that should be used instead of reaching into its ``stream``
  attribute.
* Empty HTTP request arguments are no longer ignored.  This applies to
  ``HTTPRequest.arguments`` and ``RequestHandler.get_argument[s]``
  in WSGI and non-WSGI modes.

`tornado.ioloop`
~~~~~~~~~~~~~~~~

* New function `IOLoop.current` returns the ``IOLoop`` that is running
  on the current thread (as opposed to `IOLoop.instance`, which returns a
  specific thread's (usually the main thread's) IOLoop).
* New method `IOLoop.add_future` to run a callback on the IOLoop when
  an asynchronous ``Future`` finishes.
* `IOLoop` now has a static ``configure`` method like the one on
  `AsyncHTTPClient`, which can be used to select an IOLoop implementation
  other than the default.
* The `IOLoop` poller implementations (``select``, ``epoll``, ``kqueue``)
  are now available as distinct subclasses of `IOLoop`.  Instantiating
  `IOLoop` will continue to automatically choose the best available
  implementation.
* The `IOLoop` constructor has a new keyword argument ``time_func``,
  which can be used to set the time function used when scheduling callbacks.
  This is most useful with the `time.monotonic()` function, introduced
  in Python 3.3 and backported to older versions via the ``monotime``
  module.  Using a monotonic clock here avoids problems when the system
  clock is changed.
* New function `IOLoop.time` returns the current time according to the
  IOLoop.  To use the new monotonic clock functionality, all calls to
  `IOLoop.add_timeout` must be either pass a `datetime.timedelta` or
  a time relative to `IOLoop.time`, not `time.time`.  (`time.time` will
  continue to work only as long as the IOLoop's ``time_func`` argument
  is not used).
* New convenience method `IOLoop.run_sync` can be used to start an IOLoop
  just long enough to run a single coroutine.
* New method `IOLoop.add_callback_from_signal` is safe to use in a signal
  handler (the regular `add_callback` method may deadlock).
* `IOLoop` now uses `signal.set_wakeup_fd` where available (Python 2.6+
  on Unix) to avoid a race condition that could result in Python signal
  handlers being delayed.
* Method `IOLoop.running()` has been removed.
* `IOLoop` has been refactored to better support subclassing.
* `IOLoop.add_callback` and `add_callback_from_signal` now take
  ``*args, **kwargs`` to pass along to the callback.

`tornado.iostream`
~~~~~~~~~~~~~~~~~~

* `IOStream.connect` now has an optional ``server_hostname`` argument
  which will be used for SSL certificate validation when applicable.
  Additionally, when supported (on Python 3.2+), this hostname
  will be sent via SNI (and this is supported by `tornado.simple_httpclient`)
* Much of `IOStream` has been refactored into a separate class
  `BaseIOStream`.
* New class `tornado.iostream.PipeIOStream` provides the IOStream
  interface on pipe file descriptors.
* `IOStream` now raises a new exception
  `tornado.iostream.StreamClosedError` when you attempt to read or
  write after the stream has been closed (by either side).
* `IOStream` now simply closes the connection when it gets an
  ``ECONNRESET`` error, rather than logging it as an error.
* `IOStream.error` no longer picks up unrelated exceptions.
* `IOStream.close` now has an ``exc_info`` argument (similar to the
  one used in the `logging` module) that can be used to set the stream's
  ``error`` attribute when closing it.
* `IOStream.read_until_close` now works correctly when it is called
  while there is buffered data.
* Fixed a major performance regression when run on PyPy (introduced in
  Tornado 2.3).

`tornado.log`
~~~~~~~~~~~~~

* New module containing `enable_pretty_logging` and `LogFormatter`,
  moved from the options module.
* `LogFormatter` now handles non-ascii data in messages and tracebacks better.

`tornado.netutil`
~~~~~~~~~~~~~~~~~

* New class `tornado.netutil.Resolver` provides an asynchronous
  interface to DNS resolution.  The default implementation is still
  blocking, but non-blocking implementations are available using one
  of three optional dependencies: `~tornado.netutil.ThreadedResolver`
  using the `concurrent.futures` thread pool,
  `~tornado.platform.caresresolver.CaresResolver` using the `pycares`
  library, or `~tornado.platform.twisted.TwistedResolver` using `twisted`
* New function `tornado.netutil.is_valid_ip` returns true if a given string
  is a valid IP (v4 or v6) address.
* `tornado.netutil.bind_sockets` has a new ``flags`` argument that can
  be used to pass additional flags to ``getaddrinfo``.
* `tornado.netutil.bind_sockets` no longer sets ``AI_ADDRCONFIG``; this will
  cause it to bind to both ipv4 and ipv6 more often than before.
* `tornado.netutil.bind_sockets` now works when Python was compiled
  with ``--disable-ipv6`` but IPv6 DNS resolution is available on the
  system.
* `tornado.netutil.TCPServer` has moved to its own module, `tornado.tcpserver`.

`tornado.options`
~~~~~~~~~~~~~~~~~

* The class underlying the functions in `tornado.options` is now public
  (`tornado.options.OptionParser`).  This can be used to create multiple
  independent option sets, such as for subcommands.
* `tornado.options.parse_config_file` now configures logging automatically
  by default, in the same way that `parse_command_line` does.
* New function `tornado.options.add_parse_callback` schedules a callback
  to be run after the command line or config file has been parsed.  The
  keyword argument ``final=False`` can be used on either parsing function
  to supress these callbacks.
* `tornado.options.define` now takes a ``callback`` argument.  This callback
  will be run with the new value whenever the option is changed.  This is
  especially useful for options that set other options, such as by reading
  from a config file.
* `tornado.option.parse_command_line` ``--help`` output now goes to ``stderr``
  rather than ``stdout``.
* `tornado.options.options` is no longer a subclass of `dict`; attribute-style
  access is now required.
* `tornado.options.options` (and `OptionParser` instances generally) now
  have a `mockable()` method that returns a wrapper object compatible with
  `mock.patch`.
* Function `tornado.options.enable_pretty_logging` has been moved to the
  `tornado.log` module.

`tornado.platform.caresresolver`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* New module containing an asynchronous implementation of the `Resolver`
  interface, using the `pycares` library.

`tornado.platform.twisted`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* New class `tornado.platform.twisted.TwistedIOLoop` allows Tornado
  code to be run on the Twisted reactor (as opposed to the existing
  `TornadoReactor`, which bridges the gap in the other direction).
* New class `tornado.platform.twisted.TwistedResolver` is an asynchronous
  implementation of the `Resolver` interface.

`tornado.process`
~~~~~~~~~~~~~~~~~

* New class `tornado.process.Subprocess` wraps `subprocess.Popen` with
  `PipeIOStream` access to the child's file descriptors.

`tornado.simple_httpclient`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `SimpleAsyncHTTPClient` now takes a ``resolver`` keyword argument (which
  may be passed to either the constructor or ``configure``), to allow it to
  use the new non-blocking `tornado.netutil.Resolver`.
* When following redirects, `SimpleAsyncHTTPClient` now treats a 302
  response code the same as a 303.  This is contrary to the HTTP spec
  but consistent with all browsers and other major HTTP clients
  (including `CurlAsyncHTTPClient`).
* The behavior of ``header_callback`` with `SimpleAsyncHTTPClient` has
  changed and is now the same as that of `CurlAsyncHTTPClient`.  The
  header callback now receives the first line of the response (e.g.
  ``HTTP/1.0 200 OK``) and the final empty line.
* `simple_httpclient` now accepts responses with a 304 status code that
  include a ``Content-Length`` header.
* Fixed a bug in which `SimpleAsyncHTTPClient` callbacks were being run in the
  client's ``stack_context``.

`tornado.stack_context`
~~~~~~~~~~~~~~~~~~~~~~~

* `stack_context.wrap` now runs the wrapped callback in a more consistent
  environment by recreating contexts even if they already exist on the
  stack.
* Fixed a bug in which stack contexts could leak from one callback
  chain to another.
* Yield statements inside a ``with`` statement can cause stack
  contexts to become inconsistent; an exception will now be raised
  when this case is detected.

`tornado.template`
~~~~~~~~~~~~~~~~~~

* Errors while rendering templates no longer log the generated code,
  since the enhanced stack traces (from version 2.1) should make this
  unnecessary.
* The ``{% apply %}`` directive now works properly with functions that return
  both unicode strings and byte strings (previously only byte strings were
  supported).
* Code in templates is no longer affected by Tornado's ``__future__`` imports
  (which previously included ``absolute_import`` and ``division``).


`tornado.testing`
~~~~~~~~~~~~~~~~~

* New function `tornado.testing.bind_unused_port` both chooses a port
  and binds a socket to it, so there is no risk of another process
  using the same port.  ``get_unused_port`` is now deprecated.
* New decorator `tornado.testing.gen_test` can be used to allow for
  yielding `tornado.gen` objects in tests, as an alternative to the
  ``stop`` and ``wait`` methods of `AsyncTestCase`.
* `tornado.testing.AsyncTestCase` and friends now extend ``unittest2.TestCase``
  when it is available (and continue to use the standard ``unittest`` module
  when ``unittest2`` is not available)
* `tornado.testing.ExpectLog` can be used as a finer-grained alternative
  to `tornado.testing.LogTrapTestCase`
* The command-line interface to `tornado.testing.main` now supports
  additional arguments from the underlying `unittest` module:
  ``verbose``, ``quiet``, ``failfast``, ``catch``, ``buffer``.
* The deprecated ``--autoreload`` option of `tornado.testing.main` has
  been removed.  Use ``python -m tornado.autoreload`` as a prefix command
  instead.
* The ``--httpclient`` option of `tornado.testing.main` has been moved
  to `tornado.test.runtests` so as not to pollute the application
  option namespace.  The `tornado.options` module's new callback
  support now makes it easy to add options from a wrapper script
  instead of putting all possible options in `tornado.testing.main`.
* `AsyncHTTPTestCase` no longer calls `AsyncHTTPClient.close` for tests
  that use the singleton `IOLoop.instance`.
* `LogTrapTestCase` no longer fails when run in unknown logging
  configurations.  This allows tests to be run under nose, which does its
  own log buffering (`LogTrapTestCase` doesn't do anything useful in this
  case, but at least it doesn't break things any more).

`tornado.util`
~~~~~~~~~~~~~~

* `tornado.util.b` (which was only intended for internal use) is gone.

`tornado.web`
~~~~~~~~~~~~~

* `RequestHandler.set_header` now overwrites previous header values
  case-insensitively.
* `tornado.web.RequestHandler` has new attributes ``path_args`` and
  ``path_kwargs``, which contain the positional and keyword arguments
  that are passed to the ``get``/``post``/etc method.  These attributes
  are set before those methods are called, so they are available during
  ``prepare()``
* `tornado.web.ErrorHandler` no longer requires XSRF tokens on ``POST``
  requests, so posts to an unknown url will always return 404 instead of
  complaining about XSRF tokens.
* Several methods related to HTTP status codes now take a ``reason`` keyword
  argument to specify an alternate "reason" string (i.e. the "Not Found" in
  "HTTP/1.1 404 Not Found").  It is now possible to set status codes other
  than those defined in the spec, as long as a reason string is given.
* The ``Date`` HTTP header is now set by default on all responses.
* ``Etag``/``If-None-Match`` requests now work with `StaticFileHandler`.
* `StaticFileHandler` no longer sets ``Cache-Control: public`` unnecessarily.
* When gzip is enabled in a `tornado.web.Application`, appropriate
  ``Vary: Accept-Encoding`` headers are now sent.
* It is no longer necessary to pass all handlers for a host in a single
  `Application.add_handlers` call.  Now the request will be matched
  against the handlers for any ``host_pattern`` that includes the request's
  ``Host`` header.

`tornado.websocket`
~~~~~~~~~~~~~~~~~~~

* Client-side WebSocket support is now available:
  `tornado.websocket.websocket_connect`
* `WebSocketHandler` has new methods `ping` and `on_pong` to send pings
  to the browser (not supported on the ``draft76`` protocol)