flesh out "learning from bidict" outline

This commit is contained in:
jab 2018-01-02 19:24:44 -05:00
parent 958ca8570a
commit 1a67961423
5 changed files with 122 additions and 73 deletions

View File

@ -137,12 +137,14 @@ for usage documentation.
Learning from bidict
--------------------
I have learned a surprisingly large amount
of fascinating advanced Python programming
from working on bidict,
especially in light of its relatively small codebase.
Check out :doc:`learning-from-bidict` [#fn-learning]_
if you're interested in learning from bidict too.
One of the most rewarding things about working on bidict
and also the most surprising,
especially in light of the small codebase (just ~600 sloc)
is how much fun, advanced Python material
has turned up in the process.
Check out the :doc:`learning-from-bidict` [#fn-learning]_ docs
if you're interested in taking a tour.
Contributing

View File

@ -2,7 +2,7 @@ h1, h2, h3, h4, h5 {
font-weight: bold !important;
}
div#changelog li {
ul.simple li {
margin-bottom: 8px;
}

View File

@ -49,10 +49,12 @@ extensions = [
'sphinx.ext.coverage',
'sphinx.ext.viewcode',
'sphinx.ext.intersphinx',
'sphinx.ext.todo',
'alabaster',
]
intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
todo_include_todos = True
# Add any paths that contain templates here, relative to this directory.
#templates_path = ['_templates']

View File

@ -1,27 +1,109 @@
Learning from bidict
--------------------
I have learned a surprisingly large amount
of fascinating advanced Python programming
in the process of developing bidict,
especially in light of its relatively small codebase.
Below are some of the fun Python corners I got to explore further
Below are some of the more fascinating Python corners
I got to explore further
thanks to working on bidict.
If you are interested in learning more about any of the following,
reading and contributing to bidict's code
could be a great way to do so.
TODO: Expand all of the following, and
include more specific references to bidict's usages.
If you are interested in learning more about any of the following,
reading through or even contributing to bidict's code
could be a great way to get started.
.. todo::
The following is just an outline.
Expand and provide more references and examples.
Python's data model
===================
- Making an immutable type hashable,
i.e. insertable into :class:`dict`\s and :class:`set`\s
- See :meth:`object.__hash__` and :meth:`object.__eq__` docs
- How this affects hashable ordered collections
like :class:`~bidict.FrozenOrderedBidict`
that have an order-insensitive
:meth:`~bidict.FrozenOrderedBidict.__eq__`
- All contained items must participate in the hash
- Resulting corner cases produce possibly surprising results:
- See :ref:`nan-as-key`
- See
`pywat#38 <https://github.com/cosmologicon/pywat/issues/38>`_
for some surprising results when keys of different types compare equal,
or when a hashable type's ``__eq__()`` is intransitive
(as in :class:`~collections.OrderedDict`):
- "Intransitive equality was a mistake." Raymond Hettinger
- Thus :ref:`eq-order-insensitive` for :class:`~bidict.FrozenOrderedBidict`
- Using :meth:`object.__new__` to bypass default object initialization,
e.g. for better :meth:`~bidict.bidict.copy` performance
- See `how bidict does this
<https://github.com/jab/bidict/blob/master/bidict/_frozen.py>`_
- Overriding :meth:`object.__getattribute__` for custom attribute lookup
- See :ref:`sorted-bidict-recipes` for example
- Using :meth:`object.__reduce__` to make an object pickleable
that otherwise wouldn't be,
due to e.g. using weakrefs (see below)
Using :mod:`weakref`
====================
- See :ref:`inv-avoids-reference-cycles`
:func:`~collections.namedtuple`-style dynamic class generation
==============================================================
- See `namedbidict's implementation
<https://github.com/jab/bidict/blob/master/bidict/_named.py>`_
How to efficiently implement an ordered mapping
===============================================
- Use a backing dict and doubly-linked list
`like OrderedDict
<https://github.com/python/cpython/blob/a0374d/Lib/collections/__init__.py#L71>`_
- See `OrderedBidict's implementation
<https://github.com/jab/bidict/blob/master/bidict/_ordered.py>`_
API Design
==========
- Making APIs "Pythonic"
- Integrating with :mod:`collections` via :mod:`collections.abc` and :mod:`abc`
- `Zen of Python <https://www.python.org/dev/peps/pep-0020/#id3>`_
- Extending :class:`collections.abc.Mapping` and :class:`collections.abc.MutableMapping`
- How to make virtual subclasses using
:meth:`abc.ABCMeta.register` or
:meth:`abc.ABCMeta.__subclasshook__` and
:obj:`NotImplemented`.
- Beyond :class:`collections.abc.Mapping`, bidicts implement additional APIs
that :class:`dict` and :class:`~collections.OrderedDict` implement.
- When creating a new API, making it familiar, memorable, and intuitive
is hugely important to a good user experience.
- Making APIs Pythonic
- `Zen of Python <https://www.python.org/dev/peps/pep-0020/>`_
- "Errors should never pass silently.
Unless explicitly silenced.
@ -31,73 +113,31 @@ API Design
- "Explicit is better than implicit.
There should be one—and preferably only one—obvious way to do it."
→ dropped the alternate ``.inv`` APIs that used
the ``~`` operator and the slice syntax hack
- Integrating with :mod:`collections` via :mod:`collections.abc` and :mod:`abc`
- Extending :class:`collections.abc.Mapping` and :class:`collections.abc.MutableMapping`
- Using :meth:`abc.ABCMeta.register`,
:meth:`abc.ABCMeta.__subclasshook__`, and
:obj:`NotImplemented`.
- Beyond :class:`collections.abc.Mapping`, bidicts implement as much of the
:class:`dict` and :class:`~collections.OrderedDict` APIs as possible.
When working with APIs, familiarity and memorability are hugely important.
Python's data model
===================
- Making an immutable type hashable, i.e. insertable into :class:`dict`\s and :class:`set`\s
(see :meth:`object.__hash__` and :meth:`object.__eq__`),
how that interacts with ordered vs. unordered hashable types
that may compare equal
(e.g.
:class:`~bidict.frozenbidict` and
:class:`~bidict.FrozenOrderedBidict`),
some interesting resulting corner cases
- :ref:`nan-as-key`
- equal keys of different type,
intransitive equality (as in :class:`~collections.OrderedDict`):
https://github.com/cosmologicon/pywat/issues/38
- "Intransitive equality was a mistake." Raymond Hettinger
- Using :meth:`object.__new__` to bypass default object initialization
- Using :meth:`object.__reduce__` to make an object pickleable that otherwise wouldn't be,
due to e.g. using weakrefs (see below)
- Overriding :meth:`object.__getattribute__` for custom attribute lookup
(see :ref:`sorted-bidict-recipes` for example)
Implementing an ordered mapping using a circular doubly-linked list
===================================================================
OrderedDict's
`implementation <https://github.com/python/cpython/blob/a0374d/Lib/collections/__init__.py#L71>`_
is a great reference.
the ``~`` operator and the old slice syntax
Portability
===========
- Python 2 vs. Python 3 (:class:`dict` API changes)
- Python 2 vs. Python 3 (mostly :class:`dict` API changes)
- CPython vs. PyPy
- gc / weakref differences
- gc / weakref
- http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies
- hence https://github.com/jab/bidict/blob/958ca85/tests/test_hypothesis.py#L168
- nan
Correctness, performance, code quality, etc.
============================================
bidict provided a need to learn these fantastic tools,
many of which have been indispensable:
many of which have been indispensable
(especially hypothesis see
`bidict's usage <https://github.com/jab/bidict/blob/master/tests/test_hypothesis.py>`_):
- `Pytest <https://docs.pytest.org/en/latest/>`_
- `Coverage <http://coverage.readthedocs.io/en/latest/>`_

View File

@ -67,6 +67,11 @@ will have its value overwritten in place::
OrderedBidict([(3, 4), (5, 2), (7, 8)])
.. _eq-order-insensitive:
:meth:`~bidict.FrozenOrderedBidict.__eq__` is order-insensitive
###############################################################
To ensure that equality of bidicts is transitive,
and to comply with the
`Liskov substitution principle <https://en.wikipedia.org/wiki/Liskov_substitution_principle>`_,