Spooled Temporary Files are file-like objects
that start out mapped to in-memory objects, but
automatically roll over to a temporary file once
they reach a certain (configurable) threshhold.
Unfortunately the built-in SpooledTemporaryFile
class in Python does not implement the exact API
that some common classes like StringIO do.
SpooledTemporaryFile also spools all of it's in-memory
files as cStringIO instances. cStringIO instances cannot
be deep-copied, and they don't work with the zip
library either. This along with the incompatible api makes
it useless for several use-cases.
To combat this but still gain the memory savings and
usefulness of a true spooled file-like-object, two custom
classes have been implemented which have a compatible API.
SpooledBytesIO is a spooled file-like-object that only
accepts bytes. On Python 2.x this means the 'str' type; on
Python 3.x this means the 'bytes' type. Bytes are written
in and retrieved exactly as given, but it will raise TypeErrors
if something other than bytes are written.
SpooledStringIO is a spooled file-like-object that only accepts
unicode values. On Python 2.x this means the 'unicode' type and
on Python 3.x this means the 'str' type. Values are accepted as
unicode and then coerced into utf-8 encoded bytes for storage. On
retrieval, the values are returned as unicode.
Brant Watson