
``pickle`` --- Python object serialization
******************************************

The ``pickle`` module implements a fundamental, but powerful algorithm
for serializing and de-serializing a Python object structure.
"Pickling" is the process whereby a Python object hierarchy is
converted into a byte stream, and "unpickling" is the inverse
operation, whereby a byte stream is converted back into an object
hierarchy.  Pickling (and unpickling) is alternatively known as
"serialization", "marshalling," [1] or "flattening", however, to avoid
confusion, the terms used here are "pickling" and "unpickling"..


Relationship to other Python modules
====================================

The ``pickle`` module has an transparent optimizer (``_pickle``)
written in C.  It is used whenever available.  Otherwise the pure
Python implementation is used.

Python has a more primitive serialization module called ``marshal``,
but in general ``pickle`` should always be the preferred way to
serialize Python objects.  ``marshal`` exists primarily to support
Python's ``.pyc`` files.

The ``pickle`` module differs from ``marshal`` several significant
ways:

* The ``pickle`` module keeps track of the objects it has already
  serialized, so that later references to the same object won't be
  serialized again. ``marshal`` doesn't do this.

  This has implications both for recursive objects and object sharing.
  Recursive objects are objects that contain references to themselves.
  These are not handled by marshal, and in fact, attempting to marshal
  recursive objects will crash your Python interpreter.  Object
  sharing happens when there are multiple references to the same
  object in different places in the object hierarchy being serialized.
  ``pickle`` stores such objects only once, and ensures that all other
  references point to the master copy.  Shared objects remain shared,
  which can be very important for mutable objects.

* ``marshal`` cannot be used to serialize user-defined classes and
  their instances.  ``pickle`` can save and restore class instances
  transparently, however the class definition must be importable and
  live in the same module as when the object was stored.

* The ``marshal`` serialization format is not guaranteed to be
  portable across Python versions.  Because its primary job in life is
  to support ``.pyc`` files, the Python implementers reserve the right
  to change the serialization format in non-backwards compatible ways
  should the need arise. The ``pickle`` serialization format is
  guaranteed to be backwards compatible across Python releases.

Warning: The ``pickle`` module is not intended to be secure against erroneous
  or maliciously constructed data.  Never unpickle data received from
  an untrusted or unauthenticated source.

Note that serialization is a more primitive notion than persistence;
although ``pickle`` reads and writes file objects, it does not handle
the issue of naming persistent objects, nor the (even more
complicated) issue of concurrent access to persistent objects.  The
``pickle`` module can transform a complex object into a byte stream
and it can transform the byte stream into an object with the same
internal structure.  Perhaps the most obvious thing to do with these
byte streams is to write them onto a file, but it is also conceivable
to send them across a network or store them in a database.  The module
``shelve`` provides a simple interface to pickle and unpickle objects
on DBM-style database files.


Data stream format
==================

The data format used by ``pickle`` is Python-specific.  This has the
advantage that there are no restrictions imposed by external standards
such as XDR (which can't represent pointer sharing); however it means
that non-Python programs may not be able to reconstruct pickled Python
objects.

By default, the ``pickle`` data format uses a compact binary
representation. The module ``pickletools`` contains tools for
analyzing data streams generated by ``pickle``.

There are currently 4 different protocols which can be used for
pickling.

* Protocol version 0 is the original ASCII protocol and is backwards
  compatible with earlier versions of Python.

* Protocol version 1 is the old binary format which is also compatible
  with earlier versions of Python.

* Protocol version 2 was introduced in Python 2.3.  It provides much
  more efficient pickling of *new-style class*es.

* Protocol version 3 was added in Python 3.0.  It has explicit support
  for bytes and cannot be unpickled by Python 2.x pickle modules.
  This is the current recommended protocol, use it whenever it is
  possible.

Refer to **PEP 307** for information about improvements brought by
protocol 2.  See ``pickletools``'s source code for extensive comments
about opcodes used by pickle protocols.


Module Interface
================

To serialize an object hierarchy, you first create a pickler, then you
call the pickler's ``dump()`` method.  To de-serialize a data stream,
you first create an unpickler, then you call the unpickler's
``load()`` method.  The ``pickle`` module provides the following
constant:

pickle.HIGHEST_PROTOCOL

   The highest protocol version available.  This value can be passed
   as a *protocol* value.

Note: Be sure to always open pickle files created with protocols >= 1 in
  binary mode. For the old ASCII-based pickle protocol 0 you can use
  either text mode or binary mode as long as you stay consistent.A
  pickle file written with protocol 0 in binary mode will contain lone
  linefeeds as line terminators and therefore will look "funny" when
  viewed in Notepad or other editors which do not support this format.

pickle.DEFAULT_PROTOCOL

   The default protocol used for pickling.  May be less than
   HIGHEST_PROTOCOL. Currently the default protocol is 3; a backward-
   incompatible protocol designed for Python 3.0.

The ``pickle`` module provides the following functions to make the
pickling process more convenient:

pickle.dump(obj, file[, protocol])

   Write a pickled representation of *obj* to the open file object
   *file*.  This is equivalent to ``Pickler(file,
   protocol).dump(obj)``.

   The optional *protocol* argument tells the pickler to use the given
   protocol; supported protocols are 0, 1, 2, 3.  The default protocol
   is 3; a backward-incompatible protocol designed for Python 3.0.

   Specifying a negative protocol version selects the highest protocol
   version supported.  The higher the protocol used, the more recent
   the version of Python needed to read the pickle produced.

   The *file* argument must have a write() method that accepts a
   single bytes argument.  It can thus be a file object opened for
   binary writing, a io.BytesIO instance, or any other custom object
   that meets this interface.

pickle.dumps(obj[, protocol])

   Return the pickled representation of the object as a ``bytes``
   object, instead of writing it to a file.

   The optional *protocol* argument tells the pickler to use the given
   protocol; supported protocols are 0, 1, 2, 3.  The default protocol
   is 3; a backward-incompatible protocol designed for Python 3.0.

   Specifying a negative protocol version selects the highest protocol
   version supported.  The higher the protocol used, the more recent
   the version of Python needed to read the pickle produced.

pickle.load(file[, *, encoding="ASCII", errors="strict"])

   Read a pickled object representation from the open file object
   *file* and return the reconstituted object hierarchy specified
   therein.  This is equivalent to ``Unpickler(file).load()``.

   The protocol version of the pickle is detected automatically, so no
   protocol argument is needed.  Bytes past the pickled object's
   representation are ignored.

   The argument *file* must have two methods, a read() method that
   takes an integer argument, and a readline() method that requires no
   arguments.  Both methods should return bytes.  Thus *file* can be a
   binary file object opened for reading, a BytesIO object, or any
   other custom object that meets this interface.

   Optional keyword arguments are encoding and errors, which are used
   to decode 8-bit string instances pickled by Python 2.x.  These
   default to 'ASCII' and 'strict', respectively.

pickle.loads(bytes_object[, *, encoding="ASCII", errors="strict"])

   Read a pickled object hierarchy from a ``bytes`` object and return
   the reconstituted object hierarchy specified therein

   The protocol version of the pickle is detected automatically, so no
   protocol argument is needed.  Bytes past the pickled object's
   representation are ignored.

   Optional keyword arguments are encoding and errors, which are used
   to decode 8-bit string instances pickled by Python 2.x.  These
   default to 'ASCII' and 'strict', respectively.

The ``pickle`` module defines three exceptions:

exception pickle.PickleError

   Common base class for the other pickling exceptions.  It inherits
   ``Exception``.

exception pickle.PicklingError

   Error raised when an unpicklable object is encountered by
   ``Pickler``. It inherits ``PickleError``.

   Refer to *What can be pickled and unpickled?* to learn what kinds
   of objects can be pickled.

exception pickle.UnpicklingError

   Error raised when there a problem unpickling an object, such as a
   data corruption or a security violation.  It inherits
   ``PickleError``.

   Note that other exceptions may also be raised during unpickling,
   including (but not necessarily limited to) AttributeError,
   EOFError, ImportError, and IndexError.

The ``pickle`` module exports two classes, ``Pickler`` and
``Unpickler``:

class pickle.Pickler(file[, protocol])

   This takes a binary file for writing a pickle data stream.

   The optional *protocol* argument tells the pickler to use the given
   protocol; supported protocols are 0, 1, 2, 3.  The default protocol
   is 3; a backward-incompatible protocol designed for Python 3.0.

   Specifying a negative protocol version selects the highest protocol
   version supported.  The higher the protocol used, the more recent
   the version of Python needed to read the pickle produced.

   The *file* argument must have a write() method that accepts a
   single bytes argument.  It can thus be a file object opened for
   binary writing, a io.BytesIO instance, or any other custom object
   that meets this interface.

   dump(obj)

      Write a pickled representation of *obj* to the open file object
      given in the constructor.

   persistent_id(obj)

      Do nothing by default.  This exists so a subclass can override
      it.

      If ``persistent_id()`` returns ``None``, *obj* is pickled as
      usual.  Any other value causes ``Pickler`` to emit the returned
      value as a persistent ID for *obj*.  The meaning of this
      persistent ID should be defined by
      ``Unpickler.persistent_load()``.  Note that the value returned
      by ``persistent_id()`` cannot itself have a persistent ID.

      See *Persistence of External Objects* for details and examples
      of uses.

   clear_memo()

      Deprecated.  Use the ``clear()`` method on ``memo``, instead.
      Clear the pickler's memo, useful when reusing picklers.

   fast

      Deprecated. Enable fast mode if set to a true value.  The fast
      mode disables the usage of memo, therefore speeding the pickling
      process by not generating superfluous PUT opcodes.  It should
      not be used with self-referential objects, doing otherwise will
      cause ``Pickler`` to recurse infinitely.

      Use ``pickletools.optimize()`` if you need more compact pickles.

   memo

      Dictionary holding previously pickled objects to allow shared or
      recursive objects to pickled by reference as opposed to by
      value.

It is possible to make multiple calls to the ``dump()`` method of the
same ``Pickler`` instance.  These must then be matched to the same
number of calls to the ``load()`` method of the corresponding
``Unpickler`` instance.  If the same object is pickled by multiple
``dump()`` calls, the ``load()`` will all yield references to the same
object.

Please note, this is intended for pickling multiple objects without
intervening modifications to the objects or their parts.  If you
modify an object and then pickle it again using the same ``Pickler``
instance, the object is not pickled again --- a reference to it is
pickled and the ``Unpickler`` will return the old value, not the
modified one.

class pickle.Unpickler(file[, *, encoding="ASCII", errors="strict"])

   This takes a binary file for reading a pickle data stream.

   The protocol version of the pickle is detected automatically, so no
   protocol argument is needed.

   The argument *file* must have two methods, a read() method that
   takes an integer argument, and a readline() method that requires no
   arguments.  Both methods should return bytes.  Thus *file* can be a
   binary file object opened for reading, a BytesIO object, or any
   other custom object that meets this interface.

   Optional keyword arguments are encoding and errors, which are used
   to decode 8-bit string instances pickled by Python 2.x.  These
   default to 'ASCII' and 'strict', respectively.

   load()

      Read a pickled object representation from the open file object
      given in the constructor, and return the reconstituted object
      hierarchy specified therein.  Bytes past the pickled object's
      representation are ignored.

   persistent_load(pid)

      Raise an ``UnpickingError`` by default.

      If defined, ``persistent_load()`` should return the object
      specified by the persistent ID *pid*.  If an invalid persistent
      ID is encountered, an ``UnpickingError`` should be raised.

      See *Persistence of External Objects* for details and examples
      of uses.

   find_class(module, name)

      Import *module* if necessary and return the object called *name*
      from it, where the *module* and *name* arguments are ``str``
      objects.  Note, unlike its name suggests, ``find_class()`` is
      also used for finding functions.

      Subclasses may override this to gain control over what type of
      objects and how they can be loaded, potentially reducing
      security risks. Refer to *Restricting Globals* for details.


What can be pickled and unpickled?
==================================

The following types can be pickled:

* ``None``, ``True``, and ``False``

* integers, floating point numbers, complex numbers

* strings, bytes, bytearrays

* tuples, lists, sets, and dictionaries containing only picklable
  objects

* functions defined at the top level of a module

* built-in functions defined at the top level of a module

* classes that are defined at the top level of a module

* instances of such classes whose ``__dict__`` or ``__setstate__()``
  is picklable  (see section *Pickling Class Instances* for details)

Attempts to pickle unpicklable objects will raise the
``PicklingError`` exception; when this happens, an unspecified number
of bytes may have already been written to the underlying file.  Trying
to pickle a highly recursive data structure may exceed the maximum
recursion depth, a ``RuntimeError`` will be raised in this case.  You
can carefully raise this limit with ``sys.setrecursionlimit()``.

Note that functions (built-in and user-defined) are pickled by "fully
qualified" name reference, not by value.  This means that only the
function name is pickled, along with the name of module the function
is defined in.  Neither the function's code, nor any of its function
attributes are pickled.  Thus the defining module must be importable
in the unpickling environment, and the module must contain the named
object, otherwise an exception will be raised. [2]

Similarly, classes are pickled by named reference, so the same
restrictions in the unpickling environment apply.  Note that none of
the class's code or data is pickled, so in the following example the
class attribute ``attr`` is not restored in the unpickling
environment:

   class Foo:
       attr = 'A class attribute'

   picklestring = pickle.dumps(Foo)

These restrictions are why picklable functions and classes must be
defined in the top level of a module.

Similarly, when class instances are pickled, their class's code and
data are not pickled along with them.  Only the instance data are
pickled.  This is done on purpose, so you can fix bugs in a class or
add methods to the class and still load objects that were created with
an earlier version of the class.  If you plan to have long-lived
objects that will see many versions of a class, it may be worthwhile
to put a version number in the objects so that suitable conversions
can be made by the class's ``__setstate__()`` method.


Pickling Class Instances
========================

In this section, we describe the general mechanisms available to you
to define, customize, and control how class instances are pickled and
unpickled.

In most cases, no additional code is needed to make instances
picklable.  By default, pickle will retrieve the class and the
attributes of an instance via introspection. When a class instance is
unpickled, its ``__init__()`` method is usually *not* invoked.  The
default behaviour first creates an uninitialized instance and then
restores the saved attributes.  The following code shows an
implementation of this behaviour:

   def save(obj):
       return (obj.__class__, obj.__dict__)

   def load(cls, attributes):
       obj = cls.__new__(cls)
       obj.__dict__.update(attributes)
       return obj

Classes can alter the default behaviour by providing one or severals
special methods.  In protocol 2 and newer, classes that implements the
``__getnewargs__()`` method can dictate the values passed to the
``__new__()`` method upon unpickling.  This is often needed for
classes whose ``__new__()`` method requires arguments.

Classes can further influence how their instances are pickled; if the
class defines the method ``__getstate__()``, it is called and the
returned object is pickled as the contents for the instance, instead
of the contents of the instance's dictionary.  If the
``__getstate__()`` method is absent, the instance's ``__dict__`` is
pickled as usual.

Upon unpickling, if the class defines ``__setstate__()``, it is called
with the unpickled state.  In that case, there is no requirement for
the state object to be a dictionary. Otherwise, the pickled state must
be a dictionary and its items are assigned to the new instance's
dictionary.

Note: If ``__getstate__()`` returns a false value, the ``__setstate__()``
  method will not be called.

Refer to the section *Handling Stateful Objects* for more information
about how to use the methods ``__getstate__()`` and
``__setstate__()``.

As we shall see, pickle does not use directly the methods described
above.  In fact, these methods are part of the copy protocol which
implements the ``__reduce__()`` special method.  The copy protocol
provides a unified interface for retrieving the data necessary for
pickling and copying objects. [3]

Although powerful, implementing ``__reduce__()`` directly in your
classes is error prone.  For this reason, class designers should use
the high-level interface (i.e., ``__getnewargs__()``,
``__getstate__()`` and ``__setstate__()``) whenever possible.  We will
show however cases where using ``__reduce__()`` is the only option or
leads to more efficient pickling or both.

The interface is currently defined as follow. The ``__reduce__()``
method takes no argument and shall return either a string or
preferably a tuple (the returned object is often refered as the
"reduce value").

If a string is returned, the string should be interpreted as the name
of a global variable.  It should be the object's local name relative
to its module; the pickle module searches the module namespace to
determine the object's module.  This behaviour is typically useful for
singletons.

When a tuple is returned, it must be between two and five items long.
Optional items can either be omitted, or ``None`` can be provided as
their value.  The semantics of each item are in order:

* A callable object that will be called to create the initial version
  of the object.

* A tuple of arguments for the callable object. An empty tuple must be
  given if the callable does not accept any argument.

* Optionally, the object's state, which will be passed to the object's
  ``__setstate__()`` method as previously described.  If the object
  has no such method then, the value must be a dictionary and it will
  be added to the object's ``__dict__`` attribute.

* Optionally, an iterator (and not a sequence) yielding successive
  items.  These items will be appended to the object either using
  ``obj.append(item)`` or, in batch, using
  ``obj.extend(list_of_items)``.  This is primarily used for list
  subclasses, but may be used by other classes as long as they have
  ``append()`` and ``extend()`` methods with the appropriate
  signature. (Whether ``append()`` or ``extend()`` is used depends on
  which pickle protocol version is used as well as the number of items
  to append, so both must be supported.)

* Optionally, an iterator (not a sequence) yielding successive key-
  value pairs. These items will be stored to the object using
  ``obj[key] = value``.  This is primarily used for dictionary
  subclasses, but may be used by other classes as long as they
  implement ``__setitem__()``.

Alternatively, a ``__reduce_ex__()`` method may be defined.  The only
difference is this method should take a single integer argument, the
protocol version.  When defined, pickle will prefer it over the
``__reduce__()`` method.  In addition, ``__reduce__()`` automatically
becomes a synonym for the extended version.  The main use for this
method is to provide backwards-compatible reduce values for older
Python releases.


Persistence of External Objects
-------------------------------

For the benefit of object persistence, the ``pickle`` module supports
the notion of a reference to an object outside the pickled data
stream.  Such objects are referenced by a persistent ID, which should
be either a string of alphanumeric characters (for protocol 0) [4] or
just an arbitrary object (for any newer protocol).

The resolution of such persistent IDs is not defined by the ``pickle``
module; it will delegate this resolution to the user defined methods
on the pickler and unpickler, ``persistent_id()`` and
``persistent_load()`` respectively.

To pickle objects that have an external persistent id, the pickler
must have a custom ``persistent_id()`` method that takes an object as
an argument and returns either ``None`` or the persistent id for that
object.  When ``None`` is returned, the pickler simply pickles the
object as normal.  When a persistent ID string is returned, the
pickler will pickle that object, along with a marker so that the
unpickler will recognize it as a persistent ID.

To unpickle external objects, the unpickler must have a custom
``persistent_load()`` method that takes a persistent ID object and
returns the referenced object.

Here is a comprehensive example presenting how persistent ID can be
used to pickle external objects by reference.

   # Simple example presenting how persistent ID can be used to pickle
   # external objects by reference.

   import pickle
   import sqlite3
   from collections import namedtuple

   # Simple class representing a record in our database.
   MemoRecord = namedtuple("MemoRecord", "key, task")

   class DBPickler(pickle.Pickler):

       def persistent_id(self, obj):
           # Instead of pickling MemoRecord as a regular class instance, we emit a
           # persistent ID.
           if isinstance(obj, MemoRecord):
               # Here, our persistent ID is simply a tuple, containing a tag and a
               # key, which refers to a specific record in the database.
               return ("MemoRecord", obj.key)
           else:
               # If obj does not have a persistent ID, return None. This means obj
               # needs to be pickled as usual.
               return None


   class DBUnpickler(pickle.Unpickler):

       def __init__(self, file, connection):
           super().__init__(file)
           self.connection = connection

       def persistent_load(self, pid):
           # This method is invoked whenever a persistent ID is encountered.
           # Here, pid is the tuple returned by DBPickler.
           cursor = self.connection.cursor()
           type_tag, key_id = pid
           if type_tag == "MemoRecord":
               # Fetch the referenced record from the database and return it.
               cursor.execute("SELECT * FROM memos WHERE key=?", (str(key_id),))
               key, task = cursor.fetchone()
               return MemoRecord(key, task)
           else:
               # Always raises an error if you cannot return the correct object.
               # Otherwise, the unpickler will think None is the object referenced
               # by the persistent ID.
               raise pickle.UnpicklingError("unsupported persistent object")


   def main():
       import io, pprint

       # Initialize and populate our database.
       conn = sqlite3.connect(":memory:")
       cursor = conn.cursor()
       cursor.execute("CREATE TABLE memos(key INTEGER PRIMARY KEY, task TEXT)")
       tasks = (
           'give food to fish',
           'prepare group meeting',
           'fight with a zebra',
           )
       for task in tasks:
           cursor.execute("INSERT INTO memos VALUES(NULL, ?)", (task,))

       # Fetch the records to be pickled.
       cursor.execute("SELECT * FROM memos")
       memos = [MemoRecord(key, task) for key, task in cursor]
       # Save the records using our custom DBPickler.
       file = io.BytesIO()
       DBPickler(file).dump(memos)

       print("Pickled records:")
       pprint.pprint(memos)

       # Update a record, just for good measure.
       cursor.execute("UPDATE memos SET task='learn italian' WHERE key=1")

       # Load the records from the pickle data stream.
       file.seek(0)
       memos = DBUnpickler(file, conn).load()

       print("Unpickled records:")
       pprint.pprint(memos)


   if __name__ == '__main__':
       main()


Handling Stateful Objects
-------------------------

Here's an example that shows how to modify pickling behavior for a
class. The ``TextReader`` class opens a text file, and returns the
line number and line contents each time its ``readline()`` method is
called. If a ``TextReader`` instance is pickled, all attributes
*except* the file object member are saved. When the instance is
unpickled, the file is reopened, and reading resumes from the last
location. The ``__setstate__()`` and ``__getstate__()`` methods are
used to implement this behavior.

   class TextReader:
       """Print and number lines in a text file."""

       def __init__(self, filename):
           self.filename = filename
           self.file = open(filename)
           self.lineno = 0

       def readline(self):
           self.lineno += 1
           line = self.file.readline()
           if not line:
               return None
           if line.endswith("\n"):
               line = line[:-1]
           return "%i: %s" % (self.lineno, line)

       def __getstate__(self):
           # Copy the object's state from self.__dict__ which contains
           # all our instance attributes. Always use the dict.copy()
           # method to avoid modifying the original state.
           state = self.__dict__.copy()
           # Remove the unpicklable entries.
           del state['file']
           return state

       def __setstate__(self, state):
           # Restore instance attributes (i.e., filename and lineno).
           self.__dict__.update(state)
           # Restore the previously opened file's state. To do so, we need to
           # reopen it and read from it until the line count is restored.
           file = open(self.filename)
           for _ in range(self.lineno):
               file.readline()
           # Finally, save the file.
           self.file = file

A sample usage might be something like this:

   >>> reader = TextReader("hello.txt")
   >>> reader.readline()
   '1: Hello world!'
   >>> reader.readline()
   '2: I am line number two.'
   >>> new_reader = pickle.loads(pickle.dumps(reader))
   >>> new_reader.readline()
   '3: Goodbye!'


Restricting Globals
===================

By default, unpickling will import any class or function that it finds
in the pickle data.  For many applications, this behaviour is
unacceptable as it permits the unpickler to import and invoke
arbitrary code.  Just consider what this hand-crafted pickle data
stream does when loaded:

   >>> import pickle
   >>> pickle.loads(b"cos\nsystem\n(S'echo hello world'\ntR.")
   hello world
   0

In this example, the unpickler imports the ``os.system()`` function
and then apply the string argument "echo hello world".  Although this
example is inoffensive, it is not difficult to imagine one that could
damage your system.

For this reason, you may want to control what gets unpickled by
customizing ``Unpickler.find_class()``.  Unlike its name suggests,
``find_class()`` is called whenever a global (i.e., a class or a
function) is requested.  Thus it is possible to either forbid
completely globals or restrict them to a safe subset.

Here is an example of an unpickler allowing only few safe classes from
the ``builtins`` module to be loaded:

   import builtins
   import io
   import pickle

   safe_builtins = {
       'range',
       'complex',
       'set',
       'frozenset',
       'slice',
   }

   class RestrictedUnpickler(pickle.Unpickler):

       def find_class(self, module, name):
           # Only allow safe classes from builtins.
           if module == "builtins" and name in safe_builtins:
               return getattr(builtins, name)
           # Forbid everything else.
           raise pickle.UnpicklingError("global '%s.%s' is forbidden" %
                                        (module, name))

   def restricted_loads(s):
       """Helper function analogous to pickle.loads()."""
       return RestrictedUnpickler(io.BytesIO(s)).load()

A sample usage of our unpickler working has intended:

   >>> restricted_loads(pickle.dumps([1, 2, range(15)]))
   [1, 2, range(0, 15)]
   >>> restricted_loads(b"cos\nsystem\n(S'echo hello world'\ntR.")
   Traceback (most recent call last):
     ...
   pickle.UnpicklingError: global 'os.system' is forbidden
   >>> restricted_loads(b'cbuiltins\neval\n'
   ...                  b'(S\'getattr(__import__("os"), "system")'
   ...                  b'("echo hello world")\'\ntR.')
   Traceback (most recent call last):
     ...
   pickle.UnpicklingError: global 'builtins.eval' is forbidden

As our examples shows, you have to be careful with what you allow to
be unpickled.  Therefore if security is a concern, you may want to
consider alternatives such as the marshalling API in ``xmlrpc.client``
or third-party solutions.


Example
=======

For the simplest code, use the ``dump()`` and ``load()`` functions.
Note that a self-referencing list is pickled and restored correctly.

   import pickle

   data1 = {'a': [1, 2.0, 3, 4+6j],
            'b': ("string", "string using Unicode features \u0394"),
            'c': None}

   selfref_list = [1, 2, 3]
   selfref_list.append(selfref_list)

   output = open('data.pkl', 'wb')

   # Pickle dictionary using protocol 2.
   pickle.dump(data1, output, 2)

   # Pickle the list using the highest protocol available.
   pickle.dump(selfref_list, output, -1)

   output.close()

The following example reads the resulting pickled data.  When reading
a pickle-containing file, you should open the file in binary mode
because you can't be sure if the ASCII or binary format was used.

   import pprint, pickle

   pkl_file = open('data.pkl', 'rb')

   data1 = pickle.load(pkl_file)
   pprint.pprint(data1)

   data2 = pickle.load(pkl_file)
   pprint.pprint(data2)

   pkl_file.close()

See also:

   Module ``copyreg``
      Pickle interface constructor registration for extension types.

   Module ``shelve``
      Indexed databases of objects; uses ``pickle``.

   Module ``copy``
      Shallow and deep object copying.

   Module ``marshal``
      High-performance serialization of built-in types.

-[ Footnotes ]-

[1] Don't confuse this with the ``marshal`` module

[2] The exception raised will likely be an ``ImportError`` or an
    ``AttributeError`` but it could be something else.

[3] The ``copy`` module uses this protocol for shallow and deep
    copying operations.

[4] The limitation on alphanumeric characters is due to the fact the
    persistent IDs, in protocol 0, are delimited by the newline
    character.  Therefore if any kind of newline characters occurs in
    persistent IDs, the resulting pickle will become unreadable.
