Doc/lib/libshelve.tex

   1 \section{\module{shelve} ---
   2          Python object persistence}
   3
   4 \declaremodule{standard}{shelve}
   5 \modulesynopsis{Python object persistence.}
   6
   7
   8 A ``shelf'' is a persistent, dictionary-like object.  The difference
   9 with ``dbm'' databases is that the values (not the keys!) in a shelf
  10 can be essentially arbitrary Python objects --- anything that the
  11 \refmodule{pickle} module can handle.  This includes most class
  12 instances, recursive data types, and objects containing lots of shared
  13 sub-objects.  The keys are ordinary strings.
  14 \refstmodindex{pickle}
  15
  16 \begin{funcdesc}{open}{filename\optional{,flag='c'\optional{,protocol=\code{None}\optional{,writeback=\code{False}\optional{,binary=\code{None}}}}}}
  17 Open a persistent dictionary.  The filename specified is the base filename
  18 for the underlying database.  As a side-effect, an extension may be added to
  19 the filename and more than one file may be created.  By default, the
  20 underlying database file is opened for reading and writing.  The optional
  21 {}\var{flag} pararameter has the same interpretation as the \var{flag}
  22 parameter of \function{anydbm.open}.
  23
  24 By default, version 0 pickles are used to serialize values.
  25 The version of the pickle protocol can be specified with the
  26 \var{protocol} parameter. \versionchanged[The \var{protocol}
  27 parameter was added. The \var{binary} parameter is deprecated
  28 and provided for backwards compatibility only]{2.3}
  29
  30 By default, mutations to persistent-dictionary mutable entries are not
  31 automatically written back.  If the optional \var{writeback} parameter
  32 is set to {}\var{True}, all entries accessed are cached in memory, and
  33 written back at close time; this can make it handier to mutate mutable
  34 entries in the persistent dictionary, but, if many entries are
  35 accessed, it can consume vast amounts of memory for the cache, and it
  36 can make the close operation very slow since all accessed entries are
  37 written back (there is no way to determine which accessed entries are
  38 mutable, nor which ones were actually mutated).
  39
  40 \end{funcdesc}
  41
  42 Shelve objects support all methods supported by dictionaries.  This eases
  43 the transition from dictionary based scripts to those requiring persistent
  44 storage.
  45
  46 \subsection{Restrictions}
  47
  48 \begin{itemize}
  49
  50 \item
  51 The choice of which database package will be used
  52 (such as \refmodule{dbm}, \refmodule{gdbm} or \refmodule{bsddb}) depends on
  53 which interface is available.  Therefore it is not safe to open the database
  54 directly using \refmodule{dbm}.  The database is also (unfortunately) subject
  55 to the limitations of \refmodule{dbm}, if it is used --- this means
  56 that (the pickled representation of) the objects stored in the
  57 database should be fairly small, and in rare cases key collisions may
  58 cause the database to refuse updates.
  59 \refbimodindex{dbm}
  60 \refbimodindex{gdbm}
  61 \refbimodindex{bsddb}
  62
  63 \item
  64 Depending on the implementation, closing a persistent dictionary may
  65 or may not be necessary to flush changes to disk.  The \method{__del__}
  66 method of the \class{Shelf} class calls the \method{close} method, so the
  67 programmer generally need not do this explicitly.
  68
  69 \item
  70 The \module{shelve} module does not support \emph{concurrent} read/write
  71 access to shelved objects.  (Multiple simultaneous read accesses are
  72 safe.)  When a program has a shelf open for writing, no other program
  73 should have it open for reading or writing.  \UNIX{} file locking can
  74 be used to solve this, but this differs across \UNIX{} versions and
  75 requires knowledge about the database implementation used.
  76
  77 \end{itemize}
  78
  79 \begin{classdesc}{Shelf}{dict\optional{, protocol=None\optional{, writeback=False\optional{, binary=None}}}}
  80 A subclass of \class{UserDict.DictMixin} which stores pickled values in the
  81 \var{dict} object.
  82
  83 By default, version 0 pickles are used to serialize values.  The
  84 version of the pickle protocol can be specified with the
  85 \var{protocol} parameter. See the \module{pickle} documentation for a
  86 discussion of the pickle protocols. \versionchanged[The \var{protocol}
  87 parameter was added. The \var{binary} parameter is deprecated and
  88 provided for backwards compatibility only]{2.3}
  89
  90 If the \var{writeback} parameter is \code{True}, the object will hold a
  91 cache of all entries accessed and write them back to the \var{dict} at
  92 sync and close times.  This allows natural operations on mutable entries,
  93 but can consume much more memory and make sync and close take a long time.
  94 \end{classdesc}
  95
  96 \begin{classdesc}{BsdDbShelf}{dict\optional{, protocol=None\optional{, writeback=False\optional{, binary=None}}}}
  97
  98 A subclass of \class{Shelf} which exposes \method{first},
  99 \method{next}, \method{previous}, \method{last} and
 100 \method{set_location} which are available in the \module{bsddb} module
 101 but not in other database modules.  The \var{dict} object passed to
 102 the constructor must support those methods.  This is generally
 103 accomplished by calling one of \function{bsddb.hashopen},
 104 \function{bsddb.btopen} or \function{bsddb.rnopen}.  The optional
 105 \var{protocol}, \var{writeback}, and \var{binary} parameters have the
 106 same interpretation as for the \class{Shelf} class.
 107
 108 \end{classdesc}
 109
 110 \begin{classdesc}{DbfilenameShelf}{filename\optional{, flag='c'\optional{, protocol=None\optional{, writeback=False\optional{, binary=None}}}}}
 111
 112 A subclass of \class{Shelf} which accepts a \var{filename} instead of
 113 a dict-like object.  The underlying file will be opened using
 114 {}\function{anydbm.open}.  By default, the file will be created and
 115 opened for both read and write.  The optional \var{flag} parameter has
 116 the same interpretation as for the \function{open} function.  The
 117 optional \var{protocol}, \var{writeback}, and \var{binary} parameters
 118 have the same interpretation as for the \class{Shelf} class.
 119
 120 \end{classdesc}
 121
 122 \subsection{Example}
 123
 124 To summarize the interface (\code{key} is a string, \code{data} is an
 125 arbitrary object):
 126
 127 \begin{verbatim}
 128 import shelve
 129
 130 d = shelve.open(filename) # open -- file may get suffix added by low-level
 131                           # library
 132
 133 d[key] = data   # store data at key (overwrites old data if
 134                 # using an existing key)
 135 data = d[key]   # retrieve a COPY of data at key (raise KeyError if no
 136                 # such key)
 137 del d[key]      # delete data stored at key (raises KeyError
 138                 # if no such key)
 139 flag = d.has_key(key)   # true if the key exists
 140 list = d.keys() # a list of all existing keys (slow!)
 141
 142 # as d was opened WITHOUT writeback=True, beware:
 143 d['xx'] = range(4)  # this works as expected, but...
 144 d['xx'].append(5)   # *this doesn't!* -- d['xx'] is STILL range(4)!!!
 145 # having opened d without writeback=True, you need to code carefully:
 146 temp = d['xx']      # extracts the copy
 147 temp.append(5)      # mutates the copy
 148 d['xx'] = temp      # stores the copy right back, to persist it
 149 # or, d=shelve.open(filename,writeback=True) would let you just code
 150 # d['xx'].append(5) and have it work as expected, BUT it would also
 151 # consume more memory and make the d.close() operation slower.
 152
 153 d.close()       # close it
 154 \end{verbatim}
 155
 156 \begin{seealso}
 157   \seemodule{anydbm}{Generic interface to \code{dbm}-style databases.}
 158   \seemodule{bsddb}{BSD \code{db} database interface.}
 159   \seemodule{dbhash}{Thin layer around the \module{bsddb} which provides an
 160   \function{open} function like the other database modules.}
 161   \seemodule{dbm}{Standard \UNIX{} database interface.}
 162   \seemodule{dumbdbm}{Portable implementation of the \code{dbm} interface.}
 163   \seemodule{gdbm}{GNU database interface, based on the \code{dbm} interface.}
 164   \seemodule{pickle}{Object serialization used by \module{shelve}.}
 165   \seemodule{cPickle}{High-performance version of \refmodule{pickle}.}
 166 \end{seealso}