Doc/lib/libstruct.tex

   1 \section{\module{struct} ---
   2          Interpret strings as packed binary data}
   3 \declaremodule{builtin}{struct}
   4
   5 \modulesynopsis{Interpret strings as packed binary data.}
   6
   7 \indexii{C}{structures}
   8 \indexiii{packing}{binary}{data}
   9
  10 This module performs conversions between Python values and C
  11 structs represented as Python strings.  It uses \dfn{format strings}
  12 (explained below) as compact descriptions of the lay-out of the C
  13 structs and the intended conversion to/from Python values.  This can
  14 be used in handling binary data stored in files or from network
  15 connections, among other sources.
  16
  17 The module defines the following exception and functions:
  18
  19
  20 \begin{excdesc}{error}
  21   Exception raised on various occasions; argument is a string
  22   describing what is wrong.
  23 \end{excdesc}
  24
  25 \begin{funcdesc}{pack}{fmt, v1, v2, \textrm{\ldots}}
  26   Return a string containing the values
  27   \code{\var{v1}, \var{v2}, \textrm{\ldots}} packed according to the given
  28   format.  The arguments must match the values required by the format
  29   exactly.
  30 \end{funcdesc}
  31
  32 \begin{funcdesc}{unpack}{fmt, string}
  33   Unpack the string (presumably packed by \code{pack(\var{fmt},
  34   \textrm{\ldots})}) according to the given format.  The result is a
  35   tuple even if it contains exactly one item.  The string must contain
  36   exactly the amount of data required by the format
  37   (\code{len(\var{string})} must equal \code{calcsize(\var{fmt})}).
  38 \end{funcdesc}
  39
  40 \begin{funcdesc}{calcsize}{fmt}
  41   Return the size of the struct (and hence of the string)
  42   corresponding to the given format.
  43 \end{funcdesc}
  44
  45 Format characters have the following meaning; the conversion between
  46 C and Python values should be obvious given their types:
  47
  48 \begin{tableiv}{c|l|l|c}{samp}{Format}{C Type}{Python}{Notes}
  49   \lineiv{x}{pad byte}{no value}{}
  50   \lineiv{c}{\ctype{char}}{string of length 1}{}
  51   \lineiv{b}{\ctype{signed char}}{integer}{}
  52   \lineiv{B}{\ctype{unsigned char}}{integer}{}
  53   \lineiv{h}{\ctype{short}}{integer}{}
  54   \lineiv{H}{\ctype{unsigned short}}{integer}{}
  55   \lineiv{i}{\ctype{int}}{integer}{}
  56   \lineiv{I}{\ctype{unsigned int}}{long}{}
  57   \lineiv{l}{\ctype{long}}{integer}{}
  58   \lineiv{L}{\ctype{unsigned long}}{long}{}
  59   \lineiv{q}{\ctype{long long}}{long}{(1)}
  60   \lineiv{Q}{\ctype{unsigned long long}}{long}{(1)}
  61   \lineiv{f}{\ctype{float}}{float}{}
  62   \lineiv{d}{\ctype{double}}{float}{}
  63   \lineiv{s}{\ctype{char[]}}{string}{}
  64   \lineiv{p}{\ctype{char[]}}{string}{}
  65   \lineiv{P}{\ctype{void *}}{integer}{}
  66 \end{tableiv}
  67
  68 \noindent
  69 Notes:
  70
  71 \begin{description}
  72 \item[(1)]
  73   The \character{q} and \character{Q} conversion codes are available in
  74   native mode only if the platform C compiler supports C \ctype{long long},
  75   or, on Windows, \ctype{__int64}.  They are always available in standard
  76   modes.
  77   \versionadded{2.2}
  78 \end{description}
  79
  80
  81 A format character may be preceded by an integral repeat count.  For
  82 example, the format string \code{'4h'} means exactly the same as
  83 \code{'hhhh'}.
  84
  85 Whitespace characters between formats are ignored; a count and its
  86 format must not contain whitespace though.
  87
  88 For the \character{s} format character, the count is interpreted as the
  89 size of the string, not a repeat count like for the other format
  90 characters; for example, \code{'10s'} means a single 10-byte string, while
  91 \code{'10c'} means 10 characters.  For packing, the string is
  92 truncated or padded with null bytes as appropriate to make it fit.
  93 For unpacking, the resulting string always has exactly the specified
  94 number of bytes.  As a special case, \code{'0s'} means a single, empty
  95 string (while \code{'0c'} means 0 characters).
  96
  97 The \character{p} format character can be used to encode a Pascal
  98 string.  The first byte is the length of the stored string, with the
  99 bytes of the string following.  If count is given, it is used as the
 100 total number of bytes used, including the length byte.  If the string
 101 passed in to \function{pack()} is too long, the stored representation
 102 is truncated.  If the string is too short, padding is used to ensure
 103 that exactly enough bytes are used to satisfy the count.
 104
 105 For the \character{I}, \character{L}, \character{q} and \character{Q}
 106 format characters, the return value is a Python long integer.
 107
 108 For the \character{P} format character, the return value is a Python
 109 integer or long integer, depending on the size needed to hold a
 110 pointer when it has been cast to an integer type.  A \NULL{} pointer will
 111 always be returned as the Python integer \code{0}. When packing pointer-sized
 112 values, Python integer or long integer objects may be used.  For
 113 example, the Alpha and Merced processors use 64-bit pointer values,
 114 meaning a Python long integer will be used to hold the pointer; other
 115 platforms use 32-bit pointers and will use a Python integer.
 116
 117 By default, C numbers are represented in the machine's native format
 118 and byte order, and properly aligned by skipping pad bytes if
 119 necessary (according to the rules used by the C compiler).
 120
 121 Alternatively, the first character of the format string can be used to
 122 indicate the byte order, size and alignment of the packed data,
 123 according to the following table:
 124
 125 \begin{tableiii}{c|l|l}{samp}{Character}{Byte order}{Size and alignment}
 126   \lineiii{@}{native}{native}
 127   \lineiii{=}{native}{standard}
 128   \lineiii{<}{little-endian}{standard}
 129   \lineiii{>}{big-endian}{standard}
 130   \lineiii{!}{network (= big-endian)}{standard}
 131 \end{tableiii}
 132
 133 If the first character is not one of these, \character{@} is assumed.
 134
 135 Native byte order is big-endian or little-endian, depending on the
 136 host system.  For example, Motorola and Sun processors are big-endian;
 137 Intel and DEC processors are little-endian.
 138
 139 Native size and alignment are determined using the C compiler's
 140 \keyword{sizeof} expression.  This is always combined with native byte
 141 order.
 142
 143 Standard size and alignment are as follows: no alignment is required
 144 for any type (so you have to use pad bytes);
 145 \ctype{short} is 2 bytes;
 146 \ctype{int} and \ctype{long} are 4 bytes;
 147 \ctype{long long} (\ctype{__int64} on Windows) is 8 bytes;
 148 \ctype{float} and \ctype{double} are 32-bit and 64-bit
 149 IEEE floating point numbers, respectively.
 150
 151 Note the difference between \character{@} and \character{=}: both use
 152 native byte order, but the size and alignment of the latter is
 153 standardized.
 154
 155 The form \character{!} is available for those poor souls who claim they
 156 can't remember whether network byte order is big-endian or
 157 little-endian.
 158
 159 There is no way to indicate non-native byte order (force
 160 byte-swapping); use the appropriate choice of \character{<} or
 161 \character{>}.
 162
 163 The \character{P} format character is only available for the native
 164 byte ordering (selected as the default or with the \character{@} byte
 165 order character). The byte order character \character{=} chooses to
 166 use little- or big-endian ordering based on the host system. The
 167 struct module does not interpret this as native ordering, so the
 168 \character{P} format is not available.
 169
 170 Examples (all using native byte order, size and alignment, on a
 171 big-endian machine):
 172
 173 \begin{verbatim}
 174 >>> from struct import *
 175 >>> pack('hhl', 1, 2, 3)
 176 '\x00\x01\x00\x02\x00\x00\x00\x03'
 177 >>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
 178 (1, 2, 3)
 179 >>> calcsize('hhl')
 180 8
 181 \end{verbatim}
 182
 183 Hint: to align the end of a structure to the alignment requirement of
 184 a particular type, end the format with the code for that type with a
 185 repeat count of zero.  For example, the format \code{'llh0l'}
 186 specifies two pad bytes at the end, assuming longs are aligned on
 187 4-byte boundaries.  This only works when native size and alignment are
 188 in effect; standard size and alignment does not enforce any alignment.
 189
 190 \begin{seealso}
 191   \seemodule{array}{Packed binary storage of homogeneous data.}
 192   \seemodule{xdrlib}{Packing and unpacking of XDR data.}
 193 \end{seealso}