Doc/lib/libpopen2.tex

   1 \section{\module{popen2} ---
   2          Subprocesses with accessible I/O streams}
   3
   4 \declaremodule{standard}{popen2}
   5   \platform{Unix, Windows}
   6 \modulesynopsis{Subprocesses with accessible standard I/O streams.}
   7 \sectionauthor{Drew Csillag}{drew_csillag@geocities.com}
   8
   9
  10 This module allows you to spawn processes and connect to their
  11 input/output/error pipes and obtain their return codes under
  12 \UNIX{} and Windows.
  13
  14 Note that starting with Python 2.0, this functionality is available
  15 using functions from the \refmodule{os} module which have the same
  16 names as the factory functions here, but the order of the return
  17 values is more intuitive in the \refmodule{os} module variants.
  18
  19 The primary interface offered by this module is a trio of factory
  20 functions.  For each of these, if \var{bufsize} is specified,
  21 it specifies the buffer size for the I/O pipes.  \var{mode}, if
  22 provided, should be the string \code{'b'} or \code{'t'}; on Windows
  23 this is needed to determine whether the file objects should be opened
  24 in binary or text mode.  The default value for \var{mode} is
  25 \code{'t'}.
  26
  27 The only way to retrieve the return codes for the child processes is
  28 by using the \method{poll()} or \method{wait()} methods on the
  29 \class{Popen3} and \class{Popen4} classes; these are only available on
  30 \UNIX.  This information is not available when using the
  31 \function{popen2()}, \function{popen3()}, and \function{popen4()}
  32 functions, or the equivalent functions in the \refmodule{os} module.
  33
  34 \begin{funcdesc}{popen2}{cmd\optional{, bufsize\optional{, mode}}}
  35 Executes \var{cmd} as a sub-process.  Returns the file objects
  36 \code{(\var{child_stdout}, \var{child_stdin})}.
  37 \end{funcdesc}
  38
  39 \begin{funcdesc}{popen3}{cmd\optional{, bufsize\optional{, mode}}}
  40 Executes \var{cmd} as a sub-process.  Returns the file objects
  41 \code{(\var{child_stdout}, \var{child_stdin}, \var{child_stderr})}.
  42 \end{funcdesc}
  43
  44 \begin{funcdesc}{popen4}{cmd\optional{, bufsize\optional{, mode}}}
  45 Executes \var{cmd} as a sub-process.  Returns the file objects
  46 \code{(\var{child_stdout_and_stderr}, \var{child_stdin})}.
  47 \versionadded{2.0}
  48 \end{funcdesc}
  49
  50
  51 On \UNIX, a class defining the objects returned by the factory
  52 functions is also available.  These are not used for the Windows
  53 implementation, and are not available on that platform.
  54
  55 \begin{classdesc}{Popen3}{cmd\optional{, capturestderr\optional{, bufsize}}}
  56 This class represents a child process.  Normally, \class{Popen3}
  57 instances are created using the \function{popen2()} and
  58 \function{popen3()} factory functions described above.
  59
  60 If not using one off the helper functions to create \class{Popen3}
  61 objects, the parameter \var{cmd} is the shell command to execute in a
  62 sub-process.  The \var{capturestderr} flag, if true, specifies that
  63 the object should capture standard error output of the child process.
  64 The default is false.  If the \var{bufsize} parameter is specified, it
  65 specifies the size of the I/O buffers to/from the child process.
  66 \end{classdesc}
  67
  68 \begin{classdesc}{Popen4}{cmd\optional{, bufsize}}
  69 Similar to \class{Popen3}, but always captures standard error into the
  70 same file object as standard output.  These are typically created
  71 using \function{popen4()}.
  72 \versionadded{2.0}
  73 \end{classdesc}
  74
  75
  76 \subsection{Popen3 and Popen4 Objects \label{popen3-objects}}
  77
  78 Instances of the \class{Popen3} and \class{Popen4} classes have the
  79 following methods:
  80
  81 \begin{methoddesc}{poll}{}
  82 Returns \code{-1} if child process hasn't completed yet, or its return
  83 code otherwise.
  84 \end{methoddesc}
  85
  86 \begin{methoddesc}{wait}{}
  87 Waits for and returns the status code of the child process.  The
  88 status code encodes both the return code of the process and
  89 information about whether it exited using the \cfunction{exit()}
  90 system call or died due to a signal.  Functions to help interpret the
  91 status code are defined in the \refmodule{os} module; see section
  92 \ref{os-process} for the \function{W\var{*}()} family of functions.
  93 \end{methoddesc}
  94
  95
  96 The following attributes are also available:
  97
  98 \begin{memberdesc}{fromchild}
  99 A file object that provides output from the child process.  For
 100 \class{Popen4} instances, this will provide both the standard output
 101 and standard error streams.
 102 \end{memberdesc}
 103
 104 \begin{memberdesc}{tochild}
 105 A file object that provides input to the child process.
 106 \end{memberdesc}
 107
 108 \begin{memberdesc}{childerr}
 109 Where the standard error from the child process goes is
 110 \var{capturestderr} was true for the constructor, or \code{None}.
 111 This will always be \code{None} for \class{Popen4} instances.
 112 \end{memberdesc}
 113
 114 \begin{memberdesc}{pid}
 115 The process ID of the child process.
 116 \end{memberdesc}
 117
 118
 119 \subsection{Flow Control Issues \label{popen2-flow-control}}
 120
 121 Any time you are working with any form of inter-process communication,
 122 control flow needs to be carefully thought out.  This remains the case
 123 with the file objects provided by this module (or the \refmodule{os}
 124 module equivalents).
 125
 126 % Example explanation and suggested work-arounds substantially stolen
 127 % from Martin von Löwis:
 128 % http://mail.python.org/pipermail/python-dev/2000-September/009460.html
 129
 130 When reading output from a child process that writes a lot of data to
 131 standard error while the parent is reading from the child's standard
 132 out, a dead lock can occur.  A similar situation can occur with other
 133 combinations of reads and writes.  The essential factors are that more
 134 than \constant{_PC_PIPE_BUF} bytes are being written by one process in
 135 a blocking fashion, while the other process is reading from the other
 136 process, also in a blocking fashion.
 137
 138 There are several ways to deal with this situation.
 139
 140 The simplest application change, in many cases, will be to follow this
 141 model in the parent process:
 142
 143 \begin{verbatim}
 144 import popen2
 145
 146 r, w, e = popen2.popen3('python slave.py')
 147 e.readlines()
 148 r.readlines()
 149 r.close()
 150 e.close()
 151 w.close()
 152 \end{verbatim}
 153
 154 with code like this in the child:
 155
 156 \begin{verbatim}
 157 import os
 158 import sys
 159
 160 # note that each of these print statements
 161 # writes a single long string
 162
 163 print >>sys.stderr, 400 * 'this is a test\n'
 164 os.close(sys.stderr.fileno())
 165 print >>sys.stdout, 400 * 'this is another test\n'
 166 \end{verbatim}
 167
 168 In particular, note that \code{sys.stderr} must be closed after
 169 writing all data, or \method{readlines()} won't return.  Also note
 170 that \function{os.close()} must be used, as \code{sys.stderr.close()}
 171 won't close \code{stderr} (otherwise assigning to \code{sys.stderr}
 172 will silently close it, so no further errors can be printed).
 173
 174 Applications which need to support a more general approach should
 175 integrate I/O over pipes with their \function{select()} loops, or use
 176 separate threads to read each of the individual files provided by
 177 whichever \function{popen*()} function or \class{Popen*} class was
 178 used.