Mac/Demo/plugins.html

   1 <HTML><HEAD><TITLE>Creating a C extension module on the Macintosh</TITLE></HEAD>
   2 <BODY>
   3 <H1>Creating a C extension module on the Macintosh</H1>
   4 <HR>
   5
   6 This document gives a step-by-step example of how to create a new C
   7 extension module on the mac. For this example, we will create a module
   8 to interface to the programmers' API of InterSLIP, a package that
   9 allows you to use MacTCP (and, hence, all internet services) over a
  10 modem connection. The actual example does not work anymore, as both
  11 MacTCP and Interslip died long ago, but the text is still mostly
  12 correct.<p>
  13
  14 <H2>Prerequisites</H2>
  15
  16 There are a few things you need to pull this off. First and foremost,
  17 you need a C development environment. Actually, you need a specific
  18 development environment, CodeWarrior by <A
  19 HREF="http://www.metrowerks.com/">MetroWerks</A>.  You will
  20 need Version 7 or later. You may be able to get by with an older
  21 version of CodeWarrior or with another development environment (Up to
  22 about 1994 python was developed with THINK C, and in the dim past it
  23 was compiled with MPW C) assuming you have managed to get Python to
  24 compile under your development environment, but the step-by-step
  25 character of this document will be lost. <p>
  26
  27 Next, you need to install the Developer option in the MacPython installer.
  28 You may also find that Guido's <A
  29 HREF="http://www.python.org/doc/ext/ext.html">Extending and embedding
  30 the Python interpreter</A> is a very handy piece of documentation. I
  31 will skip lots of details that are handled there, like complete
  32 descriptions of <CODE>Py_ParseTuple</CODE> and such utility routines, or
  33 the general structure of extension modules. <p>
  34
  35 <H2>InterSLIP and the C API to it</H2>
  36
  37 InterSLIP, the utility to which we are going to create a python
  38 interface, is a system extension that does all the work of connecting
  39 to the internet over a modem connection. InterSLIP is provided
  40 free-of-charge by <A
  41 HREF="http://www.intercon.com/">InterCon</A>. First it connects to
  42 your modem, then it goes through the whole process of dialling,
  43 logging in and possibly starting the SLIP software on the remote
  44 computer and finally it starts with the real work: packing up IP
  45 packets handed to it by MacTCP and sending them to the remote side
  46 (and, of course, the reverse action of receiving incoming packets,
  47 unpacking them and handing them to MacTCP). InterSLIP is a device
  48 driver, and you control it using a application supplied with it,
  49 InterSLIP Setup. The API that InterSLIP Setup uses to talk to the
  50 device driver is published in the documentation and, hence, also
  51 useable by other applications. <p>
  52
  53 I happened to have a C interface to the API, which is all ugly
  54 low-level device-driver calls by itself. The C interface is in <A
  55 HREF="interslip/InterslipLib.c">InterslipLib.c</A> and <A
  56 HREF="interslip/InterslipLib.h">InterslipLib.h</A>, we'll
  57 concentrate here on how to build the Python wrapper module around
  58 it. Note that this is the "normal" situation when you are writing a
  59 Python extension module: you have some sort of functionality available
  60 to C programmers and want to make a Python interface to it. <p>
  61
  62 <H2>Using Modulator</H2>
  63
  64 The method we describe in this document, using Modulator, is the best
  65 method for small interfaces. For large interfaces there is another
  66 tool, Bgen, which actually generates the complete module without you
  67 lifting a single finger. Bgen, however, has the disadvantage of having
  68 a very steep learning curve, so an example using it will have to wait
  69 until another document, when I have more time. <p>
  70
  71 First, let us look at the <A
  72 HREF="interslip/InterslipLib.h">InterslipLib.h</A> header file,
  73 and see that the whole interface consists of six routines:
  74 <CODE>is_open</CODE>, <CODE>is_connect</CODE>,
  75 <CODE>is_disconnect</CODE>, <CODE>is_status</CODE>,
  76 <CODE>is_getconfig</CODE> and <CODE>is_setconfig</CODE>.  Our first
  77 step will be to create a skeleton file <A
  78 HREF="interslip/@interslipmodule.c">@interslipmodule.c</A>, a
  79 dummy module that will contain all the glue code that python expects
  80 of an extension module. Creating this glue code is a breeze with
  81 modulator, a tool that we only have to tell that we want to create a
  82 module with methods of the six names above and that will create the
  83 complete skeleton C code for us. <p>
  84
  85 Why call this dummy module <CODE>@interslipmodule.c</CODE> and not
  86 <CODE>interslipmodule.c</CODE>? Self-preservation: if ever you happen
  87 to repeat the whole process after you have actually turned the
  88 skeleton module into a real module you would overwrite your
  89 hand-written code. By calling the dummy module a different name you
  90 have to make <EM>two</EM> mistakes in a row before you do this. <p>
  91
  92 If you installed Tk support when you installed Python this is extremely
  93 simple. You start modulator and are provided with a form in which you
  94 fill out the details of the module you are creating. <p>
  95
  96 <IMG SRC="html.icons/modulator.gif" ALIGN=CENTER><p>
  97
  98 You'll need to supply a module name (<CODE>interslip</CODE>, in our
  99 case), a module abbreviation (<CODE>pyis</CODE>, which is used as a
 100 prefix to all the routines and data structures modulator will create
 101 for you) and you enter the names of all the methods your module will
 102 export (the list above, with <CODE>is_</CODE> stripped off). Note that
 103 we use <CODE>pyis</CODE> as the prefix instead of the more logical
 104 <CODE>is</CODE>, since the latter would cause our routine names to
 105 collide with those in the API we are interfacing to! The method names
 106 are the names as seen by the python program, and the C routine names
 107 will have the prefix and an underscore prepended. Modulator can do
 108 much more, like generating code for objects and such, but that is a
 109 topic for a later example. <p>
 110
 111 Once you have told modulator all about the module you want to create
 112 you press "check", which checks that you haven't omitted any
 113 information and "Generate code". This will prompt you for a C output
 114 file and generate your module for you. <p>
 115
 116 <H2>Using Modulator without Tk</H2>
 117
 118
 119 Modulator actually uses a two-stage process to create your code: first
 120 the information you provided is turned into a number of python
 121 statements and then these statements are executed to generate your
 122 code. This is done so that you can even use modulator if you don't
 123 have Tk support in Python: you'll just have to write the modulator
 124 python statements by hand (about 10 lines, in our example) and
 125 modulator will generate the C code (about 150 lines, in our
 126 example). Here is the Python code you'll want to execute to generate
 127 our skeleton module: <p>
 128
 129 <CODE><PRE>
 130         import addpack
 131         addpack.addpack('Tools')
 132         addpack.addpack('modulator')
 133         import genmodule
 134
 135         m = genmodule.module()
 136         m.name = 'interslip'
 137         m.abbrev = 'pyis'
 138         m.methodlist = ['open', 'connect', 'disconnect', 'status', \
 139                 'getconfig', 'setconfig']
 140         m.objects = []
 141
 142         fp = open('@interslipmodule.c', 'w')
 143         genmodule.write(fp, m)
 144 </PRE></CODE>
 145
 146 Drop this program on the python interpreter and out will come your
 147 skeleton module. <p>
 148
 149 Now, rename the file to interslipmodule.c and you're all set to start
 150 developing. The module is complete in the sense that it should
 151 compile, and that if you import it in a python program you will see
 152 all the methods.  It is, of course, not yet complete in a functional
 153 way... <p>
 154
 155 <H2>Creating a plugin module</H2>
 156
 157 The easiest way to build a plugin module is to use the distutils package,
 158 this works fine on MacOS with CodeWarrior. See the distutils documentation
 159 for details. Keep in mind that even though you are on the Mac you specify
 160 filenames with Unix syntax: they are actually URLs, not filenames.
 161  <p>
 162
 163 Alternatively you can build the project file by hand.
 164 Go to the ":Mac:Build" folder and copy the files xx.carbon.mcp,
 165 and xx.carbon.mcp.exp to interslipmodule.carbon.mcp and
 166 interslipmodule.carbon.mcp.exp, respectively. Edit
 167 interslipmodule.carbon.mcp.exp and change the name of the exported routine
 168 "initxx" to "initinterslip".  Open interslipmodule.carbon.mcp with CodeWarrior,
 169 remove the file xxmodule.c and add interslipmodule.c and make a number
 170 of adjustments to the preferences:
 171 <UL>
 172 <LI> in PPC target, set the output file name to "interslipmodule.carbon.slb",
 173 <LI> if you are working without a source distribution (i.e. with a normal
 174 binary distribution plus a development distribution) you will not have
 175 a file <code>PythonCoreCarbon</code>. The installation process has deposited this
 176 file in the System <code>Extensions</code> folder under the name
 177 <code>PythonCoreCarbon <i>version</i></code>. Add that file to the project, replacing
 178 <code>PythonCoreCarbon</code>.
 179 <LI> you must either download and build GUSI (if your extension module uses sockets
 180 or other Unix I/O constructs) or remove GUSI references from the Access Paths
 181 settings. See the <a href="building.html">Building</a> document for where to get GUSI
 182 and how to build it.
 183 </UL>
 184 Next, compile and link your module, fire up python and test it. <p>
 185
 186 <H2>Getting the module to do real work</H2>
 187
 188 So far, so good. In half an hour or so we have created a complete new
 189 extension module for Python. The downside, however, is that the module
 190 does not do anything useful. So, in the next half hour we will turn
 191 our beautiful skeleton module into something that is at least as
 192 beautiful but also gets some serious work done.  For this once,
 193 <EM>I</EM> have spent that half hour for you, and you can see the
 194 results in <A
 195 HREF="interslip/interslipmodule.c">interslipmodule.c</A>. <p>
 196
 197 We add
 198 <CODE><PRE>
 199         #include "InterslipLib.h"
 200         #include "macglue.h"
 201 </PRE></CODE>
 202 to the top of the file, and work our way through each of the methods
 203 to add the functionality needed. Starting with open, we fill in the
 204 template docstring, the value accessible from Python by looking at
 205 <CODE>interslip.open.__doc__</CODE>.  There are not many tools using
 206 this information at the moment, but as soon as class browsers for
 207 python become available having this minimal documentation available is
 208 a good idea. We put "Load the interslip driver" as the comment
 209 here. <p>
 210
 211 Next, we tackle the body of <CODE>pyis_open()</CODE>.  Since it has no
 212 arguments and no return value we don't need to mess with that, we just
 213 have to add a call to <CODE>is_open()</CODE> and check the return for
 214 an error code, in which case we raise an error:
 215 <CODE><PRE>
 216         err = is_open();
 217         if ( err ) {
 218                 PyErr_Mac(ErrorObject, err);
 219                 return NULL;
 220         }
 221 </PRE></CODE>
 222 The routine <CODE><A NAME="PyErr_Mac">PyErr_Mac()</A></CODE> is a
 223 useful routine that raises the exception passed as its first
 224 argument. The data passed with the exception is based on the standard
 225 MacOS error code given, and PyErr_Mac() attempts to locate a textual
 226 description of the error code (which sure beats the "error -14021"
 227 messages that so many macintosh applications tell their poor
 228 users). <p>
 229
 230 We will skip pyis_connect and pyis_disconnect here, which are pretty
 231 much identical to pyis_open: no arguments, no return value, just a
 232 call and an error check. With pyis_status() things get interesting
 233 again: this call still takes 3 arguments, and all happen to be values
 234 returned (a numeric connection status indicator, a message sequence
 235 number and a pointer to the message itself, in MacOS pascal-style
 236 string form). We declare variables to receive the returned values, do
 237 the call, check the error and format the return value. <p>
 238
 239 Building the return value is done using <CODE><A
 240 NAME="Py_BuildValue">Py_BuildValue</A></CODE>:
 241 <CODE><PRE>
 242         return Py_BuildValue("iiO&", (int)status, (int)seqnum, PyMac_BuildStr255, message);
 243 </PRE></CODE>
 244 Py_BuildValue() is a very handy routine that builds tuples according
 245 to a format string, somewhat similar to the way <CODE>printf()</CODE>
 246 works.  The format string specifies the arguments expected after the
 247 string, and turns them from C objects into python objects. The
 248 resulting objects are put in a python tuple object and returned. The
 249 "i" format specifier signifies an "int" (hence the cast: status and
 250 seqnum are declared as "long", which is what the is_status() routine
 251 wants, and even though we use a 4-byte project there is really no
 252 reason not to put the cast here). Py_BuildValue and its counterpart
 253 Py_ParseTuple have format codes for all the common C types like ints,
 254 shorts, C-strings, floats, etc. Also, there is a nifty escape
 255 mechanism to format values about which is does not know. This is
 256 invoked by the "O&" format: it expects two arguments, a routine
 257 pointer and an int-sized data object. The routine is called with the
 258 object as a parameter and it should return a python objects
 259 representing the data. <CODE>Macglue.h</CODE> declares a number of
 260 such formatting routines for common MacOS objects like Str255, FSSpec,
 261 OSType, Rect, etc. See the comments in the include file for
 262 details. <p>
 263
 264 <CODE>Pyis_getconfig()</CODE> is again similar to pyis_getstatus, only
 265 two minor points are worth noting here.  First, the C API return the
 266 input and output baudrate squashed together into a single 4-byte
 267 long. We separate them out before returning the result to
 268 python. Second, whereas the status call returned us a pointer to a
 269 <CODE>Str255</CODE> it kept we are responsible for allocating the
 270 <CODE>Str255</CODE> for getconfig. This is something that would have
 271 been easy to get wrong had we not used prototypes everywhere. Morale:
 272 always try to include the header files for interfaces to libraries and
 273 other stuff, so that the compiler can catch any mistakes you make. <p>
 274
 275 <CODE>Pyis_setconfig()</CODE> finally shows off
 276 <CODE>Py_ParseTuple</CODE>, the companion function to
 277 <CODE>Py_BuildValue</CODE>.  You pass it the argument tuple "args"
 278 that your method gets as its second argument, a format string and
 279 pointers to where you want the arguments stored. Again, standard C
 280 types such as strings and integers Py_ParseTuple knows all about and
 281 through the "O&" format you can extend the functionality. For each
 282 "O&" you pass a function pointer and a pointer to a data area. The
 283 function will be called with a PyObject pointer and your data pointer
 284 and it should convert the python object to the correct C type. It
 285 should return 1 on success and 0 on failure. Again, a number of
 286 converters for standard MacOS types are provided, and declared in
 287 <CODE>macglue.h</CODE>. <p>
 288
 289 Next in our source file comes the method table for our module, which
 290 has been generated by modulator (and it did a good job too!), but
 291 which is worth looking at for a moment.  Entries are of the form
 292 <CODE><PRE>
 293         {"open",        pyis_open,      1,      pyis_open__doc__},
 294 </PRE></CODE>
 295 where the entries are python method name, C routine pointer, flags and
 296 docstring pointer.  The value to note is the 1 for the flags: this
 297 signifies that you want to use "new-style" Py_ParseTuple behaviour. If
 298 you are writing a new module always use this, but if you are modifying
 299 old code which calls something like <CODE>getargs(args, "(ii)",
 300 ...)</CODE> you will have to put zero here. See "extending and
 301 embedding" or possibly the getargs.c source file for details if you
 302 need them. <p>
 303
 304 Finally, we add some code to the init module, to put some symbolic
 305 constants (codes that can by returned by the status method) in the
 306 module dictionary, so the python program can use "interslip.RUN"
 307 instead of the cryptic "4" when it wants to check that the interslip
 308 driver is in RUN state. Modulator has already generated code to get at
 309 the module dictionary using PyModule_GetDict() to store the exception
 310 object, so we simply call
 311 <CODE><PRE>
 312         PyDict_SetItemString(d, "IDLE", PyInt_FromLong(IS_IDLE));
 313 </PRE></CODE>
 314 for each of our items. Since the last bit of code in our init routine
 315 checks for previous errors with <CODE>PyErr_Occurred()</CODE> and
 316 since <CODE>PyDict_SetItemString()</CODE> gracefully handles the case
 317 of <CODE>NULL</CODE> parameters (if <CODE>PyInt_FromLong()</CODE>
 318 failed, for instance) we don't have to do error checking here. In some
 319 other cases you may have to do error checking yourself. <p>
 320
 321 This concludes our crash-course on writing Python extensions in C on
 322 the Macintosh.  If you are not done reading yet I suggest you look
 323 back at the <A HREF="index.html">MacPython Crashcourse index</A> to
 324 find another topic to study. <p>