Mac/Demo/plugins.html

   1 <HTML><HEAD><TITLE>Creating a C extension module on the Macintosh</TITLE></HEAD>
   2 <BODY>
   3 <H1>Creating a C extension module on the Macintosh</H1>
   4 <HR>
   5
   6 This document gives a step-by-step example of how to create a new C
   7 extension module on the mac. For this example, we will create a module
   8 to interface to the programmers' API of InterSLIP, a package that
   9 allows you to use MacTCP (and, hence, all internet services) over a
  10 modem connection. <p>
  11
  12 <H2>Prerequisites</H2>
  13
  14 There are a few things you need to pull this off. First and foremost,
  15 you need a C development environment. Actually, you need a specific
  16 development environment, CodeWarrior by <A
  17 HREF="http://www.metrowerks.com/">MetroWerks</A>.  You will probably
  18 need the latest version. You may be able to get by with an older
  19 version of CodeWarrior or with another development environment (Up to
  20 about 1994 python was developed with THINK C, and in the dim past it
  21 was compiled with MPW C) assuming you have managed to get Python to
  22 compile under your development environment, but the step-by-step
  23 character of this document will be lost. <p>
  24
  25 Next, you need a <A
  26 HREF="http://www.python.org/python/Sources.html">python source
  27 distribution</A>.  For PowerPC and cfm68k development you can actually
  28 get by without a full source distribution, using the Development
  29 distribution (if I have gotten around to putting it together by the time
  30 you read this). You'll also need a functional python interpreter, and
  31 the Modulator program (which lives in <CODE>Tools:Modulator</CODE> in
  32 the standard source distribution). You may also find that Guido's <A
  33 HREF="http://www.python.org/doc/ext/ext.html">Extending and embedding
  34 the Python interpreter</A> is a very handy piece of documentation. I
  35 will skip lots of details that are handled there, like complete
  36 descriptions of <CODE>Py_ParseTuple</CODE> and such utility routines, or
  37 the general structure of extension modules. <p>
  38
  39 <H2>InterSLIP and the C API to it</H2>
  40
  41 InterSLIP, the utility to which we are going to create a python
  42 interface, is a system extension that does all the work of connecting
  43 to the internet over a modem connection. InterSLIP is provided
  44 free-of-charge by <A
  45 HREF="http://www.intercon.com/">InterCon</A>. First it connects to
  46 your modem, then it goes through the whole process of dialling,
  47 logging in and possibly starting the SLIP software on the remote
  48 computer and finally it starts with the real work: packing up IP
  49 packets handed to it by MacTCP and sending them to the remote side
  50 (and, of course, the reverse action of receiving incoming packets,
  51 unpacking them and handing them to MacTCP). InterSLIP is a device
  52 driver, and you control it using a application supplied with it,
  53 InterSLIP Setup. The API that InterSLIP Setup uses to talk to the
  54 device driver is published in the documentation and, hence, also
  55 useable by other applications. <p>
  56
  57 I happened to have a C interface to the API, which is all ugly
  58 low-level device-driver calls by itself. The C interface is in <A
  59 HREF="interslip/InterslipLib.c">InterslipLib.c</A> and <A
  60 HREF="interslip/InterslipLib.h">InterslipLib.h</A>, we'll
  61 concentrate here on how to build the Python wrapper module around
  62 it. Note that this is the "normal" situation when you are writing a
  63 Python extension module: you have some sort of functionality available
  64 to C programmers and want to make a Python interface to it. <p>
  65
  66 <H2>Using Modulator</H2>
  67
  68 The method we describe in this document, using Modulator, is the best
  69 method for small interfaces. For large interfaces there is another
  70 tool, Bgen, which actually generates the complete module without you
  71 lifting a single finger. Bgen, however, has the disadvantage of having
  72 a very steep learning curve, so an example using it will have to wait
  73 until another document, when I have more time. <p>
  74
  75 First, let us look at the <A
  76 HREF="interslip/InterslipLib.h">InterslipLib.h</A> header file,
  77 and see that the whole interface consists of six routines:
  78 <CODE>is_open</CODE>, <CODE>is_connect</CODE>,
  79 <CODE>is_disconnect</CODE>, <CODE>is_status</CODE>,
  80 <CODE>is_getconfig</CODE> and <CODE>is_setconfig</CODE>.  Our first
  81 step will be to create a skeleton file <A
  82 HREF="interslip/@interslipmodule.c">@interslipmodule.c</A>, a
  83 dummy module that will contain all the glue code that python expects
  84 of an extension module. Creating this glue code is a breeze with
  85 modulator, a tool that we only have to tell that we want to create a
  86 module with methods of the six names above and that will create the
  87 complete skeleton C code for us. <p>
  88
  89 Why call this dummy module <CODE>@interslipmodule.c</CODE> and not
  90 <CODE>interslipmodule.c</CODE>? Self-preservation: if ever you happen
  91 to repeat the whole process after you have actually turned the
  92 skeleton module into a real module you would overwrite your
  93 hand-written code. By calling the dummy module a different name you
  94 have to make <EM>two</EM> mistakes in a row before you do this. <p>
  95
  96 If you installed Tk support when you installed Python this is extremely
  97 simple. You start modulator and are provided with a form in which you
  98 fill out the details of the module you are creating. <p>
  99
 100 <IMG SRC="html.icons/modulator.gif" ALIGN=CENTER><p>
 101
 102 You'll need to supply a module name (<CODE>interslip</CODE>, in our
 103 case), a module abbreviation (<CODE>pyis</CODE>, which is used as a
 104 prefix to all the routines and data structures modulator will create
 105 for you) and you enter the names of all the methods your module will
 106 export (the list above, with <CODE>is_</CODE> stripped off). Note that
 107 we use <CODE>pyis</CODE> as the prefix instead of the more logical
 108 <CODE>is</CODE>, since the latter would cause our routine names to
 109 collide with those in the API we are interfacing to! The method names
 110 are the names as seen by the python program, and the C routine names
 111 will have the prefix and an underscore prepended. Modulator can do
 112 much more, like generating code for objects and such, but that is a
 113 topic for a later example. <p>
 114
 115 Once you have told modulator all about the module you want to create
 116 you press "check", which checks that you haven't omitted any
 117 information and "Generate code". This will prompt you for a C output
 118 file and generate your module for you. <p>
 119
 120 <H2>Using Modulator without Tk</H2>
 121
 122
 123 Modulator actually uses a two-stage process to create your code: first
 124 the information you provided is turned into a number of python
 125 statements and then these statements are executed to generate your
 126 code. This is done so that you can even use modulator if you don't
 127 have Tk support in Python: you'll just have to write the modulator
 128 python statements by hand (about 10 lines, in our example) and
 129 modulator will generate the C code (about 150 lines, in our
 130 example). Here is the Python code you'll want to execute to generate
 131 our skeleton module: <p>
 132
 133 <CODE><PRE>
 134         import addpack
 135         addpack.addpack('Tools')
 136         addpack.addpack('modulator')
 137         import genmodule
 138
 139         m = genmodule.module()
 140         m.name = 'interslip'
 141         m.abbrev = 'pyis'
 142         m.methodlist = ['open', 'connect', 'disconnect', 'status', \
 143                 'getconfig', 'setconfig']
 144         m.objects = []
 145
 146         fp = open('@interslipmodule.c', 'w')
 147         genmodule.write(fp, m)
 148 </PRE></CODE>
 149
 150 Drop this program on the python interpreter and out will come your
 151 skeleton module. <p>
 152
 153 Now, rename the file to interslipmodule.c and you're all set to start
 154 developing. The module is complete in the sense that it should
 155 compile, and that if you import it in a python program you will see
 156 all the methods.  It is, of course, not yet complete in a functional
 157 way... <p>
 158
 159 <H2>Adding a module to Classic 68K Python</H2>
 160
 161 What you do now depends on whether you're developing for PowerPC (or
 162 for CFM68K) or for "traditional" mac. For a traditional 68K Python,
 163 you will have to add your new module to the project file of the Python
 164 interpreter, and you have to edit "config.c" to add the module to the
 165 set of builtin modules. In config.c you will add the module at two
 166 places: near the start of the file there is a list of external
 167 declarations for all init() routines. Add a line of the form
 168 <CODE><PRE>
 169                 extern void initinterslip();
 170 </PRE></CODE>
 171 here. Further down the file there is an array that is initialized with
 172 modulename/initfunction pairs. Add a line of the form
 173 <CODE><PRE>
 174                 {"interslip",   initinterslip},
 175 </PRE></CODE>
 176 here. You may want to bracket these two lines with
 177 <CODE><PRE>
 178         #ifdef USE_INTERSLIP
 179         #endif
 180 </PRE></CODE>
 181 lines, that way you can easily control whether the module is
 182 incorporated into python at compile time. If you decide to do the
 183 latter edit your config file (you can find the name in the "C/C++
 184 language" section of the MW preferences dialog, it will probably be
 185 "mwerks_nonshared_config.h") and add a
 186 <CODE><PRE>
 187         #define USE_INTERSLIP
 188 </PRE></CODE>
 189
 190 Make the new interpreter and check that you can import the module, see
 191 the methods (with "dir(interslip)") and call them. <p>
 192
 193 <H2>Creating a PowerPC plugin module</H2>
 194
 195 For PowerPC development you could follow the same path, but it is
 196 actually a better idea to use a dynamically loadable module. The
 197 advantage of dynamically loadable modules is that they are not loaded
 198 until a python program actually uses them (resulting in less memory
 199 usage by the interpreter) and that development is a lot simpler (since
 200 your projects will all be smaller). Moreover, you can distribute a
 201 plugin module by itself without haveing to distribute a complete
 202 python interpreter. <p>
 203
 204 Go to the "PlugIns" folder and copy the files xx.prj,
 205 and xx.prj.exp to interslipmodule.prj and
 206 interslipmodule.prj.exp, respectively. Edit
 207 interslipmodule.prj.exp and change the name of the exported routine
 208 "initxx" to "initinterslip".  Open interslipmodule.prj with CodeWarrior,
 209 remove the file xxmodule.c and add interslipmodule.c and make a number
 210 of adjustments to the preferences:
 211 <UL>
 212 <LI> in PPC target, set the output file name to "interslipmodule.pcc.slb",
 213 <LI> in cfm68k target set the output file name to "interslipmodule.cfm68k.slb".
 214 </UL>
 215 Next, compile and link your module, fire up python and do the same
 216 tests as for 68K python. <p>
 217
 218 <H2>Getting the module to do real work</H2>
 219
 220 So far, so good. In half an hour or so we have created a complete new
 221 extension module for Python. The downside, however, is that the module
 222 does not do anything useful. So, in the next half hour we will turn
 223 our beautiful skeleton module into something that is at least as
 224 beautiful but also gets some serious work done.  For this once,
 225 <EM>I</EM> have spent that half hour for you, and you can see the
 226 results in <A
 227 HREF="interslip/interslipmodule.c">interslipmodule.c</A>. <p>
 228
 229 We add
 230 <CODE><PRE>
 231         #include "InterslipLib.h"
 232         #include "macglue.h"
 233 </PRE></CODE>
 234 to the top of the file, and work our way through each of the methods
 235 to add the functionality needed. Starting with open, we fill in the
 236 template docstring, the value accessible from Python by looking at
 237 <CODE>interslip.open.__doc__</CODE>.  There are not many tools using
 238 this information at the moment, but as soon as class browsers for
 239 python become available having this minimal documentation available is
 240 a good idea. We put "Load the interslip driver" as the comment
 241 here. <p>
 242
 243 Next, we tackle the body of <CODE>pyis_open()</CODE>.  Since it has no
 244 arguments and no return value we don't need to mess with that, we just
 245 have to add a call to <CODE>is_open()</CODE> and check the return for
 246 an error code, in which case we raise an error:
 247 <CODE><PRE>
 248         err = is_open();
 249         if ( err ) {
 250                 PyErr_Mac(ErrorObject, err);
 251                 return NULL;
 252         }
 253 </PRE></CODE>
 254 The routine <CODE><A NAME="PyErr_Mac">PyErr_Mac()</A></CODE> is a
 255 useful routine that raises the exception passed as its first
 256 argument. The data passed with the exception is based on the standard
 257 MacOS error code given, and PyErr_Mac() attempts to locate a textual
 258 description of the error code (which sure beats the "error -14021"
 259 messages that so many macintosh applications tell their poor
 260 users). <p>
 261
 262 We will skip pyis_connect and pyis_disconnect here, which are pretty
 263 much identical to pyis_open: no arguments, no return value, just a
 264 call and an error check. With pyis_status() things get interesting
 265 again: this call still takes 3 arguments, and all happen to be values
 266 returned (a numeric connection status indicator, a message sequence
 267 number and a pointer to the message itself, in MacOS pascal-style
 268 string form). We declare variables to receive the returned values, do
 269 the call, check the error and format the return value. <p>
 270
 271 Building the return value is done using <CODE><A
 272 NAME="Py_BuildValue">Py_BuildValue</A></CODE>:
 273 <CODE><PRE>
 274         return Py_BuildValue("iiO&", (int)status, (int)seqnum, PyMac_BuildStr255, message);
 275 </PRE></CODE>
 276 Py_BuildValue() is a very handy routine that builds tuples according
 277 to a format string, somewhat similar to the way <CODE>printf()</CODE>
 278 works.  The format string specifies the arguments expected after the
 279 string, and turns them from C objects into python objects. The
 280 resulting objects are put in a python tuple object and returned. The
 281 "i" format specifier signifies an "int" (hence the cast: status and
 282 seqnum are declared as "long", which is what the is_status() routine
 283 wants, and even though we use a 4-byte project there is really no
 284 reason not to put the cast here). Py_BuildValue and its counterpart
 285 Py_ParseTuple have format codes for all the common C types like ints,
 286 shorts, C-strings, floats, etc. Also, there is a nifty escape
 287 mechanism to format values about which is does not know. This is
 288 invoked by the "O&" format: it expects two arguments, a routine
 289 pointer and an int-sized data object. The routine is called with the
 290 object as a parameter and it should return a python objects
 291 representing the data. <CODE>Macglue.h</CODE> declares a number of
 292 such formatting routines for common MacOS objects like Str255, FSSpec,
 293 OSType, Rect, etc. See the comments in the include file for
 294 details. <p>
 295
 296 <CODE>Pyis_getconfig()</CODE> is again similar to pyis_getstatus, only
 297 two minor points are worth noting here.  First, the C API return the
 298 input and output baudrate squashed together into a single 4-byte
 299 long. We separate them out before returning the result to
 300 python. Second, whereas the status call returned us a pointer to a
 301 <CODE>Str255</CODE> it kept we are responsible for allocating the
 302 <CODE>Str255</CODE> for getconfig. This is something that would have
 303 been easy to get wrong had we not used prototypes everywhere. Morale:
 304 always try to include the header files for interfaces to libraries and
 305 other stuff, so that the compiler can catch any mistakes you make. <p>
 306
 307 <CODE>Pyis_setconfig()</CODE> finally shows off
 308 <CODE>Py_ParseTuple</CODE>, the companion function to
 309 <CODE>Py_BuildValue</CODE>.  You pass it the argument tuple "args"
 310 that your method gets as its second argument, a format string and
 311 pointers to where you want the arguments stored. Again, standard C
 312 types such as strings and integers Py_ParseTuple knows all about and
 313 through the "O&" format you can extend the functionality. For each
 314 "O&" you pass a function pointer and a pointer to a data area. The
 315 function will be called with a PyObject pointer and your data pointer
 316 and it should convert the python object to the correct C type. It
 317 should return 1 on success and 0 on failure. Again, a number of
 318 converters for standard MacOS types are provided, and declared in
 319 <CODE>macglue.h</CODE>. <p>
 320
 321 Next in our source file comes the method table for our module, which
 322 has been generated by modulator (and it did a good job too!), but
 323 which is worth looking at for a moment.  Entries are of the form
 324 <CODE><PRE>
 325         {"open",        pyis_open,      1,      pyis_open__doc__},
 326 </PRE></CODE>
 327 where the entries are python method name, C routine pointer, flags and
 328 docstring pointer.  The value to note is the 1 for the flags: this
 329 signifies that you want to use "new-style" Py_ParseTuple behaviour. If
 330 you are writing a new module always use this, but if you are modifying
 331 old code which calls something like <CODE>getargs(args, "(ii)",
 332 ...)</CODE> you will have to put zero here. See "extending and
 333 embedding" or possibly the getargs.c source file for details if you
 334 need them. <p>
 335
 336 Finally, we add some code to the init module, to put some symbolic
 337 constants (codes that can by returned by the status method) in the
 338 module dictionary, so the python program can use "interslip.RUN"
 339 instead of the cryptic "4" when it wants to check that the interslip
 340 driver is in RUN state. Modulator has already generated code to get at
 341 the module dictionary using PyModule_GetDict() to store the exception
 342 object, so we simply call
 343 <CODE><PRE>
 344         PyDict_SetItemString(d, "IDLE", PyInt_FromLong(IS_IDLE));
 345 </PRE></CODE>
 346 for each of our items. Since the last bit of code in our init routine
 347 checks for previous errors with <CODE>PyErr_Occurred()</CODE> and
 348 since <CODE>PyDict_SetItemString()</CODE> gracefully handles the case
 349 of <CODE>NULL</CODE> parameters (if <CODE>PyInt_FromLong()</CODE>
 350 failed, for instance) we don't have to do error checking here. In some
 351 other cases you may have to do error checking yourself. <p>
 352
 353 This concludes our crash-course on writing Python extensions in C on
 354 the Macintosh.  If you are not done reading yet I suggest you look
 355 back at the <A HREF="index.html">MacPython Crashcourse index</A> to
 356 find another topic to study. <p>