Mac/Demo/plugins.html

   1 <HTML><HEAD><TITLE>Creating a C extension module on the Macintosh</TITLE></HEAD>
   2 <BODY>
   3 <H1>Creating a C extension module on the Macintosh</H1>
   4 <HR>
   5
   6 This document gives a step-by-step example of how to create a new C
   7 extension module on the mac. For this example, we will create a module
   8 to interface to the programmers' API of InterSLIP, a package that
   9 allows you to use MacTCP (and, hence, all internet services) over a
  10 modem connection. <p>
  11
  12 <H2>Prerequisites</H2>
  13
  14 There are a few things you need to pull this off. First and foremost,
  15 you need a C development environment. Actually, you need a specific
  16 development environment, CodeWarrior by <A
  17 HREF="http://www.metrowerks.com/">MetroWerks</A>.  You will probably
  18 need the latest version. You may be able to get by with an older
  19 version of CodeWarrior or with another development environment (Up to
  20 about 1994 python was developed with THINK C, and in the dim past it
  21 was compiled with MPW C) assuming you have managed to get Python to
  22 compile under your development environment, but the step-by-step
  23 character of this document will be lost. <p>
  24
  25 Next, you need a <A
  26 HREF="http://www.python.org/python/Sources.html">python source
  27 distribution</A>.  For PowerPC and cfm68k development you can actually
  28 get by without a full source distribution, using the Development
  29 distribution. You'll also need a functional python interpreter, and
  30 the Modulator program (which lives in <CODE>Tools:Modulator</CODE> in
  31 the standard source distribution). You may also find that Guido's <A
  32 HREF="http://www.python.org/doc/ext/ext.html">Extending and embedding
  33 the Python interpreter</A> is a very handy piece of documentation. I
  34 will skip lots of details that are handled there, like complete
  35 descriptions of <CODE>Py_ParseTuple</CODE> and such utility routines, or
  36 the general structure of extension modules. <p>
  37
  38 <H2>InterSLIP and the C API to it</H2>
  39
  40 InterSLIP, the utility to which we are going to create a python
  41 interface, is a system extension that does all the work of connecting
  42 to the internet over a modem connection. InterSLIP is provided
  43 free-of-charge by <A
  44 HREF="http://www.intercon.com/">InterCon</A>. First it connects to
  45 your modem, then it goes through the whole process of dialling,
  46 logging in and possibly starting the SLIP software on the remote
  47 computer and finally it starts with the real work: packing up IP
  48 packets handed to it by MacTCP and sending them to the remote side
  49 (and, of course, the reverse action of receiving incoming packets,
  50 unpacking them and handing them to MacTCP). InterSLIP is a device
  51 driver, and you control it using a application supplied with it,
  52 InterSLIP Setup. The API that InterSLIP Setup uses to talk to the
  53 device driver is published in the documentation and, hence, also
  54 useable by other applications. <p>
  55
  56 I happened to have a C interface to the API, which is all ugly
  57 low-level device-driver calls by itself. The C interface is in <A
  58 HREF="interslip/InterslipLib.c">InterslipLib.c</A> and <A
  59 HREF="interslip/InterslipLib.h">InterslipLib.h</A>, we'll
  60 concentrate here on how to build the Python wrapper module around
  61 it. Note that this is the "normal" situation when you are writing a
  62 Python extension module: you have some sort of functionality available
  63 to C programmers and want to make a Python interface to it. <p>
  64
  65 <H2>Using Modulator</H2>
  66
  67 The method we describe in this document, using Modulator, is the best
  68 method for small interfaces. For large interfaces there is another
  69 tool, Bgen, which actually generates the complete module without you
  70 lifting a single finger. Bgen, however, has the disadvantage of having
  71 a very steep learning curve, so an example using it will have to wait
  72 until another document, when I have more time. <p>
  73
  74 First, let us look at the <A
  75 HREF="interslip/InterslipLib.h">InterslipLib.h</A> header file,
  76 and see that the whole interface consists of six routines:
  77 <CODE>is_open</CODE>, <CODE>is_connect</CODE>,
  78 <CODE>is_disconnect</CODE>, <CODE>is_status</CODE>,
  79 <CODE>is_getconfig</CODE> and <CODE>is_setconfig</CODE>.  Our first
  80 step will be to create a skeleton file <A
  81 HREF="interslip/@interslipmodule.c">@interslipmodule.c</A>, a
  82 dummy module that will contain all the glue code that python expects
  83 of an extension module. Creating this glue code is a breeze with
  84 modulator, a tool that we only have to tell that we want to create a
  85 module with methods of the six names above and that will create the
  86 complete skeleton C code for us. <p>
  87
  88 Why call this dummy module <CODE>@interslipmodule.c</CODE> and not
  89 <CODE>interslipmodule.c</CODE>? Self-preservation: if ever you happen
  90 to repeat the whole process after you have actually turned the
  91 skeleton module into a real module you would overwrite your
  92 hand-written code. By calling the dummy module a different name you
  93 have to make <EM>two</EM> mistakes in a row before you do this. <p>
  94
  95 If you installed Tk support when you installed Python this is extremely
  96 simple. You start modulator and are provided with a form in which you
  97 fill out the details of the module you are creating. <p>
  98
  99 <IMG SRC="html.icons/modulator.gif" ALIGN=CENTER><p>
 100
 101 You'll need to supply a module name (<CODE>interslip</CODE>, in our
 102 case), a module abbreviation (<CODE>pyis</CODE>, which is used as a
 103 prefix to all the routines and data structures modulator will create
 104 for you) and you enter the names of all the methods your module will
 105 export (the list above, with <CODE>is_</CODE> stripped off). Note that
 106 we use <CODE>pyis</CODE> as the prefix instead of the more logical
 107 <CODE>is</CODE>, since the latter would cause our routine names to
 108 collide with those in the API we are interfacing to! The method names
 109 are the names as seen by the python program, and the C routine names
 110 will have the prefix and an underscore prepended. Modulator can do
 111 much more, like generating code for objects and such, but that is a
 112 topic for a later example. <p>
 113
 114 Once you have told modulator all about the module you want to create
 115 you press "check", which checks that you haven't omitted any
 116 information and "Generate code". This will prompt you for a C output
 117 file and generate your module for you. <p>
 118
 119 <H2>Using Modulator without Tk</H2>
 120
 121
 122 Modulator actually uses a two-stage process to create your code: first
 123 the information you provided is turned into a number of python
 124 statements and then these statements are executed to generate your
 125 code. This is done so that you can even use modulator if you don't
 126 have Tk support in Python: you'll just have to write the modulator
 127 python statements by hand (about 10 lines, in our example) and
 128 modulator will generate the C code (about 150 lines, in our
 129 example). Here is the Python code you'll want to execute to generate
 130 our skeleton module: <p>
 131
 132 <CODE><PRE>
 133         import addpack
 134         addpack.addpack('Tools')
 135         addpack.addpack('modulator')
 136         import genmodule
 137
 138         m = genmodule.module()
 139         m.name = 'interslip'
 140         m.abbrev = 'pyis'
 141         m.methodlist = ['open', 'connect', 'disconnect', 'status', \
 142                 'getconfig', 'setconfig']
 143         m.objects = []
 144
 145         fp = open('@interslipmodule.c', 'w')
 146         genmodule.write(fp, m)
 147 </PRE></CODE>
 148
 149 Drop this program on the python interpreter and out will come your
 150 skeleton module. <p>
 151
 152 Now, rename the file to interslipmodule.c and you're all set to start
 153 developing. The module is complete in the sense that it should
 154 compile, and that if you import it in a python program you will see
 155 all the methods.  It is, of course, not yet complete in a functional
 156 way... <p>
 157
 158 <H2>Adding a module to Classic 68K Python</H2>
 159
 160 What you do now depends on whether you're developing for PowerPC (or
 161 for CFM68K) or for "traditional" mac. For a traditional 68K Python,
 162 you will have to add your new module to the project file of the Python
 163 interpreter, and you have to edit "config.c" to add the module to the
 164 set of builtin modules. In config.c you will add the module at two
 165 places: near the start of the file there is a list of external
 166 declarations for all init() routines. Add a line of the form
 167 <CODE><PRE>
 168                 extern void initinterslip();
 169 </PRE></CODE>
 170 here. Further down the file there is an array that is initialized with
 171 modulename/initfunction pairs. Add a line of the form
 172 <CODE><PRE>
 173                 {"interslip",   initinterslip},
 174 </PRE></CODE>
 175 here. You may want to bracket these two lines with
 176 <CODE><PRE>
 177         #ifdef USE_INTERSLIP
 178         #endif
 179 </PRE></CODE>
 180 lines, that way you can easily control whether the module is
 181 incorporated into python at compile time. If you decide to do the
 182 latter edit your config file (you can find the name in the "C/C++
 183 language" section of the MW preferences dialog, it will probably be
 184 "mwerks_nonshared_config.h") and add a
 185 <CODE><PRE>
 186         #define USE_INTERSLIP
 187 </PRE></CODE>
 188
 189 Make the new interpreter and check that you can import the module, see
 190 the methods (with "dir(interslip)") and call them. <p>
 191
 192 <H2>Creating a plugin module</H2>
 193
 194 For PowerPC or cfm68k development you could follow the same path, but it is
 195 actually a better idea to use a dynamically loadable module. The
 196 advantage of dynamically loadable modules is that they are not loaded
 197 until a python program actually uses them (resulting in less memory
 198 usage by the interpreter) and that development is a lot simpler (since
 199 your projects will all be smaller). Moreover, you can distribute a
 200 plugin module by itself without haveing to distribute a complete
 201 python interpreter. <p>
 202
 203 Go to the "PlugIns" folder and copy the files xx.prj,
 204 and xx.prj.exp to interslipmodule.prj and
 205 interslipmodule.prj.exp, respectively. Edit
 206 interslipmodule.prj.exp and change the name of the exported routine
 207 "initxx" to "initinterslip".  Open interslipmodule.prj with CodeWarrior,
 208 remove the file xxmodule.c and add interslipmodule.c and make a number
 209 of adjustments to the preferences:
 210 <UL>
 211 <LI> in PPC target, set the output file name to "interslipmodule.pcc.slb",
 212 <LI> in cfm68k target set the output file name to "interslipmodule.cfm68k.slb".
 213 <LI> if you are working without a source distribution (i.e. with a normal
 214 binary distribution plus a development distribution) you will not have
 215 a file <code>PythonCore</code>. The installation process has deposited this
 216 file in the System <code>Extensions</code> folder under the name
 217 <code>PythonCore <i>version</i></code>. Add that file to the project, replacing
 218 <code>PythonCore</code>.
 219 </UL>
 220 Next, compile and link your module, fire up python and do the same
 221 tests as for 68K python. <p>
 222
 223 <H2>Getting the module to do real work</H2>
 224
 225 So far, so good. In half an hour or so we have created a complete new
 226 extension module for Python. The downside, however, is that the module
 227 does not do anything useful. So, in the next half hour we will turn
 228 our beautiful skeleton module into something that is at least as
 229 beautiful but also gets some serious work done.  For this once,
 230 <EM>I</EM> have spent that half hour for you, and you can see the
 231 results in <A
 232 HREF="interslip/interslipmodule.c">interslipmodule.c</A>. <p>
 233
 234 We add
 235 <CODE><PRE>
 236         #include "InterslipLib.h"
 237         #include "macglue.h"
 238 </PRE></CODE>
 239 to the top of the file, and work our way through each of the methods
 240 to add the functionality needed. Starting with open, we fill in the
 241 template docstring, the value accessible from Python by looking at
 242 <CODE>interslip.open.__doc__</CODE>.  There are not many tools using
 243 this information at the moment, but as soon as class browsers for
 244 python become available having this minimal documentation available is
 245 a good idea. We put "Load the interslip driver" as the comment
 246 here. <p>
 247
 248 Next, we tackle the body of <CODE>pyis_open()</CODE>.  Since it has no
 249 arguments and no return value we don't need to mess with that, we just
 250 have to add a call to <CODE>is_open()</CODE> and check the return for
 251 an error code, in which case we raise an error:
 252 <CODE><PRE>
 253         err = is_open();
 254         if ( err ) {
 255                 PyErr_Mac(ErrorObject, err);
 256                 return NULL;
 257         }
 258 </PRE></CODE>
 259 The routine <CODE><A NAME="PyErr_Mac">PyErr_Mac()</A></CODE> is a
 260 useful routine that raises the exception passed as its first
 261 argument. The data passed with the exception is based on the standard
 262 MacOS error code given, and PyErr_Mac() attempts to locate a textual
 263 description of the error code (which sure beats the "error -14021"
 264 messages that so many macintosh applications tell their poor
 265 users). <p>
 266
 267 We will skip pyis_connect and pyis_disconnect here, which are pretty
 268 much identical to pyis_open: no arguments, no return value, just a
 269 call and an error check. With pyis_status() things get interesting
 270 again: this call still takes 3 arguments, and all happen to be values
 271 returned (a numeric connection status indicator, a message sequence
 272 number and a pointer to the message itself, in MacOS pascal-style
 273 string form). We declare variables to receive the returned values, do
 274 the call, check the error and format the return value. <p>
 275
 276 Building the return value is done using <CODE><A
 277 NAME="Py_BuildValue">Py_BuildValue</A></CODE>:
 278 <CODE><PRE>
 279         return Py_BuildValue("iiO&", (int)status, (int)seqnum, PyMac_BuildStr255, message);
 280 </PRE></CODE>
 281 Py_BuildValue() is a very handy routine that builds tuples according
 282 to a format string, somewhat similar to the way <CODE>printf()</CODE>
 283 works.  The format string specifies the arguments expected after the
 284 string, and turns them from C objects into python objects. The
 285 resulting objects are put in a python tuple object and returned. The
 286 "i" format specifier signifies an "int" (hence the cast: status and
 287 seqnum are declared as "long", which is what the is_status() routine
 288 wants, and even though we use a 4-byte project there is really no
 289 reason not to put the cast here). Py_BuildValue and its counterpart
 290 Py_ParseTuple have format codes for all the common C types like ints,
 291 shorts, C-strings, floats, etc. Also, there is a nifty escape
 292 mechanism to format values about which is does not know. This is
 293 invoked by the "O&" format: it expects two arguments, a routine
 294 pointer and an int-sized data object. The routine is called with the
 295 object as a parameter and it should return a python objects
 296 representing the data. <CODE>Macglue.h</CODE> declares a number of
 297 such formatting routines for common MacOS objects like Str255, FSSpec,
 298 OSType, Rect, etc. See the comments in the include file for
 299 details. <p>
 300
 301 <CODE>Pyis_getconfig()</CODE> is again similar to pyis_getstatus, only
 302 two minor points are worth noting here.  First, the C API return the
 303 input and output baudrate squashed together into a single 4-byte
 304 long. We separate them out before returning the result to
 305 python. Second, whereas the status call returned us a pointer to a
 306 <CODE>Str255</CODE> it kept we are responsible for allocating the
 307 <CODE>Str255</CODE> for getconfig. This is something that would have
 308 been easy to get wrong had we not used prototypes everywhere. Morale:
 309 always try to include the header files for interfaces to libraries and
 310 other stuff, so that the compiler can catch any mistakes you make. <p>
 311
 312 <CODE>Pyis_setconfig()</CODE> finally shows off
 313 <CODE>Py_ParseTuple</CODE>, the companion function to
 314 <CODE>Py_BuildValue</CODE>.  You pass it the argument tuple "args"
 315 that your method gets as its second argument, a format string and
 316 pointers to where you want the arguments stored. Again, standard C
 317 types such as strings and integers Py_ParseTuple knows all about and
 318 through the "O&" format you can extend the functionality. For each
 319 "O&" you pass a function pointer and a pointer to a data area. The
 320 function will be called with a PyObject pointer and your data pointer
 321 and it should convert the python object to the correct C type. It
 322 should return 1 on success and 0 on failure. Again, a number of
 323 converters for standard MacOS types are provided, and declared in
 324 <CODE>macglue.h</CODE>. <p>
 325
 326 Next in our source file comes the method table for our module, which
 327 has been generated by modulator (and it did a good job too!), but
 328 which is worth looking at for a moment.  Entries are of the form
 329 <CODE><PRE>
 330         {"open",        pyis_open,      1,      pyis_open__doc__},
 331 </PRE></CODE>
 332 where the entries are python method name, C routine pointer, flags and
 333 docstring pointer.  The value to note is the 1 for the flags: this
 334 signifies that you want to use "new-style" Py_ParseTuple behaviour. If
 335 you are writing a new module always use this, but if you are modifying
 336 old code which calls something like <CODE>getargs(args, "(ii)",
 337 ...)</CODE> you will have to put zero here. See "extending and
 338 embedding" or possibly the getargs.c source file for details if you
 339 need them. <p>
 340
 341 Finally, we add some code to the init module, to put some symbolic
 342 constants (codes that can by returned by the status method) in the
 343 module dictionary, so the python program can use "interslip.RUN"
 344 instead of the cryptic "4" when it wants to check that the interslip
 345 driver is in RUN state. Modulator has already generated code to get at
 346 the module dictionary using PyModule_GetDict() to store the exception
 347 object, so we simply call
 348 <CODE><PRE>
 349         PyDict_SetItemString(d, "IDLE", PyInt_FromLong(IS_IDLE));
 350 </PRE></CODE>
 351 for each of our items. Since the last bit of code in our init routine
 352 checks for previous errors with <CODE>PyErr_Occurred()</CODE> and
 353 since <CODE>PyDict_SetItemString()</CODE> gracefully handles the case
 354 of <CODE>NULL</CODE> parameters (if <CODE>PyInt_FromLong()</CODE>
 355 failed, for instance) we don't have to do error checking here. In some
 356 other cases you may have to do error checking yourself. <p>
 357
 358 This concludes our crash-course on writing Python extensions in C on
 359 the Macintosh.  If you are not done reading yet I suggest you look
 360 back at the <A HREF="index.html">MacPython Crashcourse index</A> to
 361 find another topic to study. <p>