Mac/Demo/mpwextensions.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
   2 <HTML>
   3
   4 <HEAD>
   5
   6 <TITLE>HOWTO: Compiling Python Modules with MPW</TITLE>
   7
   8 </HEAD>
   9
  10 <BODY>
  11
  12 <H1>HOWTO: Compiling Python Modules with MPW</H1>
  13
  14 <blockquote>
  15 This HOWTO is a slightly reformatted version of an original by
  16 <A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, whose
  17 <A  HREF="http://www.nevada.edu/~cwebster/Python/">Python page</A>
  18 may contain a more up-to-date version.
  19 </blockquote>
  20 <HR>
  21
  22 <P>
  23 The <A HREF="http://www.cwi.nl/~jack/macpython.html">Macintosh version</A>
  24 of the <A HREF="http://www.python.org/">Python programming language</A> is
  25 usually compiled with <A HREF="http://www.metrowerks.com/">Metrowerks
  26 CodeWarrior</A>.  As a result, C extension modules are also usually
  27 compiled with CodeWarrior, and the documentation and sample code reflects
  28 this.  CodeWarrior is a commercial product, and may be beyond the budgets
  29 of hobbyist hackers, making them dependent on others to compile C extension
  30 modules.  At the present time, many standard C extension modules compile
  31 "out of the box" on the Macintosh, but in only a few cases is the plugin
  32 for the Macintosh included in the distribution.
  33 </P>
  34
  35 <P>
  36 The <A HREF="http://developer.apple.com/tools/mpw-tools/">Macintosh
  37 Programmer's Workshop</A> (MPW) is Apple's development environment, and is
  38 freely available for <A
  39 HREF="ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./">download</A>
  40 from Apple, as well as on their Developer CDs.  Since Python was originally
  41 developed using MPW, before CodeWarrior became the dominant MacOS
  42 development environment, most of the idiosyncrasies of MPW are already
  43 supported, and compilation of C extension modules in MPW is possible.
  44 </P>
  45
  46 <P>
  47 This HOWTO only deals with compiling for PowerPC Macintoshes.  The process
  48 should be similar for 68k Macintoshes using the code fragment manager, but
  49 I have not attempted this - my old Mac is running NetBSD.
  50 </P>
  51
  52 <P>
  53 This way of compiling modules is still experimental.  Please read the
  54 caveats section below.
  55 </P>
  56
  57 <H2><A NAME="setup">Setting Up MPW for Compiling Python Modules</A></H2>
  58
  59 <P>
  60 This assumes that you have successfully installed both MPW and Python with
  61 the Developer's Kit on your Macintosh.
  62 </P>
  63
  64 <P>
  65 The first step is to let MPW know where you keep Python.  This step is not
  66 strictly necessary, but will make development easier and improve
  67 portability.  Create a new file in the <CODE>Startup Items</CODE> folder of
  68 MPW called <A HREF="Python"><CODE>Python</CODE></A>.  Type the lines:
  69 </P>
  70
  71 <PRE>
  72 set Python "Macintosh HD:Applications:Python 1.5.2c1:"
  73 set PythonIncludes "{Python}Include"
  74 set PythonMacIncludes "{Python}Mac:Include"
  75 set PythonCore "{Python}PythonCore"
  76
  77 export Python PythonIncludes PythonMacIncludes PythonCore
  78 </PRE>
  79
  80 <P>
  81 where <CODE>Macintosh HD:Applications:Python 1.5.2c1:</CODE> is replaced by
  82 the path to the directory where you keep your copy of Python, and the other
  83 variables reflect where you keep your header files and Python core files.
  84 The locations here are the standard for Python 1.5.2c1, but they are
  85 different for Python 1.52b2 and earlier (most notably, the PythonCore is
  86 kept in the Extensions folder).
  87 </P>
  88
  89 <P>
  90 Next, you need to update the <A HREF="config.h"><CODE>config.h</CODE></A>
  91 file for the <CODE>MrC</CODE> compiler included with MPW.  This header file
  92 is located in the <CODE>:Mac:Include</CODE> folder in the standard
  93 distribution.  You can update it by hand, by adding the lines:
  94 </P>
  95
  96 <PRE>
  97 #ifdef __MRC__
  98 #define BAD_STATIC_FORWARD
  99 #endif
 100 </PRE>
 101
 102 <P>
 103 at the after the similar defines for <CODE>__MWERKS__</CODE> and
 104 <CODE>__SC__</CODE> in the file. This step is critical: many modules,
 105 including ones in the standard distribution, will not compile properly
 106 without this modification (see common problems below).
 107 </P>
 108
 109 <P>
 110 Copies of both the <A HREF="Python"><CODE>Python</CODE></A> startup item
 111 for MPW and the <A HREF="config.h"><CODE>config.h</CODE></A> are included
 112 here for your convenience.
 113 </P>
 114
 115 <P>
 116 If you are porting Unix modules to the mac, you may find it useful to
 117 install <A
 118 HREF="http://www.iis.ee.ethz.ch/~neeri/macintosh/gusi-qa.html">GUSI</A> for
 119 your copy of MPW.  GUSI provides some amount of POSIX compatibility, and is
 120 used by Python itself for this purpose - at the very least having it's
 121 header files available may be useful.  Also of note for people porting Unix
 122 modules, the most recent alpha version (4.1a8) of <CODE>MrC</CODE> and
 123 <CODE>MrCpp</CODE> at this writing permits using unix-style pathnames for
 124 includes via the <CODE>-includes unix</CODE> command line option.  I have
 125 not experimented heavily with this, but will be doing so in the future and
 126 report my findings.
 127 </P>
 128
 129 <P>
 130 You now have MPW and Python set up to allow compilation of modules.
 131 </P>
 132
 133 <H2><A NAME="compiling">Compiling a Module</A></H2>
 134
 135 <P>
 136 This assumes that you have a C extension module ready to compile.  For
 137 instructions on how to write a module, see the Python documentation.
 138 </P>
 139
 140 <P>
 141 There are three approaches you can take to compiling in MPW: using the
 142 command line interface, using the MPW <CODE>CreateMake</CODE> command
 143 (available as the "Create build commands..." menu item, and writing a
 144 Makefile by hand.
 145 </P>
 146
 147 <P>
 148 Before you start any of these, you'll need to know:
 149 </P>
 150
 151 <UL>
 152     <LI>The names and locations of the C source files.  In the examples, this
 153     is the file <A HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>, and is in
 154     MPW's current working directory.
 155     <LI>The name that Python expects to import your module under.  In the
 156     examples, this is <CODE>xx</CODE>, so the shared library file will be
 157     called <CODE>xx.ppc.slb</CODE>.
 158     <LI>The location of any additional header files use by the C source. The
 159     example does not use any additional header files.
 160     <LI>The location of any additional shared libraries which the module needs
 161     to link to.  The example does not link to any other shared libraries.
 162     <LI>The name of the entry point to your module.  This is usually the last
 163     function in the main C source file, and the name usually starts with
 164     <CODE>init</CODE>.  In the examples, this is <CODE>initxx</CODE>.
 165 </UL>
 166
 167 <H3>Using the Command Line</H3>
 168
 169 <P>
 170 For simple modules consisting of one or two C files, it's often convenient
 171 to simply use commands in a MPW Worksheet.  Usually you will want to set
 172 MPW's working directory to the directory containing the C source code.  The
 173 following commands compile and link the standard Python test module <A
 174 HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
 175 </P>
 176
 177 <PRE>
 178 MrC "xxmodule.c" -o "xx.c.x" -w off  -d HAVE_CONFIG_H &#8706;
 179         -i "{PythonMacIncludes}" &#8706;
 180         -i "{PythonIncludes}"
 181 PPCLink &#8706;
 182         -o "xx.ppc.slb"  &#8706;
 183         "xx.c.x" &#8706;
 184         -t 'shlb' &#8706;
 185         -c 'Pyth' &#8706;
 186         -xm s &#8706;
 187         -d &#8706;
 188         "{PythonCore}" &#8706;
 189         "{SharedLibraries}InterfaceLib" &#8706;
 190         "{SharedLibraries}MathLib" &#8706;
 191         "{SharedLibraries}StdCLib" &#8706;
 192         "{PPCLibraries}StdCRuntime.o" &#8706;
 193         "{PPCLibraries}PPCCRuntime.o" &#8706;
 194         "{PPCLibraries}PPCToolLibs.o" &#8706;
 195         -export initxx
 196 </PRE>
 197
 198 <P>
 199 (Note: The last character on each line should appear as "partial
 200 derivative" symbol, which you type as <KBD>option-d</KBD> and which is
 201 MPW's line continuation symbol.)
 202 </P>
 203
 204 <P>
 205 Any additional header files should be specified by adding their directories
 206 as extra <CODE>-i</CODE> options to the <CODE>MrC</CODE> command.  Any
 207 additional shared libraries should be added before the PythonCore library
 208 in the <CODE>PPCLink</CODE> command.
 209 </P>
 210
 211 <P>
 212 If there is more than one source file, you will need to duplicate the
 213 compile command for each source file, and you will need to include all the
 214 object files in the place where <CODE>"xx.c.x"</CODE> appears in the
 215 <CODE>PPCLink</CODE> command.
 216 </P>
 217
 218 <H3>Using CreateMake</H3>
 219
 220 <P>
 221 For more complex modules, or modules that you are writing yourself, you
 222 will probably want to use a makefile. Unfortunately MPW's makefiles are
 223 incompatible with the standard Unix makefiles, so you will not be able to
 224 use any makefiles which come with a C module.
 225 </P>
 226
 227 <P>
 228 Usually, you will want the makefile to reside in the same directory as the
 229 C source code, so you should set MPW's working directory to that directory
 230 before proceeding.
 231 </P>
 232
 233 <P>
 234 To create a makefile for the standard Python test module <A
 235 HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
 236 </P>
 237
 238 <UL>
 239     <LI>Select "Create build commands..." from the "Build" Menu.
 240     <LI>Type <KBD>xx.ppc.slb</KBD> for the Program Name.
 241     <LI>Select "Shared Library" for the Program Type.
 242     <LI>Select "PowerPC Only" for the Target.
 243     <LI>Click on the "Source Files..." button, and add your module's C source
 244     files to the list.
 245     <LI>Click on the "Other Options..." button and change the creator type to
 246     "Pyth".  If you are using additional header files, you can also add their
 247     directories at this stage.  Click on "Continue" once you have done this.
 248     <LI>Click on the "Exported Symbols..." button and type <KBD>initxx</KBD>
 249     into the entry field.  Click on "Continue" once you have done this.
 250     <LI>At this stage, your CreateMake window should look like this: <IMG
 251     SRC="html.icons/createmake.png" ALT="[picture of commando window for CreateMake]">
 252     <LI>Click on the "CreateMake" button.
 253 </UL>
 254
 255 <P>
 256 You will now need to edit the makefile that was just created.  Open the
 257 file "xx.ppc.slb.make" in the current directory and make the following
 258 changes:
 259 </P>
 260
 261 <UL>
 262     <LI>Change the line
 263
 264     <PRE>
 265 Includes     =
 266 </PRE>
 267
 268     <P>
 269     to read
 270     </P>
 271
 272     <PRE>
 273 Includes     = -i "{PythonIncludes}" -i "{PythonMacIncludes}"
 274 </PRE>
 275
 276     <P>
 277     If you have any additional headers than need to be included, you can add
 278     them here as well.
 279     <LI>Change the line
 280
 281     <PRE>
 282 PPCCOptions  = {Includes} {Sym&#8226;PPC}
 283 </PRE>
 284
 285     <P>
 286     to read
 287     </P>
 288
 289     <PRE>
 290 PPCCOptions  = -w off  -d HAVE_CONFIG_H {Includes} {Sym&#8226;PPC}
 291 </PRE>
 292
 293     <P>
 294     <LI>After the line
 295
 296     <PRE>
 297                 -xm s &#8706;
 298 </PRE>
 299
 300     <P>
 301     add
 302     </P>
 303
 304     <PRE>
 305                 -d &#8706;
 306                 "{PythonCore}" &#8706;
 307 </PRE>
 308
 309     <P>
 310     If you have any other shared libraries you need to link to, add each on a
 311     line before PythonCore, terminating each line with a <CODE>&#8706;</CODE>.
 312     </P>
 313
 314 </UL>
 315
 316 <P>Save the file.  You are now ready to build.
 317 </P>
 318
 319 <P>
 320 Go to the "Build" or "Full Build" menu items, type in
 321 <KBD>xx.ppc.slb</KBD>, and MPW should take things from there.  Any time you
 322 need to rebuild the shared library, you can simply do another "Build" or
 323 "Full Build".
 324 </P>
 325
 326 <H3>Writing a Makefile by Hand</H3>
 327
 328 <P>
 329 For modules which have complex interdependencies between files, you will
 330 likely need a more sophisticated makefile than the one created by
 331 <CODE>CreateMake</CODE>.  You will need to be familiar with the MPW
 332 makefile format, but you can get a start by either using
 333 <CODE>CreateMake</CODE> to get a simple starting point, or taking another
 334 MPW makefile as a starting point.
 335 </P>
 336
 337 <P>
 338 It is beyond the scope of this HOWTO to go into the generalities of MPW
 339 makefiles. Documentation on MPW's <CODE>Make</CODE> command can be found
 340 with the MPW distribution, in particular the documents <A
 341 HREF="http://developer.apple.com/tools/mpw-tools/books.html#Building">Building
 342 and Maintaining Programs with MPW (2nd Edition)</A> and the <A
 343 HREF="http://developer.apple.com/tools/mpw-tools/books.html#CommandRef">MPW
 344 Command Reference</A>.
 345 </P>
 346
 347 <P>
 348 There are a couple of important points to keep in mind when writing a
 349 makefile by hand:</P>
 350
 351 <UL>
 352     <LI>When there are multiple symbols with the same name in object files or
 353     shared libraries, <CODE>PPCLink</CODE> used the symbol from the file which
 354     appears first in arguments of the <CODE>PPCLink</CODE> command.  For this
 355     reason, you will usually want the PythonCore and any other shared libraries
 356     which are not part of the standard MPW runtime environment to appear before
 357     the standard runtime libraries.  This is particularly the case with
 358     StdCLib.  The "-d" option turns off the (often copious) warnings about
 359     multiply defined symbols.
 360     <LI>You will want to make sure that the <CODE>HAVE_CONFIG_H</CODE>
 361     preprocessor symbol is defined for most C source files using the <CODE>-d
 362     HAVE_CONFIG_H</CODE> option to <CODE>MrC</CODE>.
 363 </UL>
 364
 365 <P>
 366 The file <A HREF="xx.ppc.slb.make.sit.hqx"><CODE>xx.ppc.slb.make</CODE></A>
 367 is included here for you to use as a starting point.
 368 </P>
 369
 370 <H2><A NAME="using">Using the Extension Module</A></H2>
 371
 372 <P>
 373 Once you have compiled your extension module, you will need to let Python
 374 know where it is.  You can either move it into a place on Python's search
 375 path - such as the <CODE>:Mac:Plugins</CODE> folder - or modify the path to
 376 include the location of your new module using the
 377 <CODE>EditPythonPrefs</CODE> applet.
 378 </P>
 379
 380 <P>
 381 Your work may not be completely done, as many extension modules have a
 382 Python wrapper around them.  If the Python was not written with portability
 383 in mind, you may need to do some more work to get that up and running.
 384 Indeed, if the Python part uses OS-specific features, like pipes, you may
 385 have to completely rewrite it if you can make it work at all.
 386 </P>
 387
 388 <H2><A NAME="problems">Common Problems</A></H2>
 389
 390 <P>
 391 There are a couple of common problems which occur when porting a module
 392 from another platform.  Fortunately, they are often easy to fix.
 393 </P>
 394
 395 <H3>Static Forward Definitions</H3>
 396
 397 <P>
 398 If you get a compiler error which looks something like:
 399 </P>
 400
 401 <PRE>
 402 File "xxmodule.c"; line 135 #Error: 'Xxo_Type' is already defined
 403 </PRE>
 404
 405 <P>
 406 then most likely either you have not set up <CODE>config.h</CODE> correctly
 407 to handle static forward definitions, or the module author has not adhered
 408 to the standard python conventions.  If the second is the case, find where
 409 the variable is first defined, and replace the <CODE>static</CODE> with
 410 <CODE>staticforward</CODE>. Then find the second place it is defined
 411 (usually the line where the compiler complained) and replace
 412 <CODE>static</CODE> with <CODE>statichere</CODE>.
 413 </P>
 414
 415 <P>
 416 If you have set up things correctly, you should now be able to compile.
 417 </P>
 418
 419 <H3>Automatic Type Conversion</H3>
 420
 421 <P>
 422 <CODE>MrC</CODE> seems to be a little pickier about automatically
 423 converting from one type to another than some other C compilers.  These can
 424 often be fixed by simply adding an explicit cast to the desired type.
 425 </P>
 426
 427 <P>
 428 XXX There may be a compiler option which relaxes this.  That would be a
 429 better solution.
 430 </P>
 431
 432 <H2><A NAME="caveats">Caveats</A></H2>
 433
 434 <P>
 435 As Jack Jansen pointed out on the Mac Python mailing list, there could
 436 potentially be conflicts between the MetroWerks C runtime which the Python
 437 core and standard modules was compiled with, and the MPW C runtime which
 438 your extension module is compiled with.  While things seem to work fine in
 439 everyday use, it is possible that there are bugs which have not been
 440 discovered yet.  Most likely these world take the form of standard C
 441 functions (most likely I/O functions due to conflicts between the SIOUX
 442 libraries and the SIOW libraries) not working as they are supposed to, or
 443 memory leaks caused by improper malloc/free.
 444 </P>
 445
 446 <P>
 447 Some such problems have been demonstrated by compiling modules with
 448 PythonCore linked after StdCLib - printf does not work properly in this
 449 setup, and I suspect that there will also be malloc/free problems in
 450 situations where the module allocates memory which is later disposed of by
 451 Python, or vice-versa. Compiling with PythonCore taking precedence over
 452 StdCLib seems to give the correct behaviour.
 453 </P>
 454
 455 <P>
 456 This method of compiling should be considered experimental for the time
 457 being.  <STRONG>Use it at your own risk.</STRONG>
 458 </P>
 459
 460 <P>
 461 If you notice any quirks in modules compiled this way, or have insight into
 462 what may go wrong or right with this situation, <A
 463 HREF="mailto:cwebster@nevada.edu">please contact me</A> so that I can add
 464 it to the HOWTO.
 465 </P>
 466
 467 <P>
 468 The ideal solution to this problem would be to get Python to compile using
 469 MPW (and a Python MPW Tool would be very neat indeed).  However, that does
 470 seem to be a major project.
 471 </P>
 472
 473 <DIV class=footer>
 474 <HR>
 475 <BR>
 476 &copy;<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, 1999. <BR>
 477 <!-- #LASTMODIFIED TEXT="Last modified" FORM="SHORT,TIME" -->
 478 Last modified 14/12/99 12:17 PM
 479 <!-- /#LASTMODIFIED -->
 480 </DIV>
 481
 482 </BODY>
 483
 484 </HTML>