| blob | 4817dcfcf8f50d4eaa3ff5f547f1109c2a646d98 |
3 % $Id$
10 \maketitle\tableofcontents
14 A new release of Python, version 2.0, will be released some time this
15 autumn. Beta versions are already available from
17 covers the exciting new features in 2.0, highlights some other useful
18 changes, and points out a few incompatible changes that may require
19 rewriting code.
21 Python's development never completely stops between releases, and a
22 steady flow of bug fixes and improvements are always being submitted.
23 A host of minor fixes, a few optimizations, additional docstrings, and
24 better error messages went into 2.0; to list them all would be
25 impossible, but they're certainly significant. Consult the
26 publicly-available CVS logs if you want to see the full list. This
27 progress is due to the five developers working for
28 PythonLabs are now getting paid to spend their days fixing bugs,
29 and also due to the improved communication resulting
30 from moving to SourceForge.
32 % ======================================================================
35 Python 1.6 can be thought of as the Contractual Obligations Python
36 release. After the core development team left CNRI in May 2000, CNRI
37 requested that a 1.6 release be created, containing all the work on
38 Python that had been performed at CNRI. Python 1.6 therefore
39 represents the state of the CVS tree as of May 2000, with the most
40 significant new feature being Unicode support. Development continued
41 after May, of course, so the 1.6 tree received a few fixes to ensure
43 of Python's evolution, and not a side branch.
45 So, should you take much interest in Python 1.6? Probably not. The
48 you have applications to maintain, there seems little point in
49 breaking things by moving to 1.6, fixing them, and then having another
50 round of breakage within a month by moving to 2.0; you're better off
51 just going straight to 2.0. Most of the really interesting features
53 done between May and September.
55 % ======================================================================
58 The most important change in Python 2.0 may not be to the code at all,
59 but to how Python is developed: in May 2000 the Python developers
60 began using the tools made available by SourceForge for storing
61 source code, tracking bug reports, and managing the queue of patch
63 bug tracking and patch manager tools available from Python's project
66 The most important of the services now hosted at SourceForge is the
67 Python CVS tree, the version-controlled repository containing the
68 source code for Python. Previously, there were roughly 7 or so people
69 who had write access to the CVS tree, and all patches had to be
70 inspected and checked in by one of the people on this short list.
71 Obviously, this wasn't very scalable. By moving the CVS tree to
72 SourceForge, it became possible to grant write access to more people;
74 fourfold increase. This makes possible large-scale changes that
75 wouldn't be attempted if they'd have to be filtered through the small
76 group of core developers. For example, one day Peter Schneider-Kamp
77 took it into his head to drop K\&R C compatibility and convert the C
78 source for Python to ANSI C. After getting approval on the python-dev
79 mailing list, he launched into a flurry of checkins that lasted about
80 a week, other developers joined in to help, and the job was done. If
81 there were only 5 people with write access, probably that task would
82 have been viewed as ``nice, but not worth the time and effort needed''
83 and it would never have gotten done.
85 The shift to using SourceForge's services has resulted in a remarkable
86 increase in the speed of development. Patches now get submitted,
87 commented on, revised by people other than the original submitter, and
88 bounced back and forth between people until the patch is deemed worth
89 checking in. Bugs are tracked in one central location and can be
90 assigned to a specific person for fixing, and we can count the number
91 of open bugs to measure progress. This didn't come without a cost:
92 developers now have more e-mail to deal with, more mailing lists to
93 follow, and special tools had to be written for the new environment.
94 For example, SourceForge sends default patch and bug notification
95 e-mail messages that are completely unhelpful, so Ka-Ping Yee wrote an
96 HTML screen-scraper that sends more useful messages.
98 The ease of adding code caused a few initial growing pains, such as
99 code was checked in before it was ready or without getting clear
100 agreement from the developer group. The approval process that has
101 emerged is somewhat similar to that used by the Apache group.
104 indifferent to the change, though with a slight positive or negative
105 slant. The most significant change from the Apache model is that the
106 voting is essentially advisory, letting Guido van Rossum, who has
107 Benevolent Dictator For Life status, know what the general opinion is.
108 He can still ignore the result of a vote, and approve or
109 reject a change even if the community disagrees with him.
111 Producing an actual patch is the last step in adding a new feature,
112 and is usually easy compared to the earlier task of coming up with a
113 good design. Discussions of new features can often explode into
114 lengthy mailing list threads, making the discussion hard to follow,
115 and no one can read every posting to python-dev. Therefore, a
116 relatively formal process has been set up to write Python Enhancement
117 Proposals (PEPs), modelled on the Internet RFC process. PEPs are
118 draft documents that describe a proposed new feature, and are
119 continually revised until the community reaches a consensus, either
120 accepting or rejecting the proposal. Quoting from the introduction to
121 PEP 1, ``PEP Purpose and Guidelines'':
124 PEP stands for Python Enhancement Proposal. A PEP is a design
125 document providing information to the Python community, or
126 describing a new feature for Python. The PEP should provide a
127 concise technical specification of the feature and a rationale for
128 the feature.
130 We intend PEPs to be the primary mechanisms for proposing new
131 features, for collecting community input on an issue, and for
132 documenting the design decisions that have gone into Python. The
133 PEP author is responsible for building consensus within the
134 community and documenting dissenting opinions.
137 Read the rest of PEP 1 for the details of the PEP editorial process,
138 style, and format. PEPs are kept in the Python CVS tree on
139 SourceForge, though they're not part of the Python 2.0 distribution,
140 and are also available in HTML form from
143 PEP 225, ``Elementwise/Objectwise Operators''.
145 % ======================================================================
148 The largest new feature in Python 2.0 is a new fundamental data type:
149 Unicode strings. Unicode uses 16-bit numbers to represent characters
151 distinct characters can be supported.
153 The final interface for Unicode support was arrived at through
154 countless often-stormy discussions on the python-dev mailing list, and
155 mostly implemented by Marc-Andr\'e Lemburg, based on a Unicode string
156 type implementation by Fredrik Lundh. A detailed explanation of the
158 distribution; it's also available on the Web at
160 This article will simply cover the most significant points about the Unicode
161 interfaces.
163 In Python source code, Unicode strings are written as
168 escapes can be used for characters up to U+01FF, which is represented
171 Unicode strings, just like regular strings, are an immutable sequence
172 type. They can be indexed and sliced, but not modified in place.
174 that returns an 8-bit string in the desired encoding. Encodings are
177 implementing and registering new encodings that are then available
178 throughout a Python program. If an encoding isn't specified, the
179 default encoding is usually 7-bit ASCII, though it can be changed for
180 your Python installation by calling the
184 Combining 8-bit and Unicode strings always coerces to Unicode, using
188 New built-in functions have been added, and existing built-ins
189 modified to support Unicode:
195 \item \code{ord(\var{u})}, where \var{u} is a 1-character regular or Unicode string, returns the number of the character as an integer.
202 value causes an exception to be raised on any encoding error, while
205 case of any problems.
209 accept Unicode strings as well as regular strings. (It's possible
210 that the process of fixing this missed some built-ins; if you find a
211 built-in function that accepts strings but doesn't accept Unicode
212 strings at all, please report it as a bug.)
218 returns the 2-character string 'Lu', the 'L' denoting it's a letter,
219 and 'u' meaning that it's uppercase.
221 an Arabic number.
224 and register new ones. Unless you want to implement a
225 new encoding, you'll most often use the
233 is an 8-bit string containing a portion (perhaps all) of the Unicode
235 how much of the Unicode string was converted.
241 8-bit string was consumed.
247 the given encoding and return Unicode strings.
253 translating them to the given encoding on output.
256 For example, the following code writes a Unicode string into a file,
257 encoding it as UTF-8:
260 import codecs
262 unistr = u'\u0660\u2000ab ...'
264 (UTF8_encode, UTF8_decode,
265 UTF8_streamreader, UTF8_streamwriter) = codecs.lookup('UTF-8')
267 output = UTF8_streamwriter( open( '/tmp/output', 'wb') )
268 output.write( unistr )
269 output.close()
272 The following code would then read UTF-8 input from the file:
275 input = UTF8_streamreader( open( '/tmp/output', 'rb') )
276 print repr(input.read())
277 input.close()
280 Unicode-aware regular expressions are available through the
282 SRE written by Fredrik Lundh of Secret Labs AB.
285 compiler to interpret all string literals as Unicode string literals.
286 This is intended to be used in testing and future-proofing your Python
287 code, since some future version of Python may drop support for 8-bit
288 strings and provide only Unicode strings.
290 % ======================================================================
293 Lists are a workhorse data type in Python, and many programs
294 manipulate a list at some point. Two common operations on lists are
295 to loop over them, and either pick out the elements that meet a
296 certain criterion, or apply some function to each element. For
297 example, given a list of strings, you might want to pull out all the
298 strings containing a given substring, or strip off trailing whitespace
299 from each line.
302 used for this purpose, but they require a function as one of their
303 arguments. This is fine if there's an existing built-in function that
304 can be passed directly, but if there isn't, you have to create a
305 little function to do the required work, and Python's scoping rules
306 make the result ugly if the little function needs additional
307 information. Take the first example in the previous paragraph,
308 finding all the strings in the list containing a given substring. You
309 could write the following to do it:
312 # Given the list L, make a list of all strings
313 # containing the substring S.
314 sublist = filter( lambda s, substring=S:
315 string.find(s, substring) != -1,
316 L)
319 Because of Python's scoping rules, a default argument is used so that
321 what substring is being searched for. List comprehensions make this
322 cleaner:
328 List comprehensions have the form:
331 [ expression for expr in sequence1
332 for expr2 in sequence2 ...
333 for exprN in sequenceN
334 if condition
338 iterated over. The sequences do not have to be the same length,
340 from left to right; this is explained more clearly in the following
341 paragraphs. The elements of the generated list will be the successive
346 To make the semantics very clear, a list comprehension is equivalent
347 to the following Python code:
350 for expr1 in sequence1:
351 for expr2 in sequence2:
352 ...
353 for exprN in sequenceN:
354 if (condition):
355 # Append the value of
356 # the expression to the
357 # resulting list.
361 the resulting list will be equal to the product of the lengths of all
362 the sequences. If you have two lists of length 3, the output list is
363 9 elements long:
366 seq1 = 'abc'
373 To avoid introducing an ambiguity into Python's grammar, if
375 parentheses. The first list comprehension below is a syntax error,
376 while the second one is correct:
379 # Syntax error
381 # Correct
385 The idea of list comprehensions originally comes from the functional
387 Ewing argued most effectively for adding them to Python and wrote the
388 initial list comprehension patch, which was then discussed for a
389 seemingly endless time on the python-dev mailing list and kept
390 up-to-date by Skip Montanaro.
392 % ======================================================================
395 Augmented assignment operators, another long-requested feature, have
396 been added to Python 2.0. Augmented assignment operators include
404 override the augmented assignment operators by defining methods named
407 new instance with an incremented value.
410 class Number:
411 def __init__(self, value):
412 self.value = value
413 def __iadd__(self, increment):
414 return Number( self.value + increment)
416 n = Number(5)
417 n += 3
418 print n.value
422 increment, and should return a new instance with an appropriately
423 modified value; this return value is bound as the new value of the
424 variable on the left-hand side.
426 Augmented assignment operators were first introduced in the C
427 programming language, and most C-derived languages, such as
429 assignment patch was implemented by Thomas Wouters.
431 % ======================================================================
436 module written in C. The addition of Unicode posed a difficulty for
438 rewritten in order to accept either 8-bit or Unicode strings. For
440 arguments, that means eight possible permutations, and correspondingly
441 complicated code.
443 Instead, Python 2.0 pushes the problem onto the string type, making
444 string manipulation functionality available through methods on both
445 8-bit strings and Unicode strings.
448 >>> 'andrew'.capitalize()
449 'Andrew'
450 >>> 'hostname'.replace('os', 'linux')
451 'hlinuxtname'
452 >>> 'moshe'.find('sh')
453 2
456 One thing that hasn't changed, a noteworthy April Fools' joke
457 notwithstanding, is that Python strings are immutable. Thus, the
458 string methods return new strings, and do not modify the string on
459 which they operate.
462 compatibility, but it mostly acts as a front-end to the new string
463 methods.
465 Two methods which have no parallel in pre-2.0 versions, although they
477 % ======================================================================
480 The C implementation of Python uses reference counting to implement
481 garbage collection. Every Python object maintains a count of the
482 number of references pointing to itself, and adjusts the count as
483 references are created or destroyed. Once the reference count reaches
484 zero, the object is no longer accessible, since you need to have a
485 reference to an object to access it, and if the count is zero, no
486 references exist any longer.
488 Reference counting has some pleasant properties: it's easy to
489 understand and implement, and the resulting implementation is
490 portable, fairly fast, and reacts well with other libraries that
491 implement their own memory handling schemes. The major problem with
492 reference counting is that it sometimes doesn't realise that objects
493 are no longer accessible, resulting in a memory leak. This happens
494 when there are cycles of references.
496 Consider the simplest possible cycle,
497 a class instance which has a reference to itself:
500 instance = SomeClass()
501 instance.myself = instance
504 After the above two lines of code have been executed, the reference
507 attribute of the instance.
512 still exists. Yet the instance is no longer accessible through Python
513 code, and it could be deleted. Several objects can participate in a
514 cycle if they have references to each other, causing all of the
515 objects to be leaked.
517 Python 2.0 fixes this problem by periodically executing a cycle
518 detection algorithm which looks for inaccessible cycles and deletes
520 perform a garbage collection, obtain debugging statistics, and tuning
521 the collector's parameters.
523 Running the cycle detection algorithm takes some time, and therefore
524 will result in some additional overhead. It is hoped that after we've
526 will be able to minimize the overhead with careful tuning. It's not
527 yet obvious how much performance is lost, because benchmarking this is
528 tricky and depends crucially on how often the program creates and
529 destroys objects. The detection of cycles can be disabled when Python
530 is compiled, if you can't afford even a tiny speed penalty or suspect
531 that the cycle collection is buggy, by specifying the
533 script.
535 Several people tackled this problem and contributed to a solution. An
536 early implementation of the cycle detection approach was written by
537 Toby Kelsey. The current algorithm was suggested by Eric Tiedemann
538 during a visit to CNRI, and Guido van Rossum and Neil Schemenauer
539 wrote two different implementations, which were later integrated by
540 Neil. Lots of other people offered suggestions along the way; the
541 March 2000 archives of the python-dev mailing list contain most of the
542 relevant discussion, especially in the threads titled ``Reference
543 cycle collection for Python'' and ``Finalization again''.
545 % ======================================================================
548 Various minor changes have been made to Python's syntax and built-in
549 functions. None of the changes are very far-reaching, but they're
550 handy conveniences.
554 A new syntax makes it more convenient to call a given function
555 with a tuple of arguments and/or a dictionary of keyword arguments.
560 is the same in 2.0, but thanks to a patch from
562 and clearer way to achieve the same effect. This syntax is
563 symmetrical with the syntax for defining functions:
565 \begin{verbatim}
566 def f(*args, **kw):
567 # args is a tuple of positional args,
568 # kw is a dictionary of keyword args
569 ...
570 \end{verbatim}
572 The \keyword{print} statement can now have its output directed to a
573 file-like object by following the \keyword{print} with
574 \verb|>> file|, similar to the redirection operator in Unix shells.
575 Previously you'd either have to use the \method{write()} method of the
576 file-like object, which lacks the convenience and simplicity of
577 \keyword{print}, or you could assign a new value to
578 \code{sys.stdout} and then restore the old value. For sending output to standard error,
579 it's much easier to write this:
581 \begin{verbatim}
582 print >> sys.stderr, "Warning: action field not supplied"
583 \end{verbatim}
585 Modules can now be renamed on importing them, using the syntax
586 \code{import \var{module} as \var{name}} or \code{from \var{module}
587 import \var{name} as \var{othername}}. The patch was submitted by
588 Thomas Wouters.
590 A new format style is available when using the \code{\%} operator;
591 '\%r' will insert the \function{repr()} of its argument. This was
592 also added from symmetry considerations, this time for symmetry with
593 the existing '\%s' format style, which inserts the \function{str()} of
594 its argument. For example, \code{'\%r \%s' \% ('abc', 'abc')} returns a
595 string containing \verb|'abc' abc|.
597 Previously there was no way to implement a class that overrode
598 Python's built-in \keyword{in} operator and implemented a custom
599 version. \code{\var{obj} in \var{seq}} returns true if \var{obj} is
600 present in the sequence \var{seq}; Python computes this by simply
601 trying every index of the sequence until either \var{obj} is found or
602 an \exception{IndexError} is encountered. Moshe Zadka contributed a
603 patch which adds a \method{__contains__} magic method for providing a
604 custom implementation for \keyword{in}. Additionally, new built-in
605 objects written in C can define what \keyword{in} means for them via a
606 new slot in the sequence protocol.
608 Earlier versions of Python used a recursive algorithm for deleting
609 objects. Deeply nested data structures could cause the interpreter to
610 fill up the C stack and crash; Christian Tismer rewrote the deletion
611 logic to fix this problem. On a related note, comparing recursive
612 objects recursed infinitely and crashed; Jeremy Hylton rewrote the
613 code to no longer crash, producing a useful result instead. For
614 example, after this code:
616 \begin{verbatim}
617 a = []
618 b = []
619 a.append(a)
620 b.append(b)
621 \end{verbatim}
623 The comparison \code{a==b} returns true, because the two recursive
624 data structures are isomorphic. See the thread ``trashcan
625 and PR\#7'' in the April 2000 archives of the python-dev mailing list
626 for the discussion leading up to this implementation, and some useful
627 relevant links.
628 % Starting URL:
629 % http://www.python.org/pipermail/python-dev/2000-April/004834.html
631 Note that comparisons can now also raise exceptions. In earlier
632 versions of Python, a comparison operation such as \code{cmp(a,b)}
633 would always produce an answer, even if a user-defined
634 \method{__cmp__} method encountered an error, since the resulting
635 exception would simply be silently swallowed.
637 Work has been done on porting Python to 64-bit Windows on the Itanium
638 processor, mostly by Trent Mick of ActiveState. (Confusingly,
639 \code{sys.platform} is still \code{'win32'} on Win64 because it seems
640 that for ease of porting, MS Visual C++ treats code as 32 bit on Itanium.)
641 PythonWin also supports Windows CE; see the Python CE page at
642 \url{http://starship.python.net/crew/mhammond/ce/} for more
643 information.
645 Another new platform is Darwin/MacOS X; inital support for it is in
646 Python 2.0. Dynamic loading works, if you specify ``configure
647 --with-dyld --with-suffix=.x''. Consult the README in the Python
648 source distribution for more instructions.
650 An attempt has been made to alleviate one of Python's warts, the
651 often-confusing \exception{NameError} exception when code refers to a
652 local variable before the variable has been assigned a value. For
653 example, the following code raises an exception on the \keyword{print}
654 statement in both 1.5.2 and 2.0; in 1.5.2 a \exception{NameError}
655 exception is raised, while 2.0 raises a new
656 \exception{UnboundLocalError} exception.
657 \exception{UnboundLocalError} is a subclass of \exception{NameError},
658 so any existing code that expects \exception{NameError} to be raised
659 should still work.
661 \begin{verbatim}
662 def f():
663 print "i=",i
664 i = i + 1
665 f()
666 \end{verbatim}
668 Two new exceptions, \exception{TabError} and
669 \exception{IndentationError}, have been introduced. They're both
670 subclasses of \exception{SyntaxError}, and are raised when Python code
671 is found to be improperly indented.
673 \subsection{Changes to Built-in Functions}
675 A new built-in, \function{zip(\var{seq1}, \var{seq2}, ...)}, has been
676 added. \function{zip()} returns a list of tuples where each tuple
677 contains the i-th element from each of the argument sequences. The
678 difference between \function{zip()} and \code{map(None, \var{seq1},
679 \var{seq2})} is that \function{map()} pads the sequences with
680 \code{None} if the sequences aren't all of the same length, while
681 \function{zip()} truncates the returned list to the length of the
682 shortest argument sequence.
684 The \function{int()} and \function{long()} functions now accept an
685 optional ``base'' parameter when the first argument is a string.
686 \code{int('123', 10)} returns 123, while \code{int('123', 16)} returns
687 291. \code{int(123, 16)} raises a \exception{TypeError} exception
688 with the message ``can't convert non-string with explicit base''.
690 A new variable holding more detailed version information has been
691 added to the \module{sys} module. \code{sys.version_info} is a tuple
692 \code{(\var{major}, \var{minor}, \var{micro}, \var{level},
693 \var{serial})} For example, in a hypothetical 2.0.1beta1,
694 \code{sys.version_info} would be \code{(2, 0, 1, 'beta', 1)}.
695 \var{level} is a string such as \code{"alpha"}, \code{"beta"}, or
696 \code{"final"} for a final release.
698 Dictionaries have an odd new method, \method{setdefault(\var{key},
699 \var{default})}, which behaves similarly to the existing
700 \method{get()} method. However, if the key is missing,
701 \method{setdefault()} both returns the value of \var{default} as
702 \method{get()} would do, and also inserts it into the dictionary as
703 the value for \var{key}. Thus, the following lines of code:
705 \begin{verbatim}
706 if dict.has_key( key ): return dict[key]
707 else:
708 dict[key] = []
709 return dict[key]
710 \end{verbatim}
712 can be reduced to a single \code{return dict.setdefault(key, [])} statement.
714 The interpreter sets a maximum recursion depth in order to catch
715 runaway recursion before filling the C stack and causing a core dump
716 or GPF.. Previously this limit was fixed when you compiled Python,
717 but in 2.0 the maximum recursion depth can be read and modified using
718 \function{sys.getrecursionlimit} and \function{sys.setrecursionlimit}.
719 The default value is 1000, and a rough maximum value for a given
720 platform can be found by running a new script,
721 \file{Misc/find_recursionlimit.py}.
723 % ======================================================================
724 \section{Porting to 2.0}
726 New Python releases try hard to be compatible with previous releases,
727 and the record has been pretty good. However, some changes are
728 considered useful enough, usually because they fix initial design decisions that
729 turned out to be actively mistaken, that breaking backward compatibility
730 can't always be avoided. This section lists the changes in Python 2.0
731 that may cause old Python code to break.
733 The change which will probably break the most code is tightening up
734 the arguments accepted by some methods. Some methods would take
735 multiple arguments and treat them as a tuple, particularly various
736 list methods such as \method{.append()} and \method{.insert()}.
737 In earlier versions of Python, if \code{L} is a list, \code{L.append(
738 1,2 )} appends the tuple \code{(1,2)} to the list. In Python 2.0 this
739 causes a \exception{TypeError} exception to be raised, with the
740 message: 'append requires exactly 1 argument; 2 given'. The fix is to
741 simply add an extra set of parentheses to pass both values as a tuple:
742 \code{L.append( (1,2) )}.
744 The earlier versions of these methods were more forgiving because they
745 used an old function in Python's C interface to parse their arguments;
746 2.0 modernizes them to use \function{PyArg_ParseTuple}, the current
747 argument parsing function, which provides more helpful error messages
748 and treats multi-argument calls as errors. If you absolutely must use
749 2.0 but can't fix your code, you can edit \file{Objects/listobject.c}
750 and define the preprocessor symbol \code{NO_STRICT_LIST_APPEND} to
751 preserve the old behaviour; this isn't recommended.
753 Some of the functions in the \module{socket} module are still
754 forgiving in this way. For example, \function{socket.connect(
755 ('hostname', 25) )} is the correct form, passing a tuple representing
756 an IP address, but \function{socket.connect( 'hostname', 25 )} also
757 works. \function{socket.connect_ex()} and \function{socket.bind()} are
758 similarly easy-going. 2.0alpha1 tightened these functions up, but
759 because the documentation actually used the erroneous multiple
760 argument form, many people wrote code which would break with the
761 stricter checking. GvR backed out the changes in the face of public
762 reaction, so for the \module{socket} module, the documentation was
763 fixed and the multiple argument form is simply marked as deprecated;
764 it \emph{will} be tightened up again in a future Python version.
766 The \code{\e x} escape in string literals now takes exactly 2 hex
767 digits. Previously it would consume all the hex digits following the
768 'x' and take the lowest 8 bits of the result, so \code{\e x123456} was
769 equivalent to \code{\e x56}.
771 The \exception{AttributeError} exception has a more friendly error message,
772 whose text will be something like \code{'Spam' instance has no attribute 'eggs'}.
773 Previously the error message was just the missing attribute name \code{eggs}, and
774 code written to take advantage of this fact will break in 2.0.
776 Some work has been done to make integers and long integers a bit more
777 interchangeable. In 1.5.2, large-file support was added for Solaris,
778 to allow reading files larger than 2Gb; this made the \method{tell()}
779 method of file objects return a long integer instead of a regular
780 integer. Some code would subtract two file offsets and attempt to use
781 the result to multiply a sequence or slice a string, but this raised a
782 \exception{TypeError}. In 2.0, long integers can be used to multiply
783 or slice a sequence, and it'll behave as you'd intuitively expect it
784 to; \code{3L * 'abc'} produces 'abcabcabc', and \code{
785 (0,1,2,3)[2L:4L]} produces (2,3). Long integers can also be used in
786 various contexts where previously only integers were accepted, such
787 as in the \method{seek()} method of file objects, and in the formats
788 supported by the \verb|%| operator (\verb|%d|, \verb|%i|, \verb|%x|,
789 etc.). For example, \code{"\%d" \% 2L**64} will produce the string
790 \samp{18446744073709551616}.
792 The subtlest long integer change of all is that the \function{str()}
793 of a long integer no longer has a trailing 'L' character, though
794 \function{repr()} still includes it. The 'L' annoyed many people who
795 wanted to print long integers that looked just like regular integers,
796 since they had to go out of their way to chop off the character. This
797 is no longer a problem in 2.0, but code which does \code{str(longval)[:-1]} and assumes the 'L' is there, will now lose
798 the final digit.
800 Taking the \function{repr()} of a float now uses a different
801 formatting precision than \function{str()}. \function{repr()} uses
802 \code{\%.17g} format string for C's \function{sprintf()}, while
803 \function{str()} uses \code{\%.12g} as before. The effect is that
804 \function{repr()} may occasionally show more decimal places than
805 \function{str()}, for certain numbers.
806 For example, the number 8.1 can't be represented exactly in binary, so
807 \code{repr(8.1)} is \code{'8.0999999999999996'}, while str(8.1) is
808 \code{'8.1'}.
810 The \code{-X} command-line option, which turned all standard
811 exceptions into strings instead of classes, has been removed; the
812 standard exceptions will now always be classes. The
813 \module{exceptions} module containing the standard exceptions was
814 translated from Python to a built-in C module, written by Barry Warsaw
815 and Fredrik Lundh.
817 % Commented out for now -- I don't think anyone will care.
818 %The pattern and match objects provided by SRE are C types, not Python
819 %class instances as in 1.5. This means you can no longer inherit from
820 %\class{RegexObject} or \class{MatchObject}, but that shouldn't be much
821 %of a problem since no one should have been doing that in the first
822 %place.
824 % ======================================================================
825 \section{Extending/Embedding Changes}
827 Some of the changes are under the covers, and will only be apparent to
828 people writing C extension modules or embedding a Python interpreter
829 in a larger application. If you aren't dealing with Python's C API,
830 you can safely skip this section.
832 The version number of the Python C API was incremented, so C
833 extensions compiled for 1.5.2 must be recompiled in order to work with
834 2.0. On Windows, it's not possible for Python 2.0 to import a third
835 party extension built for Python 1.5.x due to how Windows DLLs work,
836 so Python will raise an exception and the import will fail.
838 Users of Jim Fulton's ExtensionClass module will be pleased to find
839 out that hooks have been added so that ExtensionClasses are now
840 supported by \function{isinstance()} and \function{issubclass()}.
841 This means you no longer have to remember to write code such as
842 \code{if type(obj) == myExtensionClass}, but can use the more natural
843 \code{if isinstance(obj, myExtensionClass)}.
845 The \file{Python/importdl.c} file, which was a mass of \#ifdefs to
846 support dynamic loading on many different platforms, was cleaned up
847 and reorganised by Greg Stein. \file{importdl.c} is now quite small,
848 and platform-specific code has been moved into a bunch of
849 \file{Python/dynload_*.c} files. Another cleanup: there were also a
850 number of \file{my*.h} files in the Include/ directory that held
851 various portability hacks; they've been merged into a single file,
852 \file{Include/pyport.h}.
854 Vladimir Marangozov's long-awaited malloc restructuring was completed,
855 to make it easy to have the Python interpreter use a custom allocator
856 instead of C's standard \function{malloc()}. For documentation, read
857 the comments in \file{Include/pymem.h} and
858 \file{Include/objimpl.h}. For the lengthy discussions during which
859 the interface was hammered out, see the Web archives of the 'patches'
860 and 'python-dev' lists at python.org.
862 Recent versions of the GUSI development environment for MacOS support
863 POSIX threads. Therefore, Python's POSIX threading support now works
864 on the Macintosh. Threading support using the user-space GNU \texttt{pth}
865 library was also contributed.
867 Threading support on Windows was enhanced, too. Windows supports
868 thread locks that use kernel objects only in case of contention; in
869 the common case when there's no contention, they use simpler functions
870 which are an order of magnitude faster. A threaded version of Python
871 1.5.2 on NT is twice as slow as an unthreaded version; with the 2.0
872 changes, the difference is only 10\%. These improvements were
873 contributed by Yakov Markovitch.
875 Python 2.0's source now uses only ANSI C prototypes, so compiling Python now
876 requires an ANSI C compiler, and can no longer be done using a compiler that
877 only supports K\&R C.
879 Previously the Python virtual machine used 16-bit numbers in its
880 bytecode, limiting the size of source files. In particular, this
881 affected the maximum size of literal lists and dictionaries in Python
882 source; occasionally people who are generating Python code would run
883 into this limit. A patch by Charles G. Waldman raises the limit from
884 \verb|2^16| to \verb|2^{32}|.
886 Three new convenience functions intended for adding constants to a
887 module's dictionary at module initialization time were added:
888 \function{PyModule_AddObject()}, \function{PyModule_AddIntConstant()},
889 and \function{PyModule_AddStringConstant()}. Each of these functions
890 takes a module object, a null-terminated C string containing the name
891 to be added, and a third argument for the value to be assigned to the
892 name. This third argument is, respectively, a Python object, a C
893 long, or a C string.
895 A wrapper API was added for Unix-style signal handlers.
896 \function{PyOS_getsig()} gets a signal handler and
897 \function{PyOS_setsig()} will set a new handler.
899 % ======================================================================
900 \section{Distutils: Making Modules Easy to Install}
902 Before Python 2.0, installing modules was a tedious affair -- there
903 was no way to figure out automatically where Python is installed, or
904 what compiler options to use for extension modules. Software authors
905 had to go through an arduous ritual of editing Makefiles and
906 configuration files, which only really work on Unix and leave Windows
907 and MacOS unsupported. Python users faced wildly differing
908 installation instructions which varied between different extension
909 packages, which made adminstering a Python installation something of a
910 chore.
912 The SIG for distribution utilities, shepherded by Greg Ward, has
913 created the Distutils, a system to make package installation much
914 easier. They form the \module{distutils} package, a new part of
915 Python's standard library. In the best case, installing a Python
916 module from source will require the same steps: first you simply mean
917 unpack the tarball or zip archive, and the run ``\code{python setup.py
918 install}''. The platform will be automatically detected, the compiler
919 will be recognized, C extension modules will be compiled, and the
920 distribution installed into the proper directory. Optional
921 command-line arguments provide more control over the installation
922 process, the distutils package offers many places to override defaults
923 -- separating the build from the install, building or installing in
924 non-default directories, and more.
926 In order to use the Distutils, you need to write a \file{setup.py}
927 script. For the simple case, when the software contains only .py
928 files, a minimal \file{setup.py} can be just a few lines long:
930 \begin{verbatim}
931 from distutils.core import setup
932 setup (name = "foo", version = "1.0",
933 py_modules = ["module1", "module2"])
934 \end{verbatim}
936 The \file{setup.py} file isn't much more complicated if the software
937 consists of a few packages:
939 \begin{verbatim}
940 from distutils.core import setup
941 setup (name = "foo", version = "1.0",
942 packages = ["package", "package.subpackage"])
943 \end{verbatim}
945 A C extension can be the most complicated case; here's an example taken from
946 the PyXML package:
949 \begin{verbatim}
950 from distutils.core import setup, Extension
952 expat_extension = Extension('xml.parsers.pyexpat',
953 define_macros = [('XML_NS', None)],
954 include_dirs = [ 'extensions/expat/xmltok',
955 'extensions/expat/xmlparse' ],
956 sources = [ 'extensions/pyexpat.c',
957 'extensions/expat/xmltok/xmltok.c',
958 'extensions/expat/xmltok/xmlrole.c',
959 ]
960 )
961 setup (name = "PyXML", version = "0.5.4",
962 ext_modules =[ expat_extension ] )
963 \end{verbatim}
965 The Distutils can also take care of creating source and binary
966 distributions. The ``sdist'' command, run by ``\code{python setup.py
967 sdist}', builds a source distribution such as \file{foo-1.0.tar.gz}.
968 Adding new commands isn't difficult, ``bdist_rpm'' and
969 ``bdist_wininst'' commands have already been contributed to create an
970 RPM distribution and a Windows installer for the software,
971 respectively. Commands to create other distribution formats such as
972 Debian packages and Solaris \file{.pkg} files are in various stages of
973 development.
975 All this is documented in a new manual, \textit{Distributing Python
976 Modules}, that joins the basic set of Python documentation.
978 % ======================================================================
979 \section{XML Modules}
981 Python 1.5.2 included a simple XML parser in the form of the
982 \module{xmllib} module, contributed by Sjoerd Mullender. Since
983 1.5.2's release, two different interfaces for processing XML have
984 become common: SAX2 (version 2 of the Simple API for XML) provides an
985 event-driven interface with some similarities to \module{xmllib}, and
986 the DOM (Document Object Model) provides a tree-based interface,
987 transforming an XML document into a tree of nodes that can be
988 traversed and modified. Python 2.0 includes a SAX2 interface and a
989 stripped-down DOM interface as part of the \module{xml} package.
990 Here we will give a brief overview of these new interfaces; consult
991 the Python documentation or the source code for complete details.
992 The Python XML SIG is also working on improved documentation.
994 \subsection{SAX2 Support}
996 SAX defines an event-driven interface for parsing XML. To use SAX,
997 you must write a SAX handler class. Handler classes inherit from
998 various classes provided by SAX, and override various methods that
999 will then be called by the XML parser. For example, the
1000 \method{startElement} and \method{endElement} methods are called for
1001 every starting and end tag encountered by the parser, the
1002 \method{characters()} method is called for every chunk of character
1003 data, and so forth.
1005 The advantage of the event-driven approach is that that the whole
1006 document doesn't have to be resident in memory at any one time, which
1007 matters if you are processing really huge documents. However, writing
1008 the SAX handler class can get very complicated if you're trying to
1009 modify the document structure in some elaborate way.
1011 For example, this little example program defines a handler that prints
1012 a message for every starting and ending tag, and then parses the file
1013 \file{hamlet.xml} using it:
1015 \begin{verbatim}
1016 from xml import sax
1018 class SimpleHandler(sax.ContentHandler):
1019 def startElement(self, name, attrs):
1020 print 'Start of element:', name, attrs.keys()
1022 def endElement(self, name):
1023 print 'End of element:', name
1025 # Create a parser object
1026 parser = sax.make_parser()
1028 # Tell it what handler to use
1029 handler = SimpleHandler()
1030 parser.setContentHandler( handler )
1032 # Parse a file!
1033 parser.parse( 'hamlet.xml' )
1034 \end{verbatim}
1036 For more information, consult the Python documentation, or the XML
1037 HOWTO at \url{http://www.python.org/doc/howto/xml/}.
1039 \subsection{DOM Support}
1041 The Document Object Model is a tree-based representation for an XML
1042 document. A top-level \class{Document} instance is the root of the
1043 tree, and has a single child which is the top-level \class{Element}
1044 instance. This \class{Element} has children nodes representing
1045 character data and any sub-elements, which may have further children
1046 of their own, and so forth. Using the DOM you can traverse the
1047 resulting tree any way you like, access element and attribute values,
1048 insert and delete nodes, and convert the tree back into XML.
1050 The DOM is useful for modifying XML documents, because you can create
1051 a DOM tree, modify it by adding new nodes or rearranging subtrees, and
1052 then produce a new XML document as output. You can also construct a
1053 DOM tree manually and convert it to XML, which can be a more flexible
1054 way of producing XML output than simply writing
1055 \code{<tag1>}...\code{</tag1>} to a file.
1057 The DOM implementation included with Python lives in the
1058 \module{xml.dom.minidom} module. It's a lightweight implementation of
1059 the Level 1 DOM with support for XML namespaces. The
1060 \function{parse()} and \function{parseString()} convenience
1061 functions are provided for generating a DOM tree:
1063 \begin{verbatim}
1064 from xml.dom import minidom
1065 doc = minidom.parse('hamlet.xml')
1066 \end{verbatim}
1068 \code{doc} is a \class{Document} instance. \class{Document}, like all
1069 the other DOM classes such as \class{Element} and \class{Text}, is a
1070 subclass of the \class{Node} base class. All the nodes in a DOM tree
1071 therefore support certain common methods, such as \method{toxml()}
1072 which returns a string containing the XML representation of the node
1073 and its children. Each class also has special methods of its own; for
1074 example, \class{Element} and \class{Document} instances have a method
1075 to find all child elements with a given tag name. Continuing from the
1076 previous 2-line example:
1078 \begin{verbatim}
1079 perslist = doc.getElementsByTagName( 'PERSONA' )
1080 print perslist[0].toxml()
1081 print perslist[1].toxml()
1082 \end{verbatim}
1084 For the \textit{Hamlet} XML file, the above few lines output:
1086 \begin{verbatim}
1087 <PERSONA>CLAUDIUS, king of Denmark. </PERSONA>
1088 <PERSONA>HAMLET, son to the late, and nephew to the present king.</PERSONA>
1089 \end{verbatim}
1091 The root element of the document is available as
1092 \code{doc.documentElement}, and its children can be easily modified
1093 by deleting, adding, or removing nodes:
1095 \begin{verbatim}
1096 root = doc.documentElement
1098 # Remove the first child
1099 root.removeChild( root.childNodes[0] )
1101 # Move the new first child to the end
1102 root.appendChild( root.childNodes[0] )
1104 # Insert the new first child (originally,
1105 # the third child) before the 20th child.
1106 root.insertBefore( root.childNodes[0], root.childNodes[20] )
1107 \end{verbatim}
1109 Again, I will refer you to the Python documentation for a complete
1110 listing of the different \class{Node} classes and their various methods.
1112 \subsection{Relationship to PyXML}
1114 The XML Special Interest Group has been working on XML-related Python
1115 code for a while. Its code distribution, called PyXML, is available
1116 from the SIG's Web pages at \url{http://www.python.org/sigs/xml-sig/}.
1117 The PyXML distribution also used the package name \samp{xml}. If
1118 you've written programs that used PyXML, you're probably wondering
1119 about its compatibility with the 2.0 \module{xml} package.
1121 The answer is that Python 2.0's \module{xml} package isn't compatible
1122 with PyXML, but can be made compatible by installing a recent version
1123 PyXML. Many applications can get by with the XML support that is
1124 included with Python 2.0, but more complicated applications will
1125 require that the full PyXML package will be installed. When
1126 installed, PyXML versions 0.6.0 or greater will replace the
1127 \module{xml} package shipped with Python, and will be a strict
1128 superset of the standard package, adding a bunch of additional
1129 features. Some of the additional features in PyXML include:
1131 \begin{itemize}
1132 \item 4DOM, a full DOM implementation
1133 from FourThought, Inc.
1134 \item The xmlproc validating parser, written by Lars Marius Garshol.
1135 \item The \module{sgmlop} parser accelerator module, written by Fredrik Lundh.
1136 \end{itemize}
1138 % ======================================================================
1139 \section{Module changes}
1141 Lots of improvements and bugfixes were made to Python's extensive
1142 standard library; some of the affected modules include
1143 \module{readline}, \module{ConfigParser}, \module{cgi},
1144 \module{calendar}, \module{posix}, \module{readline}, \module{xmllib},
1145 \module{aifc}, \module{chunk, wave}, \module{random}, \module{shelve},
1146 and \module{nntplib}. Consult the CVS logs for the exact
1147 patch-by-patch details.
1149 Brian Gallew contributed OpenSSL support for the \module{socket}
1150 module. OpenSSL is an implementation of the Secure Socket Layer,
1151 which encrypts the data being sent over a socket. When compiling
1152 Python, you can edit \file{Modules/Setup} to include SSL support,
1153 which adds an additional function to the \module{socket} module:
1154 \function{socket.ssl(\var{socket}, \var{keyfile}, \var{certfile})},
1155 which takes a socket object and returns an SSL socket. The
1156 \module{httplib} and \module{urllib} modules were also changed to
1157 support ``https://'' URLs, though no one has implemented FTP or SMTP
1158 over SSL.
1160 The \module{httplib} module has been rewritten by Greg Stein to
1161 support HTTP/1.1. Backward compatibility with the 1.5 version of
1162 \module{httplib} is provided, though using HTTP/1.1 features such as
1163 pipelining will require rewriting code to use a different set of
1164 interfaces.
1166 The \module{Tkinter} module now supports Tcl/Tk version 8.1, 8.2, or
1167 8.3, and support for the older 7.x versions has been dropped. The
1168 Tkinter module now supports displaying Unicode strings in Tk widgets.
1169 Also, Fredrik Lundh contributed an optimization which makes operations
1170 like \code{create_line} and \code{create_polygon} much faster,
1171 especially when using lots of coordinates.
1173 The \module{curses} module has been greatly extended, starting from
1174 Oliver Andrich's enhanced version, to provide many additional
1175 functions from ncurses and SYSV curses, such as colour, alternative
1176 character set support, pads, and mouse support. This means the module
1177 is no longer compatible with operating systems that only have BSD
1178 curses, but there don't seem to be any currently maintained OSes that
1179 fall into this category.
1181 As mentioned in the earlier discussion of 2.0's Unicode support, the
1182 underlying implementation of the regular expressions provided by the
1183 \module{re} module has been changed. SRE, a new regular expression
1184 engine written by Fredrik Lundh and partially funded by Hewlett
1185 Packard, supports matching against both 8-bit strings and Unicode
1186 strings.
1188 % ======================================================================
1189 \section{New modules}
1191 A number of new modules were added. We'll simply list them with brief
1192 descriptions; consult the 2.0 documentation for the details of a
1193 particular module.
1195 \begin{itemize}
1197 \item{\module{atexit}}:
1198 For registering functions to be called before the Python interpreter exits.
1199 Code that currently sets
1200 \code{sys.exitfunc} directly should be changed to
1201 use the \module{atexit} module instead, importing \module{atexit}
1202 and calling \function{atexit.register()} with
1203 the function to be called on exit.
1204 (Contributed by Skip Montanaro.)
1206 \item{\module{codecs}, \module{encodings}, \module{unicodedata}:} Added as part of the new Unicode support.
1208 \item{\module{filecmp}:} Supersedes the old \module{cmp}, \module{cmpcache} and
1209 \module{dircmp} modules, which have now become deprecated.
1210 (Contributed by Gordon MacMillan and Moshe Zadka.)
1212 \item{\module{gettext}:} This module provides internationalization
1213 (I18N) and localization (L10N) support for Python programs by
1214 providing an interface to the GNU gettext message catalog library.
1215 (Integrated by Barry Warsaw, from separate contributions by Martin von
1216 Loewis, Peter Funk, and James Henstridge.)
1218 \item{\module{linuxaudiodev}:} Support for the \file{/dev/audio}
1219 device on Linux, a twin to the existing \module{sunaudiodev} module.
1220 (Contributed by Peter Bosch, with fixes by Jeremy Hylton.)
1222 \item{\module{mmap}:} An interface to memory-mapped files on both
1223 Windows and Unix. A file's contents can be mapped directly into
1224 memory, at which point it behaves like a mutable string, so its
1225 contents can be read and modified. They can even be passed to
1226 functions that expect ordinary strings, such as the \module{re}
1227 module. (Contributed by Sam Rushing, with some extensions by
1228 A.M. Kuchling.)
1230 \item{\module{pyexpat}:} An interface to the Expat XML parser.
1231 (Contributed by Paul Prescod.)
1233 \item{\module{robotparser}:} Parse a \file{robots.txt} file, which is
1234 used for writing Web spiders that politely avoid certain areas of a
1235 Web site. The parser accepts the contents of a \file{robots.txt} file,
1236 builds a set of rules from it, and can then answer questions about
1237 the fetchability of a given URL. (Contributed by Skip Montanaro.)
1239 \item{\module{tabnanny}:} A module/script to
1240 check Python source code for ambiguous indentation.
1241 (Contributed by Tim Peters.)
1243 \item{\module{UserString}:} A base class useful for deriving objects that behave like strings.
1245 \item{\module{webbrowser}:} A module that provides a platform independent
1246 way to launch a web browser on a specific URL. For each platform, various
1247 browsers are tried in a specific order. The user can alter which browser
1248 is launched by setting the \var{BROWSER} environment variable.
1249 (Originally inspired by Eric S. Raymond's patch to \module{urllib}
1250 which added similar functionality, but
1251 the final module comes from code originally
1252 implemented by Fred Drake as \file{Tools/idle/BrowserControl.py},
1253 and adapted for the standard library by Fred.)
1255 \item{\module{_winreg}:} An interface to the
1256 Windows registry. \module{_winreg} is an adaptation of functions that
1257 have been part of PythonWin since 1995, but has now been added to the core
1258 distribution, and enhanced to support Unicode.
1259 \module{_winreg} was written by Bill Tutt and Mark Hammond.
1261 \item{\module{zipfile}:} A module for reading and writing ZIP-format
1262 archives. These are archives produced by \program{PKZIP} on
1263 DOS/Windows or \program{zip} on Unix, not to be confused with
1264 \program{gzip}-format files (which are supported by the \module{gzip}
1265 module)
1266 (Contributed by James C. Ahlstrom.)
1268 \item{\module{imputil}:} A module that provides a simpler way for
1269 writing customised import hooks, in comparison to the existing
1270 \module{ihooks} module. (Implemented by Greg Stein, with much
1271 discussion on python-dev along the way.)
1273 \end{itemize}
1275 % ======================================================================
1276 \section{IDLE Improvements}
1278 IDLE is the official Python cross-platform IDE, written using Tkinter.
1279 Python 2.0 includes IDLE 0.6, which adds a number of new features and
1280 improvements. A partial list:
1282 \begin{itemize}
1283 \item UI improvements and optimizations,
1284 especially in the area of syntax highlighting and auto-indentation.
1286 \item The class browser now shows more information, such as the top
1287 level functions in a module.
1289 \item Tab width is now a user settable option. When opening an existing Python
1290 file, IDLE automatically detects the indentation conventions, and adapts.
1292 \item There is now support for calling browsers on various platforms,
1293 used to open the Python documentation in a browser.
1295 \item IDLE now has a command line, which is largely similar to
1296 the vanilla Python interpreter.
1298 \item Call tips were added in many places.
1300 \item IDLE can now be installed as a package.
1302 \item In the editor window, there is now a line/column bar at the bottom.
1304 \item Three new keystroke commands: Check module (Alt-F5), Import
1305 module (F5) and Run script (Ctrl-F5).
1307 \end{itemize}
1309 % ======================================================================
1310 \section{Deleted and Deprecated Modules}
1312 A few modules have been dropped because they're obsolete, or because
1313 there are now better ways to do the same thing. The \module{stdwin}
1314 module is gone; it was for a platform-independent windowing toolkit
1315 that's no longer developed.
1317 A number of modules have been moved to the
1318 \file{lib-old} subdirectory:
1319 \module{cmp}, \module{cmpcache}, \module{dircmp}, \module{dump},
1320 \module{find}, \module{grep}, \module{packmail},
1321 \module{poly}, \module{util}, \module{whatsound}, \module{zmod}.
1322 If you have code which relies on a module that's been moved to
1323 \file{lib-old}, you can simply add that directory to \code{sys.path}
1324 to get them back, but you're encouraged to update any code that uses
1325 these modules.
1327 \section{Acknowledgements}
1329 The authors would like to thank the following people for offering
1330 suggestions on various drafts of this article: David Bolen, Mark Hammond, Gregg Hauser,
1331 Jeremy Hylton, Fredrik Lundh, Detlef Lannert, Aahz Maruch, Skip
1332 Montanaro, Vladimir Marangozov, Guido van Rossum, Neil Schemenauer,
1333 and Russ Schmidt.
1335 \end{document}