| blob | 1cf456f76fbe8b62bea64f1c561a352ceb2463ac |
1 Project: core implementation
2 ****************************
4 Still to do
5 -----------
7 Check for conflicts between base classes. I fear that the rules used
8 to decide whether multiple bases have conflicting instance variables
9 aren't strict enough. I think that sometimes two different classes
10 adding __dict__ may be incompatible after all.
12 Check for order conflicts. Suppose there are two base classes X and
13 Y. Suppose class B derives from X and Y, and class C from Y and X (in
14 that order). Now suppose class D derives from B and C. In which
15 order should the base classes X and Y be searched? This is an order
16 conflict, and should be disallowed; currently the test for this is not
17 implemented.
19 Allow assignment to __bases__ and __dict__?
21 Make __dynamic__ the default.
23 Add __del__ handlers.
25 Add __coerce__?
27 Support pickling (via __reduce__)
29 Support mixed multiple inheritance from classic and new-style classes?
31 Done (mostly)
32 -------------
34 Fix comparisons. There's some nasty stuff here: when two types are
35 not the same, and they're not instances, the fallback code doesn't
36 account for the possibility that they might be subtypes of a common
37 base type that defines a comparison. *** I believe this is now done,
38 but it's a bit of a mess. ***
40 Allow __class__ assignment. *** done ***
42 Change __getattr__ to be more like classic __getattr__, and introduce
43 a new name for new-style __getattr__. *** Done. The new-style method
44 is called __getattribute__. ***
46 Make inspect and pydoc do the right thing for new-style classes. ***
47 done ***
49 Do binary operators properly. nb_add should try to call self.__add__
50 and other.__radd__. I think I'll exclude base types that define any
51 binary operator without setting the CHECKTYPES flag. *** This is
52 done, AFAICT. Even supports __truediv__ and __floordiv__. ***
54 Fix subtype_dealloc(). This currently searches through the list of
55 base types until it finds a type whose tp_dealloc is not
56 subtype_dealloc. I think this is not safe. I think the alloc/dealloc
57 policy needs to be rethought. *** There's an idea here that I haven't
58 worked out yet: just as object creation now has separate API's tp_new,
59 tp_alloc, and tp_init, destruction has tp_dealloc and tp_free. (Maybe
60 tp_fini should be added to correspond to tp_init?) Something
61 could/should be done with this. ***
63 Clean up isinstance(), issubclass() and their C equivalents. There
64 are a bunch of different APIs here and not all of them do the right
65 thing yet. There should be fewer APIs and their implementation should
66 be simpler. The old "abstract subclass" test should probably
67 disappear (if we want to root out ExtensionClass). *** I think I've
68 done 90% of this by creating PyType_IsSubtype() and using it
69 appropriately. For now, the old "abstract subclass" test is still
70 there, and there may be some places where PyObject_IsSubclass() is
71 called where PyType_IsSubtype() would be more appropriate. ***
73 Clean up the GC interface. Currently, tp_basicsize includes the GC
74 head size iff tp_flags includes the GC flag bit. This makes object
75 size math a pain (e.g. to see if two object types have the same
76 instance size, you can't just compare the tp_basicsize fields -- you
77 have to conditionally subtract the GC head size). Neil has a patch
78 that improves the API in this area, but it's backwards incompatible.
79 (http://sf.net/tracker/?func=detail&aid=421893&group_id=5470&atid=305470)
80 I think I know of a way to fix the incompatibility (by switching to a
81 different flag bit). *** Tim proposed a better idea: macros to access
82 tp_basicsize while hiding the nastiness. This is done now, so I think
83 the rest of this task needn't be done. *** *** Neil checked in a
84 much improved version of his idea, and it's all squared away. ***
86 Make the __dict__ of types declared with Python class statements
87 writable -- only statically declared types must have an immutable
88 dict, because they're shared between interpreter instances. Possibly
89 trap writes to the __dict__ to update the corresponding tp_<slot> if
90 an __<slot>__ name is affected. *** Done as part of the next task. ***
92 It should be an option (maybe a different metaclass, maybe a flag) to
93 *not* merge __dict__ with all the bases, but instead search the
94 __dict__ (or __introduced__?) of all bases in __mro__ order. (This is
95 needed anyway to unify classes completely.) *** Partly done.
96 Inheritance of slots from bases is still icky: (1) MRO is not always
97 respected when inheriting slots; (2) dynamic classes can't add slot
98 implementations in Python after creation (e.g., setting C.__hash__
99 doesn't set the tp_hash slot). ***
101 Universal base class (object). How can we make the object class
102 subclassable and define simple default methods for everything without
103 having these inherited by built-in types that don't want these
104 defaults? *** Done, really. ***
106 Add error checking to the MRO calculation. *** Done. ***
108 Make __new__ overridable through a Python class method (!). Make more
109 of the sub-algorithms of type construction available as methods. ***
110 After I implemented class methods, I found that in order to be able
111 to make an upcall to Base.__new__() and have it create an instance of
112 your class (rather than a Base instance), you can't use class methods
113 -- you must use static methods. So I've implemented those too. I've
114 hooked up __new__ in the right places, so the first part of this is
115 now done. I've also exported the MRO calculation and made it
116 overridable, as metamethod mro(). I believe that closes this topic
117 for now. I expect that some warts will only be really debugged when
118 we try to use this for some, eh, interesting types such as tuples. ***
120 There was a sequel to the __new__ story (see checkins). There
121 still is a problem: object.__new__ now no longer exists, because
122 it was inherited by certain extension types that could break. But
123 now when I write
125 class C(object):
126 def __new__(cls, *args):
127 "How do I call the default __new__ implementation???"
129 This was resolved nicely by putting object.__new__ back but not
130 inheriting __new__ from object when the subtype is a built-in or
131 extension type.
133 More -- I'm sure new issues will crop up as we go.
136 Project: loose ends and follow-through
137 **************************************
139 Still to do
140 -----------
142 Exceptions should be types. This changes the rules, since now almost
143 anything can be raised (as maybe it should). Or should we strive for
144 enforcement of the convention that all exceptions should be derived
145 from Exception? String exceptions will be another hassle, to be
146 deprecated and eventually ruled out.
148 Standardize a module containing names for all built-in types, and
149 standardize on names. E.g. should the official name of the string
150 type be 'str', 'string', or 'StringType'?
152 Create a hierarchy of types, so that e.g. int and long are both
153 subtypes of an abstract base type integer, which is itself a subtype
154 of number, etc. A lot of thinking can go into this!
156 *** NEW TASK??? ***
157 Implement "signature" objects. These are alluded to in PEP 252 but
158 not yet specified. Supposedly they provide an easily usable API to
159 find out about function/method arguments. Building these for Python
160 functions is simple. Building these for built-in functions will
161 require a change to the PyMethodDef structure, so that a type can
162 provide signature information for its C methods. (This would also
163 help in supporting keyword arguments for C methods with less work than
164 PyArg_ParseTupleAndKeywords() currently requires.) But should we do
165 this? It's additional work and not required for any of the other
166 parts.
168 Done (mostly)
169 -------------
171 Make more (most?) built-in types act as their own factory functions.
172 *** Done for all reasonable built-in types. ***
174 Make more (most?) built-in types subtypable -- with or without
175 overridable allocation. *** This includes descriptors! It should be
176 possible to write descriptors in Python, so metaclasses can do clever
177 things with them. *** *** Done for most reasonable built-in types,
178 except for descriptors ***
181 Project: making classes use the new machinery
182 *********************************************
184 Tasks:
186 Try to get rid of all code in classobject.c by deferring to the new
187 mechanisms. How far can we get without breaking backwards
188 compatibility? This is underspecified because I haven't thought much
189 about it yet. Can we lose the use of PyInstance_Check() everywhere?
190 I would hope so! *** I'm dropping this goal for now -- classic
191 classes will be 99% unchanged. ***
194 Project: backwards compatibility
195 ********************************
197 Tasks:
199 Make sure all code checks the proper tp_flags bit before accessing
200 type object fields.
202 Identify areas of incompatibility with Python 2.1. Design solutions.
203 Implement and test.
205 Some specific areas: a fair amount of code probably depends on
206 specific types having __members__ and/or __methods__ attributes.
207 These are currently not present (conformant to PEP 252, which proposes
208 to drop them) but we may have to add them back. This can be done in a
209 generic way with not too much effort. Tim adds: Perhaps that dir(object)
210 rarely returns anything but [] now is a consequence of this. I'm very
211 used to doing, e.g., dir([]) or dir("") in an interactive shell to jog my
212 memory; also one of the reasons test_generators failed.
214 Another area: going all the way with classes and instances means that
215 type(x) == types.InstanceType won't work any more to detect instances.
216 Should there be a mode where this still works? Maybe this should be
217 the default mode, with a warning, and an explicit way to get the new
218 way to work? (Instead of a __future__ statement, I'm thinking of a
219 module global __metaclass__ which would provide the default metaclass
220 for baseless class statements.)
223 Project: testing
224 ****************
226 Tasks:
228 Identify new functionality that needs testing. Conceive unit tests
229 for all new functionality. Conceive stress tests for critical
230 features. Run the tests. Fix bugs. Repeat until satisfied.
232 Note: this may interact with the branch integration task.
235 Project: integration with main branch *** This is done - tim ***
236 *************************************
238 Tasks:
240 Merge changes in the HEAD branch into the descr-branch. Then merge
241 the descr-branch back into the HEAD branch.
243 The longer we wait, the more effort this will be -- the descr-branch
244 forked off quite a long time ago, and there are changes everywhere in
245 the HEAD branch (e.g. the dict object has been radically rewritten).
247 On the other hand, if we do this too early, we'll have to do it again
248 later.
250 Note from Tim: We should never again wait until literally 100s of files
251 are out of synch. I don't care how often I need to do this, provided only
252 that it's a tractable task each time. Once per week sounds like a good
253 idea. As is, even the trunk change to rangeobject.c created more than its
254 proper share of merge headaches, because it confused all the other reasons
255 include file merges were getting conflicts (the more changes there are, the
256 worse diff does; indeed, I came up with the ndiff algorithm in the 80s
257 precisely because the source-control diff program Cray used at the time
258 produced minimal but *senseless* diffs, thus creating artificial conflicts;
259 paying unbounded attention to context does a much better job of putting
260 changes where they make semantic sense too; but we're stuck with Unix diff
261 here, and it isn't robust in this sense; if we don't keep its job simple,
262 it will make my job hell).
264 Done:
265 To undo or rename before final merge: Modules/spam.c has worked its
266 way into the branch Unix and Windows builds (pythoncore.dsp and
267 PC/config.c); also imported by test_descr.py. How about renaming to
268 xxsubtype.c (whatever) now? *** this is done - tim ***
271 Project: performance tuning
272 ***************************
274 Tasks:
276 Pick or create a general performance benchmark for Python. Benchmark
277 the new system vs. the old system. Profile the new system. Improve
278 hotspots. Repeat until satisfied.
280 Note: this may interact with the branch integration task.
283 Project: documentation
284 **********************
286 Tasks:
288 Update PEP 252 (descriptors). Describe more of the prototype
289 implementation
291 Update PEP 253 (subtyping). Complicated architectural wrangling with
292 metaclasses. There is an interaction between implementation and
293 description.
295 Write PEP 254 (unification of classes). This should discuss what
296 changes for ordinary classes, and how we can make it more b/w
297 compatible.
299 Other documentation. There needs to be user documentation,
300 eventually.
303 Project: community interaction
304 ******************************
306 Tasks:
308 Once the PEPs are written, solicit community feedback, and formulate
309 responses to the feedback. Give the community enough time to think
310 over this complicated proposal. Provide the community with a
311 prototype implementation to test. Try to do this *before* casting
312 everything in stone!