Lib/ni1.py

   1 """New import scheme with package support.
   2
   3 Quick Reference
   4 ---------------
   5
   6 - To enable package support, execute "import ni" before importing any
   7   packages.  Importing this module automatically installs the relevant
   8   import hooks.
   9
  10 - To create a package named spam containing sub-modules ham, bacon and
  11   eggs, create a directory spam somewhere on Python's module search
  12   path (i.e. spam's parent directory must be one of the directories in
  13   sys.path or $PYTHONPATH); then create files ham.py, bacon.py and
  14   eggs.py inside spam.
  15
  16 - To import module ham from package spam and use function hamneggs()
  17   from that module, you can either do
  18
  19     import spam.ham             # *not* "import spam" !!!
  20     spam.ham.hamneggs()
  21
  22   or
  23
  24     from spam import ham
  25     ham.hamneggs()
  26
  27   or
  28
  29     from spam.ham import hamneggs
  30     hamneggs()
  31
  32 - Importing just "spam" does not do what you expect: it creates an
  33   empty package named spam if one does not already exist, but it does
  34   not import spam's submodules.  The only submodule that is guaranteed
  35   to be imported is spam.__init__, if it exists.  Note that
  36   spam.__init__ is a submodule of package spam.  It can reference to
  37   spam's namespace via the '__.' prefix, for instance
  38
  39     __.spam_inited = 1          # Set a package-level variable
  40
  41
  42
  43 Theory of Operation
  44 -------------------
  45
  46 A Package is a module that can contain other modules.  Packages can be
  47 nested.  Package introduce dotted names for modules, like P.Q.M, which
  48 could correspond to a file P/Q/M.py found somewhere on sys.path.  It
  49 is possible to import a package itself, though this makes little sense
  50 unless the package contains a module called __init__.
  51
  52 A package has two variables that control the namespace used for
  53 packages and modules, both initialized to sensible defaults the first
  54 time the package is referenced.
  55
  56 (1) A package's *module search path*, contained in the per-package
  57 variable __path__, defines a list of *directories* where submodules or
  58 subpackages of the package are searched.  It is initialized to the
  59 directory containing the package.  Setting this variable to None makes
  60 the module search path default to sys.path (this is not quite the same
  61 as setting it to sys.path, since the latter won't track later
  62 assignments to sys.path).
  63
  64 (2) A package's *import domain*, contained in the per-package variable
  65 __domain__, defines a list of *packages* that are searched (using
  66 their respective module search paths) to satisfy imports.  It is
  67 initialized to the list cosisting of the package itself, its parent
  68 package, its parent's parent, and so on, ending with the root package
  69 (the nameless package containing all top-level packages and modules,
  70 whose module search path is None, implying sys.path).
  71
  72 The default domain implements a search algorithm called "expanding
  73 search".  An alternative search algorithm called "explicit search"
  74 fixes the import search path to contain only the root package,
  75 requiring the modules in the package to name all imported modules by
  76 their full name.  The convention of using '__' to refer to the current
  77 package (both as a per-module variable and in module names) can be
  78 used by packages using explicit search to refer to modules in the same
  79 package; this combination is known as "explicit-relative search".
  80
  81 The PackageImporter and PackageLoader classes together implement the
  82 following policies:
  83
  84 - There is a root package, whose name is ''.  It cannot be imported
  85   directly but may be referenced, e.g. by using '__' from a top-level
  86   module.
  87
  88 - In each module or package, the variable '__' contains a reference to
  89   the parent package; in the root package, '__' points to itself.
  90
  91 - In the name for imported modules (e.g. M in "import M" or "from M
  92   import ..."), a leading '__' refers to the current package (i.e.
  93   the package containing the current module); leading '__.__' and so
  94   on refer to the current package's parent, and so on.  The use of
  95   '__' elsewhere in the module name is not supported.
  96
  97 - Modules are searched using the "expanding search" algorithm by
  98   virtue of the default value for __domain__.
  99
 100 - If A.B.C is imported, A is searched using __domain__; then
 101   subpackage B is searched in A using its __path__, and so on.
 102
 103 - Built-in modules have priority: even if a file sys.py exists in a
 104   package, "import sys" imports the built-in sys module.
 105
 106 - The same holds for frozen modules, for better or for worse.
 107
 108 - Submodules and subpackages are not automatically loaded when their
 109   parent packages is loaded.
 110
 111 - The construct "from package import *" is illegal.  (It can still be
 112   used to import names from a module.)
 113
 114 - When "from package import module1, module2, ..." is used, those
 115     modules are explicitly loaded.
 116
 117 - When a package is loaded, if it has a submodule __init__, that
 118   module is loaded.  This is the place where required submodules can
 119   be loaded, the __path__ variable extended, etc.  The __init__ module
 120   is loaded even if the package was loaded only in order to create a
 121   stub for a sub-package: if "import P.Q.R" is the first reference to
 122   P, and P has a submodule __init__, P.__init__ is loaded before P.Q
 123   is even searched.
 124
 125 Caveats:
 126
 127 - It is possible to import a package that has no __init__ submodule;
 128   this is not particularly useful but there may be useful applications
 129   for it (e.g. to manipulate its search paths from the outside!).
 130
 131 - There are no special provisions for os.chdir().  If you plan to use
 132   os.chdir() before you have imported all your modules, it is better
 133   not to have relative pathnames in sys.path.  (This could actually be
 134   fixed by changing the implementation of path_join() in the hook to
 135   absolutize paths.)
 136
 137 - Packages and modules are introduced in sys.modules as soon as their
 138   loading is started.  When the loading is terminated by an exception,
 139   the sys.modules entries remain around.
 140
 141 - There are no special measures to support mutually recursive modules,
 142   but it will work under the same conditions where it works in the
 143   flat module space system.
 144
 145 - Sometimes dummy entries (whose value is None) are entered in
 146   sys.modules, to indicate that a particular module does not exist --
 147   this is done to speed up the expanding search algorithm when a
 148   module residing at a higher level is repeatedly imported (Python
 149   promises that importing a previously imported module is cheap!)
 150
 151 - Although dynamically loaded extensions are allowed inside packages,
 152   the current implementation (hardcoded in the interpreter) of their
 153   initialization may cause problems if an extension invokes the
 154   interpreter during its initialization.
 155
 156 - reload() may find another version of the module only if it occurs on
 157   the package search path.  Thus, it keeps the connection to the
 158   package to which the module belongs, but may find a different file.
 159
 160 XXX Need to have an explicit name for '', e.g. '__root__'.
 161
 162 """
 163
 164
 165 import imp
 166 import string
 167 import sys
 168 import __builtin__
 169
 170 import ihooks
 171 from ihooks import ModuleLoader, ModuleImporter
 172
 173
 174 class PackageLoader(ModuleLoader):
 175
 176     """A subclass of ModuleLoader with package support.
 177
 178     find_module_in_dir() will succeed if there's a subdirectory with
 179     the given name; load_module() will create a stub for a package and
 180     load its __init__ module if it exists.
 181
 182     """
 183
 184     def find_module_in_dir(self, name, dir):
 185         if dir is not None:
 186             dirname = self.hooks.path_join(dir, name)
 187             if self.hooks.path_isdir(dirname):
 188                 return None, dirname, ('', '', 'PACKAGE')
 189         return ModuleLoader.find_module_in_dir(self, name, dir)
 190
 191     def load_module(self, name, stuff):
 192         file, filename, info = stuff
 193         suff, mode, type = info
 194         if type == 'PACKAGE':
 195             return self.load_package(name, stuff)
 196         if sys.modules.has_key(name):
 197             m = sys.modules[name]
 198         else:
 199             sys.modules[name] = m = imp.new_module(name)
 200         self.set_parent(m)
 201         if type == imp.C_EXTENSION and '.' in name:
 202             return self.load_dynamic(name, stuff)
 203         else:
 204             return ModuleLoader.load_module(self, name, stuff)
 205
 206     def load_dynamic(self, name, stuff):
 207         file, filename, (suff, mode, type) = stuff
 208         # Hack around restriction in imp.load_dynamic()
 209         i = string.rfind(name, '.')
 210         tail = name[i+1:]
 211         if sys.modules.has_key(tail):
 212             save = sys.modules[tail]
 213         else:
 214             save = None
 215         sys.modules[tail] = imp.new_module(name)
 216         try:
 217             m = imp.load_dynamic(tail, filename, file)
 218         finally:
 219             if save:
 220                 sys.modules[tail] = save
 221             else:
 222                 del sys.modules[tail]
 223         sys.modules[name] = m
 224         return m
 225
 226     def load_package(self, name, stuff):
 227         file, filename, info = stuff
 228         if sys.modules.has_key(name):
 229             package = sys.modules[name]
 230         else:
 231             sys.modules[name] = package = imp.new_module(name)
 232         package.__path__ = [filename]
 233         self.init_package(package)
 234         return package
 235
 236     def init_package(self, package):
 237         self.set_parent(package)
 238         self.set_domain(package)
 239         self.call_init_module(package)
 240
 241     def set_parent(self, m):
 242         name = m.__name__
 243         if '.' in name:
 244             name = name[:string.rfind(name, '.')]
 245         else:
 246             name = ''
 247         m.__ = sys.modules[name]
 248
 249     def set_domain(self, package):
 250         name = package.__name__
 251         package.__domain__ = domain = [name]
 252         while '.' in name:
 253             name = name[:string.rfind(name, '.')]
 254             domain.append(name)
 255         if name:
 256             domain.append('')
 257
 258     def call_init_module(self, package):
 259         stuff = self.find_module('__init__', package.__path__)
 260         if stuff:
 261             m = self.load_module(package.__name__ + '.__init__', stuff)
 262             package.__init__ = m
 263
 264
 265 class PackageImporter(ModuleImporter):
 266
 267     """Importer that understands packages and '__'."""
 268
 269     def __init__(self, loader = None, verbose = 0):
 270         ModuleImporter.__init__(self,
 271         loader or PackageLoader(None, verbose), verbose)
 272
 273     def import_module(self, name, globals={}, locals={}, fromlist=[]):
 274         if globals.has_key('__'):
 275             package = globals['__']
 276         else:
 277             # No calling context, assume in root package
 278             package = sys.modules['']
 279         if name[:3] in ('__.', '__'):
 280             p = package
 281             name = name[3:]
 282             while name[:3] in ('__.', '__'):
 283                 p = package.__
 284                 name = name[3:]
 285             if not name:
 286                 return self.finish(package, p, '', fromlist)
 287             if '.' in name:
 288                 i = string.find(name, '.')
 289                 name, tail = name[:i], name[i:]
 290             else:
 291                 tail = ''
 292             mname = p.__name__ and p.__name__+'.'+name or name
 293             m = self.get1(mname)
 294             return self.finish(package, m, tail, fromlist)
 295         if '.' in name:
 296             i = string.find(name, '.')
 297             name, tail = name[:i], name[i:]
 298         else:
 299             tail = ''
 300         for pname in package.__domain__:
 301             mname = pname and pname+'.'+name or name
 302             m = self.get0(mname)
 303             if m: break
 304         else:
 305             raise ImportError, "No such module %s" % name
 306         return self.finish(m, m, tail, fromlist)
 307
 308     def finish(self, module, m, tail, fromlist):
 309         # Got ....A; now get ....A.B.C.D
 310         yname = m.__name__
 311         if tail and sys.modules.has_key(yname + tail): # Fast path
 312             yname, tail = yname + tail, ''
 313             m = self.get1(yname)
 314         while tail:
 315             i = string.find(tail, '.', 1)
 316             if i > 0:
 317                 head, tail = tail[:i], tail[i:]
 318             else:
 319                 head, tail = tail, ''
 320             yname = yname + head
 321             m = self.get1(yname)
 322
 323         # Got ....A.B.C.D; now finalize things depending on fromlist
 324         if not fromlist:
 325             return module
 326         if '__' in fromlist:
 327             raise ImportError, "Can't import __ from anywhere"
 328         if not hasattr(m, '__path__'): return m
 329         if '*' in fromlist:
 330             raise ImportError, "Can't import * from a package"
 331         for f in fromlist:
 332             if hasattr(m, f): continue
 333             fname = yname + '.' + f
 334             self.get1(fname)
 335         return m
 336
 337     def get1(self, name):
 338         m = self.get(name)
 339         if not m:
 340             raise ImportError, "No module named %s" % name
 341         return m
 342
 343     def get0(self, name):
 344         m = self.get(name)
 345         if not m:
 346             sys.modules[name] = None
 347         return m
 348
 349     def get(self, name):
 350         # Internal routine to get or load a module when its parent exists
 351         if sys.modules.has_key(name):
 352             return sys.modules[name]
 353         if '.' in name:
 354             i = string.rfind(name, '.')
 355             head, tail = name[:i], name[i+1:]
 356         else:
 357             head, tail = '', name
 358         path = sys.modules[head].__path__
 359         stuff = self.loader.find_module(tail, path)
 360         if not stuff:
 361             return None
 362         sys.modules[name] = m = self.loader.load_module(name, stuff)
 363         if head:
 364             setattr(sys.modules[head], tail, m)
 365         return m
 366
 367     def reload(self, module):
 368         name = module.__name__
 369         if '.' in name:
 370             i = string.rfind(name, '.')
 371             head, tail = name[:i], name[i+1:]
 372             path = sys.modules[head].__path__
 373         else:
 374             tail = name
 375             path = sys.modules[''].__path__
 376         stuff = self.loader.find_module(tail, path)
 377         if not stuff:
 378             raise ImportError, "No module named %s" % name
 379         return self.loader.load_module(name, stuff)
 380
 381     def unload(self, module):
 382         if hasattr(module, '__path__'):
 383             raise ImportError, "don't know how to unload packages yet"
 384         PackageImporter.unload(self, module)
 385
 386     def install(self):
 387         if not sys.modules.has_key(''):
 388             sys.modules[''] = package = imp.new_module('')
 389             package.__path__ = None
 390             self.loader.init_package(package)
 391             for m in sys.modules.values():
 392                 if not m: continue
 393                 if not hasattr(m, '__'):
 394                     self.loader.set_parent(m)
 395         ModuleImporter.install(self)
 396
 397
 398 def install(v = 0):
 399     ihooks.install(PackageImporter(None, v))
 400
 401 def uninstall():
 402     ihooks.uninstall()
 403
 404 def ni(v = 0):
 405     install(v)
 406
 407 def no():
 408     uninstall()
 409
 410 def test():
 411     import pdb
 412     try:
 413         testproper()
 414     except:
 415         sys.last_type, sys.last_value, sys.last_traceback = (
 416                 sys.exc_type, sys.exc_value, sys.exc_traceback)
 417         print
 418         print sys.last_type, ':', sys.last_value
 419         print
 420         pdb.pm()
 421
 422 def testproper():
 423     install(1)
 424     try:
 425         import mactest
 426         print dir(mactest)
 427         raw_input('OK?')
 428     finally:
 429         uninstall()
 430
 431
 432 if __name__ == '__main__':
 433     test()
 434 else:
 435     install()