Lib/ni.py

   1 """New import scheme with package support.
   2
   3 A Package is a module that can contain other modules.  Packages can be
   4 nested.  Package introduce dotted names for modules, like P.Q.M, which
   5 could correspond to a file P/Q/M.py found somewhere on sys.path.  It
   6 is possible to import a package itself, though this makes little sense
   7 unless the package contains a module called __init__.
   8
   9 A package has two variables that control the namespace used for
  10 packages and modules, both initialized to sensible defaults the first
  11 time the package is referenced.
  12
  13 (1) A package's *module search path*, contained in the per-package
  14 variable __path__, defines a list of *directories* where submodules or
  15 subpackages of the package are searched.  It is initialized to the
  16 directory containing the package.  Setting this variable to None makes
  17 the module search path default to sys.path (this is not quite the same
  18 as setting it to sys.path, since the latter won't track later
  19 assignments to sys.path).
  20
  21 (2) A package's *import domain*, contained in the per-package variable
  22 __domain__, defines a list of *packages* that are searched (using
  23 their respective module search paths) to satisfy imports.  It is
  24 initialized to the list cosisting of the package itself, its parent
  25 package, its parent's parent, and so on, ending with the root package
  26 (the nameless package containing all top-level packages and modules,
  27 whose module search path is None, implying sys.path).
  28
  29 The default domain implements a search algorithm called "expanding
  30 search".  An alternative search algorithm called "explicit search"
  31 fixes the import search path to contain only the root package,
  32 requiring the modules in the package to name all imported modules by
  33 their full name.  The convention of using '__' to refer to the current
  34 package (both as a per-module variable and in module names) can be
  35 used by packages using explicit search to refer to modules in the same
  36 package; this combination is known as "explicit-relative search".
  37
  38 The PackageImporter and PackageLoader classes together implement the
  39 following policies:
  40
  41 - There is a root package, whose name is ''.  It cannot be imported
  42   directly but may be referenced, e.g. by using '__' from a top-level
  43   module.
  44
  45 - In each module or package, the variable '__' contains a reference to
  46   the parent package; in the root package, '__' points to itself.
  47
  48 - In the name for imported modules (e.g. M in "import M" or "from M
  49   import ..."), a leading '__' refers to the current package (i.e.
  50   the package containing the current module); leading '__.__' and so
  51   on refer to the current package's parent, and so on.  The use of
  52   '__' elsewhere in the module name is not supported.
  53
  54 - Modules are searched using the "expanding search" algorithm by
  55   virtue of the default value for __domain__.
  56
  57 - If A.B.C is imported, A is searched using __domain__; then
  58   subpackage B is searched in A using its __path__, and so on.
  59
  60 - Built-in modules have priority: even if a file sys.py exists in a
  61   package, "import sys" imports the built-in sys module.
  62
  63 - The same holds for frozen modules, for better or for worse.
  64
  65 - Submodules and subpackages are not automatically loaded when their
  66   parent packages is loaded.
  67
  68 - The construct "from package import *" is illegal.  (It can still be
  69   used to import names from a module.)
  70
  71 - When "from package import module1, module2, ..." is used, those
  72     modules are explicitly loaded.
  73
  74 - When a package is loaded, if it has a submodule __init__, that
  75   module is loaded.  This is the place where required submodules can
  76   be loaded, the __path__ variable extended, etc.  The __init__ module
  77   is loaded even if the package was loaded only in order to create a
  78   stub for a sub-package: if "import P.Q.R" is the first reference to
  79   P, and P has a submodule __init__, P.__init__ is loaded before P.Q
  80   is even searched.
  81
  82 Caveats:
  83
  84 - It is possible to import a package that has no __init__ submodule;
  85   this is not particularly useful but there may be useful applications
  86   for it (e.g. to manipulate its search paths from the outside!).
  87
  88 - There are no special provisions for os.chdir().  If you plan to use
  89   os.chdir() before you have imported all your modules, it is better
  90   not to have relative pathnames in sys.path.  (This could actually be
  91   fixed by changing the implementation of path_join() in the hook to
  92   absolutize paths.)
  93
  94 - Packages and modules are introduced in sys.modules as soon as their
  95   loading is started.  When the loading is terminated by an exception,
  96   the sys.modules entries remain around.
  97
  98 - There are no special measures to support mutually recursive modules,
  99   but it will work under the same conditions where it works in the
 100   flat module space system.
 101
 102 - Sometimes dummy entries (whose value is None) are entered in
 103   sys.modules, to indicate that a particular module does not exist --
 104   this is done to speed up the expanding search algorithm when a
 105   module residing at a higher level is repeatedly imported (Python
 106   promises that importing a previously imported module is cheap!)
 107
 108 - Although dynamically loaded extensions are allowed inside packages,
 109   the current implementation (hardcoded in the interpreter) of their
 110   initialization may cause problems if an extension invokes the
 111   interpreter during its initialization.
 112
 113 - reload() may find another version of the module only if it occurs on
 114   the package search path.  Thus, it keeps the connection to the
 115   package to which the module belongs, but may find a different file.
 116
 117 XXX Need to have an explicit name for '', e.g. '__root__'.
 118
 119 """
 120
 121
 122 import imp
 123 import string
 124 import sys
 125 import __builtin__
 126
 127 import ihooks
 128 from ihooks import ModuleLoader, ModuleImporter
 129
 130
 131 class PackageLoader(ModuleLoader):
 132
 133     """A subclass of ModuleLoader with package support.
 134
 135     find_module_in_dir() will succeed if there's a subdirectory with
 136     the given name; load_module() will create a stub for a package and
 137     load its __init__ module if it exists.
 138
 139     """
 140
 141     def find_module_in_dir(self, name, dir):
 142         if dir is not None:
 143             dirname = self.hooks.path_join(dir, name)
 144             if self.hooks.path_isdir(dirname):
 145                 return None, dirname, ('', '', 'PACKAGE')
 146         return ModuleLoader.find_module_in_dir(self, name, dir)
 147
 148     def load_module(self, name, stuff):
 149         file, filename, info = stuff
 150         suff, mode, type = info
 151         if type == 'PACKAGE':
 152             return self.load_package(name, stuff)
 153         if sys.modules.has_key(name):
 154             m = sys.modules[name]
 155         else:
 156             sys.modules[name] = m = imp.new_module(name)
 157         self.set_parent(m)
 158         if type == imp.C_EXTENSION and '.' in name:
 159             return self.load_dynamic(name, stuff)
 160         else:
 161             return ModuleLoader.load_module(self, name, stuff)
 162
 163     def load_dynamic(self, name, stuff):
 164         file, filename, (suff, mode, type) = stuff
 165         # Hack around restriction in imp.load_dynamic()
 166         i = string.rfind(name, '.')
 167         tail = name[i+1:]
 168         if sys.modules.has_key(tail):
 169             save = sys.modules[tail]
 170         else:
 171             save = None
 172         sys.modules[tail] = imp.new_module(name)
 173         try:
 174             m = imp.load_dynamic(tail, filename, file)
 175         finally:
 176             if save:
 177                 sys.modules[tail] = save
 178             else:
 179                 del sys.modules[tail]
 180         sys.modules[name] = m
 181         return m
 182
 183     def load_package(self, name, stuff):
 184         file, filename, info = stuff
 185         if sys.modules.has_key(name):
 186             package = sys.modules[name]
 187         else:
 188             sys.modules[name] = package = imp.new_module(name)
 189         package.__path__ = [filename]
 190         self.init_package(package)
 191         return package
 192
 193     def init_package(self, package):
 194         self.set_parent(package)
 195         self.set_domain(package)
 196         self.call_init_module(package)
 197
 198     def set_parent(self, m):
 199         name = m.__name__
 200         if '.' in name:
 201             name = name[:string.rfind(name, '.')]
 202         else:
 203             name = ''
 204         m.__ = sys.modules[name]
 205
 206     def set_domain(self, package):
 207         name = package.__name__
 208         package.__domain__ = domain = [name]
 209         while '.' in name:
 210             name = name[:string.rfind(name, '.')]
 211             domain.append(name)
 212         if name:
 213             domain.append('')
 214
 215     def call_init_module(self, package):
 216         stuff = self.find_module('__init__', package.__path__)
 217         if stuff:
 218             m = self.load_module(package.__name__ + '.__init__', stuff)
 219             package.__init__ = m
 220
 221
 222 class PackageImporter(ModuleImporter):
 223
 224     """Importer that understands packages and '__'."""
 225
 226     def __init__(self, loader = None, verbose = 0):
 227         ModuleImporter.__init__(self,
 228         loader or PackageLoader(None, verbose), verbose)
 229
 230     def import_module(self, name, globals={}, locals={}, fromlist=[]):
 231         if globals.has_key('__'):
 232             package = globals['__']
 233         else:
 234             # No calling context, assume in root package
 235             package = sys.modules['']
 236         if name[:3] in ('__.', '__'):
 237             p = package
 238             name = name[3:]
 239             while name[:3] in ('__.', '__'):
 240                 p = package.__
 241                 name = name[3:]
 242             if not name:
 243                 return self.finish(package, p, '', fromlist)
 244             if '.' in name:
 245                 i = string.find(name, '.')
 246                 name, tail = name[:i], name[i:]
 247             else:
 248                 tail = ''
 249             mname = p.__name__ and p.__name__+'.'+name or name
 250             m = self.get1(mname)
 251             return self.finish(package, m, tail, fromlist)
 252         if '.' in name:
 253             i = string.find(name, '.')
 254             name, tail = name[:i], name[i:]
 255         else:
 256             tail = ''
 257         for pname in package.__domain__:
 258             mname = pname and pname+'.'+name or name
 259             m = self.get0(mname)
 260             if m: break
 261         else:
 262             raise ImportError, "No such module %s" % name
 263         return self.finish(m, m, tail, fromlist)
 264
 265     def finish(self, module, m, tail, fromlist):
 266         # Got ....A; now get ....A.B.C.D
 267         yname = m.__name__
 268         if tail and sys.modules.has_key(yname + tail): # Fast path
 269             yname, tail = yname + tail, ''
 270             m = self.get1(yname)
 271         while tail:
 272             i = string.find(tail, '.', 1)
 273             if i > 0:
 274                 head, tail = tail[:i], tail[i:]
 275             else:
 276                 head, tail = tail, ''
 277             yname = yname + head
 278             m = self.get1(yname)
 279
 280         # Got ....A.B.C.D; now finalize things depending on fromlist
 281         if not fromlist:
 282             return module
 283         if '__' in fromlist:
 284             raise ImportError, "Can't import __ from anywhere"
 285         if not hasattr(m, '__path__'): return m
 286         if '*' in fromlist:
 287             raise ImportError, "Can't import * from a package"
 288         for f in fromlist:
 289             if hasattr(m, f): continue
 290             fname = yname + '.' + f
 291             self.get1(fname)
 292         return m
 293
 294     def get1(self, name):
 295         m = self.get(name)
 296         if not m:
 297             raise ImportError, "No module named %s" % name
 298         return m
 299
 300     def get0(self, name):
 301         m = self.get(name)
 302         if not m:
 303             sys.modules[name] = None
 304         return m
 305
 306     def get(self, name):
 307         # Internal routine to get or load a module when its parent exists
 308         if sys.modules.has_key(name):
 309             return sys.modules[name]
 310         if '.' in name:
 311             i = string.rfind(name, '.')
 312             head, tail = name[:i], name[i+1:]
 313         else:
 314             head, tail = '', name
 315         path = sys.modules[head].__path__
 316         stuff = self.loader.find_module(tail, path)
 317         if not stuff:
 318             return None
 319         sys.modules[name] = m = self.loader.load_module(name, stuff)
 320         if head:
 321             setattr(sys.modules[head], tail, m)
 322         return m
 323
 324     def reload(self, module):
 325         name = module.__name__
 326         if '.' in name:
 327             i = string.rfind(name, '.')
 328             head, tail = name[:i], name[i+1:]
 329             path = sys.modules[head].__path__
 330         else:
 331             tail = name
 332             path = sys.modules[''].__path__
 333         stuff = self.loader.find_module(tail, path)
 334         if not stuff:
 335             raise ImportError, "No module named %s" % name
 336         return self.loader.load_module(name, stuff)
 337
 338     def unload(self, module):
 339         if hasattr(module, '__path__'):
 340             raise ImportError, "don't know how to unload packages yet"
 341         PackageImporter.unload(self, module)
 342
 343     def install(self):
 344         if not sys.modules.has_key(''):
 345             sys.modules[''] = package = imp.new_module('')
 346             package.__path__ = None
 347             self.loader.init_package(package)
 348             for m in sys.modules.values():
 349                 if not m: continue
 350                 if not hasattr(m, '__'):
 351                     self.loader.set_parent(m)
 352         ModuleImporter.install(self)
 353
 354
 355 def install(v = 0):
 356     ihooks.install(PackageImporter(None, v))
 357
 358 def uninstall():
 359     ihooks.uninstall()
 360
 361 def ni(v = 0):
 362     install(v)
 363
 364 def no():
 365     uninstall()
 366
 367 def test():
 368     import pdb
 369     try:
 370         testproper()
 371     except:
 372         sys.last_type, sys.last_value, sys.last_traceback = (
 373                 sys.exc_type, sys.exc_value, sys.exc_traceback)
 374         print
 375         print sys.last_type, ':', sys.last_value
 376         print
 377         pdb.pm()
 378
 379 def testproper():
 380     install(1)
 381     try:
 382         import mactest
 383         print dir(mactest)
 384         raw_input('OK?')
 385     finally:
 386         uninstall()
 387
 388
 389 if __name__ == '__main__':
 390     test()