RelNotes/Prebinding.html

   1 <HTML>
   2 <!--This file created 1/10/03 12:25 PM by Claris Home Page version 3.0-->
   3 <HEAD>
   4    <TITLE>Prebinding Notes</TITLE>
   5    <META NAME=GENERATOR CONTENT="Claris Home Page 3.0">
   6    <X-CLARIS-WINDOW TOP=57 BOTTOM=788 LEFT=183 RIGHT=843>
   7    <X-CLARIS-TAGVIEW MODE=minimal>
   8 </HEAD>
   9 <BODY BGCOLOR="#FFFFFF">
  10 <P><FONT SIZE="-1">MacOS X 10.2 Prebinding Notes Copyright &copy;
  11 2003 by Apple Computer, Inc. All Rights Reserved.</FONT></P>
  12
  13 <H1>Mac OS X 10.2 Prebinding Notes</H1>
  14
  15 <P>This document describes an optimization called <B>prebinding</B>
  16 which enables fast launching of applications on Mac OS X.</P>
  17
  18 <P>Normally, when you build an application or dynamic library, the
  19 static linker (<TT>ld</TT>) records the names of the symbols the
  20 executable links against, and marks references to these symbols as
  21 "undefined."</P>
  22
  23 <P>When an application is launched, the dynamic linker
  24 (<TT>dyld</TT>) must bind the needed undefined references from the
  25 executable and dynamic libraries to their respective definitions. The
  26 binding process can take time, since the linker must map the library
  27 to an unoccupied address range and calculate the address of each
  28 referenced symbol in the library.</P>
  29
  30 <P>Building a dynamic library with prebinding enabled eliminates the
  31 normal binding overhead by predefining the library at a specified
  32 address range. When an executable or other dynamic library is built
  33 against a prebound library, the linker can directly reference symbols
  34 in the prebound library by address, instead of leaving the addresses
  35 undefined.</P>
  36
  37 <P>The static linker also records the time stamp of libraries
  38 dependent libraries. When the program is executed, the dynamic linker
  39 checks to see that all the build time stamps match and that the
  40 prebound address ranges of all code does not overlap. If both of
  41 these conditions are met, the binding of undefined references is
  42 already done, which saves a considerable amount of time. If the time
  43 stamps don't match, or prebound executable addresses overlap, the
  44 prebinding is undone and the program is bound normally.</P>
  45
  46 <P>Note that prebinding is only applicable to Mach-O executables. CFM
  47 PEF binaries do not support prebinding.</P>
  48
  49 <H3><A NAME="#prebindreqs"></A>Requirements for Prebinding</H3>
  50
  51 <P>To work properly, libraries and executables must meet several
  52 important requirements:</P>
  53
  54 <OL>
  55    <LI>Libraries must not have overlapping preferred addresses. For
  56    each release of Mac OS X the Apple-supplied libraries are all
  57    prebound and do not have overlapping addresses. Your libraries
  58    must not overlap with any of your own libraries, and also must not
  59    overlap with any of the Apple-supplied libraries that are
  60    installed with Mac OS X. There is currently no way to
  61    automatically select an address for a library to ensure that it
  62    does not overlap the addresses of other libraries. Non-Apple
  63    libraries can use any address in the range <TT>0x00000000 </TT> to
  64    <TT>0x3FFFFFFF</TT>. For the Mac OS X 10.2 release, all addresses
  65    above <TT> 0xb0000000 </TT> are also available. To set the address
  66    of a library, pass either the <TT>-seg1addr</TT> flag or the
  67    <TT>-seg_addr_table</TT> flag to <TT>ld(1)</TT>. All executables
  68    should be at the default address, <TT>0x00000000</TT>. The default
  69    address is <TT>0x00000000</TT>. When selecting addresses for
  70    libraries, the address range of the largest executable using the
  71    libraries should be avoided.</LI>
  72
  73    <LI>There can be no undefined symbols. In most cases, this means
  74    that you must link against your dependent libraries. It also means
  75    that there can be no circular dependencies (cases where library A
  76    calls a function in library B, and a function in library B also
  77    calls a function in library A). Circular dependencies can be
  78    removed by changing the code to dynamically look up the function
  79    instead of directly referencing it.</LI>
  80
  81    <LI>You cannot override symbols that are referenced in flat
  82    namespace images used by the dependent libraries. For example, you
  83    can't define your own <TT>malloc</TT> and then prebind using flat
  84    namespace libraries.</LI>
  85 </OL>
  86
  87 <H3>Producing a Prebound Build</H3>
  88
  89 <P>All libraries must be built in dependent order and built prebound.
  90 That is, libraries must be built before the libraries (and
  91 executables) that link against them are built. To build prebound,
  92 either pass the <TT>-prebind</TT> flag to <TT>ld(1)</TT>, or define
  93 the <TT>LD_PREBIND</TT> environment variable.</P>
  94
  95 <P>You do not need to relink every time the prebinding of a library
  96 might change. You can use the <TT>redo_prebinding(1)</TT> tool and
  97 the <TT>update_prebinding(1)</TT> tool to update prebindings.
  98 However, in order to use these tools, you need to link against
  99 libraries that were initially prebound. If a library was not prebound
 100 when it was originally linked against, <TT>redo_prebinding(1)</TT>
 101 cannot prebind it. It must be initially built as a prebound library,
 102 as specified above.</P>
 103
 104 <H3>Building Your Project Prebound</H3>
 105
 106 <P>The easy way to do this is to set the environment variable
 107 <TT>LD_PREBIND</TT> before building your projects. For example, if
 108 you are using <TT>tcsh(1)</TT>:</P>
 109
 110 <P><TT>% setenv LD_PREBIND</TT></P>
 111
 112 <P>This has the same effect as passing <TT>-prebind</TT> to the
 113 <TT>ld(1)</TT>. For Project Builder framework and library projects
 114 you can add -prebind to <TT>OTHER_LDFLAGS</TT>. For application
 115 projects, Project Builder adds <TT>-prebind</TT> to the linker flags
 116 by default.</P>
 117
 118 <H3>If your project's prebinding is disabled when built, here's how
 119 to fix it:</H3>
 120
 121 <P>If prebinding fails you will see a warning message in the build
 122 log. Prebinding has a number of requirements to allow it to work
 123 <A HREF="#prebindreqs">(see above)</A>. If any of these requirements
 124 aren't met prebinding will disabled and a warning message will be
 125 printed by the static linker when building.</P>
 126
 127 <H4>The most common message you might see is:</H4>
 128
 129 <P><TT>/usr/bin/ld: warning prebinding disabled because of symbols
 130 overridden in dependent dynamic shared libraries:<BR>
 131 /BinCache1/objc4/Objects/objc4-133.obj~2/objects-optimized/objcopt.tproj/objcopt.o
 132 definition of _swap_mach_header in section (__TEXT,__text)<BR>
 133 /System/Library/Frameworks/System.framework/System(swap.o) definition
 134 of _swap_mach_header</TT></P>
 135
 136 <P>In this case, this happened for the objc4 project because it had a
 137 copy of System framework's swap.c in the project. It did this long
 138 ago as a workaround when the System framework's swap.c was out of
 139 date. The fix for this was to remove the objc4 project's copy of
 140 swap.c, thus eliminating the symbol override.</P>
 141
 142 <P>If you need to track down what object is referencing a specified
 143 symbol then <TT>ld(1)</TT>'s <TT>-ysymbol</TT> flag can be used. If
 144 you need to determine why a specific module is being linked in from a
 145 library <TT>ld(1)</TT>'s <TT>-whyload</TT> flag can be used.</P>
 146
 147 <H4>Another example:</H4>
 148
 149 <P><TT>ld: warning prebinding disabled because of symbols overridden
 150 in dependent dynamic shared libraries:<BR>
 151 /BinCache1/PBDevKit/Objects/PBDevKit-378.1.obj~2/objects-optimized/DevKit/Lowlevel.subproj_subproj.o
 152 definition of _regcomp in section (__TEXT,__text)<BR>
 153 /System/Library/Frameworks/System.framework/System(regcomp.o)
 154 definition of _regcomp</TT></P>
 155
 156 <P>In this case PBDevKit was using a different version of regcomp,
 157 regexec and regfree than the System framework's version. In this
 158 case, PBDevKit needs this different version but does not need the
 159 code in System to use it's version. The best fix for this problem is
 160 to hide the names of these three functions in PBDevKit and their
 161 uses. This can be done in a number of ways:</P>
 162
 163 <UL>
 164    <LI>PBDevKit could declare these symbols as
 165    <TT>__private_extern__</TT></LI>
 166
 167    <LI>PBDevKit could remove these symbols from the list of exported
 168    symbols using the nmedit(1) tool</LI>
 169
 170    <LI>If the System framework was built as a two-level namespace
 171    library and PBDevKit was built as a two-level namespace library,
 172    this would not be a problem, because these symbols would be
 173    declared in different namespaces and therefore not overlap.</LI>
 174 </UL>
 175
 176 <H4>Another message you might see is:</H4>
 177
 178 <P><TT>/usr/bin/ld: warning prebinding disabled because dependent
 179 library:<BR>
 180 /System/Library/Frameworks/System.framework/Versions/B/System is not
 181 prebound</TT></P>
 182
 183 <P>This happened because the project was built against a version of
 184 System.framework that was not prebound. This might be a build order
 185 problem. If you have programs or frameworks that depend on other
 186 frameworks you must build them in the right order, and you must
 187 ensure that they are prebound.</P>
 188
 189 <H4>Another example is:</H4>
 190
 191 <P><TT>/usr/bin/ld: warning prebinding disabled because dependent
 192 library:<BR>
 193 /System/Library/PrivateFrameworks/PBDevKit.framework/Versions/C/PBDevKit
 194 is not prebound</TT></P>
 195
 196 <P>In this case the prebinding of PBDevKit was disabled so that
 197 caused the prebinding of the program linking against it to be
 198 disabled (see the above example).</P>
 199
 200 <H3>Checking If A Binary Is Prebound</H3>
 201
 202 <P>To check if a binary was built prebound you can use
 203 <TT>otool(1)</TT> to look for the <TT>PREBOUND</TT> flag in the mach
 204 header. For example:</P>
 205
 206 <P><TT>% otool -hv /bin/cat<BR>
 207 /bin/cat:<BR>
 208 Mach header<BR>
 209 magic cputype cpusubtype filetype ncmds sizeofcmds flags<BR>
 210 MH_MAGIC PPC ALL EXECUTE 10 1456 NOUNDEFS DYLDLINK PREBOUND</TT></P>
 211
 212 <P>A prebound binary has the build time stamps of its dependent
 213 libraries recorded in them. To see the build time stamps again:</P>
 214
 215 <P><TT>% otool -Lv /bin/cat<BR>
 216 /bin/cat:<BR>
 217 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 218 version 50.0.0)<BR>
 219 time stamp 982996740 Fri Feb 23 22:39:00 2001</TT></P>
 220
 221 <P>And viewing the dependent libraries build time stamp can also be
 222 done with otool(1):</P>
 223
 224 <P><TT>% otool -Lv /usr/lib/libSystem.B.dylib<BR>
 225 /usr/lib/libSystem.B.dylib:<BR>
 226 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
 227 version 50.0.0)<BR>
 228 time stamp 982996740 Fri Feb 23 22:39:00 2001</TT></P>
 229
 230 <H3>Checking If Prebinding Is Successful</H3>
 231
 232 <P>Set the environment variable <TT>DYLD_PREBIND_DEBUG</TT> and run
 233 the program. For example using <TT>tcsh(1)</TT> and checking the
 234 program <TT>sync(1)</TT>:</P>
 235
 236 <P><TT>%<BR>
 237 % setenv DYLD_PREBIND_DEBUG<BR>
 238 % sync<BR>
 239 dyld: sync: prebinding enabled</TT></P>
 240
 241 <P>If a program is not built prebound but all the libraries are built
 242 prebound then dyld may try to use the prebound libraries. For
 243 example:</P>
 244
 245 <P><TT>% setenv DYLD_PREBIND_DEBUG<BR>
 246 % cc -v<BR>
 247 dyld: cc: prebinding enabled using only prebound libraries</TT></P>
 248
 249 <P>This happened because when <TT>cc(1)</TT> was built the System
 250 framework it was built against was not prebound. <TT>dyld(1)</TT> can
 251 fail trying to use prebound libraries if the program overrides
 252 symbols defined in it's dependent libraries that are used by a
 253 dependent library. For example:</P>
 254
 255 <P><TT>% /usr/bin/objcunique<BR>
 256 dyld: /usr/bin/objcunique: trying to use prebound libraries failed
 257 due to overridden symbols</TT></P>
 258
 259 <P>Currently <TT>dyld(1)</TT> only tries to use the prebound
 260 libraries if the program uses only one prebound library. This is
 261 because with flat namespace libraries the checking needed to make
 262 sure all the libraries don't override any of each other symbols is
 263 more time consuming than the optimization would generally save.</P>
 264
 265 <H3>Updating the Prebinding</H3>
 266
 267 <P>Another way prebinding may fail is that the build time stamps of
 268 the libraries a program was built against with do not match the
 269 libraries the program is run against. In the following example, sync1
 270 was built using using a different version of the
 271 System.framework:</P>
 272
 273 <P><TT>% setenv DYLD_PREBIND_DEBUG<BR>
 274 % /tmp/sync1<BR>
 275 % dyld: /tmp/sync1: prebinding disabled because time stamp of
 276 library:<BR>
 277 /System/Library/Frameworks/System.framework/Versions/B/System did not
 278 match<BR>
 279 %</TT></P>
 280
 281 <P>The program will run correctly but the prebinding optimization
 282 will be undone and the program dynamically bound as usual. Releases
 283 from Apple should never be in this state. However, if you change a
 284 framework on your system you can update the dependent programs (or
 285 frameworks) without rebuilding them by running
 286 <TT>redo_prebinding(1)</TT> on them. For example, using the case
 287 above:</P>
 288
 289 <P><TT>% redo_prebinding /tmp/sync1<BR>
 290 % setenv DYLD_PREBIND_DEBUG<BR>
 291 % /tmp/sync1<BR>
 292 dyld: /tmp/sync1: prebinding enabled<BR>
 293 % </TT></P>
 294
 295 <P>The program <TT>update_prebinding(1)</TT> can also be used to
 296 update the prebinding and is used as part of the Installer's
 297 processing during installation.</P>
 298
 299 <H3>Invalidating the Prebinding</H3>
 300
 301 <P>To force the prebinding to be out of date one can run
 302 <TT>strip(1)</TT> or <TT>nmedit(1)</TT> on one of the dependent
 303 libraries to cause the built time stamp to be changed. A common way
 304 to do this and not effect the symbols contained in the library is to
 305 run <TT>strip(1)</TT> with the <TT>-X</TT> option as all the local
 306 symbols starting with <TT>'L'</TT> are already stripped by the
 307 assembler by default.</P>
 308
 309 <H3>Prebinding and Library Initialization</H3>
 310
 311 <P>Since prebinding requires that the entire set of libraries be
 312 initialized when loaded, libraries may be initialized in a different
 313 order than they would without prebinding. This can uncover latent
 314 problems in the order of library initializations that went undetected
 315 without prebinding, particularly if there were library initialization
 316 routines which did not explicitly call the initialization routines
 317 for libraries on which they depend. As a result, unexpected problems
 318 can occur when prebinding is enabled.</P>
 319 </BODY>
 320 </HTML>