| blob | 6cffe9117f5d2226b8f90a5650e5b8eb8b3a0d43 |
1 ;;;; stuff that knows about dumping FASL files
3 ;;;; This software is part of the SBCL system. See the README file for
4 ;;;; more information.
5 ;;;;
6 ;;;; This software is derived from the CMU CL system, which was
7 ;;;; written at Carnegie Mellon University and released into the
8 ;;;; public domain. The software is in the public domain and is
9 ;;;; provided with absolutely no warranty. See the COPYING and CREDITS
10 ;;;; files for more information.
13 ;;; KLUDGE: Even though we're IN-PACKAGE SB!FASL, some of the code in
14 ;;; here is awfully chummy with the SB!C package. CMU CL didn't have
15 ;;; any separation between the two packages, and a lot of tight
16 ;;; coupling remains. -- WHN 2001-06-04
17 \f
18 ;;;; fasl dumper state
20 ;;; The FASL-OUTPUT structure represents everything we need to
21 ;;; know about dumping to a fasl file. (We need to objectify the
22 ;;; state because the fasdumper must be reentrant.)
24 #-no-ansi-print-object
28 s))))
30 ;; the stream we dump to
32 ;; hashtables we use to keep track of dumped constants so that we
33 ;; can get them from the table rather than dumping them again. The
34 ;; EQUAL-TABLE is used for lists and strings, and the EQ-TABLE is
35 ;; used for everything else. We use a separate EQ table to avoid
36 ;; performance pathologies with objects for which EQUAL degenerates
37 ;; to EQL. Everything entered in the EQUAL table is also entered in
38 ;; the EQ table.
41 ;; the table's current free pointer: the next offset to be used
43 ;; an alist (PACKAGE . OFFSET) of the table offsets for each package
44 ;; we have currently located.
46 ;; a table mapping from the ENTRY-INFO structures for dumped XEPs to
47 ;; the table offsets of the corresponding code pointers
49 ;; a table holding back-patching info for forward references to XEPs.
50 ;; The key is the ENTRY-INFO structure for the XEP, and the value is
51 ;; a list of conses (<code-handle> . <offset>), where <code-handle>
52 ;; is the offset in the table of the code object needing to be
53 ;; patched, and <offset> is the offset that must be patched.
55 ;; a list of the table handles for all of the DEBUG-INFO structures
56 ;; dumped in this file. These structures must be back-patched with
57 ;; source location information when the compilation is complete.
59 ;; This is used to keep track of objects that we are in the process
60 ;; of dumping so that circularities can be preserved. The key is the
61 ;; object that we have previously seen, and the value is the object
62 ;; that we reference in the table to find this previously seen
63 ;; object. (The value is never NIL.)
64 ;;
65 ;; Except with list objects, the key and the value are always the
66 ;; same. In a list, the key will be some tail of the value.
68 ;; a hash table of structures that are allowed to be dumped. If we
69 ;; try to dump a structure that isn't in this hash table, we lose.
72 ;;; This structure holds information about a circularity.
74 ;; the kind of modification to make to create circularity
76 ;; the object containing circularity
77 object
78 ;; index in object for circularity
80 ;; the object to be stored at INDEX in OBJECT. This is that the key
81 ;; that we were using when we discovered the circularity.
82 value
83 ;; the value that was associated with VALUE in the
84 ;; CIRCULARITY-TABLE. This is the object that we look up in the
85 ;; EQ-TABLE to locate VALUE.
86 enclosing-object)
88 ;;; a list of the CIRCULARITY structures for all of the circularities
89 ;;; detected in the current top level call to DUMP-OBJECT. Setting
90 ;;; this lobotomizes circularity detection as well, since circular
91 ;;; dumping uses the table.
94 ;;; used to inhibit table access when dumping forms to be read by the
95 ;;; cold loader
98 ;;; used to turn off the structure validation during dumping of source
99 ;;; info
101 ;;;; utilities
103 ;;; Write the byte B to the specified FASL-OUTPUT stream.
108 ;; Dump a word-sized integer.
116 ;; Dump a 32-bit integer.
124 ;;; Dump NUM to the fasl stream, represented by N bytes. This works
125 ;;; for either signed or unsigned integers. There's no range checking
126 ;;; -- if you don't specify enough bytes for the number to fit, this
127 ;;; function cheerfully outputs the low bytes.
138 ;;; Setting this variable to an (UNSIGNED-BYTE 32) value causes
139 ;;; DUMP-FOP to use it as a counter and emit a FOP-NOP4 with the
140 ;;; counter value before every ordinary fop. This can make it easier
141 ;;; to follow the progress of LOAD-AS-FASL when
142 ;;; debugging/testing/experimenting.
146 ;;; Dump the FOP code for the named FOP to the specified FASL-OUTPUT.
147 ;;;
148 ;;; FIXME: This should be a function, with a compiler macro expansion
149 ;;; for the common constant-FS case. (Among other things, that'll stop
150 ;;; it from EVALing ,FILE multiple times.)
151 ;;;
152 ;;; FIXME: Compiler macros, frozen classes, inlining, and similar
153 ;;; optimizations should be conditional on #!+SB-FROZEN.
159 #!+sb-show
167 ;;; Dump a FOP-CODE along with an integer argument, choosing the FOP
168 ;;; based on whether the argument will fit in a single byte.
169 ;;;
170 ;;; FIXME: This, like DUMP-FOP, should be a function with a
171 ;;; compiler-macro expansion.
182 ;;; Push the object at table offset Handle on the fasl stack.
188 ;;; Pop the object currently on the fasl stack top into the table, and
189 ;;; return the table index, incrementing the free pointer.
196 ;;; If X is in File's EQUAL-TABLE, then push the object and return T,
197 ;;; otherwise NIL. If *COLD-LOAD-DUMP* is true, then do nothing and
198 ;;; return NIL.
218 ;;; These functions are called after dumping an object to save the
219 ;;; object in the table. The object (also passed in as X) must already
220 ;;; be on the top of the FOP stack. If *COLD-LOAD-DUMP* is true, then
221 ;;; we don't do anything.
244 handle)
249 ;;; Record X in File's CIRCULARITY-TABLE unless *COLD-LOAD-DUMP* is
250 ;;; true. This is called on objects that we are about to dump might
251 ;;; have a circular path through them.
252 ;;;
253 ;;; The object must not currently be in this table, since the dumper
254 ;;; should never be recursively called on a circular reference.
255 ;;; Instead, the dumping function must detect the circularity and
256 ;;; arrange for the dumped object to be patched.
264 ;;; Dump FORM to a fasl file so that it evaluated at load time in normal
265 ;;; load and at cold-load time in cold load. This is used to dump package
266 ;;; frobbing forms.
275 \f
276 ;;;; opening and closing fasl files
278 ;;; Open a fasl file, write its header, and return a FASL-OUTPUT
279 ;;; object for dumping to it. Some human-readable information about
280 ;;; the source code is given by the string WHERE.
284 ;; SB-EXT:STRING-TO-OCTETS is not available while cross-compiling
285 #+sb-xc-host
291 ;; UTF-8 is safe to use, because +FASL-HEADER-STRING-STOP-CHAR-CODE+
292 ;; may not appear in UTF-8 encoded bytes
293 #-sb-xc-host
295 stream)))
301 ;; Begin the header with the constant machine-readable (and
302 ;; semi-human-readable) string which is used to identify fasl files.
304 ;; The constant string which begins the header is followed by
305 ;; arbitrary human-readable text, terminated by
306 ;; +FASL-HEADER-STRING-STOP-CHAR-CODE+.
312 "~% ~
313 compiled from ~S~% ~
314 at ~A~% ~
315 on ~A~% ~
316 using ~A version ~A~%"
317 where
324 stream)
326 ;; Finish the header by outputting fasl file implementation,
327 ;; version, and key *FEATURES*.
329 ;; The count is dumped as a 32-bit unsigned-byte even on 64-bit
330 ;; platforms. This ensures that a x86-64 SBCL can gracefully
331 ;; detect an error when trying to read a x86 fasl, instead
332 ;; of choking on a ridiculously long counted string.
333 ;; -- JES, 2005-12-30
341 res)))
343 ;;; Close the specified FASL-OUTPUT, aborting the write if ABORT-P.
348 ;; sanity checks
350 ;; End the group.
354 fasl-output)
357 ;; That's all, folks.
360 \f
361 ;;;; main entries to object dumping
363 ;;; This function deals with dumping objects that are complex enough
364 ;;; so that we want to cache them in the table, rather than repeatedly
365 ;;; dumping them. If the object is in the EQ-TABLE, then we push it,
366 ;;; otherwise, we do a type dispatch to a type specific dumping
367 ;;; function. The type specific branches do any appropriate
368 ;;; EQUAL-TABLE check and table entry.
369 ;;;
370 ;;; When we go to dump the object, we enter it in the CIRCULARITY-TABLE.
379 ;; KLUDGE: The code in this case has been hacked
380 ;; to match Douglas Crosher's quick fix to CMU CL
381 ;; (on cmucl-imp 1999-12-27), applied in sbcl-0.6.8.11
382 ;; with help from Martin Atzmueller. This is not an
383 ;; ideal solution; to quote DTC,
384 ;; The compiler locks up trying to coalesce the
385 ;; constant lists. The hack below will disable the
386 ;; coalescing of lists while dumping and allows
387 ;; the code to compile. The real fix would be to
388 ;; take a little more care while dumping these.
389 ;; So if better list coalescing is needed, start here.
390 ;; -- WHN 2000-11-07
405 ;; DUMP-ARRAY (and its callees) are responsible for
406 ;; updating the EQ and EQUAL hash tables.
417 ;; This probably never happens, since bad things tend to
418 ;; be detected during IR1 conversion.
420 x))))))
423 ;;; Dump an object of any type by dispatching to the correct
424 ;;; type-specific dumping function. We pick off immediate objects,
425 ;;; symbols and magic lists here. Other objects are handled by
426 ;;; DUMP-NON-IMMEDIATE-OBJECT.
427 ;;;
428 ;;; This is the function used for recursive calls to the fasl dumper.
429 ;;; We don't worry about creating circularities here, since it is
430 ;;; assumed that there is a top level call to DUMP-OBJECT.
445 ;;; Dump stuff to backpatch already dumped objects. INFOS is the list
446 ;;; of CIRCULARITY structures describing what to do. The patching FOPs
447 ;;; take the value to store on the stack. We compute this value by
448 ;;; fetching the enclosing object from the table, and then CDR'ing it
449 ;;; if necessary.
473 ;;; Set up stuff for circularity detection, then dump an object. All
474 ;;; shared and circular structure will be exactly preserved within a
475 ;;; single call to DUMP-OBJECT. Sharing between objects dumped by
476 ;;; separate calls is only preserved when convenient.
477 ;;;
478 ;;; We peek at the object type so that we only pay the circular
479 ;;; detection overhead on types of objects that might be circular.
490 \f
491 ;;;; LOAD-TIME-VALUE and MAKE-LOAD-FORM support
493 ;;; Emit a funcall of the function and return the handle for the
494 ;;; result.
505 ;;; Return T iff CONSTANT has already been dumped. It's been dumped if
506 ;;; it's in the EQ table.
507 ;;;
508 ;;; Note: historically (1) the above comment was "T iff ... has not been dumped",
509 ;;; (2) the test was was also true if the constant had been validated / was in
510 ;;; the valid objects table. This led to substructures occasionally skipping the
511 ;;; validation, and hence failing the "must have been validated" test.
515 ;;; Use HANDLE whenever we try to dump CONSTANT. HANDLE should have been
516 ;;; returned earlier by FASL-DUMP-LOAD-TIME-VALUE-LAMBDA.
524 ;;; Note that the specified structure can just be dumped by
525 ;;; enumerating the slots.
529 \f
530 ;;;; number dumping
564 #!+long-float
571 #-sb-xc-host
576 #-sb-xc-host
587 #!+long-float
589 ;; (There's no easy way to mix #!+LONG-FLOAT and #-SB-XC-HOST
590 ;; conditionalization at read time, so we do this SB-XC-HOST
591 ;; conditional at runtime instead.)
600 \f
601 ;;;; symbol dumping
603 ;;; Return the table index of PKG, adding the package to the table if
604 ;;; necessary. During cold load, we read the string as a normal string
605 ;;; so that we can do the package lookup at cold load time.
606 ;;;
607 ;;; FIXME: Despite the parallelism in names, the functionality of
608 ;;; this function is not parallel to other functions DUMP-FOO, e.g.
609 ;;; DUMP-SYMBOL and DUMP-LIST. The mapping between names and behavior
610 ;;; should be made more consistent.
618 #+sb-xc-host
621 file)
622 #-sb-xc-host
623 (#!+sb-unicode dump-simple-character-string
624 #!-sb-unicode dump-simple-base-string
626 file)
632 entry))))
633 \f
634 ;;; dumper for lists
636 ;;; Dump a list, setting up patching information when there are
637 ;;; circularities. We scan down the list, checking for CDR and CAR
638 ;;; circularities.
639 ;;;
640 ;;; If there is a CDR circularity, we terminate the list with NIL and
641 ;;; make a CIRCULARITY notation for the CDR of the previous cons.
642 ;;;
643 ;;; If there is no CDR circularity, then we mark the current cons and
644 ;;; check for a CAR circularity. When there is a CAR circularity, we
645 ;;; make the CAR NIL initially, arranging for the current cons to be
646 ;;; patched later.
647 ;;;
648 ;;; Otherwise, we recursively call the dumper to dump the current
649 ;;; element.
650 ;;;
651 ;;; Marking of the conses is inhibited when *COLD-LOAD-DUMP* is true.
652 ;;; This inhibits all circularity detection.
669 :object list
671 :value l
684 :object list
685 :index n
686 :value obj
712 ;;; If N > 255, must build list with one LIST operator, then LIST*
713 ;;; operators.
738 \f
739 ;;;; array dumping
741 ;;; Dump the array thing.
747 ;;; Dump the vector object. If it's not simple, then actually dump a
748 ;;; simple version of it. But we enter the original in the EQ or EQUAL
749 ;;; tables.
754 (*)))
755 x)))
757 #+sb-xc-host
762 #-sb-xc-host
767 #-sb-xc-host
769 #!+sb-unicode
773 #!-sb-unicode
784 #!+long-float
794 #!+long-float
802 ;;; Dump a SIMPLE-VECTOR, handling any circularities.
815 :object v
816 :index index
817 :value obj
824 ;;; In the grand scheme of things I don't pretend to understand any
825 ;;; more how this works, or indeed whether. But to write out specialized
826 ;;; vectors in the same format as fop-int-vector expects to read them
827 ;;; we need to be target-endian. dump-integer-as-n-bytes always writes
828 ;;; little-endian (which is correct for all other integers) so for a bigendian
829 ;;; target we need to swap octets -- CSR, after DB
832 "BITS must be a multiple of 8"
846 ;; The case which is easy to handle in a portable way is when
847 ;; the element size is a multiple of the output byte size, and
848 ;; happily that's the only case we need to be portable. (The
849 ;; cross-compiler has to output debug information (including
850 ;; (SIMPLE-ARRAY (UNSIGNED-BYTE 8) *).) The other cases are only
851 ;; needed in the target SBCL, so we let them be handled with
852 ;; unportable bit bashing.
864 floor file))))
868 ;; Note: Dumping specialized signed vectors isn't
869 ;; supported in the cross-compiler. (All cases here end
870 ;; up trying to call DUMP-RAW-BYTES, which isn't
871 ;; provided in the cross-compilation host, only on the
872 ;; target machine.)
879 #-sb-xc-host
884 ;; KLUDGE: This isn't the best way of expressing that the host
885 ;; may not have specializations for (unsigned-byte 2) and
886 ;; (unsigned-byte 4), which means that these types are
887 ;; type-equivalent to (simple-array (unsigned-byte 8) (*));
888 ;; the workaround is to remove them from the etypecase, since
889 ;; they can't be dumped from the cross-compiler anyway. --
890 ;; CSR, 2002-05-07
891 #-sb-xc-host
894 #-sb-xc-host
897 #-sb-xc-host
902 #-sb-xc-host
907 #-sb-xc-host
912 #-sb-xc-host
940 \f
941 ;;; Dump characters and string-ish things.
962 ;;; Dump a SIMPLE-BASE-STRING.
970 ;;; If we get here, it is assumed that the symbol isn't in the table,
971 ;;; but we are responsible for putting it there when appropriate. To
972 ;;; avoid too much special-casing, we always push the symbol in the
973 ;;; table, but don't record that we have done so if *COLD-LOAD-DUMP*
974 ;;; is true.
980 ;; see comment in genesis: we need this here for repeatable fasls
981 #+sb-xc-host
986 ;; special case, to work around possible xc host "design
987 ;; choice" weirdness in COMMON-LISP package
992 fop-uninterned-small-symbol-save
993 fop-uninterned-symbol-save
994 file))
995 ;; CMU CL had FOP-SYMBOL-SAVE/FOP-SMALL-SYMBOL-SAVE fops which
996 ;; used the current value of *PACKAGE*. Unfortunately that's
997 ;; broken w.r.t. ANSI Common Lisp semantics, so those are gone
998 ;; from SBCL.
999 ;;((eq pkg *package*)
1000 ;; (dump-fop* pname-length
1001 ;; fop-small-symbol-save
1002 ;; fop-symbol-save file))
1005 fop-lisp-small-symbol-save
1006 fop-lisp-symbol-save
1007 file))
1010 fop-keyword-small-symbol-save
1011 fop-keyword-symbol-save
1012 file))
1015 fop-small-symbol-in-byte-package-save
1016 fop-small-symbol-in-package-save
1017 file)
1021 fop-symbol-in-byte-package-save
1022 fop-symbol-in-package-save
1023 file)
1028 #!-sb-unicode dump-base-chars-of-string
1029 pname file)
1038 \f
1039 ;;;; component (function) dumping
1046 ;; In CMU CL there was no enforced connection between the CODE-LENGTH
1047 ;; argument and the number of bytes actually written. I added this
1048 ;; assertion while trying to debug portable genesis. -- WHN 19990902
1053 ;;; Dump all the fixups. Currently there are three flavors of fixup:
1054 ;;; - assembly routines: named by a symbol
1055 ;;; - foreign (C) symbols: named by a string
1056 ;;; - code object references: don't need a name.
1069 ;; Depending on the flavor, we may have various kinds of
1070 ;; noise before the position.
1072 (:assembly-routine
1082 (:foreign
1084 #!+linkage-table
1085 (:foreign-dataref
1092 (:code-object
1095 ;; No matter what the flavor, we'll always dump the position
1099 ;;; Dump out the constant pool and code-vector for component, push the
1100 ;;; result in the table, and return the offset.
1101 ;;;
1102 ;;; The only tricky thing is handling constant-pool references to
1103 ;;; functions. If we have already dumped the function, then we just
1104 ;;; push the code pointer. Otherwise, we must create back-patching
1105 ;;; information so that the constant will be set when the function is
1106 ;;; eventually dumped. This is a bit awkward, since we don't have the
1107 ;;; handle for the code object being dumped while we are dumping its
1108 ;;; constants.
1109 ;;;
1110 ;;; We dump trap objects in any unused slots or forward referenced slots.
1112 code-segment
1113 code-length
1114 trace-table-as-list
1115 fixups
1116 fasl-output)
1133 ;; Dump the offset of the trace table.
1135 ;; FIXME: As long as we don't have GENGC, the trace table is
1136 ;; hardwired to be empty. And SBCL doesn't have GENGC (and as
1137 ;; far as I know no modern CMU CL does either -- WHN
1138 ;; 2001-10-05). So might we be able to get rid of trace tables?
1139 ;;
1140 ;; Note that gencgc also does something with the trace table.
1142 ;; Dump the constants, noting any :ENTRY constants that have to
1143 ;; be patched.
1151 (:entry
1155 fasl-output))))
1163 (:load-time-value
1165 (:fdefinition
1171 ;; Dump the debug info.
1189 ;; These two dumps are only ones which contribute to our
1190 ;; TOTAL-LENGTH value.
1194 ;; DUMP-FIXUPS does its own internal DUMP-FOPs: the bytes it
1195 ;; dumps aren't included in the TOTAL-LENGTH passed to our
1196 ;; FOP-CODE/FOP-SMALL-CODE fop.
1206 handle))))
1223 ;;; Dump a function entry data structure corresponding to ENTRY to
1224 ;;; FILE. CODE-HANDLE is the table offset of the code object for the
1225 ;;; component.
1239 ;;; Alter the code object referenced by CODE-HANDLE at the specified
1240 ;;; OFFSET, storing the object referenced by ENTRY-HANDLE.
1249 ;;; Dump the code, constants, etc. for component. We pass in the
1250 ;;; assembler fixups, code vector and node info.
1252 code-segment
1253 code-length
1254 trace-table
1255 fixups
1256 file)
1263 #!+sb-dyncount
1269 code-segment
1270 code-length
1271 trace-table
1272 fixups
1273 file))
1284 entry-handle
1285 file))
1297 ;;; Dump a FOP-FUNCALL to call an already-dumped top level lambda at
1298 ;;; load time.
1306 ;;; Dump a FOP-FSET to arrange static linkage (at cold init) between
1307 ;;; FUN-NAME and the already-dumped function whose dump handle is
1308 ;;; FUN-DUMP-HANDLE.
1309 #+sb-xc-host
1318 ;;; Compute the correct list of DEBUG-SOURCE structures and backpatch
1319 ;;; all of the dumped DEBUG-INFO structures. We clear the
1320 ;;; FASL-OUTPUT-DEBUG-INFO, so that subsequent components with
1321 ;;; different source info may be dumped.
1335 #+sb-xc-host
1341 \f
1342 ;;;; dumping structures
1348 struct)))
1354 ;; last slot first on the stack, so that the layout is on top:
1365 :object struct
1366 :index index
1367 :value obj