| blob | 12d751dd4a5096cbf42f15074497c96a1759b4ed |
1 <HTML>
2 <HEAD>
8 </HEAD>
11 <H2>
12 HTML</H2>
13 This documents describes the complete handling of HTML in magellan. The
14 document covers the parsing process - how HTML is lexically analysized
15 and then interprted. After the parsing process is discussed we give a detailed
16 analysis of each HTML tag and the attributes that are supported, the values
17 for the attributes and how the tag is treated by magellan.
18 <H2>
19 Parsing</H2>
20 HTML is tokenized by an HTML scanner. The scanner is fed unicode data to
21 parse. Stream converters are used to translate from various encodings to
22 unicode. The scanner separates the input stream into tokens which consist
23 of:
24 <UL>
25 <LI>
26 text</LI>
28 <LI>
29 tags</LI>
31 <LI>
32 entities</LI>
34 <LI>
35 script-entities</LI>
37 <LI>
38 comments</LI>
40 <LI>
41 conditional comments</LI>
42 </UL>
43 The HTML parsing engine uses the HTML scanner for lexical anlaysis. The
44 parsing engine operates by attacking the input stream in a set of well
45 defined steps:
46 <UL>
47 <LI>
48 The parser processes the head portion of the document first, without emitting
49 any output. This is done to discover a few special features of html:</LI>
51 <UL>
52 <LI>
53 The parser processes META tags looking for META TARGET</LI>
55 <LI>
56 The parser processes META tags looking for META tags which affect the character
57 set. Nav4 handles the very first character set defining meta tag (all others
58 are ignored) by reloading the document with the proper character conversion
59 module inserted into the stream pipeline.</LI>
60 </UL>
62 <LI>
63 After the head portion is processed the parser then proceeds to process
64 the body of the document</LI>
65 </UL>
67 <H3>
68 Tag Processing</H3>
70 the tag. The HTML parser serves as the tag handler for all of the builtin
71 tags documented below. Tag attribute handling is done during translation
72 of tags into content. This mapping translates the tag attributes into content
73 data and into style data. The translation to style data is documented below
74 by indicating the mapping from tag attributes to their CSS1 (plus extensions)
75 equivalents.
76 <H3>
77 Special Hacks</H3>
78 The following list describes hacks added to the magellan parsing engine
79 to deal with navigator compatability. These are just the parser hacks,
80 not the layout or presentation hacks. Most hacks are intriduced for HTML
81 syntax error recovering. HTML doesn't specify much how to handle those
82 error conditions. Netscape has made big effort to render pages with non-prefect
83 HTML. For many reasons, new browsers need to keep compatible in thsi area.
84 <UL>
85 <LI>
86 Entities can be used as escape in quoted string. For value string in name-value
89 means the value is NOT quoted. Test line 90 shows that if the value is
90 started with a quote, then an entity quote does NOT terminate the value
91 string.</LI>
93 <LI>
94 Wrapping tags are special tags such as title, textarea, server, script,
95 style, and etc.. The comment in ns\lib\libparse\pa_parse.c says:</LI>
99 them, all other tags are ignored until the matching
102 <BR>During the searching of an end tag, comments and quoted strings are
104 6.0 handles comments now, need to add quoted string.
105 <LI>
109 <LI>
111 tags before the last table tag are ignored. We found this problem in some
115 test table201.html</A>. There table and tr tags are buffered for this recovery.
116 When a TD or CAPTION tag is open, the buffer is flushed out, because we
117 cannot buffer contents of TD or CAPTION for performance and memory constrains.
120 Nav4.0 can discard previous table in such case. <A HREF="../testhtml/tableall.html">tableall.html
123 <LI>
124 Caption is not a commonly used feature. In Nav4.0, captions can be anywhere.
125 For Captions outside cells, the first one takes effect. For captions inside
126 cells, the last one takes effect, and they also close TD and TR. In 6.0,
128 in other places are ignored, their contents are treated as text. See test
129 case table05a.html to table05o.html.</LI>
131 <LI>
133 takes effect.</FONT></LI>
135 <LI>
136 The nav4 parser notices when it hits EOF and it's in the middle of scanning
137 in a comment. When this happens, the parser goes back and looks for an
138 improperly closed comment (e.g. a simple > instead of a -->). If it finds
139 one, it reparses the input after closing out the comment.</LI>
141 <LI>
143 similar done for tags, but I don't recall what it is right now.</FONT></LI>
145 <LI>
149 is searched. the default limit is 4096 char, and there is a API HTMLScanner.setMaxTagLength()
151 </UL>
157 <UL>
158 <LI>
165 <H2>
166 Tags (Categorically sorted)</H2>
167 All line breaks are conditional. If the x coordinate is at the current
168 left margin then a soft line break does nothing. Hard line breaks are ignored
169 if the last tag did a hard line break.
173 | baseline | center | bottom | top | middle | absmiddle
176 coordinates; pct of height for odd coordinates)
178 is present any following characters are ignored!
181 trailing and any embedded whitespace that is not an actual space (e.g.
182 newlines)</FONT>
183 <H1>
184 Head objects:</H1>
186 <UL>The TITLE tag is a container tag whose contents are not HTML. The contents
187 are pure text and are processed by the parser until the closing tag is
188 found. There are no attributes on the tag and any whitespace present in
189 the tag is compressed down with leading and trailing whitespace eliminated.
190 The first TITLE tag found by the parser is used as the document's title
191 (subsequent tags are ignored).</UL>
193 <UL>Sets the base element in the head portion of the document. Defines
195 <BR>Attributes:
198 otherwise nav4 ignores it]</UL>
199 </UL>
201 <UL>Can define several header fields (content-encoding, author, etc.)
202 <BR>Attributes:
207 </UL>
208 HTTP-EQUIV values (from libnet/mkutils.c NET_ParseMimeHeader):
209 <UL>ACCEPT-RANGES
210 <BR>CONTENT-DISPOSITION
211 <BR>CONTENT-ENCODING
212 <BR>CONTENT-RANGE
213 <BR>CONTENT-TYPE [ defines character set only ]
214 <BR>CONNECTION
215 <BR>DATE
216 <BR>EXPIRES
217 <BR>EXT-CACHE
218 <BR>LOCATION
219 <BR>LAST-MODIFIED
220 <BR>LINK
221 <BR>PROXY-AUTHENTICATE
222 <BR>PROXY-CONNECTION
223 <BR>PRAGMA
224 <BR>RANGE
225 <BR>REFRESH
226 <BR>SET-COOKIE
227 <BR>SERVER
228 <BR>WWW-AUTHENTICATE
229 <BR>WWW-PROTECTION-TEMPLATE
231 Style sheets and HTML w3c spec adds this:
232 <UL>CONTENT-STYLE-TYPE [ last one wins; overrides header from server if
233 any ]</UL>
234 </UL>
236 <UL>List related resources. Used by extensions mechanism to find tag handlers.
238 <BR>Attributes:
249 </UL>
250 Note: HREF takes precedence over SRC in nav4.</UL>
253 <BR>Text in head clears in_head, and set in_body true, just as if the author
254 forgot the /HEAD tag.
257 <UL>Ignored.
260 <UL>Allowed anywhere in the document. Note that entities are not parsed
261 in the style tag's content.
262 <BR>Attributes:
270 </UL>
273 <BR>Attributes:
279 </UL>
281 <UL>Border width of zero disables edges.
282 <BR>Attributes:
289 references]
293 </UL>
295 <UL>Used when frames are disabled or for backrev browsers. Has no stylistic
296 consequences.</UL>
298 <H1>
302 <UL>The tag is only processed on open tags and it is always processed.
303 See ns\lib\layout\laytags.c, searching for "case P_BODY". During tag processing
304 the in_head flag is set to false and the in_body flag is set to true. An
305 attribute is ignored if the document already has that attribute set. Attributes
306 can be set by style sheets, or by previous BODY tags. see <A HREF="../testhtml/head02.html">test
307 head02.html</A>.
308 <BR>Attributes:
310 - 1)]
312 - 1)]
321 </UL>
323 <UL>Open layer/ilayer tag automaticly close out an open form if one is
324 open. It does something to the soft linebreak state too.
325 <BR>Attributes:
342 </UL>
344 <UL>Container for content which is used when layers are disabled or unsupported.
345 The content has no style consequences (though it could if somebody stuck
346 in some CSS1 style rules for it).</UL>
348 <UL>Closes the paragraph. If the attribute is present then an alignment
349 gets pushed on the alignment stack. All values are supported by nav4.
350 <BR>Attributes:
352 </UL>
354 <UL>There are no attributes. ADDRESS closes out the open paragraph. The
355 open tag does a conditional soft line break and then pushes a merge of
356 the current style with italics enabled onto the style stack. The close
357 always pop the style stack and also does a conditional soft line break.</UL>
359 <UL>PLAINTEXT causes the remaining content to no longer be parsed. XMP
360 causes the content to not parse entities or other tags. The XMP can be
361 closed by it's own tag (on any boundary); PLAINTEXT is not closed (html3.2
362 allows it to be closed). Both tags change the style to a fixed font of
363 a</UL>
365 <UL>Closes the paragraph. Does a hard line break on open and close. Open
366 pushes a fixed width font style of a particular font size on the style
367 stack. The close tag pops the top of the style stack.
370 <UL>Closes the paragraph. The open tag does a hard line break. A fixed
371 font style (unless VARIABLE is present) is pushed on the style stack. The
372 close tag pops the top of the style stack. It also does a hard line break.
373 <BR>Attributes:
377 value]
379 </UL>
381 <UL>This tag doesn't nest. Instead it just sets or clears a flag in the
382 state machine. It has no effect on any other state.</UL>
384 <UL>Closes the paragraph. Always does a conditional soft line break. The
385 open tag pushes an alignment on the aligment stack. The close tag pops
386 the top alignment off.
389 <UL>Closes the paragraph. Always does a conditional soft line break. COLS
390 defines the number of columns to layout in (like MULTICOL). The open tag
392 to be a MULTICOL tag). The close tag pops an aligment from the alignment
393 stack.
394 <BR>Attributes:
397 DIV is just a container]
400 (strange code)]</UL>
401 </UL>
402 </UL>
404 <UL>Closes the paragraph. The open tag does a hard line break and pushes
405 a style item which enables bold and disables fixed and italic. The close
406 tag always pops the top item from the style stack. It also does a hard
408 pushes an alignment on the alignment stack. The close tag will look at
409 the top of the alignment stack and if its a header of any kind (H1 through
410 H6) then the alignment is popped. In either case the close tag also does
411 a conditional soft line break (this happens before the hard line break).
412 <BR>Attributes:
414 </UL>
415 A note regarding closing paragraphs: Any time a close paragraph is done
416 (for any tag) if the top of the alignment stack has a tag named "P" then
417 a conditional soft line break is done and the alignment is popped.
418 <H3>
422 <UL>Close the paragraph.
423 <BR>Attributes:
427 ]
437 ]
444 The code supports more attributes in the Table attribute handler than it
445 does in the code that gets the attributes from the tag! They are border_top,
446 border_left, border_right, border_bottom, border_style (defaults to outset;
447 allows for outset/dotted/none/dashed/solid/double/groove/ridge/inset).</UL>
449 <UL>Open TR automatically closes an open table row (and an open table cell
450 if one is open). It also automatically closes a CAPTION tag.
451 <BR>Attributes:
455 is top
457 </UL>
459 <UL>If no table then the tag is ignored (open or close). If no row is currently
461 a new row is begun. Oddly enough the tag parameters for the row come from
462 the TH/TD tag in this case. An open of either of these tags will automatically
463 close the previous cell.
464 <BR>Attributes:
469 table; if not table then inherit from an outer table cell; this works because
470 the style is flattened so the outer table cell will have a color]
472 mode is inherited too and not settable by TH/TD attributes (have to use
473 style sheets for that)]
475 is top
479 </UL>
481 <UL>An open caption tag will automatically close an open table row (and
482 an open cell).
483 <BR>Attributes:
485 The code sets the vertical alignment to top w/o providing a mechanism for
486 the user to set it (there is no VALIGN attribute).</UL>
488 <UL>The open tag does a hard line break. The close tag checks to see if
489 the state machine has an open multicol and if it does then it does a conditional
490 soft line break and then continues to break until both margins are cleared
491 of floating elements. It recomputes the margins based on the list indenting
492 level (?). After the synthetic table is output the close tag does a hard
493 line break.
495 <P>This tag will treat the input as source for a table with one row and
496 COLS columns. The data is laid out using the width divided by the number
497 of columns. After the total height is known, the content is partitioned
498 as evenly as possible between the columns in the table.
499 <BR>Attributes:
503 (strange code)]</UL>
504 </UL>
506 <H3>
510 <UL>Closes the paragraph. The open tag does a hard line break. A list with
511 the empty-bullet style is pushed on the list stack (unless TYPE=cite/jwz
512 then a styled list is pushed). The close tag pops any list and does a hard
513 line break.
514 <BR>Attributes:
516 </UL>
518 <UL>For top-level lists (lists not in lists) a hard break is done on the
519 open tag, otherwise a conditional-soft-break is done. Tag always does a
520 close paragrah. The close tag does a conditional soft line break when nested;
521 when not nested the close tag does a hard line break (even if no list is
522 open). The open tag pushes the list on the list stack. The close tag pops
523 any list off the list stack.
524 <BR>Attributes:
526 | upper-roman | lower-alpha | upper-alpha | A | a | I | i [clamped: if
527 none of the above is picked and OL then the bullet type is "number" otherwise
528 the bullet type is "basic"]
531 </UL>
533 <UL>Closes the paragraph. For the open tag, if the list is nested then
534 a conditional soft line break is done otherwise a hard line break is done.
535 The open tag pushes a list on the list stack. The close tag pops any list
536 from the list stack. Closing the list acts like other lists closes.
537 <BR>Attributes:
539 </UL>
541 <UL>Closes the paragraph. The open tag does a conditional soft line break.
542 Close tags are ignored (except for closing the paragraph).
543 <BR>Attributes:
548 The magellan html parser allows the full set of list item styles from the
549 OL/DL tag instead of just the limited set that nav4 allows.</UL>
551 <UL>Closes the paragraph. Close tags are ignored (except for closing the
552 paragraph). DD outside a DL just advances the X coordinate of layout by
553 a small constant. DD inside a DL does a conditional soft line break and
554 other margin crud.
557 <UL>Closes the paragraph (open or close). Close tags are otherwise ignored.
558 Does a conditional soft line break. Moves the X layout coordinate to the
559 left margin.
562 <H3>
567 Close anchors pop as many styles off the top of the style stack that are
568 anchor tags (anchor tags don't nest in other words). In addition, any styles
569 on the stack that have the ANCHOR bit set have it cleared and fiddle with
570 the foreground and background colors.
571 <BR>Attributes:
576 </UL>
577 </UL>
578 <B>STRIKE, S, TT, CODE, SAMPLE, KBD, B, STRONG, I, EM, VAR, CITE, BLINK,
579 BIG, SMALL, U, INLINEINPUT, SPELL</B>
580 <UL>The open tag pushes onto the style stack. The close tag always pops
581 the top item from the style stack.
584 <UL>The open tag pushes a font size descrease on the style stack. The close
585 tag always pops the top of the style stack. The open and close tag impacts
586 the baselineThe only difference between SUP and SUB is how they impact
587 the baseline. Note that the baseline information is forgotten after a line
588 break; therefore a close SUP/SUB on the next line will do strange things.
591 <UL>Ignored by the navigator.
594 <UL>The open font tag with no attributes resets the font size to the base
595 font size. The open tag always pushes a style stack entry. The close tag
596 always pops the top item off the style stack.
597 <BR>Attributes:
599 7]
606 </UL>
607 A note regarding the style stack: The pop of the stack checks to see if
608 the top of the stack is an ANCHOR tag. If it is not an anchor then the
609 top item is unconditionally popped. If the top of the style stack is an
610 anchor tag then the code searches for either the bottom of the stack or
611 the first style stack entry not created by an anchor tag. If the entry
612 is followed by another entry then the entry is removed from the stack (an
613 out-of-order pop in other words). In this case the anchor style stack entry
614 is left untouched.
615 <H3>
619 <UL>These are basic content objects that get fed directly to the output.
620 In navigator the text is processed by doing line-breaking (entities have
621 been converted to text already by the parser). The line-breaking is controlled
622 by the margin settings and the list depth, the floating elements, the style
623 attributes (font size, etc.), the preformatted flag, the no-break flag
624 and so on.</UL>
626 <UL>Close tag is ignored.
627 <BR>Attributes:
640 </UL>
642 <UL>Closes the paragraph. If an open tag then does a conditional soft line
643 break. The rule inherits alignment from the parent container unless there
644 is no container (then it's centered) or if the tag defines it's own alignment.
645 After the object is inserted into the layout stream a soft line break is
646 inserted as well.
647 <BR>Attributes:
649 it's left or right only)
653 </UL>
655 <UL>Does an unconditional soft break. If clear is set then it will also
656 soft break until either the left or right or both margins are clear of
658 <BR>Attributes:
660 </UL>
662 <UL>Soft word break.
665 <UL>Close tag does nothing. Embed's operate inline just like images (they
666 don't close the paragraph).
667 <BR>Attributes:
673 gets HIDDEN automatically
677 </UL>
679 <UL>Used when EMBED's are disabled. It is a container for regular content
680 that has no stylistic consequences (no line breaking, no style stack effect,
681 etc.).</UL>
683 <UL>Applet tags don't nest (there is a notion of current_applet). The open
684 tag automatically closes an open applet tag.
685 <BR>Attributes:
697 If no width is provided:
699 window width if percentage widths are allowed, otherwise use a value of
700 600.
702 If no height is provided:
705 If the applet is hidden, then the widht/height get forced to zero.</UL>
707 <UL>The param tag is supported when contained by the APPLET tag or the
708 OBJECT tag. It has no stylistic consequences. The attribute values from
710 == PARAM</FONT>.
711 <BR>Attributes:
714 White space being stripped is done as follows: leading and trailing whitespace
715 is removed. Any embedded whitespace is left alone except if it's a non-space
716 whitespace in which case it is removed.</UL>
718 <UL>The open tag pushes an object onto the object stack. The close tag
719 pops from the object stack. I don't understand how the data stuff works.
720 <BR>Attributes:
722 supported prefixes; maybe it's a url if no prefix shown?)
725 There are more attributes that depend on the type of object being embedded
726 in the page. If the object is a java bean (?) then the applet parameters
727 are supported:
728 <UL>CLASSID
729 <BR>HIDDEN
730 <BR>ALIGN
731 <BR>CLASSID (instead of CODE)
732 <BR>CODEBASE
733 <BR>ARCHIVE
734 <BR>MAYSCRIPT
735 <BR>ID (applets use NAME)
736 <BR>WIDTH
737 <BR>HEIGHT
738 <BR>BORDER
739 <BR>HSPACE
741 </UL>
743 <UL>The open tag automatically closes an open map (maps don't nest). There
744 is no stylistic consequence of the map nor does it provide any visible
745 presentation in the normal layout case (an editor would do something different).
746 The map can be declared anywhere in the document.
747 <BR>Attributes:
749 </UL>
751 <UL>Does nothing if there is no current map or the tag is a close tag.
752 <BR>Attributes:
759 </UL>
761 <UL>A container for server-side javascript. Not evaluated by the client
762 (parsed and ignored). Note: The navigator parser doesn't expand entities
765 <UL>Close tag is ignored. Open tag provides whitespace during layout: <B>TYPE</B>=line/vert/vertical
770 <BR>Attributes:
776 </UL>
778 <H3>
782 <UL>Note: The navigator parser doesn't expand entities in a SCRIPT tag.
783 <BR>Attributes:
791 </UL>
793 <UL>Used when scripting is off or by backrev browsers. It is a container
794 that has no stylistic consequences.</UL>
796 <H3>
800 <UL>Attributes:
805 </UL>
807 <UL>This tag is a shortcut for creating a form element with a submit button
808 and a single text field. If the PROMPT attribute is not present in the
812 <P>Attributes:
818 </UL>
820 <UL>Attributes vary according to type:
822 | button | image | file | jot | readonly | object
828 <UL>font style is forced to fixed
834 to the type:
840 reset or button)
842 reset or button)</UL>
846 </UL>
848 <UL>Attributes:
855 </UL>
857 <UL>Lives inside the SELECT tag (ignored otherwise).
858 <BR>Attributes:
861 </UL>
863 <UL>Attributes:
868 not known turns into soft)</UL>
869 </UL>
871 <UL>Attributes:
876 </UL>
878 <H3>
882 <UL>Sets the base font value which +/- size values in FONT tags are relative
883 to.
884 <BR>Attributes:
886 </UL>
888 <H2>
891 <B>NSCP_CLOSE, NSCP_OPEN, NSCP_REBLOCK, MQUOTE, CELL, SUBDOC, CERTIFICATE,
892 INLINEINPUTTHICK, INLINEINPUTDOTTED, COLORMAP, HYPE, SPELL, NSDT</B>
893 <UL>These tags are unsupported because they are used internally by netscape
894 and are never seen in real content. If somebody does use them between 4.0
895 and magellan, tough beans. We never documented them so they lose.</UL>
897 </BODY>
898 </HTML>