Doc/lib/libsgmllib.tex

   1 \section{\module{sgmllib} ---
   2          Simple SGML parser}
   3
   4 \declaremodule{standard}{sgmllib}
   5 \modulesynopsis{Only as much of an SGML parser as needed to parse HTML.}
   6
   7 \index{SGML}
   8
   9 This module defines a class \class{SGMLParser} which serves as the
  10 basis for parsing text files formatted in SGML (Standard Generalized
  11 Mark-up Language).  In fact, it does not provide a full SGML parser
  12 --- it only parses SGML insofar as it is used by HTML, and the module
  13 only exists as a base for the \refmodule{htmllib} module.  Another
  14 HTML parser which supports XHTML and offers a somewhat different
  15 interface is available in the \refmodule{HTMLParser} module.
  16
  17
  18 \begin{classdesc}{SGMLParser}{}
  19 The \class{SGMLParser} class is instantiated without arguments.
  20 The parser is hardcoded to recognize the following
  21 constructs:
  22
  23 \begin{itemize}
  24 \item
  25 Opening and closing tags of the form
  26 \samp{<\var{tag} \var{attr}="\var{value}" ...>} and
  27 \samp{</\var{tag}>}, respectively.
  28
  29 \item
  30 Numeric character references of the form \samp{\&\#\var{name};}.
  31
  32 \item
  33 Entity references of the form \samp{\&\var{name};}.
  34
  35 \item
  36 SGML comments of the form \samp{<!--\var{text}-->}.  Note that
  37 spaces, tabs, and newlines are allowed between the trailing
  38 \samp{>} and the immediately preceding \samp{--}.
  39
  40 \end{itemize}
  41 \end{classdesc}
  42
  43 \class{SGMLParser} instances have the following interface methods:
  44
  45
  46 \begin{methoddesc}{reset}{}
  47 Reset the instance.  Loses all unprocessed data.  This is called
  48 implicitly at instantiation time.
  49 \end{methoddesc}
  50
  51 \begin{methoddesc}{setnomoretags}{}
  52 Stop processing tags.  Treat all following input as literal input
  53 (CDATA).  (This is only provided so the HTML tag
  54 \code{<PLAINTEXT>} can be implemented.)
  55 \end{methoddesc}
  56
  57 \begin{methoddesc}{setliteral}{}
  58 Enter literal mode (CDATA mode).
  59 \end{methoddesc}
  60
  61 \begin{methoddesc}{feed}{data}
  62 Feed some text to the parser.  It is processed insofar as it consists
  63 of complete elements; incomplete data is buffered until more data is
  64 fed or \method{close()} is called.
  65 \end{methoddesc}
  66
  67 \begin{methoddesc}{close}{}
  68 Force processing of all buffered data as if it were followed by an
  69 end-of-file mark.  This method may be redefined by a derived class to
  70 define additional processing at the end of the input, but the
  71 redefined version should always call \method{close()}.
  72 \end{methoddesc}
  73
  74 \begin{methoddesc}{get_starttag_text}{}
  75 Return the text of the most recently opened start tag.  This should
  76 not normally be needed for structured processing, but may be useful in
  77 dealing with HTML ``as deployed'' or for re-generating input with
  78 minimal changes (whitespace between attributes can be preserved,
  79 etc.).
  80 \end{methoddesc}
  81
  82 \begin{methoddesc}{handle_starttag}{tag, method, attributes}
  83 This method is called to handle start tags for which either a
  84 \method{start_\var{tag}()} or \method{do_\var{tag}()} method has been
  85 defined.  The \var{tag} argument is the name of the tag converted to
  86 lower case, and the \var{method} argument is the bound method which
  87 should be used to support semantic interpretation of the start tag.
  88 The \var{attributes} argument is a list of \code{(\var{name},
  89 \var{value})} pairs containing the attributes found inside the tag's
  90 \code{<>} brackets.  The \var{name} has been translated to lower case
  91 and double quotes and backslashes in the \var{value} have been interpreted.
  92 For instance, for the tag \code{<A HREF="http://www.cwi.nl/">}, this
  93 method would be called as \samp{unknown_starttag('a', [('href',
  94 'http://www.cwi.nl/')])}.  The base implementation simply calls
  95 \var{method} with \var{attributes} as the only argument.
  96 \end{methoddesc}
  97
  98 \begin{methoddesc}{handle_endtag}{tag, method}
  99 This method is called to handle endtags for which an
 100 \method{end_\var{tag}()} method has been defined.  The
 101 \var{tag} argument is the name of the tag converted to lower case, and
 102 the \var{method} argument is the bound method which should be used to
 103 support semantic interpretation of the end tag.  If no
 104 \method{end_\var{tag}()} method is defined for the closing element,
 105 this handler is not called.  The base implementation simply calls
 106 \var{method}.
 107 \end{methoddesc}
 108
 109 \begin{methoddesc}{handle_data}{data}
 110 This method is called to process arbitrary data.  It is intended to be
 111 overridden by a derived class; the base class implementation does
 112 nothing.
 113 \end{methoddesc}
 114
 115 \begin{methoddesc}{handle_charref}{ref}
 116 This method is called to process a character reference of the form
 117 \samp{\&\#\var{ref};}.  In the base implementation, \var{ref} must
 118 be a decimal number in the
 119 range 0-255.  It translates the character to \ASCII{} and calls the
 120 method \method{handle_data()} with the character as argument.  If
 121 \var{ref} is invalid or out of range, the method
 122 \code{unknown_charref(\var{ref})} is called to handle the error.  A
 123 subclass must override this method to provide support for named
 124 character entities.
 125 \end{methoddesc}
 126
 127 \begin{methoddesc}{handle_entityref}{ref}
 128 This method is called to process a general entity reference of the
 129 form \samp{\&\var{ref};} where \var{ref} is an general entity
 130 reference.  It looks for \var{ref} in the instance (or class)
 131 variable \member{entitydefs} which should be a mapping from entity
 132 names to corresponding translations.  If a translation is found, it
 133 calls the method \method{handle_data()} with the translation;
 134 otherwise, it calls the method \code{unknown_entityref(\var{ref})}.
 135 The default \member{entitydefs} defines translations for
 136 \code{\&amp;}, \code{\&apos}, \code{\&gt;}, \code{\&lt;}, and
 137 \code{\&quot;}.
 138 \end{methoddesc}
 139
 140 \begin{methoddesc}{handle_comment}{comment}
 141 This method is called when a comment is encountered.  The
 142 \var{comment} argument is a string containing the text between the
 143 \samp{<!--} and \samp{-->} delimiters, but not the delimiters
 144 themselves.  For example, the comment \samp{<!--text-->} will
 145 cause this method to be called with the argument \code{'text'}.  The
 146 default method does nothing.
 147 \end{methoddesc}
 148
 149 \begin{methoddesc}{handle_decl}{data}
 150 Method called when an SGML declaration is read by the parser.  In
 151 practice, the \code{DOCTYPE} declaration is the only thing observed in
 152 HTML, but the parser does not discriminate among different (or broken)
 153 declarations.  Internal subsets in a \code{DOCTYPE} declaration are
 154 not supported.  The \var{data} parameter will be the entire contents
 155 of the declaration inside the \code{<!}...\code{>} markup.  The
 156 default implementation does nothing.
 157 \end{methoddesc}
 158
 159 \begin{methoddesc}{report_unbalanced}{tag}
 160 This method is called when an end tag is found which does not
 161 correspond to any open element.
 162 \end{methoddesc}
 163
 164 \begin{methoddesc}{unknown_starttag}{tag, attributes}
 165 This method is called to process an unknown start tag.  It is intended
 166 to be overridden by a derived class; the base class implementation
 167 does nothing.
 168 \end{methoddesc}
 169
 170 \begin{methoddesc}{unknown_endtag}{tag}
 171 This method is called to process an unknown end tag.  It is intended
 172 to be overridden by a derived class; the base class implementation
 173 does nothing.
 174 \end{methoddesc}
 175
 176 \begin{methoddesc}{unknown_charref}{ref}
 177 This method is called to process unresolvable numeric character
 178 references.  Refer to \method{handle_charref()} to determine what is
 179 handled by default.  It is intended to be overridden by a derived
 180 class; the base class implementation does nothing.
 181 \end{methoddesc}
 182
 183 \begin{methoddesc}{unknown_entityref}{ref}
 184 This method is called to process an unknown entity reference.  It is
 185 intended to be overridden by a derived class; the base class
 186 implementation does nothing.
 187 \end{methoddesc}
 188
 189 Apart from overriding or extending the methods listed above, derived
 190 classes may also define methods of the following form to define
 191 processing of specific tags.  Tag names in the input stream are case
 192 independent; the \var{tag} occurring in method names must be in lower
 193 case:
 194
 195 \begin{methoddescni}{start_\var{tag}}{attributes}
 196 This method is called to process an opening tag \var{tag}.  It has
 197 preference over \method{do_\var{tag}()}.  The
 198 \var{attributes} argument has the same meaning as described for
 199 \method{handle_starttag()} above.
 200 \end{methoddescni}
 201
 202 \begin{methoddescni}{do_\var{tag}}{attributes}
 203 This method is called to process an opening tag \var{tag} that does
 204 not come with a matching closing tag.  The \var{attributes} argument
 205 has the same meaning as described for \method{handle_starttag()} above.
 206 \end{methoddescni}
 207
 208 \begin{methoddescni}{end_\var{tag}}{}
 209 This method is called to process a closing tag \var{tag}.
 210 \end{methoddescni}
 211
 212 Note that the parser maintains a stack of open elements for which no
 213 end tag has been found yet.  Only tags processed by
 214 \method{start_\var{tag}()} are pushed on this stack.  Definition of an
 215 \method{end_\var{tag}()} method is optional for these tags.  For tags
 216 processed by \method{do_\var{tag}()} or by \method{unknown_tag()}, no
 217 \method{end_\var{tag}()} method must be defined; if defined, it will
 218 not be used.  If both \method{start_\var{tag}()} and
 219 \method{do_\var{tag}()} methods exist for a tag, the
 220 \method{start_\var{tag}()} method takes precedence.