lib/perl5/5.6.1/unicode/UCDFF301.html

   1 <html>
   2
   3 <head>
   4 <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
   5 <meta name="ProgId" content="FrontPage.Editor.Document">
   6 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
   7 <link rel="stylesheet" href="http://www.unicode.org/unicode.css" type="text/css">
   8 <title>UnicodeData File Format</title>
   9 </head>
  10
  11 <body>
  12
  13 <h1>UnicodeData File Format<br>
  14 Version 3.0.1</h1>
  15 <table border="1" cellspacing="2" cellpadding="0" height="87" width="100%">
  16   <tr>
  17     <td valign="TOP" width="144">Revision</td>
  18     <td valign="TOP">3.0.1</td>
  19   </tr>
  20   <tr>
  21     <td valign="TOP" width="144">Authors</td>
  22     <td valign="TOP">Mark Davis and Ken Whistler</td>
  23   </tr>
  24   <tr>
  25     <td valign="TOP" width="144">Date</td>
  26     <td valign="TOP">2000-08-17</td>
  27   </tr>
  28   <tr>
  29     <td valign="TOP" width="144">This Version</td>
  30     <td valign="TOP"><a
  31       href="http://www.unicode.org/Public/3.0-Update1/UnicodeData-3.0.1.html">http://www.unicode.org/Public/3.0-Update1/UnicodeData-3.0.1.html</a></td>
  32   </tr>
  33   <tr>
  34     <td valign="TOP" width="144">Previous Version</td>
  35     <td valign="TOP"><a
  36       href="http://www.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.html">http://www.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.html</a></td>
  37   </tr>
  38   <tr>
  39     <td valign="TOP" width="144">Latest Version</td>
  40     <td valign="TOP"><a
  41       href="http://www.unicode.org/Public/UNIDATA/UnicodeData.html">http://www.unicode.org/Public/UNIDATA/UnicodeData.html</a></td>
  42   </tr>
  43 </table>
  44 <p align="center">Copyright © 1995-2000 Unicode, Inc. All Rights reserved.<br>
  45 <i>For more information, including Disclamer and Limitations, see <a
  46 href="UnicodeCharacterDatabase-3.0.1.html">UnicodeCharacterDatabase-3.0.1.html</a></i></p>
  47 <p>This document describes the format of the UnicodeData.txt file, which is one
  48 of the files in the Unicode Character Database. The document is divided into the
  49 following sections:
  50 <ul>
  51   <li><a href="#Field Formats">Field Formats</a>
  52     <ul>
  53       <li><a href="#General Category">General Category</a></li>
  54       <li><a href="#Bidirectional Category">Bidirectional Category</a></li>
  55       <li><a href="#Character Decomposition">Character Decomposition Mapping</a></li>
  56       <li><a href="#Canonical Combining Classes">Canonical Combining Classes</a></li>
  57       <li><a href="#Decompositions and Normalization">Decompositions and
  58         Normalization</a></li>
  59       <li><a href="#Case Mappings">Case Mappings</a></li>
  60     </ul>
  61   </li>
  62   <li><a href="#Property Invariants">Property Invariants</a></li>
  63   <li><a href="#Modification History">Modification History</a></li>
  64 </ul>
  65 <p><b>Warning: </b>the information in this file does not completely describe the
  66 use and interpretation of Unicode character properties and behavior. It must be
  67 used in conjunction with the data in the other files in the <a
  68 href="UnicodeCharacterDatabase-3.0.1.html">Unicode Character Database</a>, and
  69 relies on the notation and definitions supplied in <i><a
  70 href="http://www.unicode.org/unicode/standard/versions/Unicode3.0.html">The
  71 Unicode Standard</a></i>. All chapter references are to Version 3.0 of the
  72 standard.</p>
  73 <h2><a name="Field Formats"></a>Field Formats</h2>
  74 <p>The file consists of lines containing fields separated by semicolons. Each
  75 line represents the data for one encoded character in the Unicode Standard.
  76 Every encoded character has a data entry, with the exception of certain special
  77 ranges, as detailed below.
  78 <ul>
  79   <li>There are nine special ranges of characters that are represented only by
  80     their start and end characters, since the properties in the file are
  81     uniform, except for code values (which are all sequential and assigned).</li>
  82   <li>The names of CJK ideograph characters and the names and decompositions of
  83     Hangul syllable characters are algorithmically derivable. (See the Unicode
  84     Standard and <a href="http://www.unicode.org/unicode/reports/tr15/">Unicode
  85     Standard Annex #15</a> for more information).</li>
  86   <li>Surrogate code values and private use characters have no names.</li>
  87   <li>The Private Use character outside of the BMP (U+F0000..U+FFFFD,
  88     U+100000..U+10FFFD) are listed as distinct ranges. These correspond to surrogate pairs
  89     where the first surrogate is in the High Surrogate Private Use section.</li>
  90 </ul>
  91 <p>The exact ranges represented by start and end characters are:
  92 <ul>
  93   <li>CJK Ideographs Extension A (U+3400 - U+4DB5)</li>
  94   <li>CJK Ideographs (U+4E00 - U+9FA5)</li>
  95   <li>Hangul Syllables (U+AC00 - U+D7A3)</li>
  96   <li>Non-Private Use High Surrogates (U+D800 - U+DB7F)</li>
  97   <li>Private Use High Surrogates (U+DB80 - U+DBFF)</li>
  98   <li>Low Surrogates (U+DC00 - U+DFFF)</li>
  99   <li>The Private Use Area (U+E000 - U+F8FF)</li>
 100   <li>Plane 15 Private Use Area (U+F0000 - U+FFFFD)</li>
 101   <li>Plane 16 Private Use Area (U+100000 - U+10FFFD)</li>
 102 </ul>
 103 <p>The following table describes the format and meaning of each field in a data
 104 entry in the UnicodeData file. Fields which contain normative information are so
 105 indicated.</p>
 106 <table border="1" cellspacing="2" cellpadding="2">
 107   <tr>
 108     <th valign="top" align="LEFT">
 109       <p align="LEFT">Field</th>
 110     <th valign="top" align="LEFT">
 111       <p align="LEFT">Name</th>
 112     <th valign="top" align="LEFT">
 113       <p align="LEFT">Status</th>
 114     <th valign="top" align="LEFT">
 115       <p align="LEFT">Explanation</th>
 116   </tr>
 117   <tr>
 118     <th valign="top">0</th>
 119     <td valign="top">Code value</td>
 120     <td valign="top">normative</td>
 121     <td valign="top">Code value. For characters in the range U+0000..U+FFFD
 122       the code value uses a 4-digit hexadecimal format; for characters in the
 123       range U+10000..U+FFFFD the code value uses a 5-digit hexadecimal format;
 124       and for characters in the range U+100000..U+10FFFD the code value uses a
 125       6-digit hexadecimal format.</td>
 126   </tr>
 127   <tr>
 128     <th valign="top">1</th>
 129     <td valign="top">Character name</td>
 130     <td valign="top">normative</td>
 131     <td valign="top">These names match exactly the names published in Chapter 14
 132       of the Unicode Standard, Version 3.0.</td>
 133   </tr>
 134   <tr>
 135     <th valign="top">2</th>
 136     <td valign="top"><a href="#General Category">General Category</a></td>
 137     <td valign="top">normative / informative<br>
 138       (see below)</td>
 139     <td valign="top">This is a useful breakdown into various &quot;character
 140       types&quot; which can be used as a default categorization in
 141       implementations. See below for a brief explanation.</td>
 142   </tr>
 143   <tr>
 144     <th valign="top">3</th>
 145     <td valign="top"><a href="#Canonical Combining Classes">Canonical Combining
 146       Classes</a></td>
 147     <td valign="top">normative</td>
 148     <td valign="top">The classes used for the Canonical Ordering Algorithm in
 149       the Unicode Standard. These classes are also printed in Chapter 4 of the
 150       Unicode Standard.</td>
 151   </tr>
 152   <tr>
 153     <th valign="top">4</th>
 154     <td valign="top"><a href="#Bidirectional Category">Bidirectional Category</a></td>
 155     <td valign="top">normative</td>
 156     <td valign="top">See the list below for an explanation of the abbreviations
 157       used in this field. These are the categories required by the Bidirectional
 158       Behavior Algorithm in the Unicode Standard. These categories are
 159       summarized in Chapter 3 of the Unicode Standard.</td>
 160   </tr>
 161   <tr>
 162     <th valign="top">5</th>
 163     <td valign="top"><a href="#Character Decomposition">Character Decomposition
 164       Mapping</a></td>
 165     <td valign="top">normative</td>
 166     <td valign="top">In the Unicode Standard, not all of the mappings are full
 167       (maximal) decompositions. Recursive application of look-up for
 168       decompositions will, in all cases, lead to a maximal decomposition. The
 169       decomposition mappings match exactly the decomposition mappings published
 170       with the character names in the Unicode Standard.</td>
 171   </tr>
 172   <tr>
 173     <th valign="top">6</th>
 174     <td valign="top">Decimal digit value</td>
 175     <td valign="top">normative</td>
 176     <td valign="top">This is a numeric field. If the character has the decimal
 177       digit property, as specified in Chapter 4 of the Unicode Standard, the
 178       value of that digit is represented with an integer value in this field</td>
 179   </tr>
 180   <tr>
 181     <th valign="top">7</th>
 182     <td valign="top">Digit value</td>
 183     <td valign="top">normative</td>
 184     <td valign="top">This is a numeric field. If the character represents a
 185       digit, not necessarily a decimal digit, the value is here. This covers
 186       digits which do not form decimal radix forms, such as the compatibility
 187       superscript digits</td>
 188   </tr>
 189   <tr>
 190     <th valign="top">8</th>
 191     <td valign="top">Numeric value</td>
 192     <td valign="top">normative</td>
 193     <td valign="top">This is a numeric field. If the character has the numeric
 194       property, as specified in Chapter 4 of the Unicode Standard, the value of
 195       that character is represented with an integer or rational number in this
 196       field. This includes fractions as, e.g., &quot;1/5&quot; for U+2155 VULGAR
 197       FRACTION ONE FIFTH Also included are numerical values for compatibility
 198       characters such as circled numbers.</td>
 199   </tr>
 200   <tr>
 201     <th valign="top">9</th>
 202     <td valign="top">Mirrored</td>
 203     <td valign="top">normative</td>
 204     <td valign="top">If the character has been identified as a
 205       &quot;mirrored&quot; character in bidirectional text, this field has the
 206       value &quot;Y&quot;; otherwise &quot;N&quot;. The list of mirrored
 207       characters is also printed in Chapter 4 of the Unicode Standard.</td>
 208   </tr>
 209   <tr>
 210     <th valign="top">10</th>
 211     <td valign="top">Unicode 1.0 Name</td>
 212     <td valign="top">informative</td>
 213     <td valign="top">This is the old name as published in Unicode 1.0. This name
 214       is only provided when it is significantly different from the Unicode 3.0
 215       name for the character.</td>
 216   </tr>
 217   <tr>
 218     <th valign="top">11</th>
 219     <td valign="top">10646 comment field</td>
 220     <td valign="top">informative</td>
 221     <td valign="top">This is the ISO 10646 comment field. It appears in parentheses
 222       in the 10646 names list, or contains an asterisk to mark an Annex P note.</td>
 223   </tr>
 224   <tr>
 225     <th valign="top">12</th>
 226     <td valign="top"><a href="#Case Mappings">Uppercase Mapping</a></td>
 227     <td valign="top">informative</td>
 228     <td valign="top">Upper case equivalent mapping. If a character is part of an
 229       alphabet with case distinctions, and has an upper case equivalent, then
 230       the upper case equivalent is in this field. See the explanation below on
 231       case distinctions. These mappings are always one-to-one, not one-to-many
 232       or many-to-one. This field is informative.</td>
 233   </tr>
 234   <tr>
 235     <th valign="top">13</th>
 236     <td valign="top"><a href="#Case Mappings">Lowercase Mapping</a></td>
 237     <td valign="top">informative</td>
 238     <td valign="top">Similar to Uppercase mapping</td>
 239   </tr>
 240   <tr>
 241     <th valign="top">14</th>
 242     <td valign="top"><a href="#Case Mappings">Titlecase Mapping</a></td>
 243     <td valign="top">informative</td>
 244     <td valign="top">Similar to Uppercase mapping</td>
 245   </tr>
 246 </table>
 247 <h3><a name="General Category"></a>General Category</h3>
 248 <p>The values in this field are abbreviations for the following. Some of the
 249 values are normative, and some are informative. For more information, see the
 250 Unicode Standard.</p>
 251 <p><b>Note:</b> the standard does not assign information to control characters
 252 (except for certain cases in the Bidirectional Algorithm). Implementations will
 253 generally also assign categories to certain control characters, notably CR and
 254 LF, according to platform conventions.</p>
 255 <h4>Normative Categories</h4>
 256 <table border="0" cellspacing="2" cellpadding="0">
 257   <tr>
 258     <th>
 259       <p align="LEFT">Abbr.</th>
 260     <th>
 261       <p align="LEFT">Description</th>
 262   </tr>
 263   <tr>
 264     <td align="CENTER">Lu</td>
 265     <td>Letter, Uppercase</td>
 266   </tr>
 267   <tr>
 268     <td align="CENTER">Ll</td>
 269     <td>Letter, Lowercase</td>
 270   </tr>
 271   <tr>
 272     <td align="CENTER">Lt</td>
 273     <td>Letter, Titlecase</td>
 274   </tr>
 275   <tr>
 276     <td align="CENTER">Mn</td>
 277     <td>Mark, Non-Spacing</td>
 278   </tr>
 279   <tr>
 280     <td align="CENTER">Mc</td>
 281     <td>Mark, Spacing Combining</td>
 282   </tr>
 283   <tr>
 284     <td align="CENTER">Me</td>
 285     <td>Mark, Enclosing</td>
 286   </tr>
 287   <tr>
 288     <td align="CENTER">Nd</td>
 289     <td>Number, Decimal Digit</td>
 290   </tr>
 291   <tr>
 292     <td align="CENTER">Nl</td>
 293     <td>Number, Letter</td>
 294   </tr>
 295   <tr>
 296     <td align="CENTER">No</td>
 297     <td>Number, Other</td>
 298   </tr>
 299   <tr>
 300     <td align="CENTER">Zs</td>
 301     <td>Separator, Space</td>
 302   </tr>
 303   <tr>
 304     <td align="CENTER">Zl</td>
 305     <td>Separator, Line</td>
 306   </tr>
 307   <tr>
 308     <td align="CENTER">Zp</td>
 309     <td>Separator, Paragraph</td>
 310   </tr>
 311   <tr>
 312     <td align="CENTER">Cc</td>
 313     <td>Other, Control</td>
 314   </tr>
 315   <tr>
 316     <td align="CENTER">Cf</td>
 317     <td>Other, Format</td>
 318   </tr>
 319   <tr>
 320     <td align="CENTER">Cs</td>
 321     <td>Other, Surrogate</td>
 322   </tr>
 323   <tr>
 324     <td align="CENTER">Co</td>
 325     <td>Other, Private Use</td>
 326   </tr>
 327   <tr>
 328     <td align="CENTER">Cn</td>
 329     <td>Other, Not Assigned (no characters in the file have this property)</td>
 330   </tr>
 331 </table>
 332 <h4>Informative Categories</h4>
 333 <table border="0" cellspacing="2" cellpadding="0">
 334   <tr>
 335     <th>
 336       <p align="LEFT">Abbr.</th>
 337     <th>
 338       <p align="LEFT">Description</th>
 339   </tr>
 340   <tr>
 341     <td align="CENTER">Lm</td>
 342     <td>Letter, Modifier</td>
 343   </tr>
 344   <tr>
 345     <td align="CENTER">Lo</td>
 346     <td>Letter, Other</td>
 347   </tr>
 348   <tr>
 349     <td align="CENTER">Pc</td>
 350     <td>Punctuation, Connector</td>
 351   </tr>
 352   <tr>
 353     <td align="CENTER">Pd</td>
 354     <td>Punctuation, Dash</td>
 355   </tr>
 356   <tr>
 357     <td align="CENTER">Ps</td>
 358     <td>Punctuation, Open</td>
 359   </tr>
 360   <tr>
 361     <td align="CENTER">Pe</td>
 362     <td>Punctuation, Close</td>
 363   </tr>
 364   <tr>
 365     <td align="CENTER">Pi</td>
 366     <td>Punctuation, Initial quote (may behave like Ps or Pe depending on usage)</td>
 367   </tr>
 368   <tr>
 369     <td align="CENTER">Pf</td>
 370     <td>Punctuation, Final quote (may behave like Ps or Pe depending on usage)</td>
 371   </tr>
 372   <tr>
 373     <td align="CENTER">Po</td>
 374     <td>Punctuation, Other</td>
 375   </tr>
 376   <tr>
 377     <td align="CENTER">Sm</td>
 378     <td>Symbol, Math</td>
 379   </tr>
 380   <tr>
 381     <td align="CENTER">Sc</td>
 382     <td>Symbol, Currency</td>
 383   </tr>
 384   <tr>
 385     <td align="CENTER">Sk</td>
 386     <td>Symbol, Modifier</td>
 387   </tr>
 388   <tr>
 389     <td align="CENTER">So</td>
 390     <td>Symbol, Other</td>
 391   </tr>
 392 </table>
 393 <h3><a name="Bidirectional Category"></a>Bidirectional Category</h3>
 394 <p>Please refer to Chapter 3 for an explanation of the algorithm for
 395 Bidirectional Behavior and an explanation of the significance of these
 396 categories. An up-to-date version can be found on <a
 397 href="http://www.unicode.org/unicode/reports/tr9/">Unicode Standard Annex #9:
 398 The Bidirectional Algorithm</a>. These values are normative.</p>
 399 <table border="0" cellpadding="2">
 400   <tr>
 401     <th valign="TOP" align="LEFT">
 402       <p align="LEFT">Type</th>
 403     <th valign="TOP" align="LEFT">
 404       <p align="LEFT">Description</th>
 405   </tr>
 406   <tr>
 407     <td valign="TOP"><b>L</b></td>
 408     <td valign="TOP">Left-to-Right</td>
 409   </tr>
 410   <tr>
 411     <td valign="TOP"><b>LRE</b></td>
 412     <td valign="TOP">Left-to-Right Embedding</td>
 413   </tr>
 414   <tr>
 415     <td valign="TOP"><b>LRO</b></td>
 416     <td valign="TOP">Left-to-Right Override</td>
 417   </tr>
 418   <tr>
 419     <td valign="TOP"><b>R</b></td>
 420     <td valign="TOP">Right-to-Left</td>
 421   </tr>
 422   <tr>
 423     <td valign="TOP"><b>AL</b></td>
 424     <td valign="TOP">Right-to-Left Arabic</td>
 425   </tr>
 426   <tr>
 427     <td valign="TOP"><b>RLE</b></td>
 428     <td valign="TOP">Right-to-Left Embedding</td>
 429   </tr>
 430   <tr>
 431     <td valign="TOP"><b>RLO</b></td>
 432     <td valign="TOP">Right-to-Left Override</td>
 433   </tr>
 434   <tr>
 435     <td valign="TOP"><b>PDF</b></td>
 436     <td valign="TOP">Pop Directional Format</td>
 437   </tr>
 438   <tr>
 439     <td valign="TOP"><b>EN</b></td>
 440     <td valign="TOP">European Number</td>
 441   </tr>
 442   <tr>
 443     <td valign="TOP"><b>ES</b></td>
 444     <td valign="TOP">European Number Separator</td>
 445   </tr>
 446   <tr>
 447     <td valign="TOP"><b>ET</b></td>
 448     <td valign="TOP">European Number Terminator</td>
 449   </tr>
 450   <tr>
 451     <td valign="TOP"><b>AN</b></td>
 452     <td valign="TOP">Arabic Number</td>
 453   </tr>
 454   <tr>
 455     <td valign="TOP"><b>CS</b></td>
 456     <td valign="TOP">Common Number Separator</td>
 457   </tr>
 458   <tr>
 459     <td valign="TOP"><b>NSM</b></td>
 460     <td valign="TOP">Non-Spacing Mark</td>
 461   </tr>
 462   <tr>
 463     <td valign="TOP"><b>BN</b></td>
 464     <td valign="TOP">Boundary Neutral</td>
 465   </tr>
 466   <tr>
 467     <td valign="TOP"><b>B</b></td>
 468     <td valign="TOP">Paragraph Separator</td>
 469   </tr>
 470   <tr>
 471     <td valign="TOP"><b>S</b></td>
 472     <td valign="TOP">Segment Separator</td>
 473   </tr>
 474   <tr>
 475     <td valign="TOP"><b>WS</b></td>
 476     <td valign="TOP">Whitespace</td>
 477   </tr>
 478   <tr>
 479     <td valign="TOP"><b>ON</b></td>
 480     <td valign="TOP">Other Neutrals</td>
 481   </tr>
 482 </table>
 483 <h3><a name="Character Decomposition"></a>Character Decomposition Mapping</h3>
 484 <p>The decomposition is a normative property of a character. The tags supplied
 485 with certain decomposition mappings generally indicate formatting information.
 486 Where no such tag is given, the mapping is designated as canonical. Conversely,
 487 the presence of a formatting tag also indicates that the mapping is a
 488 compatibility mapping and not a canonical mapping. In the absence of other
 489 formatting information in a compatibility mapping, the tag is used to
 490 distinguish it from canonical mappings.</p>
 491 <p>In some instances a canonical mapping or a compatibility mapping may consist
 492 of a single character. For a canonical mapping, this indicates that the
 493 character is a canonical equivalent of another single character. For a
 494 compatibility mapping, this indicates that the character is a compatibility
 495 equivalent of another single character. The compatibility formatting tags used
 496 are:</p>
 497 <table border="0" cellspacing="2" cellpadding="0">
 498   <tr>
 499     <th>Tag</th>
 500     <th>
 501       <p align="LEFT">Description</th>
 502   </tr>
 503   <tr>
 504     <td align="CENTER">&lt;font&gt;&nbsp;&nbsp;</td>
 505     <td>A font variant (e.g. a blackletter form).</td>
 506   </tr>
 507   <tr>
 508     <td align="CENTER">&lt;noBreak&gt;&nbsp;&nbsp;</td>
 509     <td>A no-break version of a space or hyphen.</td>
 510   </tr>
 511   <tr>
 512     <td align="CENTER">&lt;initial&gt;&nbsp;&nbsp;</td>
 513     <td>An initial presentation form (Arabic).</td>
 514   </tr>
 515   <tr>
 516     <td align="CENTER">&lt;medial&gt;&nbsp;&nbsp;</td>
 517     <td>A medial presentation form (Arabic).</td>
 518   </tr>
 519   <tr>
 520     <td align="CENTER">&lt;final&gt;&nbsp;&nbsp;</td>
 521     <td>A final presentation form (Arabic).</td>
 522   </tr>
 523   <tr>
 524     <td align="CENTER">&lt;isolated&gt;&nbsp;&nbsp;</td>
 525     <td>An isolated presentation form (Arabic).</td>
 526   </tr>
 527   <tr>
 528     <td align="CENTER">&lt;circle&gt;&nbsp;&nbsp;</td>
 529     <td>An encircled form.</td>
 530   </tr>
 531   <tr>
 532     <td align="CENTER">&lt;super&gt;&nbsp;&nbsp;</td>
 533     <td>A superscript form.</td>
 534   </tr>
 535   <tr>
 536     <td align="CENTER">&lt;sub&gt;&nbsp;&nbsp;</td>
 537     <td>A subscript form.</td>
 538   </tr>
 539   <tr>
 540     <td align="CENTER">&lt;vertical&gt;&nbsp;&nbsp;</td>
 541     <td>A vertical layout presentation form.</td>
 542   </tr>
 543   <tr>
 544     <td align="CENTER">&lt;wide&gt;&nbsp;&nbsp;</td>
 545     <td>A wide (or zenkaku) compatibility character.</td>
 546   </tr>
 547   <tr>
 548     <td align="CENTER">&lt;narrow&gt;&nbsp;&nbsp;</td>
 549     <td>A narrow (or hankaku) compatibility character.</td>
 550   </tr>
 551   <tr>
 552     <td align="CENTER">&lt;small&gt;&nbsp;&nbsp;</td>
 553     <td>A small variant form (CNS compatibility).</td>
 554   </tr>
 555   <tr>
 556     <td align="CENTER">&lt;square&gt;&nbsp;&nbsp;</td>
 557     <td>A CJK squared font variant.</td>
 558   </tr>
 559   <tr>
 560     <td align="CENTER">&lt;fraction&gt;&nbsp;&nbsp;</td>
 561     <td>A vulgar fraction form.</td>
 562   </tr>
 563   <tr>
 564     <td align="CENTER">&lt;compat&gt;&nbsp;&nbsp;</td>
 565     <td>Otherwise unspecified compatibility character.</td>
 566   </tr>
 567 </table>
 568 <p><b>Reminder: </b>There is a difference between decomposition and
 569 decomposition mapping. The decomposition mappings are defined in the UnicodeData,
 570 while the decomposition (also termed &quot;full decomposition&quot;) is defined
 571 in Chapter 3 to use those mappings <i>recursively.</i>
 572 <ul>
 573   <li>The canonical decomposition is formed by recursively applying the
 574     canonical mappings, then applying the canonical reordering algorithm.</li>
 575   <li>The compatibility decomposition is formed by recursively applying the
 576     canonical <em>and</em> compatibility mappings, then applying the canonical
 577     reordering algorithm.</li>
 578 </ul>
 579 <h3><a name="Canonical Combining Classes"></a>Canonical Combining Classes</h3>
 580 <table border="0" cellspacing="2" cellpadding="0">
 581   <tr>
 582     <th>
 583       <p align="LEFT">Value</th>
 584     <th>
 585       <p align="LEFT">Description</th>
 586   </tr>
 587   <tr>
 588     <td align="RIGHT">0:</td>
 589     <td>Spacing, split, enclosing, reordrant, and Tibetan subjoined</td>
 590   </tr>
 591   <tr>
 592     <td align="RIGHT">1:</td>
 593     <td>Overlays and interior</td>
 594   </tr>
 595   <tr>
 596     <td align="RIGHT">7:</td>
 597     <td>Nuktas</td>
 598   </tr>
 599   <tr>
 600     <td align="RIGHT">8:</td>
 601     <td>Hiragana/Katakana voicing marks</td>
 602   </tr>
 603   <tr>
 604     <td align="RIGHT">9:</td>
 605     <td>Viramas</td>
 606   </tr>
 607   <tr>
 608     <td align="RIGHT">10:</td>
 609     <td>Start of fixed position classes</td>
 610   </tr>
 611   <tr>
 612     <td align="RIGHT">199:</td>
 613     <td>End of fixed position classes</td>
 614   </tr>
 615   <tr>
 616     <td align="RIGHT">200:</td>
 617     <td>Below left attached</td>
 618   </tr>
 619   <tr>
 620     <td align="RIGHT">202:</td>
 621     <td>Below attached</td>
 622   </tr>
 623   <tr>
 624     <td align="RIGHT">204:</td>
 625     <td>Below right attached</td>
 626   </tr>
 627   <tr>
 628     <td align="RIGHT">208:</td>
 629     <td>Left attached (reordrant around single base character)</td>
 630   </tr>
 631   <tr>
 632     <td align="RIGHT">210:</td>
 633     <td>Right attached</td>
 634   </tr>
 635   <tr>
 636     <td align="RIGHT">212:</td>
 637     <td>Above left attached</td>
 638   </tr>
 639   <tr>
 640     <td align="RIGHT">214:</td>
 641     <td>Above attached</td>
 642   </tr>
 643   <tr>
 644     <td align="RIGHT">216:</td>
 645     <td>Above right attached</td>
 646   </tr>
 647   <tr>
 648     <td align="RIGHT">218:</td>
 649     <td>Below left</td>
 650   </tr>
 651   <tr>
 652     <td align="RIGHT">220:</td>
 653     <td>Below</td>
 654   </tr>
 655   <tr>
 656     <td align="RIGHT">222:</td>
 657     <td>Below right</td>
 658   </tr>
 659   <tr>
 660     <td align="RIGHT">224:</td>
 661     <td>Left (reordrant around single base character)</td>
 662   </tr>
 663   <tr>
 664     <td align="RIGHT">226:</td>
 665     <td>Right</td>
 666   </tr>
 667   <tr>
 668     <td align="RIGHT">228:</td>
 669     <td>Above left</td>
 670   </tr>
 671   <tr>
 672     <td align="RIGHT">230:</td>
 673     <td>Above</td>
 674   </tr>
 675   <tr>
 676     <td align="RIGHT">232:</td>
 677     <td>Above right</td>
 678   </tr>
 679   <tr>
 680     <td align="RIGHT">233:</td>
 681     <td>Double below</td>
 682   </tr>
 683   <tr>
 684     <td align="RIGHT">234:</td>
 685     <td>Double above</td>
 686   </tr>
 687   <tr>
 688     <td align="RIGHT">240:</td>
 689     <td>Below (iota subscript)</td>
 690   </tr>
 691 </table>
 692 <p><strong>Note: </strong>some of the combining classes in this list do not
 693 currently have members but are specified here for completeness.</p>
 694 <h3><a name="Decompositions and Normalization"></a>Decompositions and
 695 Normalization</h3>
 696 <p>Decomposition is specified in Chapter 3. <a
 697 href="http://www.unicode.org/unicode/reports/tr15/"><i>Unicode Standard Annex
 698 #15: Unicode Normalization Forms</i></a> specifies the interaction between decomposition
 699 and normalization. The most up-to-date version is found on <a
 700 href="http://www.unicode.org/unicode/reports/tr15/">http://www.unicode.org/unicode/reports/tr15/</a>.
 701 That report specifies how the decompositions defined in UnicodeData.txt are used
 702 to derive normalized forms of Unicode text.</p>
 703 <p>Note that as of the 2.1.9 update of the Unicode Character Database, the
 704 decompositions in the UnicodeData.txt file can be used to recursively derive the
 705 full decomposition in canonical order, without the need to separately apply
 706 canonical reordering. However, canonical reordering of combining character
 707 sequences must still be applied in decomposition when normalizing source text
 708 which contains any combining marks.</p>
 709 <h3><a name="Case Mappings"></a>Case Mappings</h3>
 710 <p>The case mapping is an informative, default mapping. Case itself, on the
 711 other hand, has normative status. Thus, for example, 0041 LATIN CAPITAL LETTER A
 712 is normatively uppercase, but its lowercase mapping the 0061 LATIN SMALL LETTER
 713 A is informative. The reason for this is that case can be considered to be an
 714 inherent property of a particular character (and is usually, but not always,
 715 derivable from the presence of the terms &quot;CAPITAL&quot; or
 716 &quot;SMALL&quot; in the character name), but case mappings between characters
 717 are occasionally influenced by local conventions. For example, certain
 718 languages, such as Turkish, German, French, or Greek may have small deviations
 719 from the default mappings listed in UnicodeData.</p>
 720 <p>In addition to uppercase and lowercase, because of the inclusion of certain
 721 composite characters for compatibility, such as 01F1 LATIN CAPITAL LETTER DZ,
 722 there is a third case, called <i>titlecase</i>, which is used where the first
 723 letter of a word is to be capitalized (e.g. UPPERCASE, Titlecase, lowercase). An
 724 example of such a titlecase letter is 01F2 LATIN CAPITAL LETTER D WITH SMALL
 725 LETTER Z.</p>
 726 <p>The uppercase, titlecase and lowercase fields are only included for
 727 characters that have a single corresponding character of that type. Composite
 728 characters (such as &quot;339D SQUARE CM&quot;) that do not have a single
 729 corresponding character of that type can be cased by decomposition.</p>
 730 <p>For compatibility with existing parsers, UnicodeData only contains case
 731 mappings for characters where they are one-to-one mappings; it also omits
 732 information about context-sensitive case mappings. Information about these
 733 special cases can be found in a separate data file, <a
 734 href="http://www.unicode.org/Public/3.0-Update1/">SpecialCasing.txt</a>, which
 735 has been added starting with the 2.1.8 update to the Unicode data files.
 736 SpecialCasing.txt contains additional informative case mappings that are either
 737 not one-to-one or which are context-sensitive.</p>
 738 <h2><a name="Property Invariants"></a>Property Invariants</h2>
 739 <p>Values in UnicodeData.txt are subject to correction as errors are found;
 740 however, some characteristics of the categories themselves can be considered
 741 invariants. Applications may wish to take these invariants into account when
 742 choosing how to implement character properties. The following is a partial list
 743 of known invariants for the Unicode Character Database.</p>
 744 <h4>Database Fields</h4>
 745 <ul>
 746   <li>The number of fields in UnicodeData.txt is fixed.</li>
 747   <li>The order of the fields is also fixed.
 748     <ul>
 749       <li>Any additional information about character properties to be added in
 750         the future will appear in separate data tables, rather than being added
 751         on to the existing table or by subdivision or reinterpretation of
 752         existing fields.</li>
 753     </ul>
 754   </li>
 755 </ul>
 756 <h4>General Category</h4>
 757 <ul>
 758   <li>There will never be more than 32 General Category values.
 759     <ul>
 760       <li>It is very unlikely that the Unicode Technical Committee will
 761         subdivide the General Category partition any further, since that can
 762         cause implementations to misbehave. Because the General Category is
 763         limited to 32 values, 5 bits can be used to represent the information,
 764         and a 32-bit integer can be used as a bitmask to represent arbitrary
 765         sets of categories.</li>
 766     </ul>
 767   </li>
 768 </ul>
 769 <h4>Combining Classes</h4>
 770 <ul>
 771   <li>Combining classes are limited to the values 0 to 255.
 772     <ul>
 773       <li>In practice, there are far fewer than 256 values used. Implementations
 774         may take advantage of this fact for compression, since only the ordering
 775         of the non-zero values matters for the Canonical Reordering Algorithm.
 776         It is possible for up to 256 values to be used in the future; however,
 777         UTC decisions in the future may restrict the number of values to 128,
 778         since this has implementation advantages. [Signed bytes can be used
 779         without widening to ints in Java, for example.]</li>
 780     </ul>
 781   </li>
 782   <li>All characters other than those of General Category M* have the combining
 783     class 0.
 784     <ul>
 785       <li>Currently, all characters other than those of General Category Mn have
 786         the value 0. However, some characters of General Category Me or Mc may
 787         be given non-zero values in the future.</li>
 788       <li>The precise values above the value 0 are not invariant--only the
 789         relative ordering is considered normative. For example, it is not
 790         guaranteed in future versions that the class of U+05B4 will be precisely
 791         14.</li>
 792     </ul>
 793   </li>
 794 </ul>
 795 <h4>Case</h4>
 796 <ul>
 797   <li>Characters of type Lu, Lt, or Ll are called <i>cased</i>. All characters
 798     with an Upper, Lower, or Titlecase mapping are cased characters.
 799     <ul>
 800       <li>However, characters with the General Categories of Lu, Ll, or Lt may
 801         not always have case mappings, and case mappings may vary by locale.
 802         (See http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt).</li>
 803     </ul>
 804   </li>
 805 </ul>
 806 <h4>Canonical Decomposition</h4>
 807 <ul>
 808   <li>Canonical mappings are always in canonical order.</li>
 809   <li>Canonical mappings have only the first of a pair possibly further
 810     decomposing.</li>
 811   <li>Canonical decompositions are &quot;transparent&quot; to other character
 812     data:
 813     <ul>
 814       <li><tt>BIDI(a) = BIDI(principal(canonicalDecomposition(a))</tt></li>
 815       <li><tt>Category(a) = Category(principal(canonicalDecomposition(a))</tt></li>
 816       <li><tt>CombiningClass(a) =
 817         CombiningClass(principal(canonicalDecomposition(a))</tt><br>
 818         where principal(a) is the first character not of type Mn, or the first
 819         character if all characters are of type Mn.</li>
 820     </ul>
 821   </li>
 822   <li>However, because there are sometimes missing case pairs, and because of
 823     some legacy characters, it is only generally true that:
 824     <ul>
 825       <li><tt>upper(canonicalDecomposition(a)) = canonicalDecomposition(upper(a))</tt></li>
 826       <li><tt>lower(canonicalDecomposition(a)) = canonicalDecomposition(lower(a))</tt></li>
 827       <li><tt>title(canonicalDecomposition(a)) = canonicalDecomposition(title(a))</tt></li>
 828     </ul>
 829   </li>
 830 </ul>
 831 <h2><a name="Modification History"></a>Modification History</h2>
 832 <p>This section provides a summary of the changes between update versions of the
 833 Unicode Standard.</p>
 834 <h3><a
 835 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 3.0.1">Unicode
 836 3.0.1</a></h3>
 837 <p>Modifications made for Version 3.0.1 of UnicodeData.txt include:
 838 <ul>
 839   <li>Added 5- and 6-digit representation of code points past U+FFFF.</li>
 840   <li>Added Private Use range definitions for Planes 15 and 16.</li>
 841   <li>Minor additions for the 10646 comment field.</li>
 842 </ul>
 843 <h3><a
 844 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 3.0.0">Unicode
 845 3.0.0</a></h3>
 846 <p>Modifications made for Version 3.0.0 of UnicodeData.txt include many new
 847 characters and a number of property changes. These are summarized in Appendex D
 848 of <em>The Unicode Standard, Version 3.0.</em></p>
 849 <h3><a
 850 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.9">Unicode
 851 2.1.9</a></h3>
 852 <p>Modifications made for Version 2.1.9 of UnicodeData.txt include:
 853 <ul>
 854   <li>Corrected combining class for U+05AE HEBREW ACCENT ZINOR.</li>
 855   <li>Corrected combining class for U+20E1 COMBINING LEFT RIGHT ARROW ABOVE</li>
 856   <li>Corrected combining class for U+0F35 and U+0F37 to 220.</li>
 857   <li>Corrected combining class for U+0F71 to 129.</li>
 858   <li>Added a decomposition for U+0F0C TIBETAN MARK DELIMITER TSHEG BSTAR.</li>
 859   <li>Added&nbsp; decompositions for several Greek symbol letters:
 860     U+03D0..U+03D2, U+03D5, U+03D6, U+03F0..U+03F2.</li>
 861   <li>Removed&nbsp; decompositions from the conjoining jamo block:
 862     U+1100..U+11F8.</li>
 863   <li>Changes to decomposition mappings for some Tibetan vowels for consistency
 864     in normalization. (U+0F71, U+0F73, U+0F77, U+0F79, U+0F81)</li>
 865   <li>Updated the decomposition mappings for several Vietnamese characters with
 866     two diacritics (U+1EAC, U+1EAD, U+1EB6, U+1EB7, U+1EC6, U+1EC7, U+1ED8,
 867     U+1ED9), so that the recursive decomposition can be generated directly in
 868     canonically reordered form (not a normative change).</li>
 869   <li>Updated the decomposition mappings for several Arabic compatibility
 870     characters involving shadda (U+FC5E..U+FC62, U+FCF2..U+FCF4), and two Latin
 871     characters (U+1E1C, U+1E1D), so that the decompositions are generated
 872     directly in canonically reordered form (not a normative change).</li>
 873   <li>Changed BIDI category for: U+00A0 NO-BREAK SPACE, U+2007 FIGURE SPACE,
 874     U+2028 LINE SEPARATOR.</li>
 875   <li>Changed BIDI category for extenders of General Category Lm: U+3005,
 876     U+3021..U+3035, U+FF9E, U+FF9F.</li>
 877   <li>Changed General Category and BIDI category for the Greek numeral signs:
 878     U+0374, U+0375.</li>
 879   <li>Corrected General Category for U+FFE8 HALFWIDTH FORMS LIGHT VERTICAL.</li>
 880   <li>Added Unicode 1.0 names for many Tibetan characters (informative).</li>
 881 </ul>
 882 <h3><a
 883 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.8">Unicode
 884 2.1.8</a></h3>
 885 <p>Modifications made for Version 2.1.8 of UnicodeData.txt include:
 886 <ul>
 887   <li>Added combining class 240 for U+0345 COMBINING GREEK YPOGEGRAMMENI so that
 888     decompositions involving iota subscript are derivable directly in
 889     canonically reordered form; this also has a bearing on simplification of
 890     casing of polytonic Greek.</li>
 891   <li>Changes in decompositions related to Greek tonos. These result from the
 892     clarification that monotonic Greek &quot;tonos&quot; should be equated with
 893     U+0301 COMBINING ACUTE, rather than with U+030D COMBINING VERTICAL LINE
 894     ABOVE. (All Greek characters in the Greek block involving &quot;tonos&quot;;
 895     some Greek characters in the polytonic Greek in the 1FXX block.)</li>
 896   <li>Changed decompositions involving dialytika tonos. (U+0390, U+03B0)</li>
 897   <li>Changed ternary decompositions to binary. (U+0CCB, U+FB2C, U+FB2D) These
 898     changes simplify normalization.</li>
 899   <li>Removed canonical decomposition for Latin Candrabindu. (U+0310)</li>
 900   <li>Corrected error in canonical decomposition for U+1FF4.</li>
 901   <li>Added compatibility decompositions to clarify collation tables. (U+2100,
 902     U+2101, U+2105, U+2106, U+1E9A)</li>
 903   <li>A series of general category changes to assist the convergence of of
 904     Unicode definition of identifier with ISO TR 10176:
 905     <ul>
 906       <li>So &gt; Lo: U+0950, U+0AD0, U+0F00, U+0F88..U+0F8B</li>
 907       <li>Po &gt; Lo: U+0E2F, U+0EAF, U+3006</li>
 908       <li>Lm &gt; Sk: U+309B, U+309C</li>
 909       <li>Po &gt; Pc: U+30FB, U+FF65</li>
 910       <li>Ps/Pe &gt; Mn: U+0F3E, U+0F3F</li>
 911     </ul>
 912   </li>
 913   <li>A series of bidi property changes for consistency.
 914     <ul>
 915       <li>L &gt; ET: U+09F2, U+09F3</li>
 916       <li>ON &gt; L: U+3007</li>
 917       <li>L &gt; ON: U+0F3A..U+0F3D, U+037E, U+0387</li>
 918     </ul>
 919   </li>
 920   <li>Add case mapping: U+01A6 &lt;-&gt; U+0280</li>
 921   <li>Updated symmetric swapping value for guillemets: U+00AB, U+00BB, U+2039,
 922     U+203A.</li>
 923   <li>Changes to combining class values. Most Indic fixed position class
 924     non-spacing marks were changed to combining class 0. This fixes some
 925     inconsistencies in how canonical reordering would apply to Indic scripts,
 926     including Tibetan. Indic interacting top/bottom fixed position classes were
 927     merged into single (non-zero) classes as part of this change. Tibetan
 928     subjoined consonants are changed from combining class 6 to combining class
 929     0. Thai pinthu (U+0E3A) moved to combining class 9. Moved two Devanagari
 930     stress marks into generic above and below combining classes (U+0951,
 931     U+0952).</li>
 932   <li>Corrected placement of semicolon near symmetric swapping field. (U+FA0E,
 933     etc., scattered positions to U+FA29)</li>
 934 </ul>
 935 <h3>Version 2.1.7</h3>
 936 <p><i>This version was for internal change tracking only, and never publicly
 937 released.</i></p>
 938 <h3>Version 2.1.6</h3>
 939 <p><i>This version was for internal change tracking only, and never publicly
 940 released.</i></p>
 941 <h3><a
 942 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.5">Unicode
 943 2.1.5</a></h3>
 944 <p>Modifications made for Version 2.1.5 of UnicodeData.txt include:
 945 <ul>
 946   <li>Changed decomposition for U+FF9E and U+FF9F so that correct collation
 947     weighting will automatically result from the canonical equivalences.</li>
 948   <li>Removed canonical decompositions for U+04D4, U+04D5, U+04D8, U+04D9,
 949     U+04E0, U+04E1, U+04E8, U+04E9 (the implication being that no canonical
 950     equivalence is claimed between these 8 characters and similar Latin
 951     letters), and updated 4 canonical decompositions for U+04DB, U+04DC, U+04EA,
 952     U+04EB to reflect the implied difference in the base character.</li>
 953   <li>Added Pi, and Pf categories and assigned the relevant quotation marks to
 954     those categories, based on the Unicode Technical Corrigendum on Quotation
 955     Characters.</li>
 956   <li>Updating of many bidi properties, following the advice of the ad hoc
 957     committee on bidi, and to make the bidi properties of compatibility
 958     characters more consistent.</li>
 959   <li>Changed category of several Tibetan characters: U+0F3E, U+0F3F,
 960     U+0F88..U+0F8B to make them non-combining, reflecting the combined opinion
 961     of Tibetan experts.</li>
 962   <li>Added case mapping for U+03F2.</li>
 963   <li>Corrected case mapping for U+0275.</li>
 964   <li>Added titlecase mappings for U+03D0, U+03D1, U+03D5, U+03D6, U+03F0..
 965     U+03F2.</li>
 966   <li>Corrected compatibility label for U+2121.</li>
 967   <li>Add specific entries for all the CJK compatibility ideographs,
 968     U+F900..U+FA2D, so the canonical decomposition for each (the URO character
 969     it is equivalent to) can be carried in the database.</li>
 970 </ul>
 971 <h3>Version 2.1.4</h3>
 972 <p><i>This version was for internal change tracking only, and never publicly
 973 released.</i></p>
 974 <h3>Version 2.1.3</h3>
 975 <p><i>This version was for internal change tracking only, and never publicly
 976 released.</i></p>
 977 <h3><a
 978 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.2">Unicode
 979 2.1.2</a></h3>
 980 <p>Modifications made in updating UnicodeData.txt to Version 2.1.2 for the
 981 Unicode Standard, Version 2.1 (from Version 2.0) include:
 982 <ul>
 983   <li>Added two characters (U+20AC and U+FFFC).</li>
 984   <li>Amended bidi properties for U+0026, U+002E, U+0040, U+2007.</li>
 985   <li>Corrected case mappings for U+018E, U+019F, U+01DD, U+0258, U+0275,
 986     U+03C2, U+1E9B.</li>
 987   <li>Changed combining order class for U+0F71.</li>
 988   <li>Corrected canonical decompositions for U+0F73, U+1FBE.</li>
 989   <li>Changed decomposition for U+FB1F from compatibility to canonical.</li>
 990   <li>Added compatibility decompositions for U+FBE8, U+FBE9, U+FBF9..U+FBFB.</li>
 991   <li>Corrected compatibility decompositions for U+2469, U+246A, U+3358.</li>
 992 </ul>
 993 <h3>Version 2.1.1</h3>
 994 <p><i>This version was for internal change tracking only, and never publicly
 995 released.</i></p>
 996 <h3><a
 997 href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.0.0">Unicode
 998 2.0.0</a></h3>
 999 <p>The modifications made in updating UnicodeData.txt for the Unicode Standard,
1000 Version 2.0 include:
1001 <ul>
1002   <li>Fixed decompositions with TONOS to use correct NSM: 030D.</li>
1003   <li>Removed old Hangul Syllables; mapping to new characters are in a separate
1004     table.</li>
1005   <li>Marked compatibility decompositions with additional tags.</li>
1006   <li>Changed old tag names for clarity.</li>
1007   <li>Revision of decompositions to use first-level decomposition, instead of
1008     maximal decomposition.</li>
1009   <li>Correction of all known errors in decompositions from earlier versions.</li>
1010   <li>Added control code names (as old Unicode names).</li>
1011   <li>Added Hangul Jamo decompositions.</li>
1012   <li>Added Number category to match properties list in book.</li>
1013   <li>Fixed categories of Koranic Arabic marks.</li>
1014   <li>Fixed categories of precomposed characters to match decomposition where
1015     possible.</li>
1016   <li>Added Hebrew cantillation marks and the Tibetan script.</li>
1017   <li>Added place holders for ranges such as CJK Ideographic Area and the
1018     Private Use Area.</li>
1019   <li>Added categories Me, Sk, Pc, Nl, Cs, Cf, and rectified a number of
1020     mistakes in the database.</li>
1021 </ul>
1022
1023 </body>
1024
1025 </html>