mingw/info/gdbint/Testsuite.html

   1 <html lang="en">
   2 <head>
   3 <title>GDB Internals</title>
   4 <meta http-equiv="Content-Type" content="text/html">
   5 <meta name="description" content="GDB Internals">
   6 <meta name="generator" content="makeinfo 4.3">
   7 <link href="http://www.gnu.org/software/texinfo/" rel="generator-home">
   8 </head>
   9 <body>
  10 <div class="node">
  11 <p>
  12 Node:<a name="Testsuite">Testsuite</a>,
  13 Next:<a rel="next" accesskey="n" href="Hints.html#Hints">Hints</a>,
  14 Previous:<a rel="previous" accesskey="p" href="Releasing-GDB.html#Releasing%20GDB">Releasing GDB</a>,
  15 Up:<a rel="up" accesskey="u" href="index.html#Top">Top</a>
  16 <hr><br>
  17 </div>
  18
  19 <h2 class="chapter">Testsuite</h2>
  20
  21    <p>The testsuite is an important component of the GDB package.
  22 While it is always worthwhile to encourage user testing, in practice
  23 this is rarely sufficient; users typically use only a small subset of
  24 the available commands, and it has proven all too common for a change
  25 to cause a significant regression that went unnoticed for some time.
  26
  27    <p>The GDB testsuite uses the DejaGNU testing framework.
  28 DejaGNU is built using <code>Tcl</code> and <code>expect</code>.  The tests
  29 themselves are calls to various <code>Tcl</code> procs; the framework runs all the
  30 procs and summarizes the passes and fails.
  31
  32 <h3 class="section">Using the Testsuite</h3>
  33
  34    <p>To run the testsuite, simply go to the GDB object directory (or to the
  35 testsuite's objdir) and type <code>make check</code>.  This just sets up some
  36 environment variables and invokes DejaGNU's <code>runtest</code> script.  While
  37 the testsuite is running, you'll get mentions of which test file is in use,
  38 and a mention of any unexpected passes or fails.  When the testsuite is
  39 finished, you'll get a summary that looks like this:
  40
  41 <pre class="example">                     === gdb Summary ===
  42
  43      # of expected passes            6016
  44      # of unexpected failures        58
  45      # of unexpected successes       5
  46      # of expected failures          183
  47      # of unresolved testcases       3
  48      # of untested testcases         5
  49      </pre>
  50
  51    <p>The ideal test run consists of expected passes only; however, reality
  52 conspires to keep us from this ideal.  Unexpected failures indicate
  53 real problems, whether in GDB or in the testsuite.  Expected
  54 failures are still failures, but ones which have been decided are too
  55 hard to deal with at the time; for instance, a test case might work
  56 everywhere except on AIX, and there is no prospect of the AIX case
  57 being fixed in the near future.  Expected failures should not be added
  58 lightly, since you may be masking serious bugs in GDB.
  59 Unexpected successes are expected fails that are passing for some
  60 reason, while unresolved and untested cases often indicate some minor
  61 catastrophe, such as the compiler being unable to deal with a test
  62 program.
  63
  64    <p>When making any significant change to GDB, you should run the
  65 testsuite before and after the change, to confirm that there are no
  66 regressions.  Note that truly complete testing would require that you
  67 run the testsuite with all supported configurations and a variety of
  68 compilers; however this is more than really necessary.  In many cases
  69 testing with a single configuration is sufficient.  Other useful
  70 options are to test one big-endian (Sparc) and one little-endian (x86)
  71 host, a cross config with a builtin simulator (powerpc-eabi,
  72 mips-elf), or a 64-bit host (Alpha).
  73
  74    <p>If you add new functionality to GDB, please consider adding
  75 tests for it as well; this way future GDB hackers can detect
  76 and fix their changes that break the functionality you added.
  77 Similarly, if you fix a bug that was not previously reported as a test
  78 failure, please add a test case for it.  Some cases are extremely
  79 difficult to test, such as code that handles host OS failures or bugs
  80 in particular versions of compilers, and it's OK not to try to write
  81 tests for all of those.
  82
  83 <h3 class="section">Testsuite Organization</h3>
  84
  85    <p>The testsuite is entirely contained in <code>gdb/testsuite</code>.  While the
  86 testsuite includes some makefiles and configury, these are very minimal,
  87 and used for little besides cleaning up, since the tests themselves
  88 handle the compilation of the programs that GDB will run.  The file
  89 <code>testsuite/lib/gdb.exp</code> contains common utility procs useful for
  90 all GDB tests, while the directory <code>testsuite/config</code> contains
  91 configuration-specific files, typically used for special-purpose
  92 definitions of procs like <code>gdb_load</code> and <code>gdb_start</code>.
  93
  94    <p>The tests themselves are to be found in <code>testsuite/gdb.*</code> and
  95 subdirectories of those.  The names of the test files must always end
  96 with <code>.exp</code>.  DejaGNU collects the test files by wildcarding
  97 in the test directories, so both subdirectories and individual files
  98 get chosen and run in alphabetical order.
  99
 100    <p>The following table lists the main types of subdirectories and what they
 101 are for.  Since DejaGNU finds test files no matter where they are
 102 located, and since each test file sets up its own compilation and
 103 execution environment, this organization is simply for convenience and
 104 intelligibility.
 105
 106      <dl>
 107 <dt><code>gdb.base</code>
 108      <dd>This is the base testsuite.  The tests in it should apply to all
 109 configurations of GDB (but generic native-only tests may live here).
 110 The test programs should be in the subset of C that is valid K&amp;R,
 111 ANSI/ISO, and C++ (<code>#ifdef</code>s are allowed if necessary, for instance
 112 for prototypes).
 113
 114      <br><dt><code>gdb.</code><var>lang</var><code></code>
 115      <dd>Language-specific tests for any language <var>lang</var> besides C.  Examples are
 116 <code>gdb.c++</code> and <code>gdb.java</code>.
 117
 118      <br><dt><code>gdb.</code><var>platform</var><code></code>
 119      <dd>Non-portable tests.  The tests are specific to a specific configuration
 120 (host or target), such as HP-UX or eCos.  Example is <code>gdb.hp</code>, for
 121 HP-UX.
 122
 123      <br><dt><code>gdb.</code><var>compiler</var><code></code>
 124      <dd>Tests specific to a particular compiler.  As of this writing (June
 125 1999), there aren't currently any groups of tests in this category that
 126 couldn't just as sensibly be made platform-specific, but one could
 127 imagine a <code>gdb.gcc</code>, for tests of GDB's handling of GCC
 128 extensions.
 129
 130      <br><dt><code>gdb.</code><var>subsystem</var><code></code>
 131      <dd>Tests that exercise a specific GDB subsystem in more depth.  For
 132 instance, <code>gdb.disasm</code> exercises various disassemblers, while
 133 <code>gdb.stabs</code> tests pathways through the stabs symbol reader.
 134 </dl>
 135
 136 <h3 class="section">Writing Tests</h3>
 137
 138    <p>In many areas, the GDB tests are already quite comprehensive; you
 139 should be able to copy existing tests to handle new cases.
 140
 141    <p>You should try to use <code>gdb_test</code> whenever possible, since it
 142 includes cases to handle all the unexpected errors that might happen.
 143 However, it doesn't cost anything to add new test procedures; for
 144 instance, <code>gdb.base/exprs.exp</code> defines a <code>test_expr</code> that
 145 calls <code>gdb_test</code> multiple times.
 146
 147    <p>Only use <code>send_gdb</code> and <code>gdb_expect</code> when absolutely
 148 necessary, such as when GDB has several valid responses to a command.
 149
 150    <p>The source language programs do <em>not</em> need to be in a consistent
 151 style.  Since GDB is used to debug programs written in many different
 152 styles, it's worth having a mix of styles in the testsuite; for
 153 instance, some GDB bugs involving the display of source lines would
 154 never manifest themselves if the programs used GNU coding style
 155 uniformly.
 156
 157    </body></html>
 158