external/bsd/atf/dist/doc/standalone/roadmap.html

   1 <?xml version="1.0"?>
   2 <html xmlns="http://www.w3.org/1999/xhtml">
   3 <head>
   4 <meta xmlns="" http-equiv="Content-Type"
   5 content="text/html; charset=iso-8859-1" />
   6 <link xmlns="" rel="made"
   7 href="mailto:atf-devel AT NetBSD DOT org" />
   8 <link xmlns="" rel="stylesheet" type="text/css"
   9 href="standalone.css" />
  10 <title xmlns="">Rearchitecting ATF: Road trip to 1.0</title>
  11 </head>
  12 <body>
  13 <div xmlns="" class="header">
  14 <p class="title">Rearchitecting ATF: Road trip to 1.0</p>
  15 <p class="author">By Julio Merino, The NetBSD Foundation</p>
  16 </div>
  17 <div xmlns="" class="toc">
  18 <h1>Contents</h1>
  19 <ol>
  20 <li>
  21 <p>
  22 <a href="#overview">Overview</a>
  23 </p>
  24 </li>
  25 <li>
  26 <p>
  27 <a href="#plan-0-8">The plan to 0.8</a>
  28 </p>
  29 </li>
  30 <li>
  31 <p>
  32 <a href="#plan-0-9">The plan to 0.9</a>
  33 </p>
  34 </li>
  35 <li>
  36 <p>
  37 <a href="#plan-pre-1-0">The plan to 1.0 prereleases</a>
  38 </p>
  39 </li>
  40 <li>
  41 <p>
  42 <a href="#plan-1.0-rc">The plan to 1.0 release candidates</a>
  43 </p>
  44 </li>
  45 <li>
  46 <p>
  47 <a href="#plan-unknown">The plan to releases after 1.0</a>
  48 </p>
  49 </li>
  50 </ol>
  51 </div>
  52 <div xmlns="" class="contents">
  53 <h1>
  54 <a name="overview">Overview</a>
  55 </h1>
  56 <p xml:space="preserve">
  57 This document describes the major steps to be taken before getting
  58 to the first stable release of ATF, 1.0.  Some of these steps
  59 describe
  60 the transition from the old code base to the new ideas documented
  61 in the
  62 specification (TODO: Add link somehow).
  63 </p>
  64 <h1>
  65 <a name="plan-0-8">The plan to 0.8</a>
  66 </h1>
  67 <ul>
  68 <li>
  69 <p xml:space="preserve">
  70 Add atf-sh interface to build-time tests.
  71 </p>
  72 </li>
  73 <li>
  74 <p xml:space="preserve">
  75 Properly document the libraries: i.e. one page per
  76 module, detailed information of each function and type, etc.  At
  77 the
  78 very least atf-c.
  79 </p>
  80 </li>
  81 <li>
  82 <p xml:space="preserve">
  83 Add a tool to collect multiple outputs of atf-run (from
  84 different machines) and generate a single (XML-only?) log with
  85 everything.  Must allow easy conversion to HTML for online
  86 publishing.
  87 </p>
  88 </li>
  89 <li>
  90 <p xml:space="preserve">
  91 Allow grouping of test programs in tiers in an Atffile.
  92 This is to permit the user specify "dependencies" between test
  93 programs:
  94 e.g. do not run a specific test program if its dependencies have
  95 failed,
  96 because it will certainly fail also.
  97 </p>
  98 </li>
  99 <li>
 100 <p xml:space="preserve">
 101 Provide a kernel-level unit testing API (for NetBSD
 102 only, at the moment).  This should come in the form of an atf.ko
 103 module
 104 that provides functions to define and register test cases,
 105 functions for
 106 results reporting and an interface (a trivial file system?) that
 107 transports the application/X-atf-tcs output to user-space, provides
 108 information to user-space about the available test cases (a list)
 109 and
 110 allows user-space to launch the execution of test
 111 cases.
 112 </p>
 113 </li>
 114 </ul>
 115 <h1>
 116 <a name="plan-0-9">The plan to 0.9</a>
 117 </h1>
 118 <ul>
 119 <li>
 120 <p xml:space="preserve">
 121 Add a module to atf-c to manage dynamic memory.  Should
 122 provide a "mem chunk" object that can only be managed through
 123 functions
 124 (i.e. not directly) so that access to buffers can be safely
 125 controlled.
 126 Dealing with strdup and similar functions, for example, might be
 127 complex.
 128 </p>
 129 <p xml:space="preserve">
 130 See these old revisions for a start, but these did not work
 131 correctly because the use of (void **) casts brought aliasing
 132 problems:
 133 </p>
 134 <p xml:space="preserve">
 135 78eeacf3b9284493e5e96b7866fc1d163a13bc02
 136 8e200683a2b70eadca728c2e0347d4790777b3fc
 137 872393ed0177cbcc30ffacede228eb3986c42ab7
 138 </p>
 139 </li>
 140 </ul>
 141 <h1>
 142 <a name="plan-pre-1-0">The plan to 1.0 prereleases</a>
 143 </h1>
 144 <ul>
 145 <li>
 146 <p xml:space="preserve">
 147 Fix all occurrences of XXX, TODO and
 148 FIXME.
 149 </p>
 150 </li>
 151 <li>
 152 <p xml:space="preserve">
 153 Split the distfile into multiple components.  We should
 154 have a component for each language binding and a component
 155 providing the
 156 ATF tools, at the very least.  If we had this, external programs
 157 using
 158 ATF wouldn't need to depend on the tools and/or the C++ binding,
 159 because
 160 they could just require the user to build the atf-c
 161 binding.
 162 </p>
 163 </li>
 164 <li>
 165 <p xml:space="preserve">
 166 Think of a way to properly add tests for (almost?) all
 167 error paths.  Most of them are probably broken
 168 already.
 169 </p>
 170 </li>
 171 <li>
 172 <p xml:space="preserve">
 173 Improve error reporting: aside from clarifying error
 174 messages, this also implies adding more error cases to give them
 175 more
 176 semantic meaning at the source code level..
 177 </p>
 178 </li>
 179 <li>
 180 <p xml:space="preserve">
 181 Make the shell library work with 'set
 182 -e'?
 183 </p>
 184 </li>
 185 <li>
 186 <p xml:space="preserve">
 187 Shell test programs dynamically locate where the shell
 188 library is by calling atf-config (done by atf.init.subr).
 189 Contrarywise,
 190 binary test programs are directly linked against the final location
 191 of
 192 libatf.  It may be nice if the latter loaded the library
 193 dynamically
 194 based on what atf-config said, so the user could switch atf
 195 installations by simply changing its PATH (and effectively making
 196 atf
 197 relocatable on the file system).  Why could this be nice?  To
 198 painlessly
 199 run an older atf test suite against a more recent version of the
 200 code
 201 base to ensure there are no regressions even with older tests.
 202 Just a
 203 crazy idea, as maybe what the shell test programs currently do is
 204 stupid.
 205 </p>
 206 </li>
 207 <li>
 208 <p xml:space="preserve">
 209 Allow users to customize the build of atf by defining
 210 additional meta-data for test cases.  At the moment this is
 211 possible
 212 because the meta-data is not sanity-checked, but I think it should
 213 be.
 214 Following the previous item, NetBSD could add a 'netbsd.pr'
 215 variable and
 216 then use this data when generating reports to add direct links to
 217 the
 218 appropriate PRs.
 219 </p>
 220 </li>
 221 <li>
 222 <p xml:space="preserve">
 223 Make sure that the tests in tests/atf have, at the very
 224 least, the same coverage as the ones in
 225 tests/bootstrap.
 226 </p>
 227 </li>
 228 <li>
 229 <p xml:space="preserve">
 230 Document the code.
 231 </p>
 232 </li>
 233 <li>
 234 <p xml:space="preserve">
 235 Possibly add a way to automatically gain or drop
 236 privileges when require.user is set.
 237 </p>
 238 </li>
 239 <li>
 240 <p xml:space="preserve">
 241 Add a way to specify which bug/issue/whatever a given
 242 test case is stress-testing.  This information is useful when
 243 detecting
 244 regressions.
 245 </p>
 246 </li>
 247 </ul>
 248 <h1>
 249 <a name="plan-1.0-rc">The plan to 1.0 release candidates</a>
 250 </h1>
 251 <ul>
 252 <li>
 253 <p xml:space="preserve">
 254 Build libatf as a shared library and set -version-info
 255 accordingly.
 256 </p>
 257 </li>
 258 <li>
 259 <p xml:space="preserve">
 260 Set the DTDs' versions to 1.0.
 261 </p>
 262 </li>
 263 </ul>
 264 <h1>
 265 <a name="plan-unknown">The plan to releases after 1.0</a>
 266 </h1>
 267 <ul>
 268 <li>
 269 <p xml:space="preserve">
 270 Allow the parallel execution of tests.  Achieving this
 271 with a test program granularity is easy: only need to change
 272 atf-run.
 273 Lowering it to a finer granularity (test cases) is harder and maybe
 274 not
 275 worth it.
 276 </p>
 277 </li>
 278 </ul>
 279 </div>
 280 </body>
 281 </html>