www/build-farm.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   2 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml">
   4 <head>
   5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
   6 <style type="text/css"> /* <![CDATA[ */
   7   @import "branding/css/tigris.css";
   8   @import "branding/css/inst.css";
   9   /* ]]> */</style>
  10 <link rel="stylesheet" type="text/css" media="print"
  11   href="branding/css/print.css"/>
  12 <script type="text/javascript" src="branding/scripts/tigris.js"></script>
  13 <title>The Subversion Build/Test Farm</title>
  14 </head>
  15
  16 <body>
  17 <div class="app">
  18
  19 <p class="warningmark" style="font-style: italic;"><b>WARNING:</b></p>
  20
  21 <p style="font-style: italic;">This document is not complete yet.
  22 When this warning disappears, that will mean the document is ready.</p>
  23
  24 <h1 style="text-align: center;">The Subversion Build/Test Farm</h1>
  25
  26 <pre style="text-align: center;"
  27 >$LastChangedDate$</pre>
  28
  29 <p>We are designing a build/test farm for Subversion, to test
  30 portability more conveniently and consistently.  We hope this will
  31 lead to fewer surprises at Release Candidate time.</p>
  32
  33 <p>This document describes the build farm.  The intended audiences
  34 are:</p>
  35
  36 <ul>
  37    <li>People who want to join the build farm, by running Subversion
  38        builds on a box with a stable environment, and using the build
  39        farm client software to send result reports to the central
  40        server for publication.</li>
  41
  42    <li>People who administer the central build server (CollabNet's
  43        operations team has volunteered to do this, as well as to run
  44        some of the clients).</li>
  45
  46    <li>Developers who want to test a change's portability before
  47        committing it, using the build farm's "try with this patch"
  48        feature.</li>
  49 </ul>
  50
  51 <div class="h2" id="overview" title="overview">
  52 <h2>Overview</h2>
  53
  54 <p>We've tentatively decided to use <a
  55 href="https://sourceforge.net/projects/buildbot/">BuildBot</a> for
  56 build management software, in part because of its <a
  57 href="http://buildbot.sourceforge.net/manual-0.7.0.html#try">TryScheduler</a>
  58 feature, which lets a developer ask the question "What would happen
  59 if I committed this patch right now?".  It runs the unit test suite
  60 (across multiple build platforms) on the developer's current code,
  61 allowing her to make sure her changes will not break the tree when she
  62 finally commits them.  The default setup, however, is to build
  63 a repository tree, usually trunk, or a release branch or tag.</p>
  64
  65 <p>BuildBot uses two terms, <b>buildmaster</b> and <b>buildslave</b>,
  66 that mean exactly what you think they do.  There is typically one
  67 buildmaster, and multiple buildslaves.  Most people will want to read
  68 about <a href="#buildslave">buildslave configuration</a>, which tells
  69 how to set up a buildslave client.  Buildmaster setup is covered <a
  70 href="#buildmaster">here</a>, but is probably of interest only to the
  71 CollabNet administrators who run the buildmaster server.</p>
  72
  73 </div>
  74
  75
  76 <div class="h2" id="buildslave" title="buildslave">
  77 <h2>Buildslave</h2>
  78
  79 <ol>
  80
  81 <li><p>Install all the dependencies required to build Subversion from
  82        working copy source.  That's outside the scope of this
  83        document, but see <a
  84        href="http://svn.collab.net/repos/svn/trunk/INSTALL"
  85        >http://svn.collab.net/repos/svn/trunk/INSTALL</a> for
  86        details.</p>
  87 </li>
  88
  89 <li><p>Create a user <b>buildslave</b> on your system.  The buildslave
  90        client will use this user, instead of your regular development
  91        user.  This is partly for security reasons, but also because
  92        part of the point of the build farm is to make sure things work
  93        in a relatively untweaked environment.</p>
  94 </li>
  95
  96 <li><p>Create a directory in ~buildslave/ to be the base directory for
  97        Subversion builds, hereinafter known as the "BASEDIR".  For
  98        example, ~buildslave/subversion-builds/.  The buildslave will
  99        touch nothing outside this directory.</p> </li>
 100
 101 <li><p>Write a script that checks out a copy of trunk (or whatever
 102        branch or tag you want to build), runs configure, make,
 103        make&nbsp;check, make&nbsp;davcheck, make&nbsp;svncheck, etc.
 104        Make sure the build works without manual intervention.
 105        Otherwise, it will not work when BuildBot attempts to do it
 106        automatically.</p>
 107
 108     <p>We recommend having that script live in BASEDIR, and having it
 109        check out the working copy such that it creates a new
 110        subdirectory within BASEDIR.</p>
 111 </li>
 112
 113 <li><p>Obtain your slavename and password from the central
 114        administrator.</p>
 115
 116     <p class="todo">Document exactly who to contact and how.</p>
 117 </li>
 118
 119 <li><p>Initialize the buildslave (the BuildBot docs call this
 120        "creating" the buildslave):</p>
 121
 122     <pre>   buildbot slave BASEDIR MASTERHOST:PORT SLAVENAME PASSWORD</pre>
 123 </li>
 124
 125 <li><p>Fill in your "hostinfo files", so other people can see the
 126        environment your builds run in, and can contact you if something
 127        appears to be wrong.  <b>BASEDIR/info/admin</b> should contain
 128        your name and email address.  This will be visible from the
 129        build status page; you can munge it to protect from spam
 130        address harvesters.  <b>BASEDIR/info/host</b> should contain a
 131        description of the host: OS, version, memory size, CPU speed,
 132        versions of relevant libraries installed, and the version of
 133        the buildbot code which is running the buildslave.</p>
 134
 135     <p class="todo">Give examples of these files.</p>
 136
 137     <p>If you run many buildslaves, you may want to create a single
 138        ~buildslave/info file and share it among all the buildslaves
 139        with symlinks.</p>
 140
 141 </li>
 142
 143 <li><p>
 144     </p>
 145 </li>
 146
 147 <li><p>
 148     </p>
 149 </li>
 150
 151 </ol>
 152
 153 </div>
 154
 155
 156 <div class="h2" id="buildmaster" title="buildmaster">
 157 <h2>Buildmaster</h2>
 158
 159 <p>BuildBot uses the usual Python installation mechanism, and depends
 160 on the TwistedMatrix library.  If you happen to use Debian GNU/Linux,
 161 you can set these up very easily:</p>
 162
 163 <pre>
 164    # apt-get update
 165    # apt-get install&nbsp;python2.3-dev
 166    # apt-get install&nbsp;twisted
 167 </pre>
 168
 169 <p>Just follow the instructions at <a
 170 href="http://buildbot.sourceforge.net/manual-0.7.0.html#Installation"
 171 >http://buildbot.sourceforge.net/manual-0.7.0.html#Installation</a></p>
 172
 173 <p class="todo">...working here...</p>
 174
 175 <!-- More verbosely:
 176
 177    erdos:/home/kfogel# apt-get install twisted
 178    Reading package lists... Done
 179    Building dependency tree... Done
 180    Note, selecting python2.3-twisted instead of twisted
 181    The following extra packages will be installed:
 182      python2.3-twisted python2.3-twisted-bin python2.3-zopeinterface
 183    Recommended packages:
 184      python2.3-pyopenssl python2.3-pam
 185    The following NEW packages will be installed:
 186      python2.3-twisted python2.3-twisted-bin python2.3-zopeinterface
 187    0 upgraded, 3 newly installed, 0 to remove and 105 not upgraded.
 188    Need to get 1345kB of archives.
 189    After unpacking 7946kB of additional disk space will be used.
 190    Do you want to continue [Y/n]?
 191
 192 -->
 193
 194 </div>
 195
 196
 197 <div class="h2" id="try" title="try">
 198 <h2>Trying out a change</h2>
 199
 200 <p class="todo">...working here...</p>
 201
 202 </div>
 203
 204
 205 <div class="h2" id="references" title="references">
 206 <h2>References</h2>
 207
 208 <ul>
 209 <li><p>BuildBot:<br/>
 210     <a href="http://buildbot.sourceforge.net/"
 211             >http://buildbot.sourceforge.net/</a>&nbsp;(start&nbsp;here)<br/>
 212     <a href="http://buildbot.sourceforge.net/PyCon-2003/buildbot.html"
 213             >http://buildbot.sourceforge.net/PyCon-2003/buildbot.html</a>&nbsp;(great&nbsp;technical&nbsp;overview)<br/>
 214     <a href="https://sourceforge.net/projects/buildbot/"
 215                >https://sourceforge.net/projects/buildbot/</a><br/>
 216     <a href="http://buildbot.sourceforge.net/manual-0.7.0.html"
 217                >http://buildbot.sourceforge.net/manual-0.7.0.html</a></p>
 218 </li>
 219
 220 <li><p>Original mail about the system:</p>
 221
 222 <pre>
 223    <a
 224    href="http://subversion.tigris.org/servlets/ReadMsg?list=dev&amp;msgNo=109179"
 225    >http://subversion.tigris.org/servlets/ReadMsg?list=dev&amp;msgNo=109179</a>
 226    From: Karl Fogel &lt;kfogel@collab.net&gt;
 227    To: dev@subversion.tigris.org
 228    Subject: Helping CollabNet provide an automated build farm for Subversion.
 229    Date: Mon, 28 Nov 2005 15:09:48 -0600 (CST)
 230    Message-Id: &lt;200511282109.jASL9m287485@newton.ch.collab.net&gt;
 231 </pre>
 232
 233 <p>Direct thread link:</p>
 234
 235 <pre>
 236    <a
 237 href="http://subversion.tigris.org/servlets/BrowseList?list=dev&amp;by=thread&amp;from=398579">http://subversion.tigris.org/servlets/BrowseList?list=dev&amp;by=thread&amp;from=398579</a>
 238 </pre>
 239 </li>
 240
 241 </ul>
 242
 243 </div>
 244
 245 </div>
 246 </body>
 247 </html>