documentation/cvs-regression.sgml

   1   <chapter id="cvs-regression">
   2     <title>How to do regression testing using CVS</title>
   3
   4     <para>
   5       A problem that can happen sometimes is 'it used to work
   6       before, now it doesn't anymore...'. Here is a step by step
   7       procedure to try to pinpoint when the problem occurred. This is
   8       <emphasis>NOT</emphasis> for casual users.
   9     </para>
  10
  11     <orderedlist>
  12       <listitem>
  13         <para>
  14           Get the <quote>full CVS</quote> archive from winehq. This archive is
  15           the CVS tree but with the tags controlling the versioning
  16           system. It's a big file (> 40 meg) with a name like
  17           full-cvs-&lt;last update date> (it's more than 100mb
  18           when uncompressed, you can't very well do this with
  19           small, old computers or slow Internet connections).
  20         </para>
  21       </listitem>
  22       <listitem>
  23         <para>
  24           untar it into a repository directory:
  25           <screen>
  26 cd /home/gerard
  27 tar -zxf full-cvs-2003-08-18.tar.gz
  28 mv wine repository
  29           </screen>
  30         </para>
  31       </listitem>
  32       <listitem>
  33         <para>
  34           extract a new destination directory. This directory must
  35           not be in a subdirectory of the repository else
  36           <command>cvs</command> will think it's part of the
  37           repository and deny you an extraction in the repository:
  38           <screen>
  39 cd /home/gerard
  40 mv wine wine_current (-> this protects your current wine sandbox, if any)
  41 export CVSROOT=/home/gerard/repository
  42 cvs -d $CVSROOT checkout wine
  43           </screen>
  44         </para>
  45         <para>
  46           Note that it's not possible to do a checkout at a given
  47           date; you always do the checkout for the last date where
  48           the full-cvs-xxx snapshot was generated.
  49         </para>
  50         <para>
  51           Note also that it is possible to do all this with a direct
  52           CVS connection, of course. The full CVS file method is less
  53           painful for the WineHQ CVS server and probably a bit faster
  54           if you don't have a very good net connection.
  55         </para>
  56       </listitem>
  57       <listitem>
  58         <para>
  59           you will have now in the <filename>~/wine</filename>
  60           directory an image of the CVS tree, on the client side.
  61           Now update this image to the date you want:
  62           <screen>
  63 cd /home/gerard/wine
  64 cvs update -PAd -D "2004-08-23 CDT"
  65           </screen>
  66         </para>
  67         <para>
  68           The date format is <literal>YYYY-MM-DD HH:MM:SS</literal>.
  69           Using the CST date format ensure that you will be able to
  70           extract patches in a way that will be compatible with the
  71           wine-cvs archive
  72           <ulink url="http://www.winehq.org/hypermail/wine-cvs">
  73           http://www.winehq.org/hypermail/wine-cvs</ulink>
  74         </para>
  75         <para>
  76           Many messages will inform you that more recent files have
  77           been deleted to set back the client cvs tree to the date
  78           you asked, for example:
  79           <screen>
  80 cvs update: tsx11/ts_xf86dga2.c is no longer in the repository
  81           </screen>
  82         </para>
  83         <para>
  84           <command>cvs update</command> is not limited to upgrade to
  85           a <emphasis>newer</emphasis> version as I have believed for
  86           far too long :-(
  87         </para>
  88       </listitem>
  89       <listitem>
  90         <para>
  91           Now proceed as for a normal update:
  92         </para>
  93         <screen>
  94 ./configure
  95 make depend && make
  96         </screen>
  97         <para>
  98           If any non-programmer reads this, the fastest method to get
  99           at the point where the problem occurred is to use a binary
 100           search, that is, if the problem occurred in 1999, start at
 101           mid-year, then is the problem is already here, back to 1st
 102           April, if not, to 1st October, and so on.
 103         </para>
 104         <para>
 105           If you have lot of hard disk free space (a full compile currently
 106           takes 400 Mb), copy the oldest known working version before
 107           updating it, it will save time if you need to go back. (it's
 108           better to <command>make distclean</command> before going back in
 109           time, so you have to make everything if you don't backup the older
 110           version)
 111         </para>
 112         <para>
 113           When you have found the day where the problem happened, continue
 114           the search using the wine-cvs archive (sorted by date) and a
 115           more precise cvs update including hour, minute, second :
 116           <screen>
 117 cvs update -PAd -D "2004-08-23 15:17:25 CDT"
 118           </screen>
 119           This will allow you to find easily the exact patch that did it.
 120         </para>
 121       </listitem>
 122       <listitem>
 123         <para>
 124           If you find the patch that is the cause of the problem, you have
 125           almost won; report about it to
 126           <ulink url="http://bugs.winehq.org/">Wine Bugzilla</ulink>
 127           or subscribe to wine-devel and post it there. There is a chance
 128           that the author
 129           will jump in to suggest a fix; or there is always the possibility
 130           to look hard at the patch until it is coerced to reveal where is
 131           the bug :-)
 132         </para>
 133       </listitem>
 134     </orderedlist>
 135   </chapter>
 136
 137   <!-- Keep this comment at the end of the file
 138   Local variables:
 139   mode: sgml
 140   sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")
 141   End:
 142   -->