clang/www/hacking.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
   2           "http://www.w3.org/TR/html4/strict.dtd">
   3 <!-- Material used from: HTML 4.01 specs: http://www.w3.org/TR/html401/ -->
   4 <html>
   5 <head>
   6   <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   7   <title>Hacking on clang</title>
   8   <link type="text/css" rel="stylesheet" href="menu.css">
   9   <link type="text/css" rel="stylesheet" href="content.css">
  10   <style type="text/css">
  11   pre { margin-left: 1.5em; }
  12   </style>
  13 </head>
  14 <body>
  15 <!--#include virtual="menu.html.incl"-->
  16 <div id="content">
  17   <!--*********************************************************************-->
  18   <h1>Hacking on Clang</h1>
  19   <!--*********************************************************************-->
  20
  21   <p>This document provides some hints for how to get started hacking
  22   on Clang for developers who are new to the Clang and/or LLVM
  23   codebases.</p>
  24     <ul>
  25       <li><a href="#style">Coding Standards</a></li>
  26       <li><a href="#docs">Developer Documentation</a></li>
  27       <li><a href="#debugging">Debugging</a></li>
  28       <li><a href="#testing">Testing</a>
  29       <ul>
  30         <li><a href="#testingNonWindows">Testing on Unix-like Systems</a></li>
  31         <li><a href="#testingWindows">Testing using Visual Studio on Windows</a></li>
  32         <li><a href="#testingCommands">Testing on the Command Line</a></li>
  33         <li><a href="#testingLibc++">Testing changes affecting libc++</a></li>
  34       </ul>
  35       </li>
  36       <li><a href="#patches">Creating Patch Files</a></li>
  37       <li><a href="#irgen">LLVM IR Generation</a></li>
  38     </ul>
  39
  40   <!--=====================================================================-->
  41   <h2 id="style">Coding Standards</h2>
  42   <!--=====================================================================-->
  43
  44   <p>Clang follows the
  45   LLVM <a href="https://llvm.org/docs/CodingStandards.html">Coding
  46   Standards</a>. When submitting patches, please take care to follow these standards
  47   and to match the style of the code to that present in Clang (for example, in
  48   terms of indentation, bracing, and statement spacing).</p>
  49
  50   <p>Clang has a few additional coding standards:</p>
  51   <ul>
  52     <li><i>cstdio is forbidden</i>: library code should not output diagnostics
  53       or other information using <tt>cstdio</tt>; debugging routines should
  54       use <tt>llvm::errs()</tt>. Other uses of <tt>cstdio</tt> impose behavior
  55       upon clients and block integrating Clang as a library. Libraries should
  56       support <tt>raw_ostream</tt> based interfaces for textual
  57       output. See <a href="https://llvm.org/docs/CodingStandards.html#use-raw-ostream">Coding
  58       Standards</a>.</li>
  59   </ul>
  60
  61   <!--=====================================================================-->
  62   <h2 id="docs">Developer Documentation</h2>
  63   <!--=====================================================================-->
  64
  65   <p>Both Clang and LLVM use doxygen to provide API documentation. Their
  66   respective web pages (generated nightly) are here:</p>
  67     <ul>
  68       <li><a href="https://clang.llvm.org/doxygen">Clang</a></li>
  69       <li><a href="https://llvm.org/doxygen">LLVM</a></li>
  70     </ul>
  71
  72   <p>For work on the LLVM IR generation, the LLVM assembly language
  73   <a href="https://llvm.org/docs/LangRef.html">reference manual</a> is
  74   also useful.</p>
  75
  76   <!--=====================================================================-->
  77   <h2 id="debugging">Debugging</h2>
  78   <!--=====================================================================-->
  79
  80   <p>Inspecting data structures in a debugger:</p>
  81     <ul>
  82       <li>Many LLVM and Clang data structures provide
  83         a <tt>dump()</tt> method which will print a description of the
  84         data structure to <tt>stderr</tt>.</li>
  85       <li>The <a href="docs/InternalsManual.html#QualType"><tt>QualType</tt></a>
  86       structure is used pervasively. This is a simple value class for
  87       wrapping types with qualifiers; you can use
  88       the <tt>isConstQualified()</tt>, for example, to get one of the
  89       qualifiers, and the <tt>getTypePtr()</tt> method to get the
  90       wrapped <tt>Type*</tt> which you can then dump.</li>
  91       <li>For <a href="https://lldb.llvm.org"> <tt>LLDB</tt></a> users there are
  92       data formatters for clang data structures in
  93       <a href="https://github.com/llvm/llvm-project/blob/main/clang/utils/ClangDataFormat.py">
  94       <tt>clang/utils/ClangDataFormat.py</tt></a>.</li>
  95     </ul>
  96
  97   <!--=====================================================================-->
  98   <h3 id="debuggingVisualStudio">Debugging using Visual Studio</h3>
  99   <!--=====================================================================-->
 100
 101   <p>The files
 102     <a href="https://github.com/llvm/llvm-project/blob/main/llvm/utils/LLVMVisualizers/llvm.natvis">
 103       <tt>llvm/utils/LLVMVisualizers/llvm.natvis</tt></a> and
 104     <a href="https://github.com/llvm/llvm-project/blob/main/clang/utils/ClangVisualizers/clang.natvis">
 105       <tt>clang/utils/ClangVisualizers/clang.natvis</tt></a> provide debugger visualizers
 106       that make debugging of more complex data types much easier.</p>
 107   <p>For Visual Studio 2013 only, put the files into
 108     <tt>%USERPROFILE%\Documents\Visual Studio 2013\Visualizers</tt> or
 109     create a symbolic link so they update automatically.</p>
 110   <p>For later versions of Visual Studio, no installation is required.
 111     Note also that later versions of Visual Studio also display better visualizations.</p>
 112
 113   <!--=====================================================================-->
 114   <h2 id="testing">Testing</h2>
 115   <!--=====================================================================-->
 116
 117   <!--=====================================================================-->
 118   <h3 id="testingNonWindows">Testing on Unix-like Systems</h3>
 119   <!--=====================================================================-->
 120
 121   <p>Clang includes a basic regression suite in the tree which can be
 122   run with <tt>make test</tt> from the top-level clang directory, or
 123   just <tt>make</tt> in the <em>test</em> sub-directory.
 124   <tt>make VERBOSE=1</tt> can be used to show more detail
 125   about what is being run.</p>
 126
 127   <p>If you built LLVM and Clang using CMake, the test suite can be run
 128   with <tt>make clang-test</tt> from the top-level LLVM directory.</p>
 129
 130   <p>The tests primarily consist of a test runner script running the compiler
 131   under test on individual test files grouped in the directories under the
 132   test directory.  The individual test files include comments at the
 133   beginning indicating the Clang compile options to use, to be read
 134   by the test runner. Embedded comments also can do things like telling
 135   the test runner that an error is expected at the current line.
 136   Any output files produced by the test will be placed under
 137   a created Output directory.</p>
 138
 139   <p>During the run of <tt>make test</tt>, the terminal output will
 140   display a line similar to the following:</p>
 141
 142   <pre>--- Running clang tests for i686-pc-linux-gnu ---</pre>
 143
 144   <p>followed by a line continually overwritten with the current test
 145   file being compiled, and an overall completion percentage.</p>
 146
 147   <p>After the <tt>make test</tt> run completes, the absence of any
 148   <tt>Failing Tests (count):</tt> message indicates that no tests
 149   failed unexpectedly.  If any tests did fail, the
 150   <tt>Failing Tests (count):</tt> message will be followed by a list
 151   of the test source file paths that failed.  For example:</p>
 152
 153   <pre>
 154   Failing Tests (3):
 155       /home/john/llvm/tools/clang/test/SemaCXX/member-name-lookup.cpp
 156       /home/john/llvm/tools/clang/test/SemaCXX/namespace-alias.cpp
 157       /home/john/llvm/tools/clang/test/SemaCXX/using-directive.cpp
 158 </pre>
 159
 160   <p>If you used the <tt>make VERBOSE=1</tt> option, the terminal
 161   output will reflect the error messages from the compiler and
 162   test runner.</p>
 163
 164   <p>The regression suite can also be run with Valgrind by running
 165   <tt>make test VG=1</tt> in the top-level clang directory.</p>
 166
 167   <p>For more intensive changes, running
 168   the <a href="https://llvm.org/docs/TestingGuide.html#quick-start">LLVM
 169   Test Suite</a> with clang is recommended. Currently the best way to
 170   override LLVMGCC, as in: <tt>make LLVMGCC="clang -std=gnu89"
 171   TEST=nightly report</tt> (make sure <tt>clang</tt> is in your PATH or use the
 172   full path).</p>
 173
 174   <!--=====================================================================-->
 175   <h3 id="testingWindows">Testing using Visual Studio on Windows</h3>
 176   <!--=====================================================================-->
 177
 178   <p>The Clang test suite can be run from either Visual Studio or
 179   the command line.</p>
 180
 181   <p>Note that the test runner is based on
 182   Python, which must be installed.  Find Python at:
 183   <a href="https://www.python.org/downloads/">https://www.python.org/downloads/</a>.
 184   Download the latest stable version.</p>
 185
 186   <p>The GnuWin32 tools are also necessary for running the tests.
 187   Get them from <a href="http://getgnuwin32.sourceforge.net/">
 188   http://getgnuwin32.sourceforge.net/</a>.
 189   If the environment variable <tt>%PATH%</tt> does not have GnuWin32,
 190   or if other grep(s) supercedes GnuWin32 on <tt>%PATH%,</tt>
 191   you should specify <tt>LLVM_LIT_TOOLS_DIR</tt>
 192   to CMake explicitly.</p>
 193
 194   <p>The cmake build tool is set up to create Visual Studio project files
 195   for running the tests, "clang-test" being the root.  Therefore, to
 196   run the test from Visual Studio, right-click the clang-test project
 197   and select "Build".</p>
 198
 199   <p>
 200     Please see also
 201     <a href="https://llvm.org/docs/GettingStartedVS.html">Getting Started
 202     with the LLVM System using Microsoft Visual Studio</a> and
 203     <a href="https://llvm.org/docs/CMake.html">Building LLVM with CMake</a>.
 204   </p>
 205
 206   <!--=====================================================================-->
 207   <h3 id="testingCommands">Testing on the Command Line</h3>
 208   <!--=====================================================================-->
 209
 210   <p>If you want more control over how the tests are run, it may
 211   be convenient to run the test harness on the command-line directly. Before
 212   running tests from the command line, you will need to ensure that
 213   <tt>lit.site.cfg</tt> files have been created for your build.  You can do
 214   this by running the tests as described in the previous sections. Once the
 215   tests have started running, you can stop them with control+C, as the
 216   files are generated before running any tests.</p>
 217
 218   <p>Once that is done, to run all the tests from the command line,
 219   execute a command like the following:</p>
 220
 221   <pre>
 222   python (path to llvm)\llvm\utils\lit\lit.py -sv
 223   --param=build_mode=Win32 --param=build_config=Debug
 224   --param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg
 225  (path to llvm)\llvm\tools\clang\test
 226 </pre>
 227
 228   <p>For CMake builds e.g. on Windows with Visual Studio, you will need
 229   to specify your build configuration (Debug, Release, etc.) via
 230   <tt>--param=build_config=(build config)</tt>.  You may also need to specify
 231   the build mode (Win32, etc) via <tt>--param=build_mode=(build mode)</tt>.</p>
 232
 233   <p>Additionally, you will need to specify the lit site configuration which
 234   lives in (build dir)\tools\clang\test, via
 235   <tt>--param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg</tt>.
 236   </p>
 237
 238   <p>To run a single test:</p>
 239
 240   <pre>
 241   python (path to llvm)\llvm\utils\lit\lit.py -sv
 242   --param=build_mode=Win32 --param=build_config=Debug
 243   --param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg
 244   (path to llvm)\llvm\tools\clang\test\(dir)\(test)
 245 </pre>
 246
 247   <p>For example:</p>
 248
 249   <pre>
 250   python C:\Tools\llvm\utils\lit\lit.py -sv
 251   --param=build_mode=Win32 --param=build_config=Debug
 252   --param=clang_site_config=C:\Tools\build\tools\clang\test\lit.site.cfg
 253   C:\Tools\llvm\tools\clang\test\Sema\wchar.c
 254 </pre>
 255
 256   <p>The -sv option above tells the runner to show the test output if
 257   any tests failed, to help you determine the cause of failure.</p>
 258
 259   <p>You can also pass in the --no-progress-bar option if you wish to disable
 260   progress indications while the tests are running.</p>
 261
 262   <p>Your output might look something like this:</p>
 263
 264   <pre>lit.py: lit.cfg:152: note: using clang: 'C:\Tools\llvm\bin\Release\clang.EXE'
 265 -- Testing: Testing: 2534 tests, 4 threads --
 266 Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
 267 Testing Time: 81.52s
 268   Passed           : 2503
 269   Expectedly Failed:   28
 270   Unsupported      :    3
 271 </pre>
 272
 273   <p>The statistic, "Failed" (not shown if all tests pass), is the important one.</p>
 274
 275   <!--=====================================================================-->
 276   <h3 id="testingLibc++">Testing changes affecting libc++</h3>
 277   <!--=====================================================================-->
 278
 279   <p>Some changes in Clang affect <a href="https://libcxx.llvm.org">libc++</a>,
 280   for example:</p>
 281   <ul>
 282       <li>Changing the output of Clang's diagnostics.</li>
 283       <li>Changing compiler builtins, especially the builtins used for type traits
 284       or replacements of library functions like <tt>std::move</tt> or
 285       <tt>std::forward</tt>.</li>
 286   </ul>
 287
 288   <p>After adjusting libc++ to work with the changes, the next revision will be
 289   tested by libc++'s
 290   <a href="https://buildkite.com/llvm-project/libcxx-ci">pre-commit CI</a>.
 291
 292   <p>For most configurations, the pre-commit CI uses a recent
 293   <a href="https://apt.llvm.org/">nightly build</a> of Clang from LLVM's main
 294   branch. These configurations do <em>not</em> use the Clang changes in the
 295   patch. They only use the libc++ changes.</p>
 296
 297   <p>The &quot;Bootstrapping build&quot; builds Clang and uses it to build and
 298   test libc++. This build <em>does</em> use the Clang changes in the patch.</p>
 299
 300   <p>Libc++ supports multiple versions of Clang. Therefore when a patch changes
 301   the diagnostics it might be required to use a regex in the
 302   &quot;expected&quot; tests to make it pass the CI.</p>
 303
 304   <p>Libc++ has more
 305   <a href="https://libcxx.llvm.org/Contributing.html#pre-commit-ci">
 306   documentation</a> about the pre-commit CI. For questions regarding
 307   libc++, the best place to ask is the <tt>#libcxx</tt> channel on
 308   <a href="https://discord.gg/jzUbyP26tQ">LLVM's Discord server</a>.</p>
 309
 310   <!--=====================================================================-->
 311   <h2 id="patches">Creating Patch Files</h2>
 312   <!--=====================================================================-->
 313
 314   <p>To contribute changes to Clang see
 315     <a href="https://llvm.org/docs/GettingStarted.html#sending-patches">LLVM's Getting Started page</a></p>
 316
 317   <!--=====================================================================-->
 318   <h2 id="irgen">LLVM IR Generation</h2>
 319   <!--=====================================================================-->
 320
 321   <p>The LLVM IR generation part of clang handles conversion of the
 322     AST nodes output by the Sema module to the LLVM Intermediate
 323     Representation (IR). Historically, this was referred to as
 324     "codegen", and the Clang code for this lives
 325     in <tt>lib/CodeGen</tt>.</p>
 326
 327   <p>The output is most easily inspected using the <tt>-emit-llvm</tt>
 328     option to clang (possibly in conjunction with <tt>-o -</tt>). You
 329     can also use <tt>-emit-llvm-bc</tt> to write an LLVM bitcode file
 330     which can be processed by the suite of LLVM tools
 331     like <tt>llvm-dis</tt>, <tt>llvm-nm</tt>, etc. See the LLVM
 332     <a href="https://llvm.org/docs/CommandGuide/">Command Guide</a>
 333     for more information.</p>
 334
 335 </div>
 336 </body>
 337 </html>