clang/www/hacking.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
   2           "http://www.w3.org/TR/html4/strict.dtd">
   3 <!-- Material used from: HTML 4.01 specs: http://www.w3.org/TR/html401/ -->
   4 <html>
   5 <head>
   6   <META http-equiv="Content-Type" content="text/html; charset=utf-8">
   7   <title>Hacking on clang</title>
   8   <link type="text/css" rel="stylesheet" href="menu.css">
   9   <link type="text/css" rel="stylesheet" href="content.css">
  10   <style type="text/css">
  11   pre { margin-left: 1.5em; }
  12   </style>
  13 </head>
  14 <body>
  15 <!--#include virtual="menu.html.incl"-->
  16 <div id="content">
  17   <!--*********************************************************************-->
  18   <h1>Hacking on Clang</h1>
  19   <!--*********************************************************************-->
  20
  21   <p>This document provides some hints for how to get started hacking
  22   on Clang for developers who are new to the Clang and/or LLVM
  23   codebases.</p>
  24     <ul>
  25       <li><a href="#style">Coding Standards</a></li>
  26       <li><a href="#docs">Developer Documentation</a></li>
  27       <li><a href="#debugging">Debugging</a></li>
  28       <li><a href="#testing">Testing</a>
  29       <ul>
  30         <li><a href="#testingNonWindows">Testing on Unix-like Systems</a></li>
  31         <li><a href="#testingWindows">Testing using Visual Studio on Windows</a></li>
  32         <li><a href="#testingCommands">Testing on the Command Line</a></li>
  33         <li><a href="#testingLibc++">Testing changes affecting libc++</a></li>
  34       </ul>
  35       </li>
  36       <li><a href="#patches">Creating Patch Files</a></li>
  37       <li><a href="#irgen">LLVM IR Generation</a></li>
  38     </ul>
  39
  40   <!--=====================================================================-->
  41   <h2 id="style">Coding Standards</h2>
  42   <!--=====================================================================-->
  43
  44   <p>Clang follows the
  45   LLVM <a href="https://llvm.org/docs/CodingStandards.html">Coding
  46   Standards</a>. When submitting patches, please take care to follow these standards
  47   and to match the style of the code to that present in Clang (for example, in
  48   terms of indentation, bracing, and statement spacing).</p>
  49
  50   <p>Clang has a few additional coding standards:</p>
  51   <ul>
  52     <li><i>cstdio is forbidden</i>: library code should not output diagnostics
  53       or other information using <tt>cstdio</tt>; debugging routines should
  54       use <tt>llvm::errs()</tt>. Other uses of <tt>cstdio</tt> impose behavior
  55       upon clients and block integrating Clang as a library. Libraries should
  56       support <tt>raw_ostream</tt> based interfaces for textual
  57       output. See <a href="https://llvm.org/docs/CodingStandards.html#use-raw-ostream">Coding
  58       Standards</a>.</li>
  59   </ul>
  60
  61   <!--=====================================================================-->
  62   <h2 id="docs">Developer Documentation</h2>
  63   <!--=====================================================================-->
  64
  65   <p>Both Clang and LLVM use doxygen to provide API documentation. Their
  66   respective web pages (generated nightly) are here:</p>
  67     <ul>
  68       <li><a href="https://clang.llvm.org/doxygen">Clang</a></li>
  69       <li><a href="https://llvm.org/doxygen">LLVM</a></li>
  70     </ul>
  71
  72   <p>For work on the LLVM IR generation, the LLVM assembly language
  73   <a href="https://llvm.org/docs/LangRef.html">reference manual</a> is
  74   also useful.</p>
  75
  76   <!--=====================================================================-->
  77   <h2 id="debugging">Debugging</h2>
  78   <!--=====================================================================-->
  79
  80   <p>Inspecting data structures in a debugger:</p>
  81     <ul>
  82       <li>Many LLVM and Clang data structures provide
  83         a <tt>dump()</tt> method which will print a description of the
  84         data structure to <tt>stderr</tt>.</li>
  85       <li>The <a href="docs/InternalsManual.html#QualType"><tt>QualType</tt></a>
  86       structure is used pervasively. This is a simple value class for
  87       wrapping types with qualifiers; you can use
  88       the <tt>isConstQualified()</tt>, for example, to get one of the
  89       qualifiers, and the <tt>getTypePtr()</tt> method to get the
  90       wrapped <tt>Type*</tt> which you can then dump.</li>
  91       <li>For <a href="https://lldb.llvm.org"> <tt>LLDB</tt></a> users there are
  92       data formatters for clang data structures in
  93       <a href="https://github.com/llvm/llvm-project/blob/main/clang/utils/ClangDataFormat.py">
  94       <tt>clang/utils/ClangDataFormat.py</tt></a>.</li>
  95     </ul>
  96
  97   <!--=====================================================================-->
  98   <h3 id="debuggingVisualStudio">Debugging using Visual Studio</h3>
  99   <!--=====================================================================-->
 100
 101   <p>The files
 102     <a href="https://github.com/llvm/llvm-project/blob/main/llvm/utils/LLVMVisualizers/llvm.natvis">
 103       <tt>llvm/utils/LLVMVisualizers/llvm.natvis</tt></a> and
 104     <a href="https://github.com/llvm/llvm-project/blob/main/clang/utils/ClangVisualizers/clang.natvis">
 105       <tt>clang/utils/ClangVisualizers/clang.natvis</tt></a> provide debugger visualizers
 106       that make debugging of more complex data types much easier.</p>
 107   <p>Depending on how you configure the project, Visual Studio may automatically
 108   use these visualizers when debugging or you may be required to put the files
 109   into <tt>%USERPROFILE%\Documents\Visual Studio &lt;version&gt;\Visualizers</tt>
 110   or create a symbolic link so they update automatically. See
 111   <a href="https://learn.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects">
 112   Microsoft's documentation</a> for more details on use of NATVIS.</p>
 113
 114   <!--=====================================================================-->
 115   <h2 id="testing">Testing</h2>
 116   <!--=====================================================================-->
 117
 118   <!--=====================================================================-->
 119   <h3 id="testingNonWindows">Testing on Unix-like Systems</h3>
 120   <!--=====================================================================-->
 121
 122   <p>Clang includes a basic regression suite in the tree which can be
 123   run with <tt>make test</tt> from the top-level clang directory, or
 124   just <tt>make</tt> in the <em>test</em> sub-directory.
 125   <tt>make VERBOSE=1</tt> can be used to show more detail
 126   about what is being run.</p>
 127
 128   <p>If you built LLVM and Clang using CMake, the test suite can be run
 129   with <tt>make check-clang</tt> from the top-level LLVM directory.</p>
 130
 131   <p>The tests primarily consist of a test runner script running the compiler
 132   under test on individual test files grouped in the directories under the
 133   test directory.  The individual test files include comments at the
 134   beginning indicating the Clang compile options to use, to be read
 135   by the test runner. Embedded comments also can do things like telling
 136   the test runner that an error is expected at the current line.
 137   Any output files produced by the test will be placed under
 138   a created Output directory.</p>
 139
 140   <p>During the run of <tt>make test</tt>, the terminal output will
 141   display a line similar to the following:</p>
 142
 143   <pre>--- Running clang tests for i686-pc-linux-gnu ---</pre>
 144
 145   <p>followed by a line continually overwritten with the current test
 146   file being compiled, and an overall completion percentage.</p>
 147
 148   <p>After the <tt>make test</tt> run completes, the absence of any
 149   <tt>Failing Tests (count):</tt> message indicates that no tests
 150   failed unexpectedly.  If any tests did fail, the
 151   <tt>Failing Tests (count):</tt> message will be followed by a list
 152   of the test source file paths that failed.  For example:</p>
 153
 154   <pre>
 155   Failing Tests (3):
 156       /home/john/llvm/tools/clang/test/SemaCXX/member-name-lookup.cpp
 157       /home/john/llvm/tools/clang/test/SemaCXX/namespace-alias.cpp
 158       /home/john/llvm/tools/clang/test/SemaCXX/using-directive.cpp
 159 </pre>
 160
 161   <p>If you used the <tt>make VERBOSE=1</tt> option, the terminal
 162   output will reflect the error messages from the compiler and
 163   test runner.</p>
 164
 165   <p>The regression suite can also be run with Valgrind by running
 166   <tt>make test VG=1</tt> in the top-level clang directory.</p>
 167
 168   <p>For more intensive changes, running
 169   the <a href="https://llvm.org/docs/TestingGuide.html#quick-start">LLVM
 170   Test Suite</a> with clang is recommended. Currently the best way to
 171   override LLVMGCC, as in: <tt>make LLVMGCC="clang -std=gnu89"
 172   TEST=nightly report</tt> (make sure <tt>clang</tt> is in your PATH or use the
 173   full path).</p>
 174
 175   <!--=====================================================================-->
 176   <h3 id="testingWindows">Testing using Visual Studio on Windows</h3>
 177   <!--=====================================================================-->
 178
 179   <p>The Clang test suite can be run from either Visual Studio or
 180   the command line.</p>
 181
 182   <p>Note that the test runner is based on
 183   Python, which must be installed.  Find Python at:
 184   <a href="https://www.python.org/downloads/">https://www.python.org/downloads/</a>.
 185   Download the latest stable version.</p>
 186
 187   <p>The GnuWin32 tools are also necessary for running the tests.
 188   Get them from <a href="http://getgnuwin32.sourceforge.net/">
 189   http://getgnuwin32.sourceforge.net/</a>.
 190   If the environment variable <tt>%PATH%</tt> does not have GnuWin32,
 191   or if other grep(s) supercedes GnuWin32 on <tt>%PATH%,</tt>
 192   you should specify <tt>LLVM_LIT_TOOLS_DIR</tt>
 193   to CMake explicitly.</p>
 194
 195   <p>The cmake build tool is set up to create Visual Studio project files
 196   for running the tests, "check-clang" being the root.  Therefore, to
 197   run the test from Visual Studio, right-click the check-clang project
 198   and select "Build".</p>
 199
 200   <p>
 201     Please see also
 202     <a href="https://llvm.org/docs/GettingStartedVS.html">Getting Started
 203     with the LLVM System using Microsoft Visual Studio</a> and
 204     <a href="https://llvm.org/docs/CMake.html">Building LLVM with CMake</a>.
 205   </p>
 206
 207   <!--=====================================================================-->
 208   <h3 id="testingCommands">Testing on the Command Line</h3>
 209   <!--=====================================================================-->
 210
 211   <p>If you want more control over how the tests are run, it may
 212   be convenient to run the test harness on the command-line directly. Before
 213   running tests from the command line, you will need to ensure that
 214   <tt>lit.site.cfg</tt> files have been created for your build.  You can do
 215   this by running the tests as described in the previous sections. Once the
 216   tests have started running, you can stop them with control+C, as the
 217   files are generated before running any tests.</p>
 218
 219   <p>Once that is done, to run all the tests from the command line,
 220   execute a command like the following:</p>
 221
 222   <pre>
 223   python (path to llvm)\llvm\utils\lit\lit.py -sv
 224   --param=build_mode=Win32 --param=build_config=Debug
 225   --param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg
 226  (path to llvm)\llvm\tools\clang\test
 227 </pre>
 228
 229   <p>For CMake builds e.g. on Windows with Visual Studio, you will need
 230   to specify your build configuration (Debug, Release, etc.) via
 231   <tt>--param=build_config=(build config)</tt>.  You may also need to specify
 232   the build mode (Win32, etc) via <tt>--param=build_mode=(build mode)</tt>.</p>
 233
 234   <p>Additionally, you will need to specify the lit site configuration which
 235   lives in (build dir)\tools\clang\test, via
 236   <tt>--param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg</tt>.
 237   </p>
 238
 239   <p>To run a single test:</p>
 240
 241   <pre>
 242   python (path to llvm)\llvm\utils\lit\lit.py -sv
 243   --param=build_mode=Win32 --param=build_config=Debug
 244   --param=clang_site_config=(build dir)\tools\clang\test\lit.site.cfg
 245   (path to llvm)\llvm\tools\clang\test\(dir)\(test)
 246 </pre>
 247
 248   <p>For example:</p>
 249
 250   <pre>
 251   python C:\Tools\llvm\utils\lit\lit.py -sv
 252   --param=build_mode=Win32 --param=build_config=Debug
 253   --param=clang_site_config=C:\Tools\build\tools\clang\test\lit.site.cfg
 254   C:\Tools\llvm\tools\clang\test\Sema\wchar.c
 255 </pre>
 256
 257   <p>The -sv option above tells the runner to show the test output if
 258   any tests failed, to help you determine the cause of failure.</p>
 259
 260   <p>You can also pass in the --no-progress-bar option if you wish to disable
 261   progress indications while the tests are running.</p>
 262
 263   <p>Your output might look something like this:</p>
 264
 265   <pre>lit.py: lit.cfg:152: note: using clang: 'C:\Tools\llvm\bin\Release\clang.EXE'
 266 -- Testing: Testing: 2534 tests, 4 threads --
 267 Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
 268 Testing Time: 81.52s
 269   Passed           : 2503
 270   Expectedly Failed:   28
 271   Unsupported      :    3
 272 </pre>
 273
 274   <p>The statistic, "Failed" (not shown if all tests pass), is the important one.</p>
 275
 276   <!--=====================================================================-->
 277   <h3 id="testingLibc++">Testing changes affecting libc++</h3>
 278   <!--=====================================================================-->
 279
 280   <p>Some changes in Clang affect <a href="https://libcxx.llvm.org">libc++</a>,
 281   for example:</p>
 282   <ul>
 283       <li>Changing the output of Clang's diagnostics.</li>
 284       <li>Changing compiler builtins, especially the builtins used for type traits
 285       or replacements of library functions like <tt>std::move</tt> or
 286       <tt>std::forward</tt>.</li>
 287   </ul>
 288
 289   <p>After adjusting libc++ to work with the changes, the next revision will be
 290   tested by libc++'s
 291   <a href="https://buildkite.com/llvm-project/libcxx-ci">pre-commit CI</a>.
 292
 293   <p>For most configurations, the pre-commit CI uses a recent
 294   <a href="https://apt.llvm.org/">nightly build</a> of Clang from LLVM's main
 295   branch. These configurations do <em>not</em> use the Clang changes in the
 296   patch. They only use the libc++ changes.</p>
 297
 298   <p>The &quot;Bootstrapping build&quot; builds Clang and uses it to build and
 299   test libc++. This build <em>does</em> use the Clang changes in the patch.</p>
 300
 301   <p>Libc++ supports multiple versions of Clang. Therefore when a patch changes
 302   the diagnostics it might be required to use a regex in the
 303   &quot;expected&quot; tests to make it pass the CI.</p>
 304
 305   <p>Libc++ has more
 306   <a href="https://libcxx.llvm.org/Contributing.html#pre-commit-ci">
 307   documentation</a> about the pre-commit CI. For questions regarding
 308   libc++, the best place to ask is the <tt>#libcxx</tt> channel on
 309   <a href="https://discord.gg/jzUbyP26tQ">LLVM's Discord server</a>.</p>
 310
 311   <!--=====================================================================-->
 312   <h2 id="patches">Creating Patch Files</h2>
 313   <!--=====================================================================-->
 314
 315   <p>To contribute changes to Clang see
 316     <a href="https://llvm.org/docs/GettingStarted.html#sending-patches">LLVM's Getting Started page</a></p>
 317
 318   <!--=====================================================================-->
 319   <h2 id="irgen">LLVM IR Generation</h2>
 320   <!--=====================================================================-->
 321
 322   <p>The LLVM IR generation part of clang handles conversion of the
 323     AST nodes output by the Sema module to the LLVM Intermediate
 324     Representation (IR). Historically, this was referred to as
 325     "codegen", and the Clang code for this lives
 326     in <tt>lib/CodeGen</tt>.</p>
 327
 328   <p>The output is most easily inspected using the <tt>-emit-llvm</tt>
 329     option to clang (possibly in conjunction with <tt>-o -</tt>). You
 330     can also use <tt>-emit-llvm-bc</tt> to write an LLVM bitcode file
 331     which can be processed by the suite of LLVM tools
 332     like <tt>llvm-dis</tt>, <tt>llvm-nm</tt>, etc. See the LLVM
 333     <a href="https://llvm.org/docs/CommandGuide/">Command Guide</a>
 334     for more information.</p>
 335
 336 </div>
 337 </body>
 338 </html>