1 ===================================
2 Customizing LLVMC: Reference Manual
3 ===================================
5 This file was automatically generated by rst2html.
6 Please do not edit directly!
7 The ReST source lives in the directory 'tools/llvmc/doc'.
13 <div class="doc_author">
14 <p>Written by <a href="mailto:foldr@codedgers.com">Mikhail Glushenkov</a></p>
20 LLVMC is a generic compiler driver, designed to be customizable and
21 extensible. It plays the same role for LLVM as the ``gcc`` program
22 does for GCC - LLVMC's job is essentially to transform a set of input
23 files into a set of targets depending on configuration rules and user
24 options. What makes LLVMC different is that these transformation rules
25 are completely customizable - in fact, LLVMC knows nothing about the
26 specifics of transformation (even the command-line options are mostly
27 not hard-coded) and regards the transformation structure as an
28 abstract graph. The structure of this graph is completely determined
29 by plugins, which can be either statically or dynamically linked. This
30 makes it possible to easily adapt LLVMC for other purposes - for
31 example, as a build tool for game resources.
33 Because LLVMC employs TableGen_ as its configuration language, you
34 need to be familiar with it to customize LLVMC.
36 .. _TableGen: http://llvm.org/docs/TableGenFundamentals.html
42 LLVMC tries hard to be as compatible with ``gcc`` as possible,
43 although there are some small differences. Most of the time, however,
44 you shouldn't be able to notice them::
46 $ # This works as expected:
47 $ llvmc -O3 -Wall hello.cpp
51 One nice feature of LLVMC is that one doesn't have to distinguish between
52 different compilers for different languages (think ``g++`` vs. ``gcc``) - the
53 right toolchain is chosen automatically based on input language names (which
54 are, in turn, determined from file extensions). If you want to force files
55 ending with ".c" to compile as C++, use the ``-x`` option, just like you would
58 $ # hello.c is really a C++ file
59 $ llvmc -x c++ hello.c
63 On the other hand, when using LLVMC as a linker to combine several C++
64 object files you should provide the ``--linker`` option since it's
65 impossible for LLVMC to choose the right linker in that case::
69 [A lot of link-time errors skipped]
70 $ llvmc --linker=c++ hello.o
74 By default, LLVMC uses ``llvm-gcc`` to compile the source code. It is also
75 possible to choose the ``clang`` compiler with the ``-clang`` option.
81 LLVMC has some built-in options that can't be overridden in the
82 configuration libraries:
84 * ``-o FILE`` - Output file name.
86 * ``-x LANGUAGE`` - Specify the language of the following input files
87 until the next -x option.
89 * ``-load PLUGIN_NAME`` - Load the specified plugin DLL. Example:
90 ``-load $LLVM_DIR/Release/lib/LLVMCSimple.so``.
92 * ``-v`` - Enable verbose mode, i.e. print out all executed commands.
94 * ``--save-temps`` - Write temporary files to the current directory and do not
95 delete them on exit. This option can also take an argument: the
96 ``--save-temps=obj`` switch will write files into the directory specified with
97 the ``-o`` option. The ``--save-temps=cwd`` and ``--save-temps`` switches are
98 both synonyms for the default behaviour.
100 * ``--temp-dir DIRECTORY`` - Store temporary files in the given directory. This
101 directory is deleted on exit unless ``--save-temps`` is specified. If
102 ``--save-temps=obj`` is also specified, ``--temp-dir`` is given the
105 * ``--check-graph`` - Check the compilation for common errors like mismatched
106 output/input language names, multiple default edges and cycles. Because of
107 plugins, these checks can't be performed at compile-time. Exit with code zero
108 if no errors were found, and return the number of found errors
109 otherwise. Hidden option, useful for debugging LLVMC plugins.
111 * ``--view-graph`` - Show a graphical representation of the compilation graph
112 and exit. Requires that you have ``dot`` and ``gv`` programs installed. Hidden
113 option, useful for debugging LLVMC plugins.
115 * ``--write-graph`` - Write a ``compilation-graph.dot`` file in the current
116 directory with the compilation graph description in Graphviz format (identical
117 to the file used by the ``--view-graph`` option). The ``-o`` option can be
118 used to set the output file name. Hidden option, useful for debugging LLVMC
121 * ``--help``, ``--help-hidden``, ``--version`` - These options have
122 their standard meaning.
124 Compiling LLVMC plugins
125 =======================
127 It's easiest to start working on your own LLVMC plugin by copying the
128 skeleton project which lives under ``$LLVMC_DIR/plugins/Simple``::
130 $ cd $LLVMC_DIR/plugins
131 $ cp -r Simple MyPlugin
134 Makefile PluginMain.cpp Simple.td
136 As you can see, our basic plugin consists of only two files (not
137 counting the build script). ``Simple.td`` contains TableGen
138 description of the compilation graph; its format is documented in the
139 following sections. ``PluginMain.cpp`` is just a helper file used to
140 compile the auto-generated C++ code produced from TableGen source. It
141 can also contain hook definitions (see `below`__).
145 The first thing that you should do is to change the ``LLVMC_PLUGIN``
146 variable in the ``Makefile`` to avoid conflicts (since this variable
147 is used to name the resulting library)::
149 LLVMC_PLUGIN=MyPlugin
151 It is also a good idea to rename ``Simple.td`` to something less
154 $ mv Simple.td MyPlugin.td
156 To build your plugin as a dynamic library, just ``cd`` to its source
157 directory and run ``make``. The resulting file will be called
158 ``plugin_llvmc_$(LLVMC_PLUGIN).$(DLL_EXTENSION)`` (in our case,
159 ``plugin_llvmc_MyPlugin.so``). This library can be then loaded in with the
160 ``-load`` option. Example::
162 $ cd $LLVMC_DIR/plugins/Simple
164 $ llvmc -load $LLVM_DIR/Release/lib/plugin_llvmc_Simple.so
166 Compiling standalone LLVMC-based drivers
167 ========================================
169 By default, the ``llvmc`` executable consists of a driver core plus several
170 statically linked plugins (``Base`` and ``Clang`` at the moment). You can
171 produce a standalone LLVMC-based driver executable by linking the core with your
172 own plugins. The recommended way to do this is by starting with the provided
173 ``Skeleton`` example (``$LLVMC_DIR/example/Skeleton``)::
175 $ cd $LLVMC_DIR/example/
176 $ cp -r Skeleton mydriver
182 If you're compiling LLVM with different source and object directories, then you
183 must perform the following additional steps before running ``make``::
185 # LLVMC_SRC_DIR = $LLVM_SRC_DIR/tools/llvmc/
186 # LLVMC_OBJ_DIR = $LLVM_OBJ_DIR/tools/llvmc/
187 $ cp $LLVMC_SRC_DIR/example/mydriver/Makefile \
188 $LLVMC_OBJ_DIR/example/mydriver/
189 $ cd $LLVMC_OBJ_DIR/example/mydriver
192 Another way to do the same thing is by using the following command::
195 $ make LLVMC_BUILTIN_PLUGINS=MyPlugin LLVMC_BASED_DRIVER_NAME=mydriver
197 This works with both srcdir == objdir and srcdir != objdir, but assumes that the
198 plugin source directory was placed under ``$LLVMC_DIR/plugins``.
200 Sometimes, you will want a 'bare-bones' version of LLVMC that has no
201 built-in plugins. It can be compiled with the following command::
204 $ make LLVMC_BUILTIN_PLUGINS=""
207 Customizing LLVMC: the compilation graph
208 ========================================
210 Each TableGen configuration file should include the common
213 include "llvm/CompilerDriver/Common.td"
215 Internally, LLVMC stores information about possible source
216 transformations in form of a graph. Nodes in this graph represent
217 tools, and edges between two nodes represent a transformation path. A
218 special "root" node is used to mark entry points for the
219 transformations. LLVMC also assigns a weight to each edge (more on
220 this later) to choose between several alternative edges.
222 The definition of the compilation graph (see file
223 ``plugins/Base/Base.td`` for an example) is just a list of edges::
225 def CompilationGraph : CompilationGraph<[
226 Edge<"root", "llvm_gcc_c">,
227 Edge<"root", "llvm_gcc_assembler">,
230 Edge<"llvm_gcc_c", "llc">,
231 Edge<"llvm_gcc_cpp", "llc">,
234 OptionalEdge<"llvm_gcc_c", "opt", (case (switch_on "opt"),
236 OptionalEdge<"llvm_gcc_cpp", "opt", (case (switch_on "opt"),
240 OptionalEdge<"llvm_gcc_assembler", "llvm_gcc_cpp_linker",
241 (case (input_languages_contain "c++"), (inc_weight),
242 (or (parameter_equals "linker", "g++"),
243 (parameter_equals "linker", "c++")), (inc_weight))>,
248 As you can see, the edges can be either default or optional, where
249 optional edges are differentiated by an additional ``case`` expression
250 used to calculate the weight of this edge. Notice also that we refer
251 to tools via their names (as strings). This makes it possible to add
252 edges to an existing compilation graph in plugins without having to
253 know about all tool definitions used in the graph.
255 The default edges are assigned a weight of 1, and optional edges get a
256 weight of 0 + 2*N where N is the number of tests that evaluated to
257 true in the ``case`` expression. It is also possible to provide an
258 integer parameter to ``inc_weight`` and ``dec_weight`` - in this case,
259 the weight is increased (or decreased) by the provided value instead
260 of the default 2. It is also possible to change the default weight of
261 an optional edge by using the ``default`` clause of the ``case``
264 When passing an input file through the graph, LLVMC picks the edge
265 with the maximum weight. To avoid ambiguity, there should be only one
266 default edge between two nodes (with the exception of the root node,
267 which gets a special treatment - there you are allowed to specify one
268 default edge *per language*).
270 When multiple plugins are loaded, their compilation graphs are merged
271 together. Since multiple edges that have the same end nodes are not
272 allowed (i.e. the graph is not a multigraph), an edge defined in
273 several plugins will be replaced by the definition from the plugin
274 that was loaded last. Plugin load order can be controlled by using the
275 plugin priority feature described above.
277 To get a visual representation of the compilation graph (useful for
278 debugging), run ``llvmc --view-graph``. You will need ``dot`` and
279 ``gsview`` installed for this to work properly.
284 Command-line options that the plugin supports are defined by using an
287 def Options : OptionList<[
288 (switch_option "E", (help "Help string")),
289 (alias_option "quiet", "q")
293 As you can see, the option list is just a list of DAGs, where each DAG
294 is an option description consisting of the option name and some
295 properties. A plugin can define more than one option list (they are
296 all merged together in the end), which can be handy if one wants to
297 separate option groups syntactically.
299 * Possible option types:
301 - ``switch_option`` - a simple boolean switch without arguments, for example
302 ``-O2`` or ``-time``. At most one occurrence is allowed.
304 - ``parameter_option`` - option that takes one argument, for example
305 ``-std=c99``. It is also allowed to use spaces instead of the equality
306 sign: ``-std c99``. At most one occurrence is allowed.
308 - ``parameter_list_option`` - same as the above, but more than one option
309 occurence is allowed.
311 - ``prefix_option`` - same as the parameter_option, but the option name and
312 argument do not have to be separated. Example: ``-ofile``. This can be also
313 specified as ``-o file``; however, ``-o=file`` will be parsed incorrectly
314 (``=file`` will be interpreted as option value). At most one occurrence is
317 - ``prefix_list_option`` - same as the above, but more than one occurence of
318 the option is allowed; example: ``-lm -lpthread``.
320 - ``alias_option`` - a special option type for creating aliases. Unlike other
321 option types, aliases are not allowed to have any properties besides the
322 aliased option name. Usage example: ``(alias_option "preprocess", "E")``
325 * Possible option properties:
327 - ``help`` - help string associated with this option. Used for ``--help``
330 - ``required`` - this option must be specified exactly once (or, in case of
331 the list options without the ``multi_val`` property, at least
332 once). Incompatible with ``zero_or_one`` and ``one_or_more``.
334 - ``one_or_more`` - the option must be specified at least one time. Useful
335 only for list options in conjunction with ``multi_val``; for ordinary lists
336 it is synonymous with ``required``. Incompatible with ``required`` and
339 - ``zero_or_one`` - the option can be specified zero or one times. Useful
340 only for list options in conjunction with ``multi_val``. Incompatible with
341 ``required`` and ``one_or_more``.
343 - ``hidden`` - the description of this option will not appear in
344 the ``--help`` output (but will appear in the ``--help-hidden``
347 - ``really_hidden`` - the option will not be mentioned in any help
350 - ``multi_val n`` - this option takes *n* arguments (can be useful in some
351 special cases). Usage example: ``(parameter_list_option "foo", (multi_val
352 3))``. Only list options can have this attribute; you can, however, use
353 the ``one_or_more`` and ``zero_or_one`` properties.
355 - ``init`` - this option has a default value, either a string (if it is a
356 parameter), or a boolean (if it is a switch; boolean constants are called
357 ``true`` and ``false``). List options can't have this attribute. Usage
358 examples: ``(switch_option "foo", (init true))``; ``(prefix_option "bar",
361 - ``extern`` - this option is defined in some other plugin, see below.
366 Sometimes, when linking several plugins together, one plugin needs to
367 access options defined in some other plugin. Because of the way
368 options are implemented, such options must be marked as
369 ``extern``. This is what the ``extern`` option property is
373 (switch_option "E", (extern))
376 If an external option has additional attributes besides 'extern', they are
377 ignored. See also the section on plugin `priorities`__.
383 Conditional evaluation
384 ======================
386 The 'case' construct is the main means by which programmability is
387 achieved in LLVMC. It can be used to calculate edge weights, program
388 actions and modify the shell commands to be executed. The 'case'
389 expression is designed after the similarly-named construct in
390 functional languages and takes the form ``(case (test_1), statement_1,
391 (test_2), statement_2, ... (test_N), statement_N)``. The statements
392 are evaluated only if the corresponding tests evaluate to true.
396 // Edge weight calculation
398 // Increases edge weight by 5 if "-A" is provided on the
399 // command-line, and by 5 more if "-B" is also provided.
401 (switch_on "A"), (inc_weight 5),
402 (switch_on "B"), (inc_weight 5))
405 // Tool command line specification
407 // Evaluates to "cmdline1" if the option "-A" is provided on the
408 // command line; to "cmdline2" if "-B" is provided;
409 // otherwise to "cmdline3".
412 (switch_on "A"), "cmdline1",
413 (switch_on "B"), "cmdline2",
414 (default), "cmdline3")
416 Note the slight difference in 'case' expression handling in contexts
417 of edge weights and command line specification - in the second example
418 the value of the ``"B"`` switch is never checked when switch ``"A"`` is
419 enabled, and the whole expression always evaluates to ``"cmdline1"`` in
422 Case expressions can also be nested, i.e. the following is legal::
424 (case (switch_on "E"), (case (switch_on "o"), ..., (default), ...)
427 You should, however, try to avoid doing that because it hurts
428 readability. It is usually better to split tool descriptions and/or
429 use TableGen inheritance instead.
431 * Possible tests are:
433 - ``switch_on`` - Returns true if a given command-line switch is
434 provided by the user. Example: ``(switch_on "opt")``.
436 - ``parameter_equals`` - Returns true if a command-line parameter equals
438 Example: ``(parameter_equals "W", "all")``.
440 - ``element_in_list`` - Returns true if a command-line parameter
441 list contains a given value.
442 Example: ``(parameter_in_list "l", "pthread")``.
444 - ``input_languages_contain`` - Returns true if a given language
445 belongs to the current input language set.
446 Example: ``(input_languages_contain "c++")``.
448 - ``in_language`` - Evaluates to true if the input file language
449 equals to the argument. At the moment works only with ``cmd_line``
450 and ``actions`` (on non-join nodes).
451 Example: ``(in_language "c++")``.
453 - ``not_empty`` - Returns true if a given option (which should be
454 either a parameter or a parameter list) is set by the
456 Example: ``(not_empty "o")``.
458 - ``empty`` - The opposite of ``not_empty``. Equivalent to ``(not (not_empty
459 X))``. Provided for convenience.
461 - ``default`` - Always evaluates to true. Should always be the last
462 test in the ``case`` expression.
464 - ``and`` - A standard logical combinator that returns true iff all
465 of its arguments return true. Used like this: ``(and (test1),
466 (test2), ... (testN))``. Nesting of ``and`` and ``or`` is allowed,
469 - ``or`` - Another logical combinator that returns true only if any
470 one of its arguments returns true. Example: ``(or (test1),
471 (test2), ... (testN))``.
474 Writing a tool description
475 ==========================
477 As was said earlier, nodes in the compilation graph represent tools,
478 which are described separately. A tool definition looks like this
479 (taken from the ``include/llvm/CompilerDriver/Tools.td`` file)::
481 def llvm_gcc_cpp : Tool<[
483 (out_language "llvm-assembler"),
484 (output_suffix "bc"),
485 (cmd_line "llvm-g++ -c $INFILE -o $OUTFILE -emit-llvm"),
489 This defines a new tool called ``llvm_gcc_cpp``, which is an alias for
490 ``llvm-g++``. As you can see, a tool definition is just a list of
491 properties; most of them should be self-explanatory. The ``sink``
492 property means that this tool should be passed all command-line
493 options that aren't mentioned in the option list.
495 The complete list of all currently implemented tool properties follows.
497 * Possible tool properties:
499 - ``in_language`` - input language name. Can be either a string or a
500 list, in case the tool supports multiple input languages.
502 - ``out_language`` - output language name. Tools are not allowed to
503 have multiple output languages.
505 - ``output_suffix`` - output file suffix. Can also be changed
506 dynamically, see documentation on actions.
508 - ``cmd_line`` - the actual command used to run the tool. You can
509 use ``$INFILE`` and ``$OUTFILE`` variables, output redirection
510 with ``>``, hook invocations (``$CALL``), environment variables
511 (via ``$ENV``) and the ``case`` construct.
513 - ``join`` - this tool is a "join node" in the graph, i.e. it gets a
514 list of input files and joins them together. Used for linkers.
516 - ``sink`` - all command-line options that are not handled by other
517 tools are passed to this tool.
519 - ``actions`` - A single big ``case`` expression that specifies how
520 this tool reacts on command-line options (described in more detail
526 A tool often needs to react to command-line options, and this is
527 precisely what the ``actions`` property is for. The next example
528 illustrates this feature::
530 def llvm_gcc_linker : Tool<[
531 (in_language "object-code"),
532 (out_language "executable"),
533 (output_suffix "out"),
534 (cmd_line "llvm-gcc $INFILE -o $OUTFILE"),
536 (actions (case (not_empty "L"), (forward "L"),
537 (not_empty "l"), (forward "l"),
539 [(append_cmd "-dummy1"), (append_cmd "-dummy2")])
542 The ``actions`` tool property is implemented on top of the omnipresent
543 ``case`` expression. It associates one or more different *actions*
544 with given conditions - in the example, the actions are ``forward``,
545 which forwards a given option unchanged, and ``append_cmd``, which
546 appends a given string to the tool execution command. Multiple actions
547 can be associated with a single condition by using a list of actions
548 (used in the example to append some dummy options). The same ``case``
549 construct can also be used in the ``cmd_line`` property to modify the
552 The "join" property used in the example means that this tool behaves
555 The list of all possible actions follows.
559 - ``append_cmd`` - append a string to the tool invocation
561 Example: ``(case (switch_on "pthread"), (append_cmd
564 - ``error` - exit with error.
565 Example: ``(error "Mixing -c and -S is not allowed!")``.
567 - ``forward`` - forward an option unchanged.
568 Example: ``(forward "Wall")``.
570 - ``forward_as`` - Change the name of an option, but forward the
572 Example: ``(forward_as "O0", "--disable-optimization")``.
574 - ``output_suffix`` - modify the output suffix of this
576 Example: ``(output_suffix "i")``.
578 - ``stop_compilation`` - stop compilation after this tool processes
579 its input. Used without arguments.
581 - ``unpack_values`` - used for for splitting and forwarding
582 comma-separated lists of options, e.g. ``-Wa,-foo=bar,-baz`` is
583 converted to ``-foo=bar -baz`` and appended to the tool invocation
585 Example: ``(unpack_values "Wa,")``.
590 If you are adding support for a new language to LLVMC, you'll need to
591 modify the language map, which defines mappings from file extensions
592 to language names. It is used to choose the proper toolchain(s) for a
593 given input file set. Language map definition looks like this::
595 def LanguageMap : LanguageMap<
596 [LangToSuffixes<"c++", ["cc", "cp", "cxx", "cpp", "CPP", "c++", "C"]>,
597 LangToSuffixes<"c", ["c"]>,
601 For example, without those definitions the following command wouldn't work::
604 llvmc: Unknown suffix: cpp
606 The language map entries should be added only for tools that are
607 linked with the root node. Since tools are not allowed to have
608 multiple output languages, for nodes "inside" the graph the input and
609 output languages should match. This is enforced at compile-time.
617 Hooks and environment variables
618 -------------------------------
620 Normally, LLVMC executes programs from the system ``PATH``. Sometimes,
621 this is not sufficient: for example, we may want to specify tool paths
622 or names in the configuration file. This can be easily achieved via
623 the hooks mechanism. To write your own hooks, just add their
624 definitions to the ``PluginMain.cpp`` or drop a ``.cpp`` file into the
625 your plugin directory. Hooks should live in the ``hooks`` namespace
626 and have the signature ``std::string hooks::MyHookName ([const char*
627 Arg0 [ const char* Arg2 [, ...]]])``. They can be used from the
628 ``cmd_line`` tool property::
630 (cmd_line "$CALL(MyHook)/path/to/file -o $CALL(AnotherHook)")
632 To pass arguments to hooks, use the following syntax::
634 (cmd_line "$CALL(MyHook, 'Arg1', 'Arg2', 'Arg # 3')/path/to/file -o1 -o2")
636 It is also possible to use environment variables in the same manner::
638 (cmd_line "$ENV(VAR1)/path/to/file -o $ENV(VAR2)")
640 To change the command line string based on user-provided options use
641 the ``case`` expression (documented `above`__)::
646 "llvm-g++ -E -x c $INFILE -o $OUTFILE",
648 "llvm-g++ -c -x c $INFILE -o $OUTFILE -emit-llvm"))
654 How plugins are loaded
655 ----------------------
657 It is possible for LLVMC plugins to depend on each other. For example,
658 one can create edges between nodes defined in some other plugin. To
659 make this work, however, that plugin should be loaded first. To
660 achieve this, the concept of plugin priority was introduced. By
661 default, every plugin has priority zero; to specify the priority
662 explicitly, put the following line in your plugin's TableGen file::
664 def Priority : PluginPriority<$PRIORITY_VALUE>;
665 # Where PRIORITY_VALUE is some integer > 0
667 Plugins are loaded in order of their (increasing) priority, starting
668 with 0. Therefore, the plugin with the highest priority value will be
674 When writing LLVMC plugins, it can be useful to get a visual view of
675 the resulting compilation graph. This can be achieved via the command
676 line option ``--view-graph``. This command assumes that Graphviz_ and
677 Ghostview_ are installed. There is also a ``--write-graph`` option that
678 creates a Graphviz source file (``compilation-graph.dot``) in the
681 Another useful ``llvmc`` option is ``--check-graph``. It checks the
682 compilation graph for common errors like mismatched output/input
683 language names, multiple default edges and cycles. These checks can't
684 be performed at compile-time because the plugins can load code
685 dynamically. When invoked with ``--check-graph``, ``llvmc`` doesn't
686 perform any compilation tasks and returns the number of encountered
687 errors as its status code.
689 .. _Graphviz: http://www.graphviz.org/
690 .. _Ghostview: http://pages.cs.wisc.edu/~ghost/
692 Conditioning on the executable name
693 -----------------------------------
695 For now, the executable name (the value passed to the driver in ``argv[0]``) is
696 accessible only in the C++ code (i.e. hooks). Use the following code::
699 extern const char* ProgramName;
702 std::string MyHook() {
704 if (strcmp(ProgramName, "mydriver") == 0) {
709 In general, you're encouraged not to make the behaviour dependent on the
710 executable file name, and use command-line switches instead. See for example how
711 the ``Base`` plugin behaves when it needs to choose the correct linker options
712 (think ``g++`` vs. ``gcc``).
718 <a href="http://jigsaw.w3.org/css-validator/check/referer">
719 <img src="http://jigsaw.w3.org/css-validator/images/vcss-blue"
720 alt="Valid CSS" /></a>
721 <a href="http://validator.w3.org/check?uri=referer">
722 <img src="http://www.w3.org/Icons/valid-xhtml10-blue"
723 alt="Valid XHTML 1.0 Transitional"/></a>
725 <a href="mailto:foldr@codedgers.com">Mikhail Glushenkov</a><br />
726 <a href="http://llvm.org">LLVM Compiler Infrastructure</a><br />
728 Last modified: $Date: 2008-12-11 11:34:48 -0600 (Thu, 11 Dec 2008) $