doc/src/sgml/jit.sgml

   1 <!-- doc/src/sgml/jit.sgml -->
   2
   3 <chapter id="jit">
   4  <title>Just-in-Time Compilation (<acronym>JIT</acronym>)</title>
   5
   6  <indexterm zone="jit">
   7   <primary><acronym>JIT</acronym></primary>
   8  </indexterm>
   9
  10  <indexterm>
  11   <primary>Just-In-Time compilation</primary>
  12   <see><acronym>JIT</acronym></see>
  13  </indexterm>
  14
  15  <para>
  16   This chapter explains what just-in-time compilation is, and how it can be
  17   configured in <productname>PostgreSQL</productname>.
  18  </para>
  19
  20  <sect1 id="jit-reason">
  21   <title>What Is <acronym>JIT</acronym> compilation?</title>
  22
  23   <para>
  24    Just-in-Time (<acronym>JIT</acronym>) compilation is the process of turning
  25    some form of interpreted program evaluation into a native program, and
  26    doing so at run time.
  27    For example, instead of using general-purpose code that can evaluate
  28    arbitrary SQL expressions to evaluate a particular SQL predicate
  29    like <literal>WHERE a.col = 3</literal>, it is possible to generate a
  30    function that is specific to that expression and can be natively executed
  31    by the CPU, yielding a speedup.
  32   </para>
  33
  34   <para>
  35    <productname>PostgreSQL</productname> has builtin support to perform
  36    <acronym>JIT</acronym> compilation using <ulink
  37    url="https://llvm.org/"><productname>LLVM</productname></ulink> when
  38    <productname>PostgreSQL</productname> is built with
  39    <link linkend="configure-with-llvm"><literal>--with-llvm</literal></link>.
  40   </para>
  41
  42   <para>
  43    See <filename>src/backend/jit/README</filename> for further details.
  44   </para>
  45
  46   <sect2 id="jit-accelerated-operations">
  47    <title><acronym>JIT</acronym> Accelerated Operations</title>
  48    <para>
  49     Currently <productname>PostgreSQL</productname>'s <acronym>JIT</acronym>
  50     implementation has support for accelerating expression evaluation and
  51     tuple deforming.  Several other operations could be accelerated in the
  52     future.
  53    </para>
  54    <para>
  55     Expression evaluation is used to evaluate <literal>WHERE</literal>
  56     clauses, target lists, aggregates and projections. It can be accelerated
  57     by generating code specific to each case.
  58    </para>
  59    <para>
  60     Tuple deforming is the process of transforming an on-disk tuple (see <xref
  61     linkend="storage-tuple-layout"/>) into its in-memory representation.
  62     It can be accelerated by creating a function specific to the table layout
  63     and the number of columns to be extracted.
  64    </para>
  65   </sect2>
  66
  67   <sect2 id="jit-inlining">
  68    <title>Inlining</title>
  69    <para>
  70     <productname>PostgreSQL</productname> is very extensible and allows new
  71     data types, functions, operators and other database objects to be defined;
  72     see <xref linkend="extend"/>. In fact the built-in objects are implemented
  73     using nearly the same mechanisms.  This extensibility implies some
  74     overhead, for example due to function calls (see <xref linkend="xfunc"/>).
  75     To reduce that overhead, <acronym>JIT</acronym> compilation can inline the
  76     bodies of small functions into the expressions using them. That allows a
  77     significant percentage of the overhead to be optimized away.
  78    </para>
  79   </sect2>
  80
  81   <sect2 id="jit-optimization">
  82    <title>Optimization</title>
  83    <para>
  84     <productname>LLVM</productname> has support for optimizing generated
  85     code. Some of the optimizations are cheap enough to be performed whenever
  86     <acronym>JIT</acronym> is used, while others are only beneficial for
  87     longer-running queries.
  88     See <ulink url="https://llvm.org/docs/Passes.html#transform-passes"/> for
  89     more details about optimizations.
  90    </para>
  91   </sect2>
  92
  93  </sect1>
  94
  95  <sect1 id="jit-decision">
  96   <title>When to <acronym>JIT</acronym>?</title>
  97
  98   <para>
  99    <acronym>JIT</acronym> compilation is beneficial primarily for long-running
 100    CPU-bound queries. Frequently these will be analytical queries.  For short
 101    queries the added overhead of performing <acronym>JIT</acronym> compilation
 102    will often be higher than the time it can save.
 103   </para>
 104
 105   <para>
 106    To determine whether <acronym>JIT</acronym> compilation should be used,
 107    the total estimated cost of a query (see
 108    <xref linkend="planner-stats-details"/> and
 109    <xref linkend="runtime-config-query-constants"/>) is used.
 110    The estimated cost of the query will be compared with the setting of <xref
 111    linkend="guc-jit-above-cost"/>. If the cost is higher,
 112    <acronym>JIT</acronym> compilation will be performed.
 113    Two further decisions are then needed.
 114    Firstly, if the estimated cost is more
 115    than the setting of <xref linkend="guc-jit-inline-above-cost"/>, short
 116    functions and operators used in the query will be inlined.
 117    Secondly, if the estimated cost is more than the setting of <xref
 118    linkend="guc-jit-optimize-above-cost"/>, expensive optimizations are
 119    applied to improve the generated code.
 120    Each of these options increases the <acronym>JIT</acronym> compilation
 121    overhead, but can reduce query execution time considerably.
 122   </para>
 123
 124   <para>
 125    These cost-based decisions will be made at plan time, not execution
 126    time. This means that when prepared statements are in use, and a generic
 127    plan is used (see <xref linkend="sql-prepare"/>), the values of the
 128    configuration parameters in effect at prepare time control the decisions,
 129    not the settings at execution time.
 130   </para>
 131
 132   <note>
 133    <para>
 134     If <xref linkend="guc-jit"/> is set to <literal>off</literal>, or if no
 135     <acronym>JIT</acronym> implementation is available (for example because
 136     the server was compiled without <literal>--with-llvm</literal>),
 137     <acronym>JIT</acronym> will not be performed, even if it would be
 138     beneficial based on the above criteria.  Setting <xref linkend="guc-jit"/>
 139     to <literal>off</literal> has effects at both plan and execution time.
 140    </para>
 141   </note>
 142
 143   <para>
 144    <xref linkend="sql-explain"/> can be used to see whether
 145    <acronym>JIT</acronym> is used or not.  As an example, here is a query that
 146    is not using <acronym>JIT</acronym>:
 147 <screen>
 148 =# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class;
 149                                                  QUERY PLAN
 150 -------------------------------------------------------------------&zwsp;------------------------------------------
 151  Aggregate  (cost=16.27..16.29 rows=1 width=8) (actual time=0.303..0.303 rows=1 loops=1)
 152    ->  Seq Scan on pg_class  (cost=0.00..15.42 rows=342 width=4) (actual time=0.017..0.111 rows=356 loops=1)
 153  Planning Time: 0.116 ms
 154  Execution Time: 0.365 ms
 155 (4 rows)
 156 </screen>
 157    Given the cost of the plan, it is entirely reasonable that no
 158    <acronym>JIT</acronym> was used; the cost of <acronym>JIT</acronym> would
 159    have been bigger than the potential savings. Adjusting the cost limits
 160    will lead to <acronym>JIT</acronym> use:
 161 <screen>
 162 =# SET jit_above_cost = 10;
 163 SET
 164 =# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class;
 165                                                  QUERY PLAN
 166 -------------------------------------------------------------------&zwsp;------------------------------------------
 167  Aggregate  (cost=16.27..16.29 rows=1 width=8) (actual time=6.049..6.049 rows=1 loops=1)
 168    ->  Seq Scan on pg_class  (cost=0.00..15.42 rows=342 width=4) (actual time=0.019..0.052 rows=356 loops=1)
 169  Planning Time: 0.133 ms
 170  JIT:
 171    Functions: 3
 172    Options: Inlining false, Optimization false, Expressions true, Deforming true
 173    Timing: Generation 1.259 ms (Deform 0.000 ms), Inlining 0.000 ms, Optimization 0.797 ms, Emission 5.048 ms, Total 7.104 ms
 174  Execution Time: 7.416 ms
 175 </screen>
 176    As visible here, <acronym>JIT</acronym> was used, but inlining and
 177    expensive optimization were not. If <xref
 178    linkend="guc-jit-inline-above-cost"/> or <xref
 179    linkend="guc-jit-optimize-above-cost"/> were also lowered,
 180    that would change.
 181   </para>
 182  </sect1>
 183
 184  <sect1 id="jit-configuration" xreflabel="JIT Configuration">
 185   <title>Configuration</title>
 186
 187   <para>
 188    The configuration variable
 189    <xref linkend="guc-jit"/> determines whether <acronym>JIT</acronym>
 190    compilation is enabled or disabled.
 191    If it is enabled, the configuration variables
 192    <xref linkend="guc-jit-above-cost"/>, <xref
 193    linkend="guc-jit-inline-above-cost"/>, and <xref
 194    linkend="guc-jit-optimize-above-cost"/> determine
 195    whether <acronym>JIT</acronym> compilation is performed for a query,
 196    and how much effort is spent doing so.
 197   </para>
 198
 199   <para>
 200    <xref linkend="guc-jit-provider"/> determines which <acronym>JIT</acronym>
 201    implementation is used. It is rarely required to be changed. See <xref
 202    linkend="jit-pluggable"/>.
 203   </para>
 204
 205   <para>
 206    For development and debugging purposes a few additional configuration
 207    parameters exist, as described in
 208    <xref linkend="runtime-config-developer"/>.
 209   </para>
 210  </sect1>
 211
 212  <sect1 id="jit-extensibility">
 213   <title>Extensibility</title>
 214
 215   <sect2 id="jit-extensibility-bitcode">
 216    <title>Inlining Support for Extensions</title>
 217    <para>
 218     <productname>PostgreSQL</productname>'s <acronym>JIT</acronym>
 219     implementation can inline the bodies of functions
 220     of types <literal>C</literal> and <literal>internal</literal>, as well as
 221     operators based on such functions.  To do so for functions in extensions,
 222     the definitions of those functions need to be made available.
 223     When using <link linkend="extend-pgxs">PGXS</link> to build an extension
 224     against a server that has been compiled with LLVM JIT support, the
 225     relevant files will be built and installed automatically.
 226    </para>
 227
 228    <para>
 229     The relevant files have to be installed into
 230     <filename>$pkglibdir/bitcode/$extension/</filename> and a summary of them
 231     into <filename>$pkglibdir/bitcode/$extension.index.bc</filename>, where
 232     <literal>$pkglibdir</literal> is the directory returned by
 233     <literal>pg_config --pkglibdir</literal> and <literal>$extension</literal>
 234     is the base name of the extension's shared library.
 235
 236     <note>
 237      <para>
 238       For functions built into <productname>PostgreSQL</productname> itself,
 239       the bitcode is installed into
 240       <literal>$pkglibdir/bitcode/postgres</literal>.
 241      </para>
 242     </note>
 243    </para>
 244   </sect2>
 245
 246   <sect2 id="jit-pluggable">
 247    <title>Pluggable <acronym>JIT</acronym> Providers</title>
 248
 249    <para>
 250     <productname>PostgreSQL</productname> provides a <acronym>JIT</acronym>
 251     implementation based on <productname>LLVM</productname>.  The interface to
 252     the <acronym>JIT</acronym> provider is pluggable and the provider can be
 253     changed without recompiling (although currently, the build process only
 254     provides inlining support data for <productname>LLVM</productname>).
 255     The active provider is chosen via the setting
 256     <xref linkend="guc-jit-provider"/>.
 257    </para>
 258
 259    <sect3 id="jit-pluggable-provider-interface">
 260     <title><acronym>JIT</acronym> Provider Interface</title>
 261     <para>
 262      A <acronym>JIT</acronym> provider is loaded by dynamically loading the
 263      named shared library. The normal library search path is used to locate
 264      the library. To provide the required <acronym>JIT</acronym> provider
 265      callbacks and to indicate that the library is actually a
 266      <acronym>JIT</acronym> provider, it needs to provide a C function named
 267      <function>_PG_jit_provider_init</function>. This function is passed a
 268      struct that needs to be filled with the callback function pointers for
 269      individual actions:
 270 <programlisting>
 271 struct JitProviderCallbacks
 272 {
 273     JitProviderResetAfterErrorCB reset_after_error;
 274     JitProviderReleaseContextCB release_context;
 275     JitProviderCompileExprCB compile_expr;
 276 };
 277
 278 extern void _PG_jit_provider_init(JitProviderCallbacks *cb);
 279 </programlisting>
 280     </para>
 281    </sect3>
 282   </sect2>
 283  </sect1>
 284
 285 </chapter>