1 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
7 This document describes LLVM BPF backend relocation types.
12 LLVM BPF backend records each relocation with the following 16-byte
17 Elf64_Addr r_offset; // Offset from the beginning of section.
18 Elf64_Xword r_info; // Relocation type and symbol index.
21 For example, for the following code::
23 int g1 __attribute__((section("sec")));
24 int g2 __attribute__((section("sec")));
25 static volatile int l1 __attribute__((section("sec")));
26 static volatile int l2 __attribute__((section("sec")));
28 return g1 + g2 + l1 + l2;
31 Compiled with ``clang --target=bpf -O2 -c test.c``, the following is
32 the code with ``llvm-objdump -dr test.o``::
34 0: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
35 0000000000000000: R_BPF_64_64 g1
36 2: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
37 3: 18 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r2 = 0 ll
38 0000000000000018: R_BPF_64_64 g2
39 5: 61 20 00 00 00 00 00 00 r0 = *(u32 *)(r2 + 0)
40 6: 0f 10 00 00 00 00 00 00 r0 += r1
41 7: 18 01 00 00 08 00 00 00 00 00 00 00 00 00 00 00 r1 = 8 ll
42 0000000000000038: R_BPF_64_64 sec
43 9: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
44 10: 0f 10 00 00 00 00 00 00 r0 += r1
45 11: 18 01 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 r1 = 12 ll
46 0000000000000058: R_BPF_64_64 sec
47 13: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
48 14: 0f 10 00 00 00 00 00 00 r0 += r1
49 15: 95 00 00 00 00 00 00 00 exit
51 There are four relocations in the above for four ``LD_imm64`` instructions.
52 The following ``llvm-readelf -r test.o`` shows the binary values of the four
55 Relocation section '.rel.text' at offset 0x190 contains 4 entries:
56 Offset Info Type Symbol's Value Symbol's Name
57 0000000000000000 0000000600000001 R_BPF_64_64 0000000000000000 g1
58 0000000000000018 0000000700000001 R_BPF_64_64 0000000000000004 g2
59 0000000000000038 0000000400000001 R_BPF_64_64 0000000000000000 sec
60 0000000000000058 0000000400000001 R_BPF_64_64 0000000000000000 sec
62 Each relocation is represented by ``Offset`` (8 bytes) and ``Info`` (8 bytes).
63 For example, the first relocation corresponds to the first instruction
64 (Offset 0x0) and the corresponding ``Info`` indicates the relocation type
65 of ``R_BPF_64_64`` (type 1) and the entry in the symbol table (entry 6).
66 The following is the symbol table with ``llvm-readelf -s test.o``::
68 Symbol table '.symtab' contains 8 entries:
69 Num: Value Size Type Bind Vis Ndx Name
70 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
71 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS test.c
72 2: 0000000000000008 4 OBJECT LOCAL DEFAULT 4 l1
73 3: 000000000000000c 4 OBJECT LOCAL DEFAULT 4 l2
74 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 sec
75 5: 0000000000000000 128 FUNC GLOBAL DEFAULT 2 test
76 6: 0000000000000000 4 OBJECT GLOBAL DEFAULT 4 g1
77 7: 0000000000000004 4 OBJECT GLOBAL DEFAULT 4 g2
79 The 6th entry is global variable ``g1`` with value 0.
81 Similarly, the second relocation is at ``.text`` offset ``0x18``, instruction 3,
82 has a type of ``R_BPF_64_64`` and refers to entry 7 in the symbol table.
83 The second relocation resolves to global variable ``g2`` which has a symbol
84 value 4. The symbol value represents the offset from the start of ``.data``
85 section where the initial value of the global variable ``g2`` is stored.
87 The third and fourth relocations refer to static variables ``l1``
88 and ``l2``. From the ``.rel.text`` section above, it is not clear
89 to which symbols they really refer as they both refer to
90 symbol table entry 4, symbol ``sec``, which has ``STT_SECTION`` type
91 and represents a section. So for a static variable or function,
92 the section offset is written to the original insn
93 buffer, which is called ``A`` (addend). Looking at
94 above insn ``7`` and ``11``, they have section offset ``8`` and ``12``.
95 From symbol table, we can find that they correspond to entries ``2``
96 and ``3`` for ``l1`` and ``l2``.
98 In general, the ``A`` is 0 for global variables and functions,
99 and is the section offset or some computation result based on
100 section offset for static variables/functions. The non-section-offset
101 case refers to function calls. See below for more details.
103 Different Relocation Types
104 ==========================
106 Six relocation types are supported. The following is an overview and
107 ``S`` represents the value of the symbol in the symbol table::
109 Enum ELF Reloc Type Description BitSize Offset Calculation
111 1 R_BPF_64_64 ld_imm64 insn 32 r_offset + 4 S + A
112 2 R_BPF_64_ABS64 normal data 64 r_offset S + A
113 3 R_BPF_64_ABS32 normal data 32 r_offset S + A
114 4 R_BPF_64_NODYLD32 .BTF[.ext] data 32 r_offset S + A
115 10 R_BPF_64_32 call insn 32 r_offset + 4 (S + A) / 8 - 1
117 For example, ``R_BPF_64_64`` relocation type is used for ``ld_imm64`` instruction.
118 The actual to-be-relocated data (0 or section offset)
119 is stored at ``r_offset + 4`` and the read/write
120 data bitsize is 32 (4 bytes). The relocation can be resolved with
121 the symbol value plus implicit addend. Note that the ``BitSize`` is 32 which
122 means the section offset must be less than or equal to ``UINT32_MAX`` and this
123 is enforced by LLVM BPF backend.
125 In another case, ``R_BPF_64_ABS64`` relocation type is used for normal 64-bit data.
126 The actual to-be-relocated data is stored at ``r_offset`` and the read/write data
127 bitsize is 64 (8 bytes). The relocation can be resolved with
128 the symbol value plus implicit addend.
130 Both ``R_BPF_64_ABS32`` and ``R_BPF_64_NODYLD32`` types are for 32-bit data.
131 But ``R_BPF_64_NODYLD32`` specifically refers to relocations in ``.BTF`` and
132 ``.BTF.ext`` sections. For cases like bcc where llvm ``ExecutionEngine RuntimeDyld``
133 is involved, ``R_BPF_64_NODYLD32`` types of relocations should not be resolved
134 to actual function/variable address. Otherwise, ``.BTF`` and ``.BTF.ext``
135 become unusable by bcc and kernel.
137 Type ``R_BPF_64_32`` is used for call instruction. The call target section
138 offset is stored at ``r_offset + 4`` (32bit) and calculated as
144 Types ``R_BPF_64_64`` and ``R_BPF_64_32`` are used to resolve ``ld_imm64``
145 and ``call`` instructions. For example::
147 __attribute__((noinline)) __attribute__((section("sec1")))
148 int gfunc(int a, int b) {
151 static __attribute__((noinline)) __attribute__((section("sec1")))
152 int lfunc(int a, int b) {
155 int global __attribute__((section("sec2")));
156 int test(int a, int b) {
157 return gfunc(a, b) + lfunc(a, b) + global;
160 Compiled with ``clang --target=bpf -O2 -c test.c``, we will have
161 following code with `llvm-objdump -dr test.o``::
163 Disassembly of section .text:
165 0000000000000000 <test>:
166 0: bf 26 00 00 00 00 00 00 r6 = r2
167 1: bf 17 00 00 00 00 00 00 r7 = r1
168 2: 85 10 00 00 ff ff ff ff call -1
169 0000000000000010: R_BPF_64_32 gfunc
170 3: bf 08 00 00 00 00 00 00 r8 = r0
171 4: bf 71 00 00 00 00 00 00 r1 = r7
172 5: bf 62 00 00 00 00 00 00 r2 = r6
173 6: 85 10 00 00 02 00 00 00 call 2
174 0000000000000030: R_BPF_64_32 sec1
175 7: 0f 80 00 00 00 00 00 00 r0 += r8
176 8: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
177 0000000000000040: R_BPF_64_64 global
178 10: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
179 11: 0f 10 00 00 00 00 00 00 r0 += r1
180 12: 95 00 00 00 00 00 00 00 exit
182 Disassembly of section sec1:
184 0000000000000000 <gfunc>:
185 0: bf 20 00 00 00 00 00 00 r0 = r2
186 1: 2f 10 00 00 00 00 00 00 r0 *= r1
187 2: 95 00 00 00 00 00 00 00 exit
189 0000000000000018 <lfunc>:
190 3: bf 20 00 00 00 00 00 00 r0 = r2
191 4: 0f 10 00 00 00 00 00 00 r0 += r1
192 5: 95 00 00 00 00 00 00 00 exit
194 The first relocation corresponds to ``gfunc(a, b)`` where ``gfunc`` has a value of 0,
195 so the ``call`` instruction offset is ``(0 + 0)/8 - 1 = -1``.
196 The second relocation corresponds to ``lfunc(a, b)`` where ``lfunc`` has a section
197 offset ``0x18``, so the ``call`` instruction offset is ``(0 + 0x18)/8 - 1 = 2``.
198 The third relocation corresponds to ld_imm64 of ``global``, which has a section
201 The following is an example to show how R_BPF_64_ABS64 could be generated::
203 int global() { return 0; }
204 struct t { void *g; } gbl = { global };
206 Compiled with ``clang --target=bpf -O2 -g -c test.c``, we will see a
207 relocation below in ``.data`` section with command
208 ``llvm-readelf -r test.o``::
210 Relocation section '.rel.data' at offset 0x458 contains 1 entries:
211 Offset Info Type Symbol's Value Symbol's Name
212 0000000000000000 0000000700000002 R_BPF_64_ABS64 0000000000000000 global
214 The relocation says the first 8-byte of ``.data`` section should be
215 filled with address of ``global`` variable.
217 With ``llvm-readelf`` output, we can see that dwarf sections have a bunch of
218 ``R_BPF_64_ABS32`` and ``R_BPF_64_ABS64`` relocations::
220 Relocation section '.rel.debug_info' at offset 0x468 contains 13 entries:
221 Offset Info Type Symbol's Value Symbol's Name
222 0000000000000006 0000000300000003 R_BPF_64_ABS32 0000000000000000 .debug_abbrev
223 000000000000000c 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str
224 0000000000000012 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str
225 0000000000000016 0000000600000003 R_BPF_64_ABS32 0000000000000000 .debug_line
226 000000000000001a 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str
227 000000000000001e 0000000200000002 R_BPF_64_ABS64 0000000000000000 .text
228 000000000000002b 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str
229 0000000000000037 0000000800000002 R_BPF_64_ABS64 0000000000000000 gbl
230 0000000000000040 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str
233 The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
235 Relocation section '.rel.BTF' at offset 0x538 contains 1 entries:
236 Offset Info Type Symbol's Value Symbol's Name
237 0000000000000084 0000000800000004 R_BPF_64_NODYLD32 0000000000000000 gbl
239 Relocation section '.rel.BTF.ext' at offset 0x548 contains 2 entries:
240 Offset Info Type Symbol's Value Symbol's Name
241 000000000000002c 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text
242 0000000000000040 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text
244 .. _btf-co-re-relocations:
250 From object file point of view CO-RE mechanism is implemented as a set
251 of CO-RE specific relocation records. These relocation records are not
252 related to ELF relocations and are encoded in .BTF.ext section.
253 See :ref:`Documentation/bpf/btf.rst <BTF_Ext_Section>` for more
254 information on .BTF.ext structure.
256 CO-RE relocations are applied to BPF instructions to update immediate
257 or offset fields of the instruction at load time with information
258 relevant for target kernel.
260 Field to patch is selected basing on the instruction class:
262 * For BPF_ALU, BPF_ALU64, BPF_LD `immediate` field is patched;
263 * For BPF_LDX, BPF_STX, BPF_ST `offset` field is patched;
264 * BPF_JMP, BPF_JMP32 instructions **should not** be patched.
269 There are several kinds of CO-RE relocations that could be split in
272 * Field-based - patch instruction with field related information, e.g.
273 change offset field of the BPF_LDX instruction to reflect offset
274 of a specific structure field in the target kernel.
276 * Type-based - patch instruction with type related information, e.g.
277 change immediate field of the BPF_ALU move instruction to 0 or 1 to
278 reflect if specific type is present in the target kernel.
280 * Enum-based - patch instruction with enum related information, e.g.
281 change immediate field of the BPF_LD_IMM64 instruction to reflect
282 value of a specific enum literal in the target kernel.
284 The complete list of relocation kinds is represented by the following enum:
288 enum bpf_core_relo_kind {
289 BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */
290 BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */
291 BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */
292 BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
293 BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
294 BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
295 BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
296 BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */
297 BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */
298 BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */
299 BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
300 BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */
301 BPF_CORE_TYPE_MATCHES = 12, /* type match in target kernel */
306 * ``BPF_CORE_FIELD_LSHIFT_U64`` and ``BPF_CORE_FIELD_RSHIFT_U64`` are
307 supposed to be used to read bitfield values using the following
312 // To read bitfield ``f`` from ``struct s``
313 is_signed = relo(s->f, BPF_CORE_FIELD_SIGNED)
314 off = relo(s->f, BPF_CORE_FIELD_BYTE_OFFSET)
315 sz = relo(s->f, BPF_CORE_FIELD_BYTE_SIZE)
316 l = relo(s->f, BPF_CORE_FIELD_LSHIFT_U64)
317 r = relo(s->f, BPF_CORE_FIELD_RSHIFT_U64)
318 // define ``v`` as signed or unsigned integer of size ``sz``
319 v = *({s|u}<sz> *)((void *)s + off)
323 * The ``BPF_CORE_TYPE_MATCHES`` queries matching relation, defined as
326 * for integers: types match if size and signedness match;
327 * for arrays & pointers: target types are recursively matched;
328 * for structs & unions:
330 * local members need to exist in target with the same name;
332 * for each member we recursively check match unless it is already behind a
333 pointer, in which case we only check matching names and compatible kind;
337 * local variants have to have a match in target by symbolic name (but not
340 * size has to match (but enum may match enum64 and vice versa);
342 * for function pointers:
344 * number and position of arguments in local type has to match target;
345 * for each argument and the return value we recursively check match.
347 CO-RE Relocation Record
348 =======================
350 Relocation record is encoded as the following structure:
354 struct bpf_core_relo {
357 __u32 access_str_off;
358 enum bpf_core_relo_kind kind;
361 * ``insn_off`` - instruction offset (in bytes) within a code section
362 associated with this relocation;
364 * ``type_id`` - BTF type ID of the "root" (containing) entity of a
365 relocatable type or field;
367 * ``access_str_off`` - offset into corresponding .BTF string section.
368 String interpretation depends on specific relocation kind:
370 * for field-based relocations, string encodes an accessed field using
371 a sequence of field and array indices, separated by colon (:). It's
372 conceptually very close to LLVM's `getelementptr <GEP_>`_ instruction's
373 arguments for identifying offset to a field. For example, consider the
381 struct { int c[10]; };
382 } __attribute__((preserve_access_index));
385 * Access to ``s[0].a`` would be encoded as ``0:0``:
387 * ``0``: first element of ``s`` (as if ``s`` is an array);
388 * ``0``: index of field ``a`` in ``struct sample``.
390 * Access to ``s->a`` would be encoded as ``0:0`` as well.
391 * Access to ``s->b`` would be encoded as ``0:1``:
393 * ``0``: first element of ``s``;
394 * ``1``: index of field ``b`` in ``struct sample``.
396 * Access to ``s[1].c[5]`` would be encoded as ``1:2:0:5``:
398 * ``1``: second element of ``s``;
399 * ``2``: index of anonymous structure field in ``struct sample``;
400 * ``0``: index of field ``c`` in anonymous structure;
401 * ``5``: access to array element #5.
403 * for type-based relocations, string is expected to be just "0";
405 * for enum value-based relocations, string contains an index of enum
406 value within its enum type;
408 * ``kind`` - one of ``enum bpf_core_relo_kind``.
410 .. _GEP: https://llvm.org/docs/LangRef.html#getelementptr-instruction
412 .. _btf_co_re_relocation_examples:
414 CO-RE Relocation Examples
415 =========================
417 For the following C code:
425 } __attribute__((preserve_access_index));
429 With the following BTF definitions:
434 [2] STRUCT 'foo' size=8 vlen=2
435 'a' type_id=3 bits_offset=0
436 'b' type_id=3 bits_offset=32
437 'c' type_id=4 bits_offset=64 bitfield_size=15
438 [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
439 [4] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
441 [16] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
445 Field offset relocations are generated automatically when
446 ``__attribute__((preserve_access_index))`` is used, for example:
450 void alpha(struct foo *s, volatile unsigned long *g) {
456 0: r3 = *(s32 *)(r1 + 0x0)
457 00: CO-RE <byte_off> [2] struct foo::a (0:0)
458 1: *(u64 *)(r2 + 0x0) = r3
459 2: *(u32 *)(r1 + 0x0) = 0x1
460 10: CO-RE <byte_off> [2] struct foo::a (0:0)
464 All relocation kinds could be requested via built-in functions.
465 E.g. field-based relocations:
469 void bravo(struct foo *s, volatile unsigned long *g) {
470 *g = __builtin_preserve_field_info(s->b, 0 /* field byte offset */);
471 *g = __builtin_preserve_field_info(s->b, 1 /* field byte size */);
472 *g = __builtin_preserve_field_info(s->b, 2 /* field existence */);
473 *g = __builtin_preserve_field_info(s->b, 3 /* field signedness */);
474 *g = __builtin_preserve_field_info(s->c, 4 /* bitfield left shift */);
475 *g = __builtin_preserve_field_info(s->c, 5 /* bitfield right shift */);
480 20: CO-RE <byte_off> [2] struct foo::b (0:1)
481 5: *(u64 *)(r2 + 0x0) = r1
483 30: CO-RE <byte_sz> [2] struct foo::b (0:1)
484 7: *(u64 *)(r2 + 0x0) = r1
486 40: CO-RE <field_exists> [2] struct foo::b (0:1)
487 9: *(u64 *)(r2 + 0x0) = r1
489 50: CO-RE <signed> [2] struct foo::b (0:1)
490 11: *(u64 *)(r2 + 0x0) = r1
492 60: CO-RE <lshift_u64> [2] struct foo::c (0:2)
493 13: *(u64 *)(r2 + 0x0) = r1
495 70: CO-RE <rshift_u64> [2] struct foo::c (0:2)
496 15: *(u64 *)(r2 + 0x0) = r1
500 Type-based relocations:
504 void charlie(struct foo *s, volatile unsigned long *g) {
505 *g = __builtin_preserve_type_info(*s, 0 /* type existence */);
506 *g = __builtin_preserve_type_info(*s, 1 /* type size */);
507 *g = __builtin_preserve_type_info(*s, 2 /* type matches */);
508 *g = __builtin_btf_type_id(*s, 0 /* type id in this object file */);
509 *g = __builtin_btf_type_id(*s, 1 /* type id in target kernel */);
514 88: CO-RE <type_exists> [2] struct foo
515 18: *(u64 *)(r2 + 0x0) = r1
517 98: CO-RE <type_size> [2] struct foo
518 20: *(u64 *)(r2 + 0x0) = r1
520 a8: CO-RE <type_matches> [2] struct foo
521 22: *(u64 *)(r2 + 0x0) = r1
523 b8: CO-RE <local_type_id> [2] struct foo
524 25: *(u64 *)(r2 + 0x0) = r1
526 d0: CO-RE <target_type_id> [2] struct foo
527 28: *(u64 *)(r2 + 0x0) = r1
530 Enum-based relocations:
534 void delta(struct foo *s, volatile unsigned long *g) {
535 *g = __builtin_preserve_enum_value(*(enum bar *)U, 0 /* enum literal existence */);
536 *g = __builtin_preserve_enum_value(*(enum bar *)V, 1 /* enum literal value */);
541 f0: CO-RE <enumval_exists> [16] enum bar::U = 0
542 32: *(u64 *)(r2 + 0x0) = r1
544 108: CO-RE <enumval_value> [16] enum bar::V = 1
545 35: *(u64 *)(r2 + 0x0) = r1