2012-05-05 10:18:41 +02:00
|
|
|
#!/bin/sh
|
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 15:07:57 +01:00
|
|
|
# SPDX-License-Identifier: GPL-2.0
|
2012-05-05 10:18:41 +02:00
|
|
|
#
|
|
|
|
|
# link vmlinux
|
|
|
|
|
#
|
2022-09-25 03:19:14 +09:00
|
|
|
# vmlinux is linked from the objects in vmlinux.a and $(KBUILD_VMLINUX_LIBS).
|
|
|
|
|
# vmlinux.a contains objects that are linked unconditionally.
|
2019-01-17 09:10:04 +09:00
|
|
|
# $(KBUILD_VMLINUX_LIBS) are archives which are linked conditionally
|
|
|
|
|
# (not within --whole-archive), and do not require symbol indexes added.
|
2012-05-05 10:18:41 +02:00
|
|
|
#
|
|
|
|
|
# vmlinux
|
|
|
|
|
# ^
|
|
|
|
|
# |
|
2022-09-25 03:19:14 +09:00
|
|
|
# +--< vmlinux.a
|
2012-05-05 10:18:41 +02:00
|
|
|
# |
|
2017-06-20 01:52:05 +10:00
|
|
|
# +--< $(KBUILD_VMLINUX_LIBS)
|
|
|
|
|
# | +--< lib/lib.a + more
|
|
|
|
|
# |
|
2012-05-05 10:18:41 +02:00
|
|
|
# +-< ${kallsymso} (see description in KALLSYMS section)
|
|
|
|
|
#
|
|
|
|
|
# vmlinux version (uname -v) cannot be updated during normal
|
|
|
|
|
# descending-into-subdirs phase since we do not yet know if we need to
|
|
|
|
|
# update vmlinux.
|
|
|
|
|
# Therefore this step is delayed until just before final link of vmlinux.
|
|
|
|
|
#
|
|
|
|
|
# System.map is generated to document addresses of all kernel symbols
|
|
|
|
|
|
|
|
|
|
# Error out on error
|
|
|
|
|
set -e
|
|
|
|
|
|
kbuild: do not export LDFLAGS_vmlinux
When you clean the build tree for ARCH=arm, you may see the following
error message from 'nm' command:
$ make -j24 ARCH=arm clean
CLEAN arch/arm/crypto
CLEAN arch/arm/kernel
CLEAN arch/arm/mach-at91
CLEAN arch/arm/mach-omap2
CLEAN arch/arm/vdso
CLEAN certs
CLEAN lib
CLEAN usr
CLEAN net/wireless
CLEAN drivers/firmware/efi/libstub
nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
/bin/sh: 1: arithmetic expression: expecting primary: " "
CLEAN arch/arm/boot/compressed
CLEAN drivers/scsi
CLEAN drivers/tty/vt
CLEAN arch/arm/boot
CLEAN vmlinux.symvers modules.builtin modules.builtin.modinfo
Even if you rerun the same command, the error message will not be
shown despite vmlinux is already gone.
To reproduce it, the parallel option -j is needed. Single thread
cleaning always executes 'archclean', 'vmlinuxclean' in this order,
so vmlinux still exists when arch/arm/boot/compressed/ is cleaned.
Looking at arch/arm/boot/compressed/Makefile does not help understand
the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are
assigned with '=' operator, hence, they are not expanded unless used.
Obviously, 'make clean' does not use them.
In fact, the root cause exists in the top Makefile:
export LDFLAGS_vmlinux
Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in
arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean
has a command to execute. This is why the error message shows up only
when there exist build artifacts in arch/arm/boot/compressed/.
Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile
will fix it as far as ARCH=arm is concerned, but I think the proper fix
is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile.
LDFLAGS_vmlinux in the top Makefile contains linker flags for the top
vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for
arch/arm/boot/compressed/vmlinux. They just happen to have the same
variable name, but are used for different purposes. Stop shadowing
LDFLAGS_vmlinux.
This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a
command line parameter instead of via an environment variable. LD and
KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway,
they must be included in cmd_link-vmlinux to allow if_changed to detect
the changes in LD or KBUILD_LDFLAGS.
The following Makefiles are not affected:
arch/arm/boot/compressed/Makefile
arch/h8300/boot/compressed/Makefile
arch/nios2/boot/compressed/Makefile
arch/parisc/boot/compressed/Makefile
arch/s390/boot/compressed/Makefile
arch/sh/boot/compressed/Makefile
arch/sh/boot/romimage/Makefile
arch/x86/boot/compressed/Makefile
They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the
top Makefile.
We need to take a closer look at the impact to unicore32 and xtensa.
arch/unicore32/boot/compressed/Makefile only uses '+=' operator for
LDFLAGS_vmlinux. So, the decompressor previously inherited the linker
flags from the top Makefile.
However, commit 70fac51feaf2 ("unicore32 additional architecture files:
boot process") was merged before commit 1f2bfbd00e46 ("kbuild: link of
vmlinux moved to a script"). So, I rather consider this is a bug fix of
1f2bfbd00e46.
arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also
considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux
when commit 4bedea945451 ("[PATCH] xtensa: Architecture support for
Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux),
which is now empty.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2020-07-02 04:29:36 +09:00
|
|
|
LD="$1"
|
|
|
|
|
KBUILD_LDFLAGS="$2"
|
|
|
|
|
LDFLAGS_vmlinux="$3"
|
2025-03-11 12:06:18 +01:00
|
|
|
VMLINUX="$4"
|
kbuild: do not export LDFLAGS_vmlinux
When you clean the build tree for ARCH=arm, you may see the following
error message from 'nm' command:
$ make -j24 ARCH=arm clean
CLEAN arch/arm/crypto
CLEAN arch/arm/kernel
CLEAN arch/arm/mach-at91
CLEAN arch/arm/mach-omap2
CLEAN arch/arm/vdso
CLEAN certs
CLEAN lib
CLEAN usr
CLEAN net/wireless
CLEAN drivers/firmware/efi/libstub
nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
/bin/sh: 1: arithmetic expression: expecting primary: " "
CLEAN arch/arm/boot/compressed
CLEAN drivers/scsi
CLEAN drivers/tty/vt
CLEAN arch/arm/boot
CLEAN vmlinux.symvers modules.builtin modules.builtin.modinfo
Even if you rerun the same command, the error message will not be
shown despite vmlinux is already gone.
To reproduce it, the parallel option -j is needed. Single thread
cleaning always executes 'archclean', 'vmlinuxclean' in this order,
so vmlinux still exists when arch/arm/boot/compressed/ is cleaned.
Looking at arch/arm/boot/compressed/Makefile does not help understand
the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are
assigned with '=' operator, hence, they are not expanded unless used.
Obviously, 'make clean' does not use them.
In fact, the root cause exists in the top Makefile:
export LDFLAGS_vmlinux
Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in
arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean
has a command to execute. This is why the error message shows up only
when there exist build artifacts in arch/arm/boot/compressed/.
Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile
will fix it as far as ARCH=arm is concerned, but I think the proper fix
is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile.
LDFLAGS_vmlinux in the top Makefile contains linker flags for the top
vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for
arch/arm/boot/compressed/vmlinux. They just happen to have the same
variable name, but are used for different purposes. Stop shadowing
LDFLAGS_vmlinux.
This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a
command line parameter instead of via an environment variable. LD and
KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway,
they must be included in cmd_link-vmlinux to allow if_changed to detect
the changes in LD or KBUILD_LDFLAGS.
The following Makefiles are not affected:
arch/arm/boot/compressed/Makefile
arch/h8300/boot/compressed/Makefile
arch/nios2/boot/compressed/Makefile
arch/parisc/boot/compressed/Makefile
arch/s390/boot/compressed/Makefile
arch/sh/boot/compressed/Makefile
arch/sh/boot/romimage/Makefile
arch/x86/boot/compressed/Makefile
They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the
top Makefile.
We need to take a closer look at the impact to unicore32 and xtensa.
arch/unicore32/boot/compressed/Makefile only uses '+=' operator for
LDFLAGS_vmlinux. So, the decompressor previously inherited the linker
flags from the top Makefile.
However, commit 70fac51feaf2 ("unicore32 additional architecture files:
boot process") was merged before commit 1f2bfbd00e46 ("kbuild: link of
vmlinux moved to a script"). So, I rather consider this is a bug fix of
1f2bfbd00e46.
arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also
considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux
when commit 4bedea945451 ("[PATCH] xtensa: Architecture support for
Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux),
which is now empty.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2020-07-02 04:29:36 +09:00
|
|
|
|
2021-12-14 11:53:52 +09:00
|
|
|
is_enabled() {
|
|
|
|
|
grep -q "^$1=y" include/config/auto.conf
|
|
|
|
|
}
|
|
|
|
|
|
2012-05-05 10:18:41 +02:00
|
|
|
# Nice output in kbuild format
|
|
|
|
|
# Will be supressed by "make -s"
|
|
|
|
|
info()
|
|
|
|
|
{
|
2021-05-17 16:03:14 +09:00
|
|
|
printf " %-7s %s\n" "${1}" "${2}"
|
2012-05-05 10:18:41 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Link of vmlinux
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
# ${1} - output file
|
2012-05-05 10:18:41 +02:00
|
|
|
vmlinux_link()
|
|
|
|
|
{
|
2019-09-05 10:59:38 -07:00
|
|
|
local output=${1}
|
2021-08-19 09:57:37 +09:00
|
|
|
local objs
|
|
|
|
|
local libs
|
2021-08-19 09:57:39 +09:00
|
|
|
local ld
|
|
|
|
|
local ldflags
|
|
|
|
|
local ldlibs
|
2012-05-05 10:18:41 +02:00
|
|
|
|
2019-09-20 08:36:47 -07:00
|
|
|
info LD ${output}
|
|
|
|
|
|
2019-09-05 10:59:38 -07:00
|
|
|
# skip output file argument
|
|
|
|
|
shift
|
|
|
|
|
|
2025-09-17 09:04:05 -07:00
|
|
|
if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT ||
|
|
|
|
|
is_enabled CONFIG_KLP_BUILD; then
|
2021-08-19 09:57:37 +09:00
|
|
|
# Use vmlinux.o instead of performing the slow LTO link again.
|
|
|
|
|
objs=vmlinux.o
|
|
|
|
|
libs=
|
|
|
|
|
else
|
2022-09-25 03:19:14 +09:00
|
|
|
objs=vmlinux.a
|
2021-08-19 09:57:37 +09:00
|
|
|
libs="${KBUILD_VMLINUX_LIBS}"
|
|
|
|
|
fi
|
|
|
|
|
|
2024-09-23 16:56:03 +09:00
|
|
|
if is_enabled CONFIG_GENERIC_BUILTIN_DTB; then
|
|
|
|
|
objs="${objs} .builtin-dtbs.o"
|
|
|
|
|
fi
|
|
|
|
|
|
2025-09-18 10:05:52 +02:00
|
|
|
objs="${objs} .vmlinux.export.o"
|
kbuild: build init/built-in.a just once
Kbuild builds init/built-in.a twice; first during the ordinary
directory descending, second from scripts/link-vmlinux.sh.
We do this because UTS_VERSION contains the build version and the
timestamp. We cannot update it during the normal directory traversal
since we do not yet know if we need to update vmlinux. UTS_VERSION is
temporarily calculated, but omitted from the update check. Otherwise,
vmlinux would be rebuilt every time.
When Kbuild results in running link-vmlinux.sh, it increments the
version number in the .version file and takes the timestamp at that
time to really fix UTS_VERSION.
However, updating the same file twice is a footgun. To avoid nasty
timestamp issues, all build artifacts that depend on init/built-in.a
are atomically generated in link-vmlinux.sh, where some of them do not
need rebuilding.
To fix this issue, this commit changes as follows:
[1] Split UTS_VERSION out to include/generated/utsversion.h from
include/generated/compile.h
include/generated/utsversion.h is generated just before the
vmlinux link. It is generated under include/generated/ because
some decompressors (s390, x86) use UTS_VERSION.
[2] Split init_uts_ns and linux_banner out to init/version-timestamp.c
from init/version.c
init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary
directory descending, they are compiled with __weak and used to
determine if vmlinux needs relinking. Just before the vmlinux link,
they are compiled without __weak to embed the real version and
timestamp.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-08-28 11:39:53 +09:00
|
|
|
objs="${objs} init/version-timestamp.o"
|
|
|
|
|
|
2021-08-19 09:57:39 +09:00
|
|
|
if [ "${SRCARCH}" = "um" ]; then
|
|
|
|
|
wl=-Wl,
|
|
|
|
|
ld="${CC}"
|
|
|
|
|
ldflags="${CFLAGS_vmlinux}"
|
|
|
|
|
ldlibs="-lutil -lrt -lpthread"
|
|
|
|
|
else
|
|
|
|
|
wl=
|
|
|
|
|
ld="${LD}"
|
|
|
|
|
ldflags="${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}"
|
|
|
|
|
ldlibs=
|
|
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
|
|
|
|
|
|
2020-03-03 18:18:34 -08:00
|
|
|
# The kallsyms linking does not need debug symbols included.
|
2024-06-10 20:25:16 +09:00
|
|
|
if [ -n "${strip_debug}" ] ; then
|
2021-08-19 09:57:39 +09:00
|
|
|
ldflags="${ldflags} ${wl}--strip-debug"
|
2020-03-03 18:18:34 -08:00
|
|
|
fi
|
|
|
|
|
|
2025-03-13 19:26:03 +09:00
|
|
|
if [ -n "${generate_map}" ]; then
|
|
|
|
|
ldflags="${ldflags} ${wl}-Map=vmlinux.map"
|
2021-03-05 10:27:07 +01:00
|
|
|
fi
|
|
|
|
|
|
2021-08-19 09:57:39 +09:00
|
|
|
${ld} ${ldflags} -o ${output} \
|
|
|
|
|
${wl}--whole-archive ${objs} ${wl}--no-whole-archive \
|
|
|
|
|
${wl}--start-group ${libs} ${wl}--end-group \
|
2024-10-30 12:38:44 +05:30
|
|
|
${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
|
2012-05-05 10:18:41 +02:00
|
|
|
}
|
|
|
|
|
|
2024-06-10 20:25:16 +09:00
|
|
|
# Create ${2}.o file with all symbols from the ${1} object file
|
2012-05-05 10:18:41 +02:00
|
|
|
kallsyms()
|
|
|
|
|
{
|
|
|
|
|
local kallsymopt;
|
|
|
|
|
|
2021-12-14 11:53:52 +09:00
|
|
|
if is_enabled CONFIG_KALLSYMS_ALL; then
|
2012-09-06 22:11:25 +01:00
|
|
|
kallsymopt="${kallsymopt} --all-symbols"
|
2012-05-05 10:18:41 +02:00
|
|
|
fi
|
|
|
|
|
|
2024-06-10 20:25:16 +09:00
|
|
|
info KSYMS "${2}.S"
|
|
|
|
|
scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S"
|
|
|
|
|
|
|
|
|
|
info AS "${2}.o"
|
|
|
|
|
${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
|
|
|
|
|
${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} -c -o "${2}.o" "${2}.S"
|
|
|
|
|
|
|
|
|
|
kallsymso=${2}.o
|
2012-05-05 10:18:41 +02:00
|
|
|
}
|
|
|
|
|
|
2024-06-10 20:25:18 +09:00
|
|
|
# Perform kallsyms for the given temporary vmlinux.
|
|
|
|
|
sysmap_and_kallsyms()
|
2019-08-13 08:15:32 -07:00
|
|
|
{
|
2024-06-10 20:25:18 +09:00
|
|
|
mksysmap "${1}" "${1}.syms"
|
|
|
|
|
kallsyms "${1}.syms" "${1}.kallsyms"
|
2019-08-13 08:15:32 -07:00
|
|
|
|
2024-06-10 20:25:18 +09:00
|
|
|
kallsyms_sysmap=${1}.syms
|
2019-08-13 08:15:32 -07:00
|
|
|
}
|
|
|
|
|
|
2012-05-05 10:18:41 +02:00
|
|
|
# Create map file with all symbols from ${1}
|
|
|
|
|
# See mksymap for additional details
|
|
|
|
|
mksysmap()
|
|
|
|
|
{
|
2022-09-26 18:02:26 +09:00
|
|
|
info NM ${2}
|
2024-06-05 13:58:09 -04:00
|
|
|
${NM} -n "${1}" | sed -f "${srctree}/scripts/mksysmap" > "${2}"
|
2012-05-05 10:18:41 +02:00
|
|
|
}
|
|
|
|
|
|
2019-12-04 08:46:31 +08:00
|
|
|
sorttable()
|
2012-05-28 10:32:28 -07:00
|
|
|
{
|
scripts/sorttable: Zero out weak functions in mcount_loc table
When a function is annotated as "weak" and is overridden, the code is not
removed. If it is traced, the fentry/mcount location in the weak function
will be referenced by the "__mcount_loc" section. This will then be added
to the available_filter_functions list. Since only the address of the
functions are listed, to find the name to show, a search of kallsyms is
used.
Since kallsyms will return the function by simply finding the function
that the address is after but before the next function, an address of a
weak function will show up as the function before it. This is because
kallsyms does not save names of weak functions. This has caused issues in
the past, as now the traced weak function will be listed in
available_filter_functions with the name of the function before it.
At best, this will cause the previous function's name to be listed twice.
At worse, if the previous function was marked notrace, it will now show up
as a function that can be traced. Note that it only shows up that it can
be traced but will not be if enabled, which causes confusion.
https://lore.kernel.org/all/20220412094923.0abe90955e5db486b7bca279@kernel.org/
The commit b39181f7c6907 ("ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid
adding weak function") was a workaround to this by checking the function
address before printing its name. If the address was too far from the
function given by the name then instead of printing the name it would
print: __ftrace_invalid_address___<invalid-offset>
The real issue is that these invalid addresses are listed in the ftrace
table look up which available_filter_functions is derived from. A place
holder must be listed in that file because set_ftrace_filter may take a
series of indexes into that file instead of names to be able to do O(1)
lookups to enable filtering (many tools use this method).
Even if kallsyms saved the size of the function, it does not remove the
need of having these place holders. The real solution is to not add a weak
function into the ftrace table in the first place.
To solve this, the sorttable.c code that sorts the mcount regions during
the build is modified to take a "nm -S vmlinux" input, sort it, and any
function listed in the mcount_loc section that is not within a boundary of
the function list given by nm is considered a weak function and is zeroed
out.
Note, this does not mean they will remain zero when booting as KASLR
will still shift those addresses. To handle this, the entries in the
mcount_loc section will be ignored if they are zero or match the
kaslr_offset() value.
Before:
~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l
551
After:
~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l
0
Cc: bpf <bpf@vger.kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nicolas Schier <nicolas@fjasle.eu>
Cc: Zheng Yejian <zhengyejian1@huawei.com>
Cc: Martin Kelly <martin.kelly@crowdstrike.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Link: https://lore.kernel.org/20250218200022.883095980@goodmis.org
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-02-18 14:59:22 -05:00
|
|
|
${NM} -S ${1} > .tmp_vmlinux.nm-sort
|
|
|
|
|
${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1}
|
2012-05-28 10:32:28 -07:00
|
|
|
}
|
|
|
|
|
|
2012-05-05 10:18:41 +02:00
|
|
|
cleanup()
|
|
|
|
|
{
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
rm -f .btf.*
|
scripts/sorttable: Zero out weak functions in mcount_loc table
When a function is annotated as "weak" and is overridden, the code is not
removed. If it is traced, the fentry/mcount location in the weak function
will be referenced by the "__mcount_loc" section. This will then be added
to the available_filter_functions list. Since only the address of the
functions are listed, to find the name to show, a search of kallsyms is
used.
Since kallsyms will return the function by simply finding the function
that the address is after but before the next function, an address of a
weak function will show up as the function before it. This is because
kallsyms does not save names of weak functions. This has caused issues in
the past, as now the traced weak function will be listed in
available_filter_functions with the name of the function before it.
At best, this will cause the previous function's name to be listed twice.
At worse, if the previous function was marked notrace, it will now show up
as a function that can be traced. Note that it only shows up that it can
be traced but will not be if enabled, which causes confusion.
https://lore.kernel.org/all/20220412094923.0abe90955e5db486b7bca279@kernel.org/
The commit b39181f7c6907 ("ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid
adding weak function") was a workaround to this by checking the function
address before printing its name. If the address was too far from the
function given by the name then instead of printing the name it would
print: __ftrace_invalid_address___<invalid-offset>
The real issue is that these invalid addresses are listed in the ftrace
table look up which available_filter_functions is derived from. A place
holder must be listed in that file because set_ftrace_filter may take a
series of indexes into that file instead of names to be able to do O(1)
lookups to enable filtering (many tools use this method).
Even if kallsyms saved the size of the function, it does not remove the
need of having these place holders. The real solution is to not add a weak
function into the ftrace table in the first place.
To solve this, the sorttable.c code that sorts the mcount regions during
the build is modified to take a "nm -S vmlinux" input, sort it, and any
function listed in the mcount_loc section that is not within a boundary of
the function list given by nm is considered a weak function and is zeroed
out.
Note, this does not mean they will remain zero when booting as KASLR
will still shift those addresses. To handle this, the entries in the
mcount_loc section will be ignored if they are zero or match the
kaslr_offset() value.
Before:
~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l
551
After:
~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l
0
Cc: bpf <bpf@vger.kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nicolas Schier <nicolas@fjasle.eu>
Cc: Zheng Yejian <zhengyejian1@huawei.com>
Cc: Martin Kelly <martin.kelly@crowdstrike.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Link: https://lore.kernel.org/20250218200022.883095980@goodmis.org
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-02-18 14:59:22 -05:00
|
|
|
rm -f .tmp_vmlinux.nm-sort
|
2012-05-05 10:18:41 +02:00
|
|
|
rm -f System.map
|
|
|
|
|
rm -f vmlinux
|
2021-03-05 10:27:07 +01:00
|
|
|
rm -f vmlinux.map
|
2012-05-05 10:18:41 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Use "make V=1" to debug this script
|
|
|
|
|
case "${KBUILD_VERBOSE}" in
|
|
|
|
|
*1*)
|
|
|
|
|
set -x
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
|
|
|
|
|
if [ "$1" = "clean" ]; then
|
|
|
|
|
cleanup
|
|
|
|
|
exit 0
|
|
|
|
|
fi
|
|
|
|
|
|
kbuild: build init/built-in.a just once
Kbuild builds init/built-in.a twice; first during the ordinary
directory descending, second from scripts/link-vmlinux.sh.
We do this because UTS_VERSION contains the build version and the
timestamp. We cannot update it during the normal directory traversal
since we do not yet know if we need to update vmlinux. UTS_VERSION is
temporarily calculated, but omitted from the update check. Otherwise,
vmlinux would be rebuilt every time.
When Kbuild results in running link-vmlinux.sh, it increments the
version number in the .version file and takes the timestamp at that
time to really fix UTS_VERSION.
However, updating the same file twice is a footgun. To avoid nasty
timestamp issues, all build artifacts that depend on init/built-in.a
are atomically generated in link-vmlinux.sh, where some of them do not
need rebuilding.
To fix this issue, this commit changes as follows:
[1] Split UTS_VERSION out to include/generated/utsversion.h from
include/generated/compile.h
include/generated/utsversion.h is generated just before the
vmlinux link. It is generated under include/generated/ because
some decompressors (s390, x86) use UTS_VERSION.
[2] Split init_uts_ns and linux_banner out to init/version-timestamp.c
from init/version.c
init_uts_ns and linux_banner contain UTS_VERSION. During the ordinary
directory descending, they are compiled with __weak and used to
determine if vmlinux needs relinking. Just before the vmlinux link,
they are compiled without __weak to embed the real version and
timestamp.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-08-28 11:39:53 +09:00
|
|
|
${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init init/version-timestamp.o
|
|
|
|
|
|
2024-10-30 12:38:44 +05:30
|
|
|
arch_vmlinux_o=
|
|
|
|
|
if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
|
|
|
|
|
arch_vmlinux_o=arch/${SRCARCH}/tools/vmlinux.arch.o
|
|
|
|
|
fi
|
|
|
|
|
|
2024-06-10 20:25:16 +09:00
|
|
|
btf_vmlinux_bin_o=
|
resolve_btfids: Change in-place update with raw binary output
Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
Change resolve_btfids behavior to enable BTF transformations as part
of its main operation. To achieve this, in-place ELF write in
resolve_btfids is replaced with generation of the following binaries:
* ${1}.BTF with .BTF section data
* ${1}.BTF_ids with .BTF_ids section data if it existed in ${1}
* ${1}.BTF.base with .BTF.base section data for out-of-tree modules
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The motivation for emitting binary data is that it allows simplifying
resolve_btfids implementation by delegating ELF update to the $OBJCOPY
tool [1], which is already widely used across the codebase.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking. The patch doesn't change this approach, only
the implementation is a little different.
Before this patch it worked as follows:
* pahole consumed .tmp_vmlinux1 [2] and added .BTF section with
llvm-objcopy [3] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [4],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with
raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For kernel modules, creating a special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with objcopy.
With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows deleting elf_update() call and satellite
code (like compressed_section_fix [5]).
Endianness handling of .BTF_ids data is also changed. Previously the
"flags" part of the section was bswapped in sets_patch() [6], and then
Elf_Type was modified before elf_update() to signal to libelf that
bswap may be necessary. With this patch we explicitly bswap entire
data buffer on load and on dump.
[1] https://lore.kernel.org/bpf/131b4190-9c49-4f79-a99d-c00fac97fa44@linux.dev/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n110
[3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c?h=v1.31#n1803
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n284
[5] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
[6] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20251219181825.1289460-3-ihor.solodrai@linux.dev
2025-12-19 10:18:25 -08:00
|
|
|
btfids_vmlinux=
|
2024-06-10 20:25:16 +09:00
|
|
|
kallsymso=
|
|
|
|
|
strip_debug=
|
2025-03-13 19:26:03 +09:00
|
|
|
generate_map=
|
2024-06-10 20:25:16 +09:00
|
|
|
|
tracing: Add a tracepoint verification check at build time
If a tracepoint is defined via DECLARE_TRACE() or TRACE_EVENT() but never
called (via the trace_<tracepoint>() function), its metadata is still
around in memory and not discarded.
When created via TRACE_EVENT() the situation is worse because the
TRACE_EVENT() creates metadata that can be around 5k per trace event.
Having unused trace events causes several thousand of wasted bytes.
Add a verifier that injects a string of the name of the tracepoint it
calls that is added to the discarded section "__tracepoint_check".
For every builtin tracepoint, its name (which is saved in the in-memory
section "__tracepoint_strings") will have its name also in the
"__tracepoint_check" section if it is used.
Add a new program that is run on build called tracepoint-update. This is
executed on the vmlinux.o before the __tracepoint_check section is
discarded (the section is discarded before vmlinux is created). This
program will create an array of each string in the __tracepoint_check
section and then sort it. Then it will walk the strings in the
__tracepoint_strings section and do a binary search to check if its name
is in the __tracepoint_check section. If it is not, then it is unused and
a warning is printed.
Note, this currently only handles tracepoints that are builtin and not in
modules.
Enabling this currently with a given config produces:
warning: tracepoint 'sched_move_numa' is unused.
warning: tracepoint 'sched_stick_numa' is unused.
warning: tracepoint 'sched_swap_numa' is unused.
warning: tracepoint 'pelt_hw_tp' is unused.
warning: tracepoint 'pelt_irq_tp' is unused.
warning: tracepoint 'rcu_preempt_task' is unused.
warning: tracepoint 'rcu_unlock_preempted_task' is unused.
warning: tracepoint 'xdp_bulk_tx' is unused.
warning: tracepoint 'xdp_redirect_map' is unused.
warning: tracepoint 'xdp_redirect_map_err' is unused.
warning: tracepoint 'vma_mas_szero' is unused.
warning: tracepoint 'vma_store' is unused.
warning: tracepoint 'hugepage_set_pmd' is unused.
warning: tracepoint 'hugepage_set_pud' is unused.
warning: tracepoint 'hugepage_update_pmd' is unused.
warning: tracepoint 'hugepage_update_pud' is unused.
warning: tracepoint 'block_rq_remap' is unused.
warning: tracepoint 'xhci_dbc_handle_event' is unused.
warning: tracepoint 'xhci_dbc_handle_transfer' is unused.
warning: tracepoint 'xhci_dbc_gadget_ep_queue' is unused.
warning: tracepoint 'xhci_dbc_alloc_request' is unused.
warning: tracepoint 'xhci_dbc_free_request' is unused.
warning: tracepoint 'xhci_dbc_queue_request' is unused.
warning: tracepoint 'xhci_dbc_giveback_request' is unused.
warning: tracepoint 'tcp_ao_wrong_maclen' is unused.
warning: tracepoint 'tcp_ao_mismatch' is unused.
warning: tracepoint 'tcp_ao_key_not_found' is unused.
warning: tracepoint 'tcp_ao_rnext_request' is unused.
warning: tracepoint 'tcp_ao_synack_no_key' is unused.
warning: tracepoint 'tcp_ao_snd_sne_update' is unused.
warning: tracepoint 'tcp_ao_rcv_sne_update' is unused.
Some of the above is totally unused but others are not used due to their
"trace_" functions being inside configs, in which case, the defined
tracepoints should also be inside those same configs. Others are
architecture specific but defined in generic code, where they should
either be moved to the architecture or be surrounded by #ifdef for the
architectures they are for.
This tool could be updated to process modules in the future.
I'd like to thank Mathieu Desnoyers for suggesting using strings instead
of pointers, as using pointers in vmlinux.o required handling relocations
and it required implementing almost a full feature linker to do so.
To enable this check, run the build with: make UT=1
Note, when all the existing unused tracepoints are removed from the build,
the "UT=1" will be removed and this will always be enabled when
tracepoints are configured to warn on any new tracepoints. The reason this
isn't always enabled now is because it will introduce a lot of warnings
for the current unused tracepoints, and all bisects would end at this
commit for those warnings.
Link: https://lore.kernel.org/all/20250528114549.4d8a5e03@gandalf.local.home/
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nicolas Schier <nicolas.schier@linux.dev>
Cc: Nick Desaulniers <nick.desaulniers+lkml@gmail.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Link: https://lore.kernel.org/20251022004452.920728129@kernel.org
Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> # for using strings instead of pointers
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-10-21 20:43:40 -04:00
|
|
|
# Use "make UT=1" to trigger warnings on unused tracepoints
|
|
|
|
|
case "${WARN_ON_UNUSED_TRACEPOINTS}" in
|
|
|
|
|
*1*)
|
|
|
|
|
${objtree}/scripts/tracepoint-update vmlinux.o
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
|
2024-06-10 20:25:17 +09:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2024-11-26 16:58:01 +01:00
|
|
|
true > .tmp_vmlinux0.syms
|
|
|
|
|
kallsyms .tmp_vmlinux0.syms .tmp_vmlinux0.kallsyms
|
2024-06-10 20:25:18 +09:00
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
|
|
|
|
|
|
|
|
|
|
# The kallsyms linking does not need debug symbols, but the BTF does.
|
|
|
|
|
if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
|
|
|
|
|
strip_debug=1
|
|
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
vmlinux_link .tmp_vmlinux1
|
2024-06-10 20:25:17 +09:00
|
|
|
fi
|
|
|
|
|
|
2021-12-14 11:53:52 +09:00
|
|
|
if is_enabled CONFIG_DEBUG_INFO_BTF; then
|
2025-12-31 10:39:29 -08:00
|
|
|
info BTF .tmp_vmlinux1
|
2026-01-21 10:16:17 -08:00
|
|
|
if ! ${CONFIG_SHELL} ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
|
2019-11-27 08:14:10 -08:00
|
|
|
echo >&2 "Failed to generate BTF for vmlinux"
|
|
|
|
|
echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
|
|
|
|
|
exit 1
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
fi
|
resolve_btfids: Change in-place update with raw binary output
Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
Change resolve_btfids behavior to enable BTF transformations as part
of its main operation. To achieve this, in-place ELF write in
resolve_btfids is replaced with generation of the following binaries:
* ${1}.BTF with .BTF section data
* ${1}.BTF_ids with .BTF_ids section data if it existed in ${1}
* ${1}.BTF.base with .BTF.base section data for out-of-tree modules
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The motivation for emitting binary data is that it allows simplifying
resolve_btfids implementation by delegating ELF update to the $OBJCOPY
tool [1], which is already widely used across the codebase.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking. The patch doesn't change this approach, only
the implementation is a little different.
Before this patch it worked as follows:
* pahole consumed .tmp_vmlinux1 [2] and added .BTF section with
llvm-objcopy [3] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [4],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with
raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For kernel modules, creating a special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with objcopy.
With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows deleting elf_update() call and satellite
code (like compressed_section_fix [5]).
Endianness handling of .BTF_ids data is also changed. Previously the
"flags" part of the section was bswapped in sets_patch() [6], and then
Elf_Type was modified before elf_update() to signal to libelf that
bswap may be necessary. With this patch we explicitly bswap entire
data buffer on load and on dump.
[1] https://lore.kernel.org/bpf/131b4190-9c49-4f79-a99d-c00fac97fa44@linux.dev/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n110
[3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c?h=v1.31#n1803
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n284
[5] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
[6] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20251219181825.1289460-3-ihor.solodrai@linux.dev
2025-12-19 10:18:25 -08:00
|
|
|
btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
|
|
|
|
|
btfids_vmlinux=.tmp_vmlinux1.BTF_ids
|
btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.
Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).
Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).
If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.
v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;
v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-12 11:39:47 -07:00
|
|
|
fi
|
|
|
|
|
|
2021-12-14 11:53:52 +09:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2012-05-05 10:18:41 +02:00
|
|
|
|
|
|
|
|
# kallsyms support
|
|
|
|
|
# Generate section listing all symbols and add it into vmlinux
|
2024-06-10 20:25:17 +09:00
|
|
|
# It's a four step process:
|
|
|
|
|
# 0) Generate a dummy __kallsyms with empty symbol list.
|
2024-11-26 16:58:01 +01:00
|
|
|
# 1) Link .tmp_vmlinux1.kallsyms so it has all symbols and sections,
|
2024-06-10 20:25:17 +09:00
|
|
|
# with a dummy __kallsyms.
|
2024-11-26 16:58:01 +01:00
|
|
|
# Running kallsyms on that gives us .tmp_vmlinux1.kallsyms.o with
|
2012-05-05 10:18:41 +02:00
|
|
|
# the right size
|
2024-11-26 16:58:01 +01:00
|
|
|
# 2) Link .tmp_vmlinux2.kallsyms so it now has a __kallsyms section of
|
2012-05-05 10:18:41 +02:00
|
|
|
# the right size, but due to the added section, some
|
|
|
|
|
# addresses have shifted.
|
2024-11-26 16:58:01 +01:00
|
|
|
# From here, we generate a correct .tmp_vmlinux2.kallsyms.o
|
2016-11-24 03:41:37 +11:00
|
|
|
# 3) That link may have expanded the kernel image enough that
|
|
|
|
|
# more linker branch stubs / trampolines had to be added, which
|
|
|
|
|
# introduces new names, which further expands kallsyms. Do another
|
|
|
|
|
# pass if that is the case. In theory it's possible this results
|
|
|
|
|
# in even more stubs, but unlikely.
|
|
|
|
|
# KALLSYMS_EXTRA_PASS=1 may also used to debug or work around
|
|
|
|
|
# other bugs.
|
|
|
|
|
# 4) The correct ${kallsymso} is linked into the final vmlinux.
|
2012-05-05 10:18:41 +02:00
|
|
|
#
|
|
|
|
|
# a) Verify that the System.map from vmlinux matches the map from
|
|
|
|
|
# ${kallsymso}.
|
|
|
|
|
|
2024-06-10 20:25:16 +09:00
|
|
|
# The kallsyms linking does not need debug symbols included.
|
|
|
|
|
strip_debug=1
|
2012-05-05 10:18:41 +02:00
|
|
|
|
2024-06-10 20:25:18 +09:00
|
|
|
sysmap_and_kallsyms .tmp_vmlinux1
|
2024-06-10 20:25:16 +09:00
|
|
|
size1=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
|
|
|
|
|
|
2024-06-10 20:25:18 +09:00
|
|
|
vmlinux_link .tmp_vmlinux2
|
|
|
|
|
sysmap_and_kallsyms .tmp_vmlinux2
|
2019-08-13 08:15:32 -07:00
|
|
|
size2=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" ${kallsymso})
|
2016-11-24 03:41:37 +11:00
|
|
|
|
|
|
|
|
if [ $size1 -ne $size2 ] || [ -n "${KALLSYMS_EXTRA_PASS}" ]; then
|
2024-06-10 20:25:18 +09:00
|
|
|
vmlinux_link .tmp_vmlinux3
|
|
|
|
|
sysmap_and_kallsyms .tmp_vmlinux3
|
2012-05-05 10:18:41 +02:00
|
|
|
fi
|
|
|
|
|
fi
|
|
|
|
|
|
2024-06-10 20:25:16 +09:00
|
|
|
strip_debug=
|
|
|
|
|
|
2025-03-13 19:26:03 +09:00
|
|
|
if is_enabled CONFIG_VMLINUX_MAP; then
|
|
|
|
|
generate_map=1
|
|
|
|
|
fi
|
|
|
|
|
|
2025-03-11 12:06:18 +01:00
|
|
|
vmlinux_link "${VMLINUX}"
|
kbuild: add ability to generate BTF type info for vmlinux
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.
The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.
This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.
BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.
[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 09:49:50 -07:00
|
|
|
|
2024-09-14 02:37:52 +09:00
|
|
|
if is_enabled CONFIG_DEBUG_INFO_BTF; then
|
2025-12-31 10:39:29 -08:00
|
|
|
info BTFIDS ${VMLINUX}
|
2025-12-30 17:25:57 -08:00
|
|
|
${RESOLVE_BTFIDS} --patch_btfids ${btfids_vmlinux} ${VMLINUX}
|
2020-07-11 23:53:24 +02:00
|
|
|
fi
|
|
|
|
|
|
2025-03-11 12:06:18 +01:00
|
|
|
mksysmap "${VMLINUX}" System.map
|
2021-12-12 19:33:58 +08:00
|
|
|
|
2021-12-14 11:53:52 +09:00
|
|
|
if is_enabled CONFIG_BUILDTIME_TABLE_SORT; then
|
2025-03-11 12:06:18 +01:00
|
|
|
info SORTTAB "${VMLINUX}"
|
|
|
|
|
if ! sorttable "${VMLINUX}"; then
|
2019-12-04 08:46:33 +08:00
|
|
|
echo >&2 Failed to sort kernel tables
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
2012-05-28 10:32:28 -07:00
|
|
|
fi
|
|
|
|
|
|
2012-05-05 10:18:41 +02:00
|
|
|
# step a (see comment above)
|
2021-12-14 11:53:52 +09:00
|
|
|
if is_enabled CONFIG_KALLSYMS; then
|
2024-06-10 20:25:18 +09:00
|
|
|
if ! cmp -s System.map "${kallsyms_sysmap}"; then
|
2012-07-07 23:04:40 +02:00
|
|
|
echo >&2 Inconsistent kallsyms data
|
2023-03-06 13:32:53 -08:00
|
|
|
echo >&2 'Try "make KALLSYMS_EXTRA_PASS=1" as a workaround'
|
2012-05-05 10:18:41 +02:00
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
fi
|
2021-03-05 11:02:12 +01:00
|
|
|
|
|
|
|
|
# For fixdep
|
2025-03-11 12:06:18 +01:00
|
|
|
echo "${VMLINUX}: $0" > ".${VMLINUX}.d"
|