Commit Graph

59 Commits

Author SHA1 Message Date
Eugene Syromiatnikov
04a4af896d find-provides.ksyms, find-requires.ksyms: avoid iterating over sections
Since the section retrieval code is implemented inside the awk script
now, the next step is to eliminate external iteration over the section
list and just retrieve the required sections inside awk on demand.  This
allows calling (and processing the output of) objdump -t just once,
which saves around 8-12% of time on kernels that store modversions
as non-absolute symbols:

    $ for i in ./lib/modules/*; do \
        echo "====== $i ====="; \
        diff -u <(find $i | ./find-provides.ksyms.old) <(find $i | ./find-provides.ksyms.new); \
        echo -n "old: "; find $i | time ./find-provides.ksyms.old > /dev/null; \
        echo -n "new: "; find $i | time ./find-provides.ksyms.new > /dev/null; \
    done
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  4.41s user 3.46s system 136% cpu 5.756 total
    new: ../find-provides.ksyms.new > /dev/null  3.90s user 2.94s system 129% cpu 5.279 total
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  6.04s user 3.88s system 124% cpu 7.993 total
    new: ../find-provides.ksyms.new > /dev/null  6.08s user 3.91s system 124% cpu 8.012 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  5.09s user 3.51s system 133% cpu 6.452 total
    new: ../find-provides.ksyms.new > /dev/null  4.65s user 2.89s system 126% cpu 5.949 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  6.52s user 4.14s system 123% cpu 8.638 total
    new: ../find-provides.ksyms.new > /dev/null  6.64s user 4.12s system 123% cpu 8.690 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  4.45s user 3.29s system 136% cpu 5.661 total
    new: ../find-provides.ksyms.new > /dev/null  3.84s user 2.54s system 127% cpu 4.980 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  7.34s user 4.33s system 129% cpu 9.019 total
    new: ../find-provides.ksyms.new > /dev/null  6.67s user 3.51s system 122% cpu 8.278 total

* find-provides.ksyms: Remove "objdump -t" section loop, do not supply
sectname to the awk script, treat sectdata as an array keyed on section
name, convert section retrieval to a function with a section name
as an argument, call it if a section name is not present in sectdata.
* find-requires.ksyms (all_provides): Likewise.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 15:03:16 +02:00
Eugene Syromiatnikov
cd6eedad11 find-provides.ksyms, find-requires.ksyms: avoid calling nm twice
Instead of running the nm output through grep and then running it
through awk again, run it directly through awk, and bail of immediately
if a non-absolute symbol is found.

Also, while we're here, avoid executing the indirect parsing code
if no __crc_* symbols are present at all, which shaves around 8-18% off
of the run time:

    % for i in ./lib/modules/*; do \
        echo "====== $i ====="; \
        diff -u <(find $i | ./find-provides.ksyms.old) <(find $i | ./find-provides.ksyms.new); \
        echo old:; find $i | time ./find-provides.ksyms.old > /dev/null; \
        echo new:; find $i | time ./find-provides.ksyms.new > /dev/null; \
    done
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  5.10s user 4.52s system 150% cpu 6.412 total
    new: ../find-provides.ksyms.new > /dev/null  4.41s user 3.48s system 136% cpu 5.764 total
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  7.80s user 6.19s system 143% cpu 9.740 total
    new: ../find-provides.ksyms.new > /dev/null  6.07s user 3.97s system 124% cpu 8.066 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  5.87s user 4.35s system 143% cpu 7.113 total
    new: ../find-provides.ksyms.new > /dev/null  5.12s user 3.56s system 132% cpu 6.533 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  8.44s user 6.78s system 144% cpu 10.535 total
    new: ../find-provides.ksyms.new > /dev/null  6.55s user 4.17s system 123% cpu 8.666 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.s390x =====
    old: ../find-provides.ksyms.old > /dev/null  5.04s user 3.95s system 145% cpu 6.166 total
    new: ../find-provides.ksyms.new > /dev/null  4.64s user 3.17s system 136% cpu 5.719 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.x86_64 =====
    old: ../find-provides.ksyms.old > /dev/null  8.49s user 5.54s system 139% cpu 10.043 total
    new: ../find-provides.ksyms.new > /dev/null  7.41s user 4.31s system 129% cpu 9.046 total

* find-provides.ksyms: Rewrite the "if nm | grep -q; then nm | awk"
with straight "nm | awk", update the awk script to return the exit code
accordingly, execute indirect symbol parsing only if inderect __crc_*
symbols are present.
* find-requires.ksyms (all_provides): Likewise.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 15:03:09 +02:00
Eugene Syromiatnikov
7cba732d24 find-provides.ksyms, find-requires.ksyms: do section data parsing inside awk
Previous approach with passing the section contents as a command line
option proven to be error-prone, as it hits the command line option size
limit on some kmods.  Move the reading and processing of the section
contents inside the awk script itself;  it also saves a bit (around 3—4%)
of time on the kernels with indirect __crc_* symbols:

    $ for i in ./lib/modules/*; do \
        echo "====== $i ====="; \
        diff -u <(find $i | ./find-provides.ksyms.old) <(find $i | ./find-provides.ksyms.new); \
        echo -n "old: "; find $i | time ./find-provides.ksyms.old > /dev/null; \
        echo -n "new: "; find $i | time ./find-provides.ksyms.new > /dev/null; \
    done
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.s390x =====
    old: ./find-provides.ksyms.old > /dev/null  5.38s user 4.66s system 150% cpu 6.693 total
    new: ./find-provides.ksyms.new > /dev/null  5.20s user 4.52s system 149% cpu 6.484 total
    ====== ./lib/modules/4.18.0-372.57.1.el8_6.x86_64 =====
    old: ./find-provides.ksyms.old > /dev/null  7.85s user 6.34s system 143% cpu 9.864 total
    new: ./find-provides.ksyms.new > /dev/null  7.75s user 6.32s system 143% cpu 9.809 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.s390x =====
    old: ./find-provides.ksyms.old > /dev/null  6.19s user 4.68s system 144% cpu 7.506 total
    new: ./find-provides.ksyms.new > /dev/null  5.93s user 4.46s system 143% cpu 7.219 total
    ====== ./lib/modules/5.14.0-284.15.1.el9_2.x86_64 =====
    old: ./find-provides.ksyms.old > /dev/null  8.47s user 6.71s system 144% cpu 10.523 total
    new: ./find-provides.ksyms.new > /dev/null  8.44s user 6.59s system 144% cpu 10.435 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.s390x =====
    old: ./find-provides.ksyms.old > /dev/null  5.21s user 4.09s system 145% cpu 6.383 total
    new: ./find-provides.ksyms.new > /dev/null  5.04s user 3.85s system 145% cpu 6.120 total
    ====== ./lib/modules/6.4.0-0.rc1.20230511git80e62bc8487b.19.eln126.x86_64 =====
    old: ./find-provides.ksyms.old > /dev/null  8.68s user 5.77s system 139% cpu 10.326 total
    new: ./find-provides.ksyms.new > /dev/null  8.47s user 5.57s system 139% cpu 10.067 total

* find-provides.ksyms: Add assign to revbytes the result of endianness
chack test instead of the code snippet;  do not generate SECTDATA
variable;  do not pass SECTDATA to the awk script;  pass module
and revbytes to the awk script;  retrieve the section data
into the sectdata variable and perform the byte re-shuffling
in accordance to the revbytes value in the BEGIN section of the awk
script.
* find-requires.ksyms (all_provides): Likewise.

Fixes: fd79794323 "find-provides.ksyms, find-requires.ksyms: rewrite indirect CRC parsing"
Resolves: #2178935
Co-Authored-by: Denys Vlasenko <dvlasenk@redhat.com>
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:36:55 +02:00
Eugene Syromiatnikov
f40a274ef8 find-provides.ksyms, find-requires.ksyms: rewrite section data matching
Currently, it just matches the 2nd to 5th fields with awk, which leads
to occasional capturing of some garbage at the end of the section data
and minor issues when supplying it to the symbol extracting awk script:

    awk: cmd. line:1: (FILENAME=- FNR=19) fatal: argument index with `$' must be > 0
    awk: warning: escape sequence `\.' treated as plain `.'
    awk: cmd. line:1: (FILENAME=- FNR=9) fatal: not enough arguments to satisfy format string
    	`78b624dbc125415351be965ex.$..%ASQ..^'
    	                              ^ ran out for this one

While at it, also eliminate grep and sed calls (with the pair of awk and
sed being the most offending).

* find-provides.ksyms: Generate part of the awk script that performs
the byte reversal based on the endianness in the readelf -h output,
inject it into the awk script;  use substr for matching parts
of the section data instead of fields.
* find-requires.ksyms (all_provides): Likewise.

Resolves: #2115811
Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:36:25 +02:00
Eugene Syromiatnikov
a553de1c74 find-provides.ksyms: factor out the output sorting
To bring it more in line with the implementation in find-requires.ksyms.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:32:34 +02:00
Eugene Syromiatnikov
60d6d64fd3 find-requires.ksyms: set LC_ALL=C globally
To bring it more in line with the way it is implemented
in find-provides.ksyms and avoid the inconsistencies prodiced by missing
the environment setting.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:31:57 +02:00
Eugene Syromiatnikov
45ce51d5ff find-provides.ksyms, find-requires.ksyms: use "read -r"
To avoid all kinds of fun that might stem from interpreting the user-supplied
input.

* find-provides.ksyms: Supply -r option to the read call.
* find-requires.ksyms (all_provides): Likewise.

Fixes: fd79794323 "find-provides.ksyms, find-requires.ksyms: rewrite indirect CRC parsing"
Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:31:36 +02:00
Eugene Syromiatnikov
c0a9b49b96 find-requires.ksyms: call readelf -R on the sectname and not hard-coded .rodata
* find-requires.ksyms (all_provides): Pass $sectname to the readelf -R call.

Fixes: fd79794323 "find-provides.ksyms, find-requires.ksyms: rewrite indirect CRC parsing"
Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:30:31 +02:00
Eugene Syromiatnikov
4ecf0dd22f find-provides.ksyms, find-requires.ksyms: match only the section being processed
Instead of ignoring the section name in the objdump -t output, section
name has to be checked in the awk script to produce only the meaningful
output;  hit when both __kcrctab and __kcrctab_gpl are present.

* find-requires.ksyms (all_provides): Pass $sectname to the awk script,
check that the relevant field of the input matches it.
* find-provides.ksyms: Likewise.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-12 14:30:19 +02:00
Eugene Syromiatnikov
25b14c8700 find-provides.ksyms, find-requires.ksyms: filter local __crc_* symbols as well
Apparently, these are local in the recent kernels.

* find-provides.ksyms (all_provides): Filter local __crc_* symbols
in additional to global in the objdump -t call output.
* find-requires.ksyms: Likewise.

Resolves: #2178935
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-09 14:11:02 +02:00
Eugene Syromiatnikov
3770b4674a kabi: handle symvers.xz
Check for symvers.xz support in addition to symvers.gz,
due to unnanounced unilateral change[1][2] in the kernel packaging.

[1] 2644665657
[2] 2db77a072a

* kabi.attr: Add xz extension to the file matching pattern
* kabi.sh: Add xz extension to the matching pattern;  determine catenation
program based on extension.

Resolves: #2209253
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-08 15:31:48 +02:00
Eugene Syromiatnikov
dc499f2b3a find-provides.ksyms, find-requires.ksyms: avoid awaking debuginfod
Starting with RHEL 9.2 and Fedora 35, debuginfod is enabled by default,
which leads to significant startup times of various tools
from the elfutils package (namely, readelf and objdump).
Unset DEBUGINFOD_URLS to avoid it.

Resolves: #2209253
Suggested-by: Panu Matilainen <pmatilai@redhat.com>
Co-Authored-by: Denys Vlasenko <dvlasenk@redhat.com>
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2023-06-08 15:30:42 +02:00
Eugene Syromiatnikov
83f4510a4c kernel-srpm-macros.spec: bump RPM release
* kernel-srpm-macros.spec (Release): Bump to 12.
(%changelog): Add a record.
2023-02-08 18:11:55 +01:00
Eugene Syromiatnikov
aed29eb0f7 kernel-srpm-macros.spec: update Requires: for binaries used in scripts
Add "Requies:" tags for binearies used in various scripts, namely
brp-kmod-compress, brp-kmod-set-exec-bit, find-provides.ksyms,
find-requires.ksyms, and kmodtool.

* kernel-srpm-macros.spec (%package -n kernel-rpm-macros) <Requires>:
Add %{_bindir}/xz, %{_bindir}/find, %{_bindir}/sed, %{_bindir}/awk,
%{_bindir}/grep, %{_bindir}/nm, %{_bindir}/objdump, %{_bindir}/readelf,
%{_sbindir}/modinfo, %{_sbindir}/modprobe.

Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
Resolves: #2135047
2023-02-08 17:35:20 +01:00
Eugene Syromiatnikov
fd79794323 find-provides.ksyms, find-requires.ksyms: rewrite indirect CRC parsing
Linux commit v5.19-rc1~139^2~2 ("kbuild: link symbol CRCs at final link,
removing CONFIG_MODULE_REL_CRCS") has broken the assumption aobut
the placement of non-absolute CRC symbols in .rodata (and also affects
the architectures that used to have these symbols as absolute);  rewrite
the parsing by utilising "objdump -t" output to figure out
the section(s) where the __crc_* symbols are stored and process
it instead of the hard-coded ".rodata" section.  The change also speeds
up the processing a bit, around 33% on synthetic tests:

    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.ppc64le -name "*.ko.xz" | find-provides.ksyms.old > /dev/null 2> /dev/null'
    sh -c   10.36s user 5.58s system 137% cpu 11.613 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.ppc64le -name "*.ko.xz" | find-provides.ksyms.new > /dev/null 2> /dev/null'
    sh -c   7.82s user 4.59s system 142% cpu 8.686 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.ppc64le -name "*.ko.xz" -exec sh -c "echo {} | find-provides.ksyms.old" \; > /dev/null 2> /dev/null'
    sh -c   11.85s user 6.76s system 129% cpu 14.318 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.ppc64le -name "*.ko.xz" -exec sh -c "echo {} | find-provides.ksyms.new" \; > /dev/null 2> /dev/null'
    sh -c   8.91s user 5.51s system 135% cpu 10.647 total

* find-provides.ksyms: Process "objdump -t" output to get the list
of sections where __crc_* symbol contents are placed;  retrieve
each one with "readelf -R" and supply it to an awk script that cuts
the required part of it for each __crc_* symbol in that section.
* find-requires.ksyms (all_provides): Likewise.

Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
Resolves: #2135047
2023-02-08 17:13:59 +01:00
Eugene Syromiatnikov
594e5a5a30 find-provides.ksyms, find-requires.ksyms: rewrite absolute CRC parsing
Avoid running sed on the whole "nm" output and also avoid "sed | awk"
pipe in favor of a single awk call.  Overall, this gives around 20%
speedup on some quick synthetic tests:

    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.x86_64 -name "*.ko.xz" | find-provides.ksyms.old > /dev/null 2> /dev/null'
    sh -c   14.20s user 8.93s system 144% cpu 16.014 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.x86_64 -name "*.ko.xz" | find-provides.ksyms.new > /dev/null 2> /dev/null'
    sh -c   12.01s user 7.46s system 143% cpu 13.567 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.x86_64 -name "*.ko.xz" -exec sh -c "echo {} | find-provides.ksyms.old" \; > /dev/null 2> /dev/null'
    sh -c   16.31s user 10.77s system 134% cpu 20.092 total
    $ time sh -c 'find ./lib/modules/5.14.0-258.el9.x86_64 -name "*.ko.xz" -exec sh -c "echo {} | find-provides.ksyms.new" \; > /dev/null 2> /dev/null'
    sh -c   13.95s user 8.92s system 135% cpu 16.836 total

* find-provides.ksyms: Check presence of absolute __crc_* symbols with
"grep -q" exit code and not presence of sed output;  rewrite awk script to match
the __crc_* symbols instead of preprocessing the nm output with sed.
* find-requires.ksyms: Likewise.

Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
Resolves: #2135047
2023-02-08 16:49:55 +01:00
Eugene Syromiatnikov
1745d287de find-provides.ksyms: set LC_ALL=C globally
This makes the sort order a bit more predictable across various
environments and also speeds up grep/sed/sort a bit as they no longer
need to deal with non-trivial collations and multibyte sequences
that may appear when some garbage is being parsed accidentally.

* find-provides.ksyms: Add "export LC_ALL=C", remove "LC_ALL=C" from the
specific calls.

Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
Resolves: #2135047
2023-02-08 16:38:58 +01:00
Eugene Syromiatnikov
5a4884e3a7 gating.yaml: switch to BaseOS CI
* gating.yaml: Use baseos-ci.brew-build.tier1-gating.functional
for PassingTestCaseRule.
2022-02-17 05:06:21 +01:00
Eugene Syromiatnikov
24312e21be find-provides.ksyms, find-requires.ksyms: work around printed __crc_* type change
Commit binutils-2_33~1385[1] has changed (and binutils-2_35~1768[2]
has not reverted) the calculated type for symbols in read-write .rodata section
from 'R' to 'D' on architectures where CONFIG_MODULE_REL_CRCS is enabled
(namely, ppc64* and s390*), since, apparently, many kernel modules have it indeed
read-write.  Work around it by matching both D and R as possible __crc_* symbol
type.

[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=a288c270991d
[2] https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=49d9fd42acef

* find-provides.ksyms: Match D as possible __crc_* symbol type in
addition to R.
* find-requires.ksyms: Likewise.
* kernel-srpm-macros.spec (Release): Bump to 11%{?dist}.
(%changelog): Mention the change.

Resolves: #2055464
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2022-02-17 04:41:45 +01:00
Eugene Syromiatnikov
12ad0a269c kernel-srpm-macros.spec: add conflicts on kernel-rpm-macros < 185-9
Add conflicts of kernel-srpm-macros with kernel-rpm-macros < 185-9
as macros.kmp, kmodtool, and rpmsort were moved from the latter
to the former.

* kernel-srpm-macros.spec (Conflicts): Add kernel-rpm-macros < 185-9.
(%changelog): New entry.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 15:36:08 +01:00
Eugene Syromiatnikov
5c5e2e9503 kernel-srpm-macros.spec: bump the release
* kernel-srpm-macros.spec (Release): Change to 9.
(%package -n kernel-rpm-macros) <Release>: Fix the release.
(%changelog): New record.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
daf10d17ca rpmsort: update the FSF address
* rpmsort: Chnge the FSF address from "59 Temple Place, Suite 330,
Boston, MA  02111-1307  USA" to "51 Franklin Street, Fifth Floor,
Boston, MA  02110-1301 USA".

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
5240efc112 modalias.prov: stricter module aliase/version mangling
* modalias.prov: Replace everyithing that is not "0-9a-zA-Z.:"
in a kernel module version with underscores;  replace everything
that is not "][0-9a-zA-Z.:*_?/-" in a kernel module alias with
underscores.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
c07c7ceda8 modalias.prov: change interpreter to /bin/bash -efu
* modalias.prov: Chnge interpreter from "/bin/sh" to "/bin/bash" because
it uses bashisms;  add "-efu" flags to keep things in check; initialise
is_kernel_package variable; check for presence of /sbin/modinfo, sed,
and sort.
(combine_modaliases): Initialise the "variants" and "pos" variables.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
bebabb191d Add zstd kmod compression support
* brp-kmod-restore-perms: Add chech for "$RPM_BUILD_ROOT/$path.zst".
* find-provides.ksyms: Add "\.zst" to the kernel module path matching
regular expression; process kernel module with zstd if its file name
ends with "zst".
* find-requires.ksyms: Likewise.
* firmware.prov: Add "\.zst" to the kernel module path matching
regular expression.
* kmodtool (%post, %preun): Likewise.
* modalias.prov: Likewise.
* modalias.attr (%__modalias_path): Add "ko\.zst" to the kernel module
path matching regular expression.
* provided_ksyms.attr (%__provided_ksyms_path): Likewise.
* required_ksyms.attr (%__required_ksyms_path): Likewise.
* kmod.attr (%__kmod_path): Add ".*\.ko\.zst" to the kernel module path
matching regular expression.
(strip_compress_sfx): Strip "\.zst" suffix, if present.

Resolves: #1942537
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
6bc05248ec Improve Provides: kernel() generation for built-in modules
After split of the kmods into a separate packages (like kernel-modules
and kernel-modules-extra), kernel() provides for the inbox kmods are generated
for incorrect package, as they were handled by kabi.sh that uses symvers
as a basis for the dependency list.  Stop using it for kmod dependencies
(but continue using it for the symbols provided by vmlinux itself) and
employ find-provides.sh for that purpose.

* kabi.sh: Filter only those symbols that are exported by vmlinux.
* find-provides.ksyms: Generate tags with the "kernel" prefix for kernel
modules inside /lib/modules/[1-9][^/]*/kernel.

Resolves: #1942563
Resolves: #1975927
Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:50:11 +01:00
Eugene Syromiatnikov
e607db68c7 kernel-srpm-macros.spec: move macros for SRPM generation to kernel-srpm-macros
redhat-rpm-macros has broken kmod SRPM generation when stopped providing
in the base buildroot by moving macros.kmp to kernel-rpm-macros
and not depending on the latter due to superficial reasons.  Since
it depends on kernel-srpm-macros now, we can fix that breakage.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:49:46 +01:00
Eugene Syromiatnikov
4c68fd7e56 Switch kmodtool-generated spec to internal dependency generator
It leads to some regressions in requires generation precision, though,
as it is no longer possible to exclude requires that are satisfied
by other kmods in the package.  And calling a script on each file (instead
of calling it on the whole file list at once) does not make things faster,
either (so much for "sanity and benefit").

* find-provides.ksyms: Check for "$@" as well.
* find-requires.ksyms: Likewise.
* kernel-srpm-macros.spec (Source104, Source105, Source106): New
attribute files.
(%install): Install firmware.prov and modalias.prov with executable bit;
install provided_ksyms.attr, required_ksyms.attr, and modalias.attr into
"%{buildroot}%{_fileattrsdir}".
(%files -n kernel-rpm-macros): Add provided_ksyms.attr,
required_ksyms.attr, and modalias.attr.
kmodtool: Remove "%global _use_internal_dependency_generator 0".
macros.kmp: Remove %__find_provides and %__find_requires.
* modalias.attr: New file.
* provided_ksyms.attr: Likewise.
* required_ksyms.attr: Likewise.

Resolves: #1942072
Resolves: #1942563
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:49:04 +01:00
Eugene Syromiatnikov
b341b38e28 kmodtool: call weak-modules for compressed kmods too
* kmodtool (%post, %preun): Add "(\.gz|\.bz2|\.xz)?" to the kernel module
path matching regular expression.

Resolves: #1942537
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
9a84b5ba8e brp-kmod-restore-perms: handle kmod compression done in post-install hooks
If a kernel module are compressed by a post-install hook, then its name
would differ from the one saved by brp-kmod-set-exec-bit;  try to
recover from this by guessing a possible name of the compressed kmod and
using it instead.

* brp-kmod-restore-perms (while read perm path): Check for
"$RPM_BUILD_ROOT/$path.gz", "$RPM_BUILD_ROOT/$path.bz2",
and "$RPM_BUILD_ROOT/$path.xz" if "$RPM_BUILD_ROOT/$path"
does not exist, add the respective suffix to the $path.

Resolves: #1942537
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
116fb03e8a modalias.prov: process compressed kmods
* modalias.prov: Add "(\.gz|\.bz2|\.xz)?" to the end of the module path
matching regular expression, use "$@" instead of $*.

Resolves: #1942537
Resolves: #2024117
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
f9fcd7463c kmod.attr: add compressed kmod support
* kmod.attr (%__kmod_path): Add ".*\.ko\.gz|.*\.ko\.bz2|.*\.ko\.xz"
to he module path matching regular expression endings.
(%__kmod_provides) <strip_compress_sfx>: New function.
(%__kmod_provides): Generate dependencies with module file name without
compressed extension (just ".ko") for compressed modules.

Resolves: #1942537
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
2aa8019903 macros.kmp: add post-install kmod processing hooks
The enablement of debuginfo generation broke custom %install scripts
that perform module signing, and in an attempt to address that,
additional hooks are added that can be used for module compression,
signing, and other means of kmod processing.

* macros.kmp (redhat_kmp_has_post_hooks): New macro, set to 1.
(kernel_module_package_release): Add __brp_kmod_pre_sign_process,
__brp_kmod_sign, __brp_kmod_post_sign_process, __brp_kmod_compress,
and __brp_kmod_post_compress_process to the generated
__spec_install_post.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
836b42876b kmodtool: bump revision
* kmodtool (myver): Change revision to "rhel9", may be useful
for external users.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
711c0be6ea Enable extraction of debug information for kmods
This hack temporary adds executable permission for the *.ko files
during post-install stage so they are picked up by find-debuginfo.sh.
Since the intention is to avoid changing of behaviour in non-kmod
workloads, it's done by adding two additional brp-* scripts to the
__spec_install_post macro when kernel_module_package is instantiated.

* brp-kmod-restore-perms: New file.
* brp-kmod-set-exec-bit: Likewise.
* macros.kmp (%__brp_kmod_set_exec_bit, %__brp_kmod_restore_perms,
%__kmod_brps_added): New macros.
(%kernel_module_package): Rewrite __spec_install_post macro.
* redhat-rpm-config.spec (Source701, Source702): Add
brp-kmod-restore-perms and brp-kmod-set-exec-bit.
(Requires): Add find requirement for brp-kmod-set-exec-bit.
(%files): Explicitly list brp-* scripts for redhat-rpm-config and
kernel-rpm-macros packages.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
d7490ffefa macros.kmp: avoid error in latest_kernel when no kernel is installed
If no kernel-devel packages are installed (this is possible, for example,
during SRPM generation in an isolated environment), no output would be
generated for latest_kernel macro, which would lead to an error (since
an RPM macro can't have an empty definition).  Append %nil to the input
of the "head" command to avoid this situation.

* macros.kmp (kernel_module_package): Feed additional line with %nil
to the input of head command in the latest_kernel macro definition.

Resolves: #1971748
Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
a60fc1e943 kmodtool: add "kernel${dashvariant}-modules" Provides
Providing "kernel-modules" with "${verrel}${dotvariant}" looks strange,
as it differs from the package and version that are actualy used for
flavored kernels.

* kmodtool (get_rpmtemplate): Add "kernel${dashvariant}-modules >= ${verrel}"
Provides to the generated spec.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
fc8ffa826b kmodtool, macros.kmp: support "+" for the kernel variant in uname
A patch[1] to the kernel spec file changed the separator that is used
for kernel flavor designation from "." to "+", which broke kmodtool and
%kernel_module_package in multiple places.  Try to accomodate it by
accepting both "+" and "." during parsing, and guessing which one should
be used for kernel osurce path generation.

PS. It looks like the analogous breakage was before, when it was changed
from "-" to ".". May be we should start support it back again as well.

[1] https://lists.fedoraproject.org/pipermail/kernel/2013-June/004262.html

* kmodtool (get_verrel, get_variant): Accept both "." and "+" as separators.
(get_variant_char, print_variant_char, print_kernel_source): New functions.
(get_filelist): Convert "+%1" in file list to $dotvariant, in addition
to ".%1" and "-%1".
(get_rpmtemplate): Determine a char that is used as a variant separator.
Use it in $dotvariant.
(usage): Mention "veriant_char" and "kernel_source" commands.
* macros.kmp (%kernel_module_package): Use "kernel_source" command in
"%kernel_source" definition when non-default variant is used; fall back
to ls-based heuristics when it is not available (for example, when a
version of kmodtool that doesn't support this command is used).

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
1c5fb80a23 kmodtool: enquote $@ usage
* kmodtool: Put $@ instances into quotes as otherwise arguments
will undergo word splitting.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
b3f88632de kmodtool, macros.kmp: update list of build requirements
The following packages are added:
 - kernel-abi-stablelists is used by find-requires.ksym in order to
   provide information regarding kABI symbol usage. It's purely
   informational and optional, however.
 - redhat-rpm-config and kernel-rpm-macros are added since the generted
   spec files use macros from these packages.
 - elfutils-libelf-devel is used during kmod build for ORC metadata
   generation (used by kernel for stack unwinding).
 - kmod programs are used by find-requires.ksyms and find-proivides.ksyms
   for parsing *.ko files during dependency generation.

* kmodtool (get_rpm_template): Update the list of generated
BuildRequires.
* macros.kmp (%kernel_module_package_buildreqs): Update the list of
default BuildRequires.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
0e8d1900c9 kmodtool: add "zfcpdump" to the list of known kernel variants
In RHEL 9, there's a s390x-specific "zfcpdump" kernel variant (which
replaces previous "kdump¨ variant). Add it to the list of the known
kernel variants.

* kmodtool (knownvariants): Add "zfcpdump".

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
3caf087dca kmodtool: update dependencies, weak-modules path
Add Requires(post) and Requires(postun) on weak-modules, use
/usr/sbin/weak-modules path, add relevant build requirements.

* kmodtool (get_rpmtemplate): Add weak-modules to Requires(post),
Requires(postun). Prefix path to weak-modules with /usr.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
808ba4b04d Re-add redhat-rpm-config-9.0.3-latest-kernel.patch to RHEL 9
Apply contents of redhat-rpm-config-9.0.3-latest-kernel.patch
from RHEL 7 branch.

Commit message from the original commit:

    commit 5523411675de5be36f6a2adc0f99c2c5021c87ca
    Author: Florian Festi <ffesti@redhat.com>
    Date:   Wed May 11 17:09:31 2016 +0200

        - Fix latest_kernel macro
        - Resolves: #1323087

Resolves: #1971748
Resolves: #1999607
Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
e743522ed4 Re-add redhat-rpm-config-9.1.0-fix-depmod-path.patch to RHEL 9
Apply contents of redhat-rpm-config-9.1.0-fix-depmod-path.patch from
RHEL 7 branch.

Commit message from the original commit:

    commit 98a159714814bfa24cb912eb3eb4c50186006ca4
    Author: Weiping Pan <wpan@redhat.com>
    Date:   Sun Sep 22 10:53:10 2013 +0800

        fix depmod path

        Since after kmod 5.1, depmod is in /usr/sbin/ instead of /sbin/,
        we have to follow it.
        kmod dis-git commit d398533b871d(version 5).

        Resolves: #1002618

        Signed-off-by: Weiping Pan <wpan@redhat.com>

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
78df0af42a Re-add contents of redhat-rpm-config-9.1.0-kmod-kabi.patch to RHEL 9
Apply redhat-rpm-config-9.1.0-kmod-kabi.patch from RHEL 7 branch.

Applied only changes to kmodtool and macros (now macros.kmp), as all
others are applied already. macros.kmp also had a merge conflict due to
commits f41a8a73aa6f ("Fix kernel_source macro to match the directory
that kernel sources are installed in") and d95e9bc7b618 ("Fix directory
name mismatch in kernel_source macro (#648996)") being applied already.

Commit message from the original commit:

    commit 29b4b765391b8f11ae64708e86ac3f7475c62661
    Author: Panu Matilainen <pmatilai@redhat.com>
    Date:   Tue Jul 9 09:57:29 2013 +0300

        - Re-add various kABI-related bits (Jon Masters)
        - Allow kmod building against z-stream kernels (Jiri Benc)

Resolves: #1971748
Resolves: #2002887
Resolves: #2015909
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
f6c02f2f49 Add support for compressed kernel modules
- Add support for compressed kernel modules to find-provides.ksyms,
  find-requires.ksyms, firmware.prov (#1622019)

* find-provides.ksyms: Try to process files whose names end with
something else than "*.ko" by guessing a decompressor and uncompressing
it in a temporary file and parsing it instead.
* find-requires.ksyms (all_provides): Likewise.
Add "(\.gz|\.bz2|\.xz)?" to the end of the module path matching regular
expression.
* firmware.prov: Add "(\.gz|\.bz2|\.xz)?" to the end of the module path
matching regular expression.

Resolves: #1942537
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
ddcb08281b find-requires.ksym: fix external kmod deps for symbols with the same name
Generate colon-separated pairs of symbol_name:version and then run joins
over them instead over just symbol names.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
670da3c03d find-requires.ksyms: import RHEL-specific changes from RHEL 7.6
* find-requires.ksyms: Add file comment.
(check_kabi): New function.
(if [ ${#modules[@]} -gt 0 ]): Call it.

Resolves: #2002887
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Eugene Syromiatnikov
cc8b47062a Re-instantiate support for old symvers path
As it breaks bz1004930-detect-kabi-provides test[1] otherwise.

[1] http://pkgs.devel.redhat.com/cgit/tests/redhat-rpm-config/tree/Sanity/bz1004930-detect-kabi-provides

- Re-instantiate support for old symvers path

* kabi.attr (%__kabi_path): Add "/boot/symvers-.*" to the path matching
regular expression.
* kabi.sh: Likewise.

Resolves: #1942563
Resolves: #2008554

Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:22:00 +01:00
Panu Matilainen
fa3dd2ca59 Add dependency generator for kABI provides
The only difference from RHEL 7 is that we're looking for symvers file
in /lib/modules now, per #1609695[1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1609695

- Add dependency generator for kABI provides

Resolves: #1942563
Resolves: #2008554
Signed-off-by: Eugene Syromiatnikov <esyr@redhat.com>
2021-11-18 14:21:51 +01:00