When doing in-place upgrading using leapp on x86_64, kdumpcl can't
acquire instance lock when running in %post RPM scriplet on x86_64,
localhost upgrade[1306]: /bin/kdumpctl: line 49: /var/lock/kdump: No such file or directory
localhost upgrade[1306]: kdump: Create file lock failed
and running "touch /var/lock/dkump" also fails with
"No such file or directory". Thus kdumpctl can't be run in %post
scriptlet. But kdumpctl can be run in %posttrans RPM scriplet.
Besides, it's better to update crashkernel after the kernel has been
updated. So let's update kernel crashkernel in the %posttrans
scriptlet which will be run in the end of a transaction i.e. after
the kernel has been updated.
Note for %posttrans scriptlet, "$1 == 1" means both installing a new
package and upgrading a package.
[1] https://github.com/apptainer/singularity/issues/2386#issuecomment-474747054
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Resolves: bz2025860
Upstream: git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
commit 186e7b0752d8fce1618fa37519671c834c46340e
Author: Alexander Egorenkov <egorenar@linux.ibm.com>
Date: Wed Dec 15 18:48:53 2021 +0100
s390: handle R_390_PLT32DBL reloc entries in machine_apply_elf_rel()
Starting with gcc 11.3, the C compiler will generate PLT-relative function
calls even if they are local and do not require it. Later on during linking,
the linker will replace all PLT-relative calls to local functions with
PC-relative ones. Unfortunately, the purgatory code of kexec/kdump is
not being linked as a regular executable or shared library would have been,
and therefore, all PLT-relative addresses remain in the generated purgatory
object code unresolved. This in turn lets kexec-tools fail with
"Unknown rela relocation: 0x14 0x73c0901c" for such relocation types.
Furthermore, the clang C compiler has always behaved like described above
and this commit should fix the purgatory code built with the latter.
Because the purgatory code is no regular executable or shared library,
contains only calls to local functions and has no PLT, all R_390_PLT32DBL
relocation entries can be resolved just like a R_390_PC32DBL one.
* https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x1633.html#AEN1699
Relocation entries of purgatory code generated with gcc 11.3
------------------------------------------------------------
$ readelf -r purgatory/purgatory.o
Relocation section '.rela.text' at offset 0x6e8 contains 27 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000c 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000001a 001000000014 R_390_PLT32DBL 0000000000000000 sha256_starts + 2
000000000030 001100000014 R_390_PLT32DBL 0000000000000000 sha256_update + 2
000000000046 001200000014 R_390_PLT32DBL 0000000000000000 sha256_finish + 2
000000000050 000300000013 R_390_PC32DBL 0000000000000000 .data + 102
00000000005a 001300000014 R_390_PLT32DBL 0000000000000000 memcmp + 2
...
000000000118 001600000014 R_390_PLT32DBL 0000000000000000 setup_arch + 2
00000000011e 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000012c 000f00000014 R_390_PLT32DBL 0000000000000000 verify_sha256_digest + 2
000000000142 001700000014 R_390_PLT32DBL 0000000000000000
post_verification[...] + 2
Relocation entries of purgatory code generated with gcc 11.2
------------------------------------------------------------
$ readelf -r purgatory/purgatory.o
Relocation section '.rela.text' at offset 0x6e8 contains 27 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000e 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000001c 001000000013 R_390_PC32DBL 0000000000000000 sha256_starts + 2
000000000036 001100000013 R_390_PC32DBL 0000000000000000 sha256_update + 2
000000000048 001200000013 R_390_PC32DBL 0000000000000000 sha256_finish + 2
000000000052 000300000013 R_390_PC32DBL 0000000000000000 .data + 102
00000000005c 001300000013 R_390_PC32DBL 0000000000000000 memcmp + 2
...
00000000011a 001600000013 R_390_PC32DBL 0000000000000000 setup_arch + 2
000000000120 000300000013 R_390_PC32DBL 0000000000000000 .data + 122
000000000130 000f00000013 R_390_PC32DBL 0000000000000000 verify_sha256_digest + 2
000000000146 001700000013 R_390_PC32DBL 0000000000000000 post_verification[...] + 2
Corresponding s390 kernel discussion:
* https://lore.kernel.org/linux-s390/20211208105801.188140-1-egorenar@linux.ibm.com/T/#u
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reported-by: Tao Liu <ltao@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
[hca@linux.ibm.com: changed commit message as requested by Philipp Rudo]
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
v2:
- Moved patch 601 -> 401
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
fadump is a ppc only feature, mkfadumprd is only needed for fadump, drop
it for other arch.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Kairui Song <kasong@tencent.com>
kexec-tools runs hostname binary in the case of fence_kdump. Since this
is a trival dependency and should not block the kexec-tools installation
if non-existent, using weak-dependency to resolve it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
This patch will make zstd as recommended instead of required for
kexec-tools. If zstd command/package is unavaliable, it can failback to invoke
gzip when making kdump initramfs.
Fixes: 0311f6e ("Set zstd as the default compression method for kdump initrd")
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
When a file doesn't exist or isn't readable, grep complains as follows,
grep: /proc/cmdline: No such file or directory
grep: /etc/kernel/cmdline: No such file or directory
/proc/cmdline doesn't exist when installing package for an OS image and
/etc/kernel/cmdline may not exist if osbuild doesn't want set custom
kernel cmdline.
Use "-s" to suppress the error messages.
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Fixes: ddd428a ("set up kernel crashkernel for osbuild in kernel hook")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
zstd has better compression ratio and time consumption balance.
When no customized compression method specified in kdump.conf,
we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following
command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \
ls -ail /boot/initramfs-*kdump.img
**The test result:
Bare metal x86_64 machine:
dracut with squash module
zlib lzo xz lz4 zstd
real 10.6282 11.0398 11.395 8.6424 10.1676
user 9.8932 11.9072 14.2304 2.8286 8.6468
sys 3.523 3.4626 3.6028 3.5 3.4942
size of
kdump.img 30575616 31419392 27102208 36666368 29236224
dracut without squash module
zlib lzo xz lz4 zstd
real 9.509 19.4876 11.6724 9.0338 10.267
user 10.6028 14.516 17.8662 4.0476 9.0936
sys 2.942 2.9184 3.0662 2.9232 3.0662
size of
kdump.img 19247949 19958120 14505056 21112544 17007764
PowerVM hosted ppc64le VM:
dracut with squash module | dracut without sqaush module
zlib zstd | zlib zstd
real 10.6742 10.7572 | 9.7676 10.5722
user 18.754 19.8338 | 20.7932 13.179
sys 1.8358 1.864 | 1.637 1.663
|
size of |
kdump.img 36917248 35467264 | 21441323 19007108
**discussion
zstd has a better compression ratio and time consumption balance.
v1 -> v2:
Use kdump_get_conf_val() to get dracut_args values of kdump.conf
v2 -> v3:
Attached testing benchmark
v3 -> v4:
Re-measured and re-attached the testing benchmark of x86_64 and ppc64le.
Changed regex '.*[[:space:]]' to '(^|[[:space:]])'
v4 -> v5:
Attacked lzo/xz/lz4 testing benchmark.
v5 -> v6:
Add zstd as required in kexec-tools.spec
Hello Coiby, you may use "RELEASE=34 make test-run", for
CONFIG_RD_ZSTD is enabled since fc-cloud-34
Acked-by: Coiby Xu <coxu@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
kexec-tools could update the default crashkernel value.
When auto_reset_crashkernel=yes, reset kernel to new crashkernel
value in the following two cases,
- crashkernel=auto is found in the kernel cmdline
- the kernel crashkernel was previously set by kexec-tools i.e.
the kernel is using old default crashkernel value
To tell if the user is using a custom value for the kernel crashkernel
or not, we assume the user would never use the default crashkernel value
as custom value. When kexec-tools gets updated,
1. save the default crashkernel value of the older package to
/tmp/crashkernel (for POWER system, /tmp/crashkernel_fadump is saved
as well).
2. If auto_reset_crashkernel=yes, iterate all installed kernels.
For each kernel, compare its crashkernel value with the old
default crashkernel and reset it if yes
The implementation makes use of two RPM scriptlets [2],
- %pre is run before a package is installed so we can use it to save
old default crashkernel value
- %post is run after a package installed so we can use it to try to reset
kernel crashkernel
There are several problems when running kdumpctl in the RPM scripts
for CoreOS/Atomic/Silverblue, for example, the lock can't be acquired by
kdumpctl, "rpm-ostree kargs" can't be run and etc.. So don't enable this
feature for CoreOS/Atomic/Silverblue.
Note latest shellcheck (0.8.0) gives false positives about the
associative array as of this commit. And Fedora's shellcheck is 0.7.2
and can't even correctly parse the shell code because of the associative
array.
[1] https://github.com/koalaman/shellcheck/issues/2399
[2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
When kexec-tools is newly installed, kdump migration action hasn't
registered and the following error could occur,
INF dnf.rpm: Could not find a registered notification tool with the specified command ('/usr/lib/kdump/kdump-migrate-action.sh').
"servicelog_notify --list" could list registered notification tools for
a command but it outputs the above error as well. So simply redirect the
error to /dev/null when running "servicelog_notify --remove".
Fixes: commit 146f662622
("kdump/ppc64: migration action registration clean up")
Acked-by: Tao Liu <ltao@redhat.com>
Acked-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
When secureboot is enabled, kdumpctl needs to use keyctl to add/remove
a key to/from the .ima keyring.
Fixes: commit 596fa0a07f
("kdumpctl: enable secure boot on ppc64le LPARs")
Signed-off-by: Coiby Xu <coxu@redhat.com>
The Zstandard (zstd) compression method is not enabled:
$ makedumpfile -v
makedumpfile: version 1.7.0 (released on 8 Nov 2021)
lzo enabled
snappy enabled
zstd disabled
This patch will enable it when building kexec-tools rpm package.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
kdump-error-handler.sh does nothing except calling three functions,
it can be easily merged into kdump.sh by using a parameter to run the
error handling routine.
kdump-lib-initramfs.sh was created to hold the three shared functions
and related code, so by merging these two files, kdump-lib-initramfs.sh
can be simplified by a lot.
Following up commits will clean up kdump-lib-initramfs.sh.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
At least, this is a plausible suspect for #1993505 - thanks to
@kevin for identifying it - and fixing it should be safe and
correct, so we may as well do it and see if it helps.
While kdump migration action is registered for LPM event, ensure it is
cleared as appropriate to avoid duplicate/stale notification entries.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
Dump capture initramfs needs rebuild after partition migration (LPM).
Use servicelog notification mechanism to invoke kdump rebuild after
migration.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Packaging guidelines have been amended to not require systemd for scriptlets,
see https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_scriptlets.
The comment duplicates what the macro contains.
systemd-sysv-convert binary was removed in 2013, trying to call it is
unlikely to succeed.
chkconfig binary is provided by the chkconfig package, which is not in
Requires. (And makes little sense to call nowadays anyway.)
To track and manage kernel's crashkernel usage by kernel version,
each kernel package will include a crashkernel.default containing the
default `crashkernel=` value of that kernel. So we can use a hook to
update the kernel cmdline of new installed kernel accordingly.
Put it after all other grub boot loader setup hooks, so it can simply
call grubby to modify the kernel cmdline.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
In case of fadump, the initramfs image has to be built to boot into
the production environment as well as to offload the active crash dump
to the specified dump target (for boot after crash). As the same image
would be used for both boot scenarios, it could not be built optimally
while accommodating both cases.
Use --include to include the initramfs image built for offloading
active crash dump to the specified dump target. Also, introduce a new
out-of-tree dracut module (99zz-fadumpinit) that installs a customized
init program while moving the default /init to /init.dracut. This
customized init program is leveraged to isolate fadump image within
the default initramfs image by kicking off default boot process
(exec /init.dracut) for regular boot scenario and activating fadump
initramfs image, if the system is booting after a crash.
If squash is available, ensure default initramfs image is also built
with squash module to reduce memory consumption in capture kernel.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 9a6f589d99dcef114c89fde992157f5467028c8f
Author: Tao Liu <ltao@redhat.com>
Date: Fri Jun 18 18:28:04 2021 +0800
[PATCH] check for invalid physical address of /proc/kcore when making ELF dumpfile
Previously when executing makedumpfile with -E option against
/proc/kcore, makedumpfile will fail:
# makedumpfile -E -d 31 /proc/kcore kcore.dump
...
write_elf_load_segment: Can't convert physaddr(ffffffffffffffff) to an offset.
makedumpfile Failed.
It's because /proc/kcore contains PT_LOAD program headers which have
physaddr (0xffffffffffffffff). With -E option, makedumpfile will
try to convert the physaddr to an offset and fails.
Skip the PT_LOAD program headers which have such physaddr.
Signed-off-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 38d921a2ef50ebd36258097553626443ffe27496
Author: Coiby Xu <coxu@redhat.com>
Date: Tue Jun 15 18:26:31 2021 +0800
[PATCH] check for invalid physical address of /proc/kcore when finding max_paddr
Kernel commit 464920104bf7adac12722035bfefb3d772eb04d8 ("/proc/kcore:
update physical address for kcore ram and text") sets an invalid paddr
(0xffffffffffffffff = -1) for PT_LOAD segments of not direct mapped
regions:
$ readelf -l /proc/kcore
...
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x0000000000000120 0x0000000000000000 0x0000000000000000
0x0000000000002320 0x0000000000000000 0x0
LOAD 0x1000000000010000 0xd000000000000000 0xffffffffffffffff
^^^^^^^^^^^^^^^^^^
0x0001f80000000000 0x0001f80000000000 RWE 0x10000
makedumpfile uses max_paddr to calculate the number of sections for
sparse memory model thus wrong number is obtained based on max_paddr
(-1). This error could lead to the failure of copying /proc/kcore
for RHEL-8.5 on ppc64le machine [1]:
$ makedumpfile /proc/kcore vmcore1
get_mem_section: Could not validate mem_section.
get_mm_sparsemem: Can't get the address of mem_section.
makedumpfile Failed.
Let's check if the phys_start of the segment is a valid physical
address to fix this problem.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1965267
Reported-by: Xiaoying Yan <yiyan@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 646456862df8926ba10dd7330abf3bf0f887e1b6
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Wed May 26 14:31:26 2021 +0900
[PATCH] Increase SECTION_MAP_LAST_BIT to 5
* Required for kernel 5.12
Kernel commit 1f90a3477df3 ("mm: teach pfn_to_online_page() about
ZONE_DEVICE section collisions") added a section flag
(SECTION_TAINT_ZONE_DEVICE) and causes makedumpfile an error on
some machines like this:
__vtop4_x86_64: Can't get a valid pmd_pte.
readmem: Can't convert a virtual address(ffffe2bdc2000000) to physical address.
readmem: type_addr: 0, addr:ffffe2bdc2000000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
Increase SECTION_MAP_LAST_BIT to 5 to fix this. The bit had not
been used until the change, so we can just increase the value.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The wrapper is introduced in commit 002337c, according to the commit
message, the only usage of the wrapper is when dracut-initqueue calls
"systemctl start emergency" directly. In that case, emergency
is started, but not in a isolation mode, which means dracut-initqueue
is still running. On the other hand, emergency will call
"systemctl start dracut-initqueue" again when default action is dump_to_rootfs.
systemd would block on the last dracut-initqueue, waiting for the first
instance to exit, which leaves us hang.
In previous commit we added initqueue status detect in dump_to_rootfs,
so now even without the wrapper, it will not hang.
And actually, previously, with the wrapper, emergency might still hang
for like 30s. When dracut called emergency service because initqueue
timed out, dump_to_rootfs will try start initqueue again and timeout
again. Now with the wrapper removed, we can avoid these two kinds of
hangs, bacause without the isolation we can detect initqueue service
status correctly in such case.
Also remove the invalid header comments in service file, the service
is not part of systemd code. And sync the service spec with dracut.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Backport from upstream:
commit 0ef2ca6c9fa2f61f217a4bf5d7fd70f24e12b2eb
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Thu Feb 4 16:29:06 2021 +0900
[PATCH] Show write byte size in report messages
Show write byte size in report messages. This value can be different
from the size of the actual file because of some holes on dumpfile
data structure.
$ makedumpfile --show-stats -l -d 1 vmcore dump.ld1
...
Total pages : 0x0000000000080000
Write bytes : 377686445
...
# ls -l dump.ld1
-rw------- 1 root root 377691573 Feb 4 16:28 dump.ld1
Note that this value should not be used with /proc/kcore to determine
how much disk space is needed for crash dump, because the real memory
usage when a crash occurs can vary widely.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream:
commit 6f3e75a558ed50d6ff0b42e3f61c099b2005b7bb
Author: Julien Thierry <jthierry@redhat.com>
Date: Tue Nov 24 10:45:25 2020 +0000
[PATCH 2/2] Add shorthand --show-stats option to show report stats
Provide shorthand --show-stats option to enable report messages
without needing to set a particular value for message-level.
Signed-off-by: Julien Thierry <jthierry@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 3422e1d6bc3511c5af9cb05ba74ad97dd93ffd7f
Author: Julien Thierry <jthierry@redhat.com>
Date: Tue Nov 24 10:45:24 2020 +0000
[PATCH 1/2] Add --dry-run option to prevent writing the dumpfile
Add a --dry-run option to run all operations without writing the
dump to the output file.
Signed-off-by: Julien Thierry <jthierry@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The `/boot` directory on some operating systems might be read-only.
If we cannot write to `$KDUMP_BOOTDIR` when generating the kdump
initrd, attempt to place the generated initrd at `/var/lib/kdump`
instead.
Signed-off by: Kelvin Fan <kelvinfan001@gmail.com>
Acked-by: Kairui Song <kasong@redhat.com>
Coiby Xu
Philipp Rudo
Kairui Song
Fedora Release Engineering
Pingfan Liu
Tao Liu
Kairui Song
Adam Williamson
Hari Bathini
Zbigniew Jędrzejewski-Szmek
Kelvin Fan