Currently $boot_img can get bad data if running on a platform
that doesn't set BOOT_IMAGE in the kernel command line. For
example, currently:
- s390x Fedora CoreOS machine:
```
[root@cosa-devsh ~]# sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\) .*/\2/" /proc/cmdline
mitigations=auto,nosmt ignition.platform.id=qemu ostree=/ostree/boot.0/fedora-coreos/2a72567ac8f7ed678c3ac89408f795e6ccd4e97b41e14af5f471b6a807e858b9/0 root=UUID=2a88436a-3b6b-4706-b33a-b8270bd87cde rw rootflags=prjquota boot=UUID=f4b2eaa5-9317-4798-85cf-308c477fee4c crashkernel=600M
```
where on a platform that uses GRUB we get:
- x86_64 Fedora CoreOS machine:
```
[root@cosa-devsh ~]# sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\) .*/\2/" /proc/cmdline
/ostree/fedora-coreos-af4f6cc7b9ff486cfa647680b180e989c72c8eed03a34a42e7328e49332bd20e/vmlinuz-5.18.5-200.fc36.x86_64
```
We should change the setting of the boot_img variable such that it will
be empty if BOOT_IMAGE doesn't exist.
With this change on the s390x machine:
```
[root@cosa-devsh ~]# grep -P -o '^BOOT_IMAGE=(\S+)' /proc/cmdline | sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\)/\2/"
[root@cosa-devsh ~]#
```
This change mattered much more before the change in c5bdd2d which changed
the following line from [[ -n $boot_img ]] to [[ "$boot_img" == *"$kdump_kernelver" ]].
Still I think this change has merit.
Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
Acked-by: Coiby Xu <coxu@redhat.com>
There are many variants on OSTree based systems these days so
we should probably refer to the class of systems as "OSTree
based systems". Also, Atomic Host is dead.
Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
Acked-by: Coiby Xu <coxu@redhat.com>
On RHEL9 and Fedora, the arm64 platform only supports 4KB page size.
the reserved memory size can be aligned to that on x86_64.
Introducing a new formula for 4KB on arm64, which bases on x86_64 plus
extra 64MB.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Kdump uses currently running kernel as default, but when currently
running kernel is a debug kernel, it will consume more memory,
which may cause out-of-memory and fail to collect vmcore.
Now we will try to use non-debug kernels first if possible.
Also extract the logic of determine KDUMP_KERNEL from
prepare_kdump_bootinfo into a function. This function will return
KDUMP_KERNEL given a kernel version.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
For CoreOS based systems we use Ignition for provisioning machines
in the initramfs on first boot. We trigger Ignition right now by
the presence of `ignition.firstboot` in the kernel command line. The
kernel argument is only present on first boot so after a reboot it
no longer is in the kernel command line.
If a kernel crash happens before the first reboot of a machine we
want the `ignition.firstboot` kernel argument to be removed and not
passed on to the crash kernel.
This patch rewrites get_recommend_size to get rid of the following
limitations,
1. only supports ranges in crashkernel sorted in increasing order
2. the first entry of crashkernel should have only a single digit and
it's in gigabytes
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Recently, it's found 'kdumpctl estimate' returns 512M while the system
reserves 1024M kdump memory in a case. This happens because the ranges
in /proc/iomem are inclusively. For example, "0-1: System RAM" means 2
bytes of system memory other than 1 byte. Fix this error by adding one
more byte.
Note
1. the function has been simplified as well.
2. define PROC_IOMEM as /proc/iomem for the sake of unit tests
Reported-by: Ruowen Qin <ruqin@redhat.com>
Fixes: 1813189 ("kdump-lib.sh: introduce functions to return recommened mem size")
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Make log saving the last step of kdump.sh, so it can catch more info,
for example, the output of post.d hooks will be covered by the log now.
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
1. yum is deprecated so use dnf instead
2. use the "kdumpctl reset-crashkernel --fadump=on" API
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
1. yum is deprecated so use dnf instead
2. use the "kdumpctl reset-crashkernel" API
3. ask the users to refer to crashkernel-howto.txt for setting custom
crashkernel value
4. fix a typo
Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
1. clean up left crashkernel.default
2. fix a few typos and grammar mistakes
3. ask the users to refer to `man kdumpctl` for reset-crashkernel
Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This test prevents the mistake of adding an option to kdump.conf
without changing check_config as is the case with commit 73ced7f
("introduce the auto_reset_crashkernel option to kdump.conf").
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
kdump_get_conf_val allows to retrieves config value defined in
kdump.conf and it also supports sed regex like
"ext[234]\|xfs\|btrfs\|minix\|raw\|nfs\|ssh".
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This commit adds a relatively thorough test suite for
kdumpctl reset-crashkernel [--fadump=[on|off|nocma]] [--kernel=path_to_kernel] [--reboot]
as implemented in commit 140da74 ("rewrite reset_crashkernel to support
fadump and to used by RPM scriptlet").
grubby have a few options to support its own testing,
- --no-etc-grub-update, not update /etc/default/grub
- --bad-image-okay, don't check the validity of the image
- --env, specify custom grub2 environment block file to avoid modifying
the default /boot/grub2/grubenv
- --bls-directory, specify custom BootLoaderSpec config files to avoid
modifying the default /boot/loader/entries
So the grubby called by kdumpctl is mocked as
@grubby --grub2 --no-etc-grub-update --bad-image-okay --env=$SPEC_TEST_DIR/env_temp -b $SPEC_TEST_DIR/boot_load_entries "$@"
in the tests. To be able to call the actual grubby in the mock function [1],
ShellSpec provides the following command
$ shellspec --gen-bin @grubby
to generate spec/support/bins/@grubby which is used to call the actual grubby.
kdumpctl has implemented its own version of updating /etc/default/grub
in _update_kernel_cmdline_in_grub_etc_default. To avoiding writing to
/etc/default/grub, this function is mocked as outputting its name and
received arguments similar to python unitest's assert_called_with.
[1] https://github.com/shellspec/shellspec#execute-the-actual-command-within-a-mock-function
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
AfterAll is an example group hook [1] which would be run after the group
tests are executed. Use this hook to clean up the files created by mktemp.
[1] https://github.com/shellspec/shellspec#beforeall-afterall---example-group-hook
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Currently there are two issues with unit-testing the functions defined
in kdumpctl and other shell scripts after sourcing them,
- kdumpctl would call main which requires root permission and would
create single instance lock (/var/lock/kdump)
- kdumpctl and other shell scripts directly source files under /usr/lib/kdump/
When ShellSpec load a script via "Include", it defines the__SOURCED__
variable. By making use of __SOURCED__, we can
1. let kdumpctl not call main when kdumpctl is "Include"d by ShellSpec
2. instruct kdumpctl and kdump-lib.sh to source the files in the repo
when running ShelSpec tests
Note coverage/ is added to .gitignore because ShellSpec generates code
coverage results in this folder.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Make use of the new ${OPT[]} array and simplify local_fs_dump_target to
remove one more file operations.
While at it rename the local_fs_dump_target to is_local_target
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
With the introduction of ${OPT[fstype]} this call to kdump_get_conf_val
can be removed now as well.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[_target]}. Thus drop DUMP_TARGET and
use ${OPT[_target]} instead.
In order to be able to distinguish between the different target types
introduce the internal ${OPT[_fstype]}.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[sshkey]}. Thus drop
SSH_KEY_LOCATION and use ${OPT[sshkey]} instead.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[path]}. Thus drop SAVE_PATH and
use ${OPT[path]} instead.
Also make sure that ${OPT[path]} is always set to the default value when
no entry in kdump.conf is found.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Every call to kdump_get_conf_val parses kdump.conf although the file has
already been parsed in check_config. Thus store the values parsed in
check_config in an array and use them later instead of re-parsing the
file over and over again.
While at it rename check_config to parse_config.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
check_config and check_ssh_config both parse /etc/kdump.conf and are
usually used together. The difference between both is that
check_ssh_config does some extra checks on the format of the provided
ssh destination but ignores invalid or deprecated options in the config.
Thus merge check_ssh_config into check_config. Leave the additional
checks on the ssh destination in check_ssh_config but treat it like the
checks done for e.g. the failure_action.
This slightly changes the behavior of 'kdumpctl propagate', which now
fails if kdump.conf contains an invalid value unrelated to ssh. This
change in behavior isn't problematic because 'kdumpctl propagate' always
needs to be followed by a 'kdumpctl start' to have a working kdump
environment. For the situations where 'propagate' fails now the 'start'
would have failed in the past. So the failure only moved one step ahead
in the sequence.
While at it drop check_ssh_target and call check_and_wait_network_ready
directly.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function has multiple problems:
1) SSH_{USER,SERVER} aren't defined local
2) Weird use of cut and sed to parse the DUMP_TARGET for the user and
host although check_ssh_config guarantees that it has the format
<user>@<host>.
3) Unnecessary use of a variable for the return value
4) Weird behavior to first unpack the DUMP_TARGET to SSH_USER and
SSH_SERVER and then putting it back together again
5) Definition of variable errmsg that is only used once but breaks
grep-ability of error message.
6) Wrong order when redirecting output of ssh-keygen, see SC2069 [1]
Fix them now.
While at it also improve the error messages in the function.
[1] https://www.shellcheck.net/wiki/SC2069
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
For ssh targets kdumpctl only verifies that the config value has the
correct <user>@<host> format itself. For all other tests, e.g. if the
destination can be reached, it relies on ssh. This allows users to
provide a <host> that isn't the proper hostname but an alias defined in
the ssh_config without failing the tests. If this is done
dracut-module-setup.sh:kdump_get_remote_ip will fail to obtain the
targets ip address. This failure is not detected and thus will not fail
the initramfs creation. The resulting initramfs however doesn't have the
necessary information for setting up the network and thus will fail to
boot.
Prevent the use of alias hostnames by verifying that the given hostname
is the same one ssh would use after parsing the ssh_config.
Note: Don't use getent ahosts to verify that the given host can be
resolved as this requires the network to be up which cannot be
guaranteed when the kdump.conf is parsed.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The time out was increased to 180 seconds in 680c0d3 ("kdumpctl:
distinguish the failed reason of ssh"). Update the comment to reflect
that change.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
There are currently three identical definitions for the default ssh key.
Combine them into one in kdump-lib-initramfs.sh.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
in prepare_kdump_bootinfo s/defaut/default/.
While at it declare it with the other local variables as local.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Using syslog for StandardOutput in a service file was deprecated in
systemd v246 with commit f3dc6af20f ("core: automatically update
StandardOuput=syslog to =journal (and similar for StandardError=)").
Thus the following warnings are printed in the crash kernel when
creating a dump.
systemd[1]: /usr/lib/systemd/system/kdump-capture.service:23: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
systemd[1]: /usr/lib/systemd/system/kdump-capture.service:24: Standard output type syslog+console is obsolete, automatically updating to journal+console. Please update your unit file, and consider removing the setting altogether.
Fix this by redirecting the stdout and stderr to the journal.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
do_estimate prints the warning that the reserved crashkernel is lower
than the recommended one even then when both values are identical. This
might cause confusion. So omit printing the warning when both values are
equal.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
There is a system-wide sync call at the end of mkdumprd, move it to
kdumpctl after rebuild initrd and add another one for mkfadumprd.
Sync only the $TARGET_INITRD to avoid a system-wide sync taking too
long on a system with high disk activity.
Also update the sync in kdumpctl:restore_default_initrd which will
mv the $DEFAULT_INITRD_BAK to $DEFAULT_INITRD.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
If GRUB_ETC_DEFAULT use crashkernel=auto or
crashkernel=OLD_DEFAULT_CRASHKERNEL, it should be updated as well.
Add a helper function to read kernel cmdline parameter from
GRUB_ETC_DEFAULT. This function is used to read kernel cmdline
parameter like fadump or crashkernel.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
There is the case where there are multiple entries of the same parameter on
the command line, e.g.
GRUB_CMDLINE_LINUX="crashkernel=110M crashkernel=220M fadump=on crashkernel=330M".
In such an situation _update_kernel_cmdline_in_grub_etc_default only
updates/removes the last entry which is usually not what you want as the
kernel (for crashkernel) takes the last entry it can find.
Thus make sure the case with multiple entries of the same parameter is
handled properly by removing all occurrences of given parameter first.
Note
1. sed command group and conditional control has been used to get rid of
grep.
2. Fully supporting kernel cmdline as documented in
Documentation/admin-guide/kernel-parameters.rst is complex and in
foreseeable future a full implementation is not needed. So simply
document the unsupported cases instead.
Fixes: 140da74 ("rewrite reset_crashkernel to support fadump and to used by RPM scriptlet")
Reported-by: Philipp Rudo <prudo@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
When doing in-place upgrading using leapp on x86_64, kdumpcl can't
acquire instance lock when running in %post RPM scriplet on x86_64,
localhost upgrade[1306]: /bin/kdumpctl: line 49: /var/lock/kdump: No such file or directory
localhost upgrade[1306]: kdump: Create file lock failed
and running "touch /var/lock/dkump" also fails with
"No such file or directory". Thus kdumpctl can't be run in %post
scriptlet. But kdumpctl can be run in %posttrans RPM scriplet.
Besides, it's better to update crashkernel after the kernel has been
updated. So let's update kernel crashkernel in the %posttrans
scriptlet which will be run in the end of a transaction i.e. after
the kernel has been updated.
Note for %posttrans scriptlet, "$1 == 1" means both installing a new
package and upgrading a package.
[1] https://github.com/apptainer/singularity/issues/2386#issuecomment-474747054
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Previously the output of blkid is not checked. If the output
is empty, the eval will report the following error message:
/lib/kdump/kdump-lib.sh: eval: line 925: syntax error near unexpected token `;'
/lib/kdump/kdump-lib.sh: eval: line 925: `; echo $TYPE'
For example, we can observe such a failing when blkid is invoked
against a lvm thinpool block device:
$ blkid -u filesystem,crypto -o export -- "/dev/block/253\:2"
$ echo $?
2
$ udevadm info /dev/block/253\:2|grep S\:
S: mapper/vg00-thinpoll_tmeta
In this patch, we will use sed instead of eval, to output the
fstype of block device if any.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
It's found that the kernel cmdline crashkernel=auto doesn't get updated
when upgrading kexec-tools. This happens because _get_all_kernels_from_grubby
is called with no argument by reset_crashkernel_after_update. When retrieving
all kernel paths on the system, "grubby --info ALL" should be used. Fix this
error by passing "ALL" argument.
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
