The following scenario is observed:
kdump: kdump_pre script exited with non-zero status!
[ 5.104841] systemd[1]: Shutting down.
[ 5.122162] printk: systemd-shutdow: 27 output lines suppressed due to ratelimiting
kdump: dump target is /dev/mapper/rhel_hpe--dl380pgen8--02--vm--12-root
kdump: saving to /sysroot//var/crash/127.0.0.1-2020-06-27-03:55:01/
kdump: saving vmcore-dmesg.txt
kdump: saving vmcore-dmesg.txt complete
kdump: saving vmcore
Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | [ 5.516573] systemd-shutdown[1]: Syncing filesystems and block devices.
[ 5.519515] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
It is caused by the following script
if [ $? -ne 0 ]; then
echo "kdump: kdump_pre script exited with non-zero status!"
do_final_action
fi
When do_final_action runs, a systemd service is forked for reboot, then the
subshell returns, and parent continues to execute. Place "exit 1" to stop
executing and make kdump service failure.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Checking modification against a file can not detect a removing file in
"/etc/kdump/post.d/ /etc/kdump/pre.d/". Hence it also needs the
modified time of directory to detect such changes.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Even if the directory "/etc/kdump/pre.d/" is empty, the following false
alarm can be observed during building kdump.initrd:
"/etc/kdump/pre.d/* is not executable"
Suppress it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If the directory /etc/kdump/pre.d is optional, then it is hard
to tell between the following cases:
-1. no directory "/etc/kdump/pre.d"
-2. "rm -rf /etc/kdump/pre.d", which removes all scripts under pre.d
For the second case, kdump.img should be rebuilt.
To bail out from this corner case, always creating pre.d and post.d
during rpm installation.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In secure boot mode, kexec_load syscall is disabled. So, if in secure
boot mode, load kdump kernel with kexec_file_load syscall instead.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 66ff48ed68.
Since the code logic is changed(which related to the option
'KDUMP_FILE_LOAD=on|off'), accordingly, need to remove the
option from the sysconfig file.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 6a20bd5447.
Let's restore the logic of secureboot status check, and remove the
option 'KDUMP_FILE_LOAD=on|off'. We will use the option KEXEC_ARGS="-s"
to enable the kexec file load later, which can avoid failures when
the secureboot is enabled.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since commit ca01cbd, mkdumprd can generate the initramfs even if dump
target is not mounted. But it will refuse to work if the dump target is
not marked as noauto, and not mounted.
If a target is not marked as noauto in fstab, it should be mounted
automatically on boot. If it's not mounted, most likely something is
wrong and user should fix that. So mkdumprd will just show an error
message and quit to avoid further failure.
This commit make the error message more clear and less confusing.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
If target is in format of UUID= or LABEL=, current code will fail as it
expects target to be the absoluta path to the block device. Check for
the fstype instead.
Also simplify the code logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut get_persistent_dev function don't recognize UUID= or LABEL=
format, so caller should conver it to the path to the block device
before calling it. There is already such a helper
"kdump_get_persistent_dev", just move it to kdump-lib.sh and rename
it to reuse it,
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently, kernel has supported the kexec_file_load() syscall on s390x,
so I follow this commit:
6a20bd5447 ("kdump-lib: switch to the kexec_file_load() syscall on
x86_64 by default")
And also enable the kexec_file_load() syscall for kdump on s390x by
default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Dracut expects --add-drivers with a single parameter, so mkdumprd
should quota the module list.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
commit 61e0169 changed definition of dump_fs function, so
need to do a mount target conversion before calling it.
Signed-off-by: Kairui Song <kasong@redhat.com>
This patch executes the binary and script files in /etc/kdump/{pre.d,post.d}
just like kdump_pre or kdump_post directive written in /etc/kdump.conf.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch adds the binary and script files in /etc/kdump/{pre.d,post.d}
to modified checklist in order to update kdump initramfs when one adds
new scripts or binaries or removes the existing ones under
/etc/kdump/{pre.d, post.d}.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch installs the binary and script files under /etc/kdump/{pre.d,post.d}
into new initramfs of kdump.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Switch root is never used for kdump image, and this will be helpful to
reduce the initramfs size.
Also increase dracut dependency version and the function is
dracut_no_switch_root is new introduced.
This commit is applied to RHEL some time ago, but missing in Fedora as
Fedora's Dracut didn't backport this feature at that time. Now apply
this missing commit.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
On journaling filesystems like XFS, bootloader is likely to pick up
older initrd (without vmcore capture scripts) if system crashes right
after initrd update, as the bootloader (read GRUB) may not replay
filesystem log before reading the initrd from disk. Added steps to
workaround that problem.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Usually kdump kernel don't need a swiotlb. There are two cases known
common case a swiotlb is needed:
- SEV/SME is active.
- crashkernel high reservation is used, or there are crashkernel memory
above 4G.
For both case, if swiotlb is set to a large value in first kernel, kdump
kernel will likely to fail with OOM. So don't inherit swiotlb parameter
by default.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Upstream dracut now use network-manager module by default and since
upstream commit 3dcaa97, network-manager expects user to pass "rd.neednet"
to indicate network is required.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit cee618593c.
Upstream dracut have provided a parameter for adding mandantory network
requirement by appending "rd.neednet" parameter, so we should use that
instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The option 'log_buf_len' in the kernel command line will make kernel
dynamically allocate memory (system ram) from memblock and always hold
the memory allocated. If the size of this option is misused such as
'log_buf_len=64M'. This may cause a short of memory for kdump kernel,
because the size of memory is usually limited in kdump kernel.
In order to avoid this problem, need to remove this option from the
kdump kernel command line.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previous commit ca01cbd ('mkdumprd: generate usable kdump initramfs
even target is not mounted') applied a wrong version of patch, let me
just fix it.
Signed-off-by: Kairui Song <kasong@redhat.com>
When adding doc for the non-mounted dump target support, I found the
document are a bit uneasy to read due to lack of a proper format, this
commit should it make looks better.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently kexec-tools always depend on dump target to be mounted, which
caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs
even if target is not mounted.
When a mounted user configured target is used, the behavior is not
changed.
When a unmounted user configured target is used, mkdumprd will look for
corresponding mount info in fstab, and a entry with noauto option is
founded, mkdumprd will try to mount it inplace with optoins specified
in fstab and do basic checks on the device, then umount it.
If there is no fstab entry, mkdumprd will try to mount it in temporary
path with defaults option, do same basic check and umount it.
If there is a fstab entry but "noauto" option is not used, then there
must be some reason that the target device is not mounted, mkdumprd will
error out.
When path based target is used, there is no behavior change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use get_mount_info so that fstab is used as a failback when look for
mount info.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Ignore mount check in kdumpctl, mkdumprd will still fail building and
exit if target is not mounted.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use is_mounted helper instaed of calling findmnt directly or checking if
"mount" value is empty.
If findmnt looks for fstab as well, some non mounted entry will also
return value. Required to support non-mounted target.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
User a helper to get the path to mount dump target in kdump kernel, and
fix duplicated '/' in the mount path problem.
Fixes: bz1785371
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
By this point, there is still an unresolved vfs kernel issue that blocks
systemd from mounting the dump target properly from time to time. To
prevent systemd from failing by mounting the dump target, we can add
nofail option to the kdump mount point.
But adding nofail will wipe out default dependency of the mount point,
see commit 94a7b43, so systemd randomize the order of calling kdump.sh
and mounting the dump target and lead to unexpected behavior.
However we can use x-systemd.before to ensure the mount is done
in right order.
In dracut-kdump-capture.service, we have "After=initrd.target",
and look at dracut.bootup.7, systems start processing of fstab by
initrd-fs.target, so set "After=initrd.target" could ensure the mount
is ready before pre-pivot and kdump-capture service.
Now with both nofail and x-systemd.before=initrd.target, systemd
will try to mount the dump target before calling kdump, and even if the
mount failed, kdump.sh will still be called and try to mount again. See
dump_fs function, which will try to mount if the target is not mounted.
Kdump will only fail if both mount attemp fails.
Else if the kdump target mount failed or unstable, systemd will directly
jump to kdump failure action, and kdump fails.
This should improve the robustness in general with no other risk.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
In second kernel, kdump always prints redundant '/':
kdump: saving to /sysroot//var/crash/127.0.0.1-2020-03-12-21:32:54/
Just trim it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This partially reverts commit 6dee286467.
There are reports that NFSv3 is failing after this commit, and after
more debug, I found NFSv4 may not work properly if
"nfs4_disable_idmapping" is set to 0.
The cause of the failure is that kdump.sh runs after dracut's pre-pivot
and clean up hook, many dracut module will install hooks to kill some
running services, so if the dump target requires a service to be running
but it's killed, mount will fail.
Dracut ensures the configured mount points are ready before pre-pivot.
After pre-pivot, any further mounting operation may not work as expected.
Although there is no report of other type of dump target failure except
NFSv3, it's better to revert this, to avoid other potential risk, and wait
for a proper fix for that systemd/kernel issue.
Else, this may bring more trouble for further development.
But still keep the change in kdump-lib-initramfs.sh for better
robustness.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Add recommendations on how much memory is required for FADump. Also,
mention the optimizations applied to default initrd when FADump is
used and how to workaround it.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
In most cases, it always provides a persistent MAC address. But for
the s390 Arch, sometimes, kernel could run in the LPAR mode and it
doesn't provide a persistent MAC address, which caused the kdump
failure.
Currently, some rules rely on the persistent MAC address, for the
above case, which won't work in kdump kernel because non-persistent
MAC could not match with udev rules.
To fix this issue, need to add a new option 'rd.znet_ifname' in order
to provide extra parameters such as 'ifname' and 'subchannels' for
some rules, which ensures kdump can also work appropriately without
the persistent MAC. Please refer to the following commit in dracut:
872eb69936bd ("95znet: Add a rd.znet_ifname= option")
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Since commit 6dee286467 ("Don't mount the dump target unless needed"),
dump_fs function unmounts the dump target just after saving vmcore.
This broke the condition that it's mounted when executing "kdump_post",
which had been stable since RHEL5, and a certain tool which uses the
kdump_post hook to save information of 2nd kernel to the dump target
started to fail.
As unmounting it is done by systemd-shutdown before reboot without
the umount command as below, so let's don't unmount it in dump_fs.
systemd-shutdown[1]: Unmounting file systems.
[547]: Remounting '/sysroot' read-only in with options '(null)'.
EXT4-fs (dm-0): re-mounted. Opts: (null)
[548]: Unmounting '/sysroot'.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
pcs cluster and cluster cib-upgrade may throw some information and disturb
the parsing. Mute them
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
There is a race issue between "pcs" and "kdumpctl restart"
-1. set up cluster
# pcs cluster setup --start mycluster node1 node2
# pcs stonith create kdump fence_kdump pcmk_reboot_action="off"
# pcs stonith level add 1 node1 kdump
# pcs stonith level add 1 node2 kdump
-2. Then here comes the command _immediately_ in kdumpctl
# pcs cluster cib
But due to some pcs internal mechanism, "pcs cluster cib" can not
fetch the updated info in time.
Fix these issue by forcing the upgrade of cib.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If user configured target is used, path should be used as the absolute
path within the dump target direct, and user should be fully aware of
the path structure within the target device. The adjust_bind_mount_path
call here make it very hard to control the behavior.
Especially, if it's a cross device bind mount, this will likely create a
invalid path in the target. And for atomic case, adjust_bind_mount_path call
here assumes user will always pass root device as the explicitly configured
dump target, which is not true.
If user configured target device is used, the path is always be the
absolute path inside of given target. If user don't know about the path
structure in the target device, then user should either use the path
based config, or carefully exam the target device before using it as a
dump target.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
This commit remove almost all special workaround for atomic, and treat
all bind mounts in any environment equally.
Use a helper get_bind_mount_directory_from_path to get the bind mount
source path of given path.
is_atomic function now only used to determine the right /boot path
for atomic/silverblue environment.
And remove get_mntpoint_from_path(), it's the only function that never
ignore bind mount, and it have no caller after this clean up.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
