Make the test script print following line when the test is finished and vmcore is successfully dumped:
You can retrive the verify the vmcore file using following command:
./scripts/copy-from-image.sh \
/home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img \
/var/crash/192.168.77.62-2020-09-02-05:16:26/vmcore.flat ./
Kernel package verion is: kernel-core-5.6.6-300.fc32.x86_64
Also add a helper to copy files out of the VM image.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Now, by execute `make test-run` in tests/, kexec-tools sanity tests
will be performed using VMs. There are currently 3 test cases, for local
kdump, nfs kdump and ssh kdump.
For each test VM, the selftest framework will create a snapshot layer,
do setup as required by the test case, this ensure each test runs in a
clean VM.
This framework will install a custom systemd service that starts when
system have finished booting, and the service will do basic routine
(fetch and set boot counter, etc..), then call the test case which is
installed in /kexec-kdump-test/test.sh in VM.
Each VM will have two serial consoles, one for ordinary console usage,
one for the test communication and log. The test script will watch the
second test console to know the test status.
The test cases are located in tests/scripts/testcases, documents about
the test cases structure will be provided in following commits.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The Makefile In tests/ could help build a VM image using Fedora cloud
image as base image, or, user can specify a base image using
BASE_IMAGE=<path/to/file>. The current repo will be packeged and
installed in the image, so the image could be used as a test image to
test kexec-tools.
The image building is splited into two steps:
The first step, it either convert the base image to qcow2 or create
a snapshot on it, and install basic packages (dracut, grubby, ...)
and do basic setups (setup crashkernel=, disable selinux, ...).
See tests/scripts/build-scripts/base-image.sh for detail.
The second step, it creates a snapshot on top of the image produced by
the previous step, and install the packaged kexec-tools of current
repo. See tests/scripts/build-scripts/test-base-image.sh for detail.
In this way, if repo's content is changes, `make` will detect it and
only rebuild the second snapshot which speed up the rebuild by a lot.
The image will be located as tests/output/test-base-image, and in qcow2
format. And default user/password is set to root/fedora.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Encrypted target have many issues, so let user check
kexec-kdump-howto.txt, which have more details.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Now all atomic special workaround is removed, we can remove the atomic
detection function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
kernel installation is not always in a fixed location /boot, there are
multiple different style of kernel installation, and initramfs location
changes with kernel. The two files should be detected together and adapt
to different style.
To do so we use a list of known installation destinations, and a list
of possible kernel image and initrd names. Iterate the two list to
detect the installation location of the two files. If GRUB is in use,
the BOOT_IMAGE= cmdline from GRUB will also be considered. And also
prefers user specified config if given.
Previous atomic workaround is no longer needed as the new detection
method can cover that case.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
There is no need to follow user's configuration when installtion the
kernel and initramfs for early kdump, just use a fixed a symbol link is
enough, this help avoid a check_boot_dir call.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Kdump service will create kdump initramfs when needed, but it won't
clean up the kdump initramfs on kernel uninstall. So create a kernel
install hook to do the clean up job.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit e5cbf4d1ad.
Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
This reverts commit 8884af04b6.
Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
It is hard to detect the time that /etc/kdump is removed. And this failure
may cause out-of-date kdump.initrd. To keep things simple, just exit if
/etc/kdump/pre.d and post.d does not exist.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The following scenario is observed:
kdump: kdump_pre script exited with non-zero status!
[ 5.104841] systemd[1]: Shutting down.
[ 5.122162] printk: systemd-shutdow: 27 output lines suppressed due to ratelimiting
kdump: dump target is /dev/mapper/rhel_hpe--dl380pgen8--02--vm--12-root
kdump: saving to /sysroot//var/crash/127.0.0.1-2020-06-27-03:55:01/
kdump: saving vmcore-dmesg.txt
kdump: saving vmcore-dmesg.txt complete
kdump: saving vmcore
Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | [ 5.516573] systemd-shutdown[1]: Syncing filesystems and block devices.
[ 5.519515] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
It is caused by the following script
if [ $? -ne 0 ]; then
echo "kdump: kdump_pre script exited with non-zero status!"
do_final_action
fi
When do_final_action runs, a systemd service is forked for reboot, then the
subshell returns, and parent continues to execute. Place "exit 1" to stop
executing and make kdump service failure.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Checking modification against a file can not detect a removing file in
"/etc/kdump/post.d/ /etc/kdump/pre.d/". Hence it also needs the
modified time of directory to detect such changes.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Even if the directory "/etc/kdump/pre.d/" is empty, the following false
alarm can be observed during building kdump.initrd:
"/etc/kdump/pre.d/* is not executable"
Suppress it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If the directory /etc/kdump/pre.d is optional, then it is hard
to tell between the following cases:
-1. no directory "/etc/kdump/pre.d"
-2. "rm -rf /etc/kdump/pre.d", which removes all scripts under pre.d
For the second case, kdump.img should be rebuilt.
To bail out from this corner case, always creating pre.d and post.d
during rpm installation.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In secure boot mode, kexec_load syscall is disabled. So, if in secure
boot mode, load kdump kernel with kexec_file_load syscall instead.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 66ff48ed68.
Since the code logic is changed(which related to the option
'KDUMP_FILE_LOAD=on|off'), accordingly, need to remove the
option from the sysconfig file.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 6a20bd5447.
Let's restore the logic of secureboot status check, and remove the
option 'KDUMP_FILE_LOAD=on|off'. We will use the option KEXEC_ARGS="-s"
to enable the kexec file load later, which can avoid failures when
the secureboot is enabled.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since commit ca01cbd, mkdumprd can generate the initramfs even if dump
target is not mounted. But it will refuse to work if the dump target is
not marked as noauto, and not mounted.
If a target is not marked as noauto in fstab, it should be mounted
automatically on boot. If it's not mounted, most likely something is
wrong and user should fix that. So mkdumprd will just show an error
message and quit to avoid further failure.
This commit make the error message more clear and less confusing.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
If target is in format of UUID= or LABEL=, current code will fail as it
expects target to be the absoluta path to the block device. Check for
the fstype instead.
Also simplify the code logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut get_persistent_dev function don't recognize UUID= or LABEL=
format, so caller should conver it to the path to the block device
before calling it. There is already such a helper
"kdump_get_persistent_dev", just move it to kdump-lib.sh and rename
it to reuse it,
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently, kernel has supported the kexec_file_load() syscall on s390x,
so I follow this commit:
6a20bd5447 ("kdump-lib: switch to the kexec_file_load() syscall on
x86_64 by default")
And also enable the kexec_file_load() syscall for kdump on s390x by
default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Dracut expects --add-drivers with a single parameter, so mkdumprd
should quota the module list.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
commit 61e0169 changed definition of dump_fs function, so
need to do a mount target conversion before calling it.
Signed-off-by: Kairui Song <kasong@redhat.com>
This patch executes the binary and script files in /etc/kdump/{pre.d,post.d}
just like kdump_pre or kdump_post directive written in /etc/kdump.conf.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch adds the binary and script files in /etc/kdump/{pre.d,post.d}
to modified checklist in order to update kdump initramfs when one adds
new scripts or binaries or removes the existing ones under
/etc/kdump/{pre.d, post.d}.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch installs the binary and script files under /etc/kdump/{pre.d,post.d}
into new initramfs of kdump.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Switch root is never used for kdump image, and this will be helpful to
reduce the initramfs size.
Also increase dracut dependency version and the function is
dracut_no_switch_root is new introduced.
This commit is applied to RHEL some time ago, but missing in Fedora as
Fedora's Dracut didn't backport this feature at that time. Now apply
this missing commit.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
On journaling filesystems like XFS, bootloader is likely to pick up
older initrd (without vmcore capture scripts) if system crashes right
after initrd update, as the bootloader (read GRUB) may not replay
filesystem log before reading the initrd from disk. Added steps to
workaround that problem.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Usually kdump kernel don't need a swiotlb. There are two cases known
common case a swiotlb is needed:
- SEV/SME is active.
- crashkernel high reservation is used, or there are crashkernel memory
above 4G.
For both case, if swiotlb is set to a large value in first kernel, kdump
kernel will likely to fail with OOM. So don't inherit swiotlb parameter
by default.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Upstream dracut now use network-manager module by default and since
upstream commit 3dcaa97, network-manager expects user to pass "rd.neednet"
to indicate network is required.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit cee618593c.
Upstream dracut have provided a parameter for adding mandantory network
requirement by appending "rd.neednet" parameter, so we should use that
instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The option 'log_buf_len' in the kernel command line will make kernel
dynamically allocate memory (system ram) from memblock and always hold
the memory allocated. If the size of this option is misused such as
'log_buf_len=64M'. This may cause a short of memory for kdump kernel,
because the size of memory is usually limited in kdump kernel.
In order to avoid this problem, need to remove this option from the
kdump kernel command line.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previous commit ca01cbd ('mkdumprd: generate usable kdump initramfs
even target is not mounted') applied a wrong version of patch, let me
just fix it.
Signed-off-by: Kairui Song <kasong@redhat.com>
