Commit Graph

1220 Commits

Author SHA1 Message Date
Lianbo Jiang
88a8b94de9 kdumpctl: add the '-d' option to enable the kexec loading debugging messages
Currently, the kexec option '--debug/-d' is not enabled by default, which
means that users need to set it manually and wait for the next failure to
capture the additional information.

Therefore, let's enable the option '-d' for kexec loading by default.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-10-27 17:34:03 +08:00
Lianbo Jiang
d001bd177f kdump.sysconfig: add the kdump logger configurations
The kdump logger has the default values of the log levels, but
sometimes, need to change the value of log level in order to
get more debugging messages for troubleshooting.

Here, user will have a chance to reconfigure it.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-10-27 17:33:58 +08:00
Lianbo Jiang
3b743ae6ae enable the logger for kdump
Since the logger was introduced into kdump, let's enable it for kdump
so that we can output kdump messages according the log level and save
these messages for debugging.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-10-27 17:33:54 +08:00
Lianbo Jiang
41b3da3996 introduce the kdump logger from the dracut
Currently, all messages are directly printed to the console, sometimes,
we also need to output these messages to the journal log according to
the log level.

In view of this, introduce the kdump logger from the dracut module.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-10-27 17:31:54 +08:00
Kairui Song
08276e9f7a Rework check_config and warn on any duplicated option
Instead of read and parse the kdump.conf multiple times, only read once
and use a single loop to handle the error check, which is faster.

Also check for any duplicated config otion, and error out if there are
duplicated ones.

Now it checks for following errors, most are unchanged from before:
 - Any duplicated config options. (New added)
 - Deprecated/Invalid kdump config option.
 - Duplicated kdump target, will have a different error message of
   other duplicated config options.
 - Duplicated --mount options in dracut_args.
 - Empty config values. All kdump configs should be in
   "<config_opt> <config_value>" format.
 - Check If raw target is used in fadump mode.

And removed detect of lines start with space, it will not break kdump
anyway.

The performance is measurable better than before for the check_config
function.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-10-27 17:07:54 +08:00
Pingfan Liu
21f5f95b06 kdump-lib.sh: detect secure boot on s390
On s390, if Secure-IPL is enabled, then "kexec -s -l" is required.
Otherwise kdump kernel can not be loaded.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-10-19 16:45:17 +08:00
Kairui Song
041ba89902 Don't drop journalctl content if failure action is "shell"
If failure action is set to "shell", user will need more debug info
available in kdump kernel. Especially when serial console is not
available, manually retrieve the log from journalctl is very useful
for debugging kdump issue.

Else, we can still drop journalctl content to save memory assuming
nothing will use it.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
2020-09-17 10:43:07 +08:00
Kairui Song
bcaa4358b1 dracut-module-install: Move systemd conf install code to a function
No feature change.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
2020-09-17 10:43:07 +08:00
Kairui Song
c44cdb6703 selftest: Show the path of dumped vmcore on test end
Make the test script print following line when the test is finished and vmcore is successfully dumped:

  You can retrive the verify the vmcore file using following command:
  ./scripts/copy-from-image.sh \
      /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img \
      /var/crash/192.168.77.62-2020-09-02-05:16:26/vmcore.flat ./
  Kernel package verion is: kernel-core-5.6.6-300.fc32.x86_64

Also add a helper to copy files out of the VM image.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-09-17 10:43:02 +08:00
Kairui Song
978a849765 selftest: Add document for selftests
Acked-by: Dave Young <dyoung@redhat.com>
2020-09-17 10:42:58 +08:00
Kairui Song
a8dbd281f7 selftest: Add basic test framework
Now, by execute `make test-run` in tests/, kexec-tools sanity tests
will be performed using VMs. There are currently 3 test cases, for local
kdump, nfs kdump and ssh kdump.

For each test VM, the selftest framework will create a snapshot layer,
do setup as required by the test case, this ensure each test runs in a
clean VM.

This framework will install a custom systemd service that starts when
system have finished booting, and the service will do basic routine
(fetch and set boot counter, etc..), then call the test case which is
installed in /kexec-kdump-test/test.sh in VM.

Each VM will have two serial consoles, one for ordinary console usage,
one for the test communication and log. The test script will watch the
second test console to know the test status.

The test cases are located in tests/scripts/testcases, documents about
the test cases structure will be provided in following commits.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-09-17 10:42:54 +08:00
Kairui Song
2457f22baf selftest: Add basic infrastructure to build test image
The Makefile In tests/ could help build a VM image using Fedora cloud
image as base image, or, user can specify a base image using
BASE_IMAGE=<path/to/file>. The current repo will be packeged and
installed in the image, so the image could be used as a test image to
test kexec-tools.

The image building is splited into two steps:
The first step, it either convert the base image to qcow2 or create
a snapshot on it, and install basic packages (dracut, grubby, ...)
and do basic setups (setup crashkernel=, disable selinux, ...).
See tests/scripts/build-scripts/base-image.sh for detail.

The second step, it creates a snapshot on top of the image produced by
the previous step, and install the packaged kexec-tools of current
repo.  See tests/scripts/build-scripts/test-base-image.sh for detail.

In this way, if repo's content is changes, `make` will detect it and
only rebuild the second snapshot which speed up the rebuild by a lot.

The image will be located as tests/output/test-base-image, and in qcow2
format. And default user/password is set to root/fedora.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-09-17 10:42:34 +08:00
Kairui Song
3d5d9f0c0f Release 2.0.20-18
Signed-off-by: Kairui Song <kasong@redhat.com>
2020-08-27 13:14:59 +08:00
Kairui Song
5ec98c2ec7 mkdumprd: Improve the warning message when using encrypted target
Encrypted target have many issues, so let user check
kexec-kdump-howto.txt, which have more details.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-08-27 13:10:42 +08:00
Kairui Song
ff46cfb19e kdump-lib.sh: Remove is_atomic
Now all atomic special workaround is removed, we can remove the atomic
detection function.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-08-27 11:35:01 +08:00
Kairui Song
a37f36ad4d Refactor kernel image and initrd detection code
kernel installation is not always in a fixed location /boot, there are
multiple different style of kernel installation, and initramfs location
changes with kernel. The two files should be detected together and adapt
to different style.

To do so we use a list of known installation destinations, and a list
of possible kernel image and initrd names. Iterate the two list to
detect the installation location of the two files. If GRUB is in use,
the BOOT_IMAGE= cmdline from GRUB will also be considered. And also
prefers user specified config if given.

Previous atomic workaround is no longer needed as the new detection
method can cover that case.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-08-27 11:29:17 +08:00
Kairui Song
c76820bddd early-kdump: Use consistent symbol link for kernel and initramfs
There is no need to follow user's configuration when installtion the
kernel and initramfs for early kdump, just use a fixed a symbol link is
enough, this help avoid a check_boot_dir call.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-08-27 11:23:45 +08:00
Kairui Song
a7199c141c Add a kernel install hook to clean up kdump initramfs
Kdump service will create kdump initramfs when needed, but it won't
clean up the kdump initramfs on kernel uninstall. So create a kernel
install hook to do the clean up job.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-08-27 11:22:03 +08:00
Kairui Song
f8f8d79d7f Release 2.0.20-17
Signed-off-by: Kairui Song <kasong@redhat.com>
2020-08-04 10:50:46 +08:00
Kairui Song
d0d9ff1eb2 Drop static lib dependencies
Signed-off-by: Kairui Song <kasong@redhat.com>
2020-08-04 10:47:51 +08:00
Kairui Song
073c30973c Revert "x86_64: enable the kexec file load by default"
This reverts commit e5cbf4d1ad.

Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.

Signed-off-by: Kairui Song <kasong@redhat.com>
2020-08-04 10:47:51 +08:00
Kairui Song
fa8aa52d94 Revert "s390x: enable the kexec file load by default"
This reverts commit 8884af04b6.

Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.

Signed-off-by: Kairui Song <kasong@redhat.com>
2020-08-04 10:47:51 +08:00
Fedora Release Engineering
21b4996e3a - Second attempt - Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2020-08-01 03:43:49 +00:00
Pingfan Liu
f96172d353 kdumpctl: exit if either pre.d or post.d is missing
It is hard to detect the time that /etc/kdump is removed. And this failure
may cause out-of-date kdump.initrd.  To keep things simple, just exit if
/etc/kdump/pre.d and post.d does not exist.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-30 16:47:10 +08:00
Fedora Release Engineering
1844d088f7 - Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2020-07-28 03:00:49 +00:00
Pingfan Liu
bc67c13651 kdump_pre: make notes more precise
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-21 10:34:36 +08:00
Pingfan Liu
bda677c3d1 dracut-kdump.sh: exit shell when machine reboot
The following scenario is observed:

  kdump: kdump_pre script exited with non-zero status!
  [    5.104841] systemd[1]: Shutting down.
  [    5.122162] printk: systemd-shutdow: 27 output lines suppressed due to ratelimiting
  kdump: dump target is /dev/mapper/rhel_hpe--dl380pgen8--02--vm--12-root
  kdump: saving to /sysroot//var/crash/127.0.0.1-2020-06-27-03:55:01/
  kdump: saving vmcore-dmesg.txt
  kdump: saving vmcore-dmesg.txt complete
  kdump: saving vmcore
   Checking for memory holes                         : [  0.0 %] /                   Checking for memory holes                         : [100.0 %] |                  [    5.516573] systemd-shutdown[1]: Syncing filesystems and block devices.
  [    5.519515] systemd-shutdown[1]: Sending SIGTERM to remaining processes...

It is caused by the following script
  if [ $? -ne 0 ]; then
      echo "kdump: kdump_pre script exited with non-zero status!"
      do_final_action
  fi

When do_final_action runs, a systemd service is forked for reboot, then the
subshell returns, and parent continues to execute. Place "exit 1" to stop
executing and make kdump service failure.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-21 10:34:07 +08:00
Pingfan Liu
25824d64cd kdumpctl: detect modification of scripts by its directory's timestamp
Checking modification against a file can not detect a removing file in
"/etc/kdump/post.d/ /etc/kdump/pre.d/".  Hence it also needs the
modified time of directory to detect such changes.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-20 16:18:42 +08:00
Pingfan Liu
bdbddbff73 module-setup.sh: suppress false alarm
Even if the directory "/etc/kdump/pre.d/" is empty, the following false
alarm can be observed during building kdump.initrd:
  "/etc/kdump/pre.d/* is not executable"

Suppress it.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-20 16:17:53 +08:00
Pingfan Liu
8bef2d943b kexec-tools.spec: make the existence of pre.d and post.d mandatory
If the directory /etc/kdump/pre.d is optional, then it is hard
to tell between the following cases:
  -1. no directory "/etc/kdump/pre.d"
  -2. "rm -rf /etc/kdump/pre.d", which removes all scripts under pre.d
For the second case, kdump.img should be rebuilt.

To bail out from this corner case, always creating pre.d and post.d
during rpm installation.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-20 16:12:04 +08:00
Hari Bathini
5521a70cc7 ppc64/kdump: use kexec_file_load when secureboot is enabled
In secure boot mode, kexec_load syscall is disabled. So, if in secure
boot mode, load kdump kernel with kexec_file_load syscall instead.

Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-07-20 15:21:54 +08:00
Kairui Song
070d362846 Release 2.0.20-14
Also fix a timestamp issue.

Signed-off-by: Kairui Song <kasong@redhat.com>
2020-07-01 17:15:38 +08:00
Lianbo Jiang
8884af04b6 s390x: enable the kexec file load by default
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-07-01 17:07:46 +08:00
Lianbo Jiang
e5cbf4d1ad x86_64: enable the kexec file load by default
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-07-01 17:07:46 +08:00
Lianbo Jiang
bf20a948ef Revert "s390x: add kdump sysconfig option to use the kexec_file_load() syscall"
This reverts commit 66ff48ed68.

Since the code logic is changed(which related to the option
'KDUMP_FILE_LOAD=on|off'), accordingly, need to remove the
option from the sysconfig file.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-07-01 17:07:46 +08:00
Lianbo Jiang
073646998f Revert "kdump-lib: switch to the kexec_file_load() syscall on x86_64 by default"
This reverts commit 6a20bd5447.

Let's restore the logic of secureboot status check, and remove the
option 'KDUMP_FILE_LOAD=on|off'. We will use the option KEXEC_ARGS="-s"
to enable the kexec file load later, which can avoid failures when
the secureboot is enabled.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-07-01 17:07:46 +08:00
Pingfan Liu
f2824e7b74 kdump.conf: fix a grammar issue
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-07-01 17:07:03 +08:00
Pingfan Liu
6dd2b9fb41 man: improve description about /etc/kdump/{pre.d,post.d}interface
The description is identical to the notes in kdump.conf

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-07-01 16:49:41 +08:00
Kairui Song
61d4fa72b5 mkdumprd: Improve the error message for umounted dump target
Since commit ca01cbd, mkdumprd can generate the initramfs even if dump
target is not mounted. But it will refuse to work if the dump target is
not marked as noauto, and not mounted.

If a target is not marked as noauto in fstab, it should be mounted
automatically on boot. If it's not mounted, most likely something is
wrong and user should fix that. So mkdumprd will just show an error
message and quit to avoid further failure.

This commit make the error message more clear and less confusing.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-06-24 14:15:49 +08:00
Kairui Song
807f0b5e48 mkdumprd: Fix nfs detection in to_mount
If target is in format of UUID= or LABEL=, current code will fail as it
expects target to be the absoluta path to the block device. Check for
the fstype instead.

Also simplify the code logic.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-06-23 09:34:53 +08:00
Kairui Song
a29de38da5 Always wrap up call to dracut get_persistent_dev function
Dracut get_persistent_dev function don't recognize UUID= or LABEL=
format, so caller should conver it to the path to the block device
before calling it. There is already such a helper
"kdump_get_persistent_dev", just move it to kdump-lib.sh and rename
it to reuse it,

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2020-06-22 19:58:08 +08:00
Lianbo Jiang
66ff48ed68 s390x: add kdump sysconfig option to use the kexec_file_load() syscall
Currently, kernel has supported the kexec_file_load() syscall on s390x,
so I follow this commit:
6a20bd5447 ("kdump-lib: switch to the kexec_file_load() syscall on
x86_64 by default")

And also enable the kexec_file_load() syscall for kdump on s390x by
default.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2020-06-15 18:20:20 +08:00
Kairui Song
5d4a5896c9 mkdumprd: Fix dracut error on multiple extra_modules
Dracut expects --add-drivers with a single parameter, so mkdumprd
should quota the module list.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-15 17:42:35 +08:00
Kairui Song
1a5d44d7f4 Fix kdump failure when mount target specified by dracut_args
commit 61e0169 changed definition of dump_fs function, so
need to do a mount target conversion before calling it.

Signed-off-by: Kairui Song <kasong@redhat.com>
2020-06-11 14:00:54 +08:00
onitsuka.shinic@fujitsu.com
077a7f19bb kdump.conf: Specify /etc/kdump/{pre.d,post.d}interface
This patch discribe /etc/kdump/{pre.d,post.d} interface.

Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-11 13:18:26 +08:00
onitsuka.shinic@fujitsu.com
45e02e73fa dracut-kdump.sh: Execute the binary and script filesin /etc/kdump/{pre.d,post.d}
This patch executes the binary and script files in /etc/kdump/{pre.d,post.d}
just like kdump_pre or kdump_post directive written in /etc/kdump.conf.

Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-11 12:59:21 +08:00
onitsuka.shinic@fujitsu.com
bdd57a5864 kdumpctl: Check the update of the binary and script files in /etc/kdump/{pre.d,post.d}
This patch adds the binary and script files in /etc/kdump/{pre.d,post.d}
to modified checklist in order to update kdump initramfs when one adds
new scripts or binaries or removes the existing ones under
/etc/kdump/{pre.d, post.d}.

Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-11 12:59:15 +08:00
onitsuka.shinic@fujitsu.com
4246f26725 dracut-module-setup.sh: Install files under /etc/kdump/{pre.d,post.d} into kdump initramfs
This patch installs the binary and script files under /etc/kdump/{pre.d,post.d}
into new initramfs of kdump.

Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-11 12:58:48 +08:00
Kairui Song
e05c550144 Drop switch root capability for non fadump initramfs
Switch root is never used for kdump image, and this will be helpful to
reduce the initramfs size.

Also increase dracut dependency version and the function is
dracut_no_switch_root is new introduced.

This commit is applied to RHEL some time ago, but missing in Fedora as
Fedora's Dracut didn't backport this feature at that time. Now apply
this missing commit.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-10 22:37:34 +08:00
Hari Bathini
8fcbb4d265 fadump: update fadump-howto.txt with some more troubleshooting help
On journaling filesystems like XFS, bootloader is likely to pick up
older initrd (without vmcore capture scripts) if system crashes right
after initrd update, as the bootloader (read GRUB) may not replay
filesystem log before reading the initrd from disk. Added steps to
workaround that problem.

Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
2020-06-04 11:45:05 +08:00