There is a system-wide sync call at the end of mkdumprd, move it to
kdumpctl after rebuild initrd and add another one for mkfadumprd.
Sync only the $TARGET_INITRD to avoid a system-wide sync taking too
long on a system with high disk activity.
Also update the sync in kdumpctl:restore_default_initrd which will
mv the $DEFAULT_INITRD_BAK to $DEFAULT_INITRD.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
This patch will make zstd as recommended instead of required for
kexec-tools. If zstd command/package is unavaliable, it can failback to invoke
gzip when making kdump initramfs.
Fixes: 0311f6e ("Set zstd as the default compression method for kdump initrd")
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
zstd has better compression ratio and time consumption balance.
When no customized compression method specified in kdump.conf,
we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following
command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \
ls -ail /boot/initramfs-*kdump.img
**The test result:
Bare metal x86_64 machine:
dracut with squash module
zlib lzo xz lz4 zstd
real 10.6282 11.0398 11.395 8.6424 10.1676
user 9.8932 11.9072 14.2304 2.8286 8.6468
sys 3.523 3.4626 3.6028 3.5 3.4942
size of
kdump.img 30575616 31419392 27102208 36666368 29236224
dracut without squash module
zlib lzo xz lz4 zstd
real 9.509 19.4876 11.6724 9.0338 10.267
user 10.6028 14.516 17.8662 4.0476 9.0936
sys 2.942 2.9184 3.0662 2.9232 3.0662
size of
kdump.img 19247949 19958120 14505056 21112544 17007764
PowerVM hosted ppc64le VM:
dracut with squash module | dracut without sqaush module
zlib zstd | zlib zstd
real 10.6742 10.7572 | 9.7676 10.5722
user 18.754 19.8338 | 20.7932 13.179
sys 1.8358 1.864 | 1.637 1.663
|
size of |
kdump.img 36917248 35467264 | 21441323 19007108
**discussion
zstd has a better compression ratio and time consumption balance.
v1 -> v2:
Use kdump_get_conf_val() to get dracut_args values of kdump.conf
v2 -> v3:
Attached testing benchmark
v3 -> v4:
Re-measured and re-attached the testing benchmark of x86_64 and ppc64le.
Changed regex '.*[[:space:]]' to '(^|[[:space:]])'
v4 -> v5:
Attacked lzo/xz/lz4 testing benchmark.
v5 -> v6:
Add zstd as required in kexec-tools.spec
Hello Coiby, you may use "RELEASE=34 make test-run", for
CONFIG_RD_ZSTD is enabled since fc-cloud-34
Acked-by: Coiby Xu <coxu@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
nfs service will append extra mount options to kernel mount options.
Such as mountaddr/mountproto options. These options only represent
current mounting details of the 1st kernel, but may not appropriate
for the 2nd kernel for the same reason as commit
d4f04afa47 ("mkdumprd: drop some nfs
mount options when reading from kernel"). This patch will remove
these options.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
All non-POSIX syntax in second kernel are gone, tested on Fedora 34
with latest dracut, dash now works fine.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This is a batch update done with:
shfmt -s -w mkfadumprd mkdumprd kdumpctl *-module-setup.sh
Clean up code style and reduce code base size, no behaviour change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Declare and assign separately to avoid masking return values:
https://github.com/koalaman/shellcheck/wiki/SC2155
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
kdumpctl, mkdumprd, *-module-setup.sh only target bash, since they
only run in first kernel and depend on dracut, and dracut depends
on bash. So use '[[ ]]' to replace '[ ]'.
This is a batch update done with following command:
`sed -i -e 's/\(\s\)\[\s\([^]]*\)\s\]/\1\[\[\ \2 \]\]/g' kdumpctl, mkdumprd, *-module-setup.sh`
and replaced [ ... -a ... ] with [[ ... ]] && [[ ... ]] manually.
See https://tldp.org/LDP/abs/html/testconstructs.html for more details
on '[[ ]]', it's more versatile, safer, and slightly faster than '[ ]'.
This will also help shfmt to clean up the code in later commits.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Use bash builtin string substitution instead, as suggested by:
https://github.com/koalaman/shellcheck/wiki/SC2001
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Some `cat` calls are useless, remove them to make it cleaner.
See: https://github.com/koalaman/shellcheck/wiki/SC2002
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
For storing arguments, plain string is not a good choice. Array is
preferred:
See: https://github.com/koalaman/shellcheck/wiki/SC2089
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Currently get_ssh_size is not working as expected, it should return the
target's available space, but it will include df's header row string as
the result. Fix this issue by only use the last output line.
And the _opt variable will be used as args so it should be an array.
Also remove the awk call, just use `df --output=avail` instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
To make arguments list work as expected, array is preferred.
Use xargs only to parse the "dracut_args" config value, and pass the
array directly to dracut.
Check following link for details:
https://github.com/koalaman/shellcheck/wiki/SC2089
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Add a helper `kdump_read_conf` to replace read_strip_comments.
`kdump_read_conf` does a few more things:
- remove trailing spaces.
- format the content, remove duplicated spaces between name and value.
- read from KDUMP_CONFIG_FILE (/etc/kdump.conf) directly, avoid pasting
"/etc/kdump.conf" path everywhere in the code.
- check if config file exists, just in case.
Also unify the environmental variable, now KDUMP_CONFIG_FILE stands for
the default config location.
This helps avoid some shell pitfalls about spaces when reading config.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
In case of fadump, the initramfs image has to be built to boot into
the production environment as well as to offload the active crash dump
to the specified dump target (for boot after crash). As the same image
would be used for both boot scenarios, it could not be built optimally
while accommodating both cases.
Use --include to include the initramfs image built for offloading
active crash dump to the specified dump target. Also, introduce a new
out-of-tree dracut module (99zz-fadumpinit) that installs a customized
init program while moving the default /init to /init.dracut. This
customized init program is leveraged to isolate fadump image within
the default initramfs image by kicking off default boot process
(exec /init.dracut) for regular boot scenario and activating fadump
initramfs image, if the system is booting after a crash.
If squash is available, ensure default initramfs image is also built
with squash module to reduce memory consumption in capture kernel.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
When kdump service fails, the current errors do not display the
absolute path of dump location(marked it as "^"), for example:
kdump: kexec: unloaded kdump kernel
kdump: Stopping kdump: [OK]
kdump: Detected change(s) in the following file(s): /etc/kdump.conf
kdump: Rebuilding /boot/initramfs-4.18.0-304.el8.x86_64kdump.img
kdump: Dump path "/var1/crash" does not exist in dump target "UUID=c202ef45-3ac3-4adb-85e7-307a916757f0"
^^^^^^^^^^^
kdump: mkdumprd: failed to make kdump initrd
kdump: Starting kdump: [FAILED]
Here, it should output the absolute path of dump location with this
format: "<mount path>/<path>". To fix it, let's extend the relative
pathname to the absolute pathname in check_user_configured_target().
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Simplfy the code and also improve the performance. udevadm call is
heavy.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
When nfs-utils is not installed, mounting as NFS fs would fail.
Currently, the error message is not user-friendly,
mount: /tmp/mkdumprd.HyPGpS/target: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program.
kdump: Failed to mount on xxx for kdump preflight check.
kdump: mkdumprd: failed to make kdump initrd
Prompt the user to install nfs-utilsa in the error message,
kdump: Failed to mount on xxx for kdump preflight check. Please make sure nfs-utils has been installed.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Sourcing logger file in kdump-lib.sh will leak kdump helper to dracut,
because module-setup.sh will source kdump-lib.sh. This will make kdump's
function override dracut's ones, and lead to unexpected behaviours.
So include kdump-logger.sh individually and only source it where it really
needed. for module-setup.sh, simply use dracut's logger helper is good
enough so just source kdump-logger.sh in kdump only scripts.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
That commit is trying to workaround a kernel VFS bug. Now,
the VFS issue should have been fixed in all recent releases, so
remove this workaround.
This reverts commit 539bff4083.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
depend() in module-setup.sh is a better place to setup dracut module
dependency, it will do early check, and fail early if needed module is
missing. Also remove a unneeded helper add_dracut_module.
Also remove the unnecessary return in depend() function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Dracut only check if a module failed installtion if the module is listed
in --add params. Without this param, if kdumpbase failed to install due
to any reason, dracut will still build the initramfs only print a
warning. Add this param to ensure it fail early.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since the logger was introduced into kdump, let's enable it for kdump
so that we can output kdump messages according the log level and save
these messages for debugging.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Encrypted target have many issues, so let user check
kexec-kdump-howto.txt, which have more details.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Since commit ca01cbd, mkdumprd can generate the initramfs even if dump
target is not mounted. But it will refuse to work if the dump target is
not marked as noauto, and not mounted.
If a target is not marked as noauto in fstab, it should be mounted
automatically on boot. If it's not mounted, most likely something is
wrong and user should fix that. So mkdumprd will just show an error
message and quit to avoid further failure.
This commit make the error message more clear and less confusing.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
If target is in format of UUID= or LABEL=, current code will fail as it
expects target to be the absoluta path to the block device. Check for
the fstype instead.
Also simplify the code logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut get_persistent_dev function don't recognize UUID= or LABEL=
format, so caller should conver it to the path to the block device
before calling it. There is already such a helper
"kdump_get_persistent_dev", just move it to kdump-lib.sh and rename
it to reuse it,
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut expects --add-drivers with a single parameter, so mkdumprd
should quota the module list.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previous commit ca01cbd ('mkdumprd: generate usable kdump initramfs
even target is not mounted') applied a wrong version of patch, let me
just fix it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Currently kexec-tools always depend on dump target to be mounted, which
caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs
even if target is not mounted.
When a mounted user configured target is used, the behavior is not
changed.
When a unmounted user configured target is used, mkdumprd will look for
corresponding mount info in fstab, and a entry with noauto option is
founded, mkdumprd will try to mount it inplace with optoins specified
in fstab and do basic checks on the device, then umount it.
If there is no fstab entry, mkdumprd will try to mount it in temporary
path with defaults option, do same basic check and umount it.
If there is a fstab entry but "noauto" option is not used, then there
must be some reason that the target device is not mounted, mkdumprd will
error out.
When path based target is used, there is no behavior change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use get_mount_info so that fstab is used as a failback when look for
mount info.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use is_mounted helper instaed of calling findmnt directly or checking if
"mount" value is empty.
If findmnt looks for fstab as well, some non mounted entry will also
return value. Required to support non-mounted target.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
User a helper to get the path to mount dump target in kdump kernel, and
fix duplicated '/' in the mount path problem.
Fixes: bz1785371
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
By this point, there is still an unresolved vfs kernel issue that blocks
systemd from mounting the dump target properly from time to time. To
prevent systemd from failing by mounting the dump target, we can add
nofail option to the kdump mount point.
But adding nofail will wipe out default dependency of the mount point,
see commit 94a7b43, so systemd randomize the order of calling kdump.sh
and mounting the dump target and lead to unexpected behavior.
However we can use x-systemd.before to ensure the mount is done
in right order.
In dracut-kdump-capture.service, we have "After=initrd.target",
and look at dracut.bootup.7, systems start processing of fstab by
initrd-fs.target, so set "After=initrd.target" could ensure the mount
is ready before pre-pivot and kdump-capture service.
Now with both nofail and x-systemd.before=initrd.target, systemd
will try to mount the dump target before calling kdump, and even if the
mount failed, kdump.sh will still be called and try to mount again. See
dump_fs function, which will try to mount if the target is not mounted.
Kdump will only fail if both mount attemp fails.
Else if the kdump target mount failed or unstable, systemd will directly
jump to kdump failure action, and kdump fails.
This should improve the robustness in general with no other risk.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This partially reverts commit 6dee286467.
There are reports that NFSv3 is failing after this commit, and after
more debug, I found NFSv4 may not work properly if
"nfs4_disable_idmapping" is set to 0.
The cause of the failure is that kdump.sh runs after dracut's pre-pivot
and clean up hook, many dracut module will install hooks to kill some
running services, so if the dump target requires a service to be running
but it's killed, mount will fail.
Dracut ensures the configured mount points are ready before pre-pivot.
After pre-pivot, any further mounting operation may not work as expected.
Although there is no report of other type of dump target failure except
NFSv3, it's better to revert this, to avoid other potential risk, and wait
for a proper fix for that systemd/kernel issue.
Else, this may bring more trouble for further development.
But still keep the change in kdump-lib-initramfs.sh for better
robustness.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This commit remove almost all special workaround for atomic, and treat
all bind mounts in any environment equally.
Use a helper get_bind_mount_directory_from_path to get the bind mount
source path of given path.
is_atomic function now only used to determine the right /boot path
for atomic/silverblue environment.
And remove get_mntpoint_from_path(), it's the only function that never
ignore bind mount, and it have no caller after this clean up.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
For user specified target, the config value is used as the dump target,
and SAVE_PATH (path in kdump.conf) value is used as the dump path within
the dump target, no need to do anything extra with the path value.
Current code logic is not only complicated, it also wrongly generate
an redundantly long path in atomic/silverblue environment.
The right way is only check two things, and do nothing else:
1. The path exists within the target;
2. The target is large enough to hold to contain the vmcore.
Currently checking the target still requires it to be mounted so it will
error out if it's not mounted. Will implement some auto mount as next
step.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
get_save_path provides default value fail back and error check, no need
to repeat it again.
Also remove a redundant echo and grep in get_save_path
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
When building kdump initramfs for a SSH dump target, mkdumprd would
check whether it has the write permission on the SSH Server's
$DUMP_TARGET.
However $DUMP_TARGET is missing in the actual error message when the
user doesn't not have the write permission. For example:
# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
Could not create temporary directory on :/home/bhsharma/test. Make
sure user has write permission on destination
mkdumprd: failed to make kdump initrd
Starting kdump: [FAILED]
This patch using $1 value passed to mkdumprd, to print the
$DUMP_TARGET inside mkdir_save_path_ssh() function to
fix the issue.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In order to check whether the specified makedumpfile parameters are
valid or not when generating initramfs, use the --check-params option,
which was recently added.
With the patch, kdumpctl can point out mistakes in core_collector
option and failed. For example, if there is an practical mistake
that dump_level is -1:
# cat /etc/kdump.conf
core_collector makedumpfile -l --message-level 1 -d -1
# kdumpctl start
Detected change(s) in the following file(s):
/etc/kdump.conf
Rebuilding /boot/initramfs-5.4.19-200.fc31.x86_64kdump.img
Dump_level(-1) is invalid.
makedumpfile parameter check failed.
mkdumprd: failed to make kdump initrd
Starting kdump: [FAILED]
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
Previous commit f13eab6 ('mkdumprd: simplify dracut args parsing')
break dracut arguments parsing for some use case, this should fix it
well.
Passed nfs/local/iscsi/ssh dump test, and with extra dracut_argss.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Previously dracut_args is stored as an array in the shell code,
so we can just pass this array as argument to dracut.
But when trying to append new dracut argument, we have to manually
handle the quotes and spaces to ensure the appended argument is
splitted into array elements in the right way.
This is complex and hard to read or maintain. Instead, just store the
dracut_args as a plain string, and let xargs help parse it and call
dracut.
Simply passing $dracut_args or "$dracut_args" will either let dracut
consider the whole string as a single argument, or loss all the
quotes. xargs can handle it well and cover more corner cases.
Eg. one corner case before, if we have:
dracut_args --mount "/dev/sda1 /mnt/test xfs rw"
Kdump will fail, because the function add_dracut_arg() will wrongly
change the arguments into: (Notice the extra space.)
dracut_args --mount " /dev/sda1 /mnt/test xfs rw"
Instead of fixing it just use xargs instead. Tested with above config
and multiple other dracut_args values.
Resolves: bz1700136
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
check_size checks if the specified dump path on the ssh target have
enough space. And if the path doesn't exits, it will fail and exit.
mkdir_save_path_ssh should be called first to check if the path
exists, and create the path if it doesn't exits, so the size check
can always work properly.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Lichen Liu
Tao Liu
Tao Liu
Kairui Song
Hari Bathini
Lianbo Jiang
Coiby Xu
Bhupesh Sharma
Kazuhito Hagio