Description of problem
(https://bugzilla.redhat.com/show_bug.cgi?id=1465735):
Run `kdumpctl status` as normal user, get below error messages:
Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
...
The bug is caused by behavior difference between bash
and sh (bash in posix).
In the function single_instance_lock in kdumpctl script,
there is
exec 9>/var/lock/kdump
which will fail in user mode. However, this fail will cause
script exiting under bash but not exiting under sh, causing
infinite loop because the flock will always fail.
According to the 16th item in
ftp://ftp.gnu.org/old-gnu/Manuals/bash-2.02/html_node/bashref_66.html
If a POSIX.2 special builtin returns an error status, a non-
interactive shell exits.
And according to
https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html
exec is one of the POSIX.2 special builtin's.
This patch fixes the bug by checking exec return value.
Fixes: 9fb2996d05 ("kdumpctl: change the shebang header to use /bin/bash")
Signed-off-by: Ziyue Yang <ziyang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
This patch fixes the whitespace errors reported by
'rpmlint' or 'fedpkg lint' when they are run on kexec-tools srpm:
kexec-tools.spec:242: W: mixed-use-of-spaces-and-tabs (spaces: line 107,
tab: line 242)
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The /sys/modules/*/drivers sysfs entries do not exist anymore on newer
kernels which means that the DRM moduels would never be included.
Instead check if there is any device with a "drm" sysfs directory to
decide on whether DRM modules need to be included.
Acked-by: Dave Young <dyoung@redhat.com>
fence_kdump_nodes should include list of cluster node(s) except localhost.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
kdump should not send fence_kdump notifications to local host, because
the role of the falied node (i.e local host) is to send fence_kdump
notifications to other nodes to tell them I'm kdumping, tell to itself is
nonsense. And we have excluded hostname of local host but when one use ip
address we also need exclude it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We met one issue that when changing softlink of "/usr/bin/sh"
to point to "ksh" instead of the default "bash", kdumpctl will
not work and go wrong.
kdumpctl is expected to run under bash like dracut, we should
change its shebang header from "#!/bin/sh" to "#!/bin/bash".
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The "time kdumpctl start" command shows that strip_comments()
consumes lots of cpu time. By only calling it when necessary,
it saves us nearly half second.
Tested on my Fedora kvm machine.
Before this patch:
$ time kdumpctl start
kexec: loaded kdump kernel
Starting kdump: [OK]
real 0m1.849s
user 0m1.497s
sys 0m0.462s
After this patch:
$ time kdumpctl start
kexec: loaded kdump kernel
Starting kdump: [OK]
real 0m1.344s
user 0m1.195s
sys 0m0.195s
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1449801
"cat $KDUMP_CONFIG_FILE|grep -v "^#"|while read ..." use pipes
to invoke subshells, as a result we met "the dreaded inaccessible
variables within a subshell problem" as described in the book
"Advanced Bash-Scripting Guide".
It cause regressions, so revert it.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
I found using "cat $KDUMP_CONFIG_FILE|grep -v "^#"|while read ..."
instead of "while read ... do ...; done < $KDUMP_CONFIG_FILE" will
make the script run faster, it saves us nearly half second.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We use faster "lsinitrd XXX -f usr/lib/dracut/build-parameter.txt"
instead of "lsinitrd XXX | grep "^Arguments:" | head -1", this can
save us around one second.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Use the logic of dracut 04watchdog/module-setup.sh to check,
then we only need to compare the content of 00-watchdog.conf,
so we can save one operation of lsinitrd.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
handle_mode_switch() can ensure the correct logic, so remove
the needless is_mode_switched(). This helps to save one slow
lsinitrd operation for each boot.
Improved backup_default_initrd() to judge DEFAULT_INITRD.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Some cloud people complained that VM boot speed is slower than
Ubuntu and other distributions, after some debugging, we found
that one of the causes is kdump service starts too slow(7seconds
according to the test result on the test VM), actually there is
no "crashkernel=X" specified. Although kdump service is parallel,
it affects the speed more or less especially on VMs with few cpus,
which is unacceptable. It is even worse when kdump initramfs is
built out in case of no reserved memory at first boot.
Commit afa4a35d3 ("kdumpctrl: kdump feasibility should fail if no
crash memory") can actually solve this issue.
This patch is a supplement of above-mentioned commit, we bail out
start() even earlier in case of no reserved memory.
Also made some cosmatic changes for check_crash_mem_reserved().
1) Before this patch
$ time kdumpctl start
No memory reserved for crash kernel.
Starting kdump: [FAILED]
real 0m0.282s
user 0m0.184s
sys 0m0.146s
2) After this patch
$ time kdumpctl start
No memory reserved for crash kernel
Starting kdump: [FAILED]
real 0m0.010s
user 0m0.008s
sys 0m0.001s
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This patch introduces the 'force_no_rebuild' option
inside the 'kdump.conf' and its handling inside the 'kdumpctl'
script.
There might be several use cases, where a system admin
decides that he doesn't need to rebuild the kdump initrd
and wants to use an existing version of the same. In such cases,
he can set the 'force_no_rebuild' option inside 'kdump.conf'
to 1, to force the 'kdumpctl' script not to rebuild the kdump
initrd.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The kdump-capture.service will fail, if the following conds are meet up.
-1. boot up a VM with the following cmd:
qemu-kvm -name 'avocado-vt-vm1' -sandbox off -machine pc -nodefaults -vga cirrus \
-drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=$guest_img \
-device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04 \
-device virtio-net-pci,mac=9a:4d:4e:4f:50:51,id=id3DveCw,vectors=4,netdev=idgW5YRp,bus=pci.0,addr=05 \
-netdev tap,id=idgW5YRp \
-m 2048 \
-smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \
-cpu 'SandyBridge',+kvm_pv_unhalt \
-vnc :0 \
-rtc base=utc,clock=host,driftfix=slew \
-boot order=cdn,once=c,menu=off,strict=off \
-enable-kvm \
-monitor stdio \
-qmp tcp:localhost:4444,server,nowait
-2. in kernel cmdline with the following options: console=tty0 console=ttyS0,
Because the "-nodefaults" option in qemu cmd excludes the emulation of serial port, the ttyS0 will
have no real backend device. We can observe such issue in 1st kernel by:
echo teststring > /dev/console or
echo teststring > /dev/ttyS0,
It gets the error "-bash: echo: write error: Input/output error".
Such conds cause small issue in 1st kernel, but it is a big problem for kdump-capture and emergency
service.
This patch aims to work aroundthe issue in kdump-capture service:
dump_fs() return value will affect the following code in dracut-kdump.sh
DUMP_RETVAL=$? <---
do_kdump_post $DUMP_RETVAL
if [ $? -ne 0 ]; then
echo "kdump: kdump_post script exited with non-zero status!"
fi
Although kdump-capture saves the vmcore successfully, but it exit 1 and
fall on emergency service.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
KASLR is to enhance security on OS kernel. While kdump kernel is
working after normal kernel corrupted. There's no need to do kaslr
in kdump kernel, so add 'nokaslr' to disable kaslr.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 892bea7aa
We already eliminated the root filesystem by removing "root=X"
in case of non-root dumping, and for livecd it must be non-root
dumping according to "live-image-kdump-howto.txt".
So it's time to revert this commit.
Also update "live-image-kdump-howto.txt", make sure users do not
configure "default dump_to_rootfs".
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by:Dave Young <dyoung@redhat.com>
Since the current dracut of Fedora already supports not always
mounting root device, we can remove "root=X" from the command
line directly, and always get the dump target specified in
"/etc/kdump.conf" and mount it. If the dump target is located
at root filesystem, we will add the root mount info explicitly
from kdump side instead of from dracut side.
For example, in case of nfs/ssh/usb/raw/etc(non-root) dumping,
kdump will not mount the unnecessary root fs after this change.
This patch removes "root=X" via the "KDUMP_COMMANDLINE_REMOVE"
(if "default dump_to_rootfs" is specified, don't remove "root=X"),
and mounts non-root target under "/kdumproot", the root target
still under "/sysroot"(to be align with systemd sysroot.mount).
After removing "root=X", we now add root fs mount information
explicitly from the kdump side.
Changed check_dump_fs_modified() a little to avoid rebuild when
dump target is root, since we add root fs mount explicitly now.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by:Dave Young <dyoung@redhat.com>
For the following scripts:
cmdline="root=/dev/mapper/fedora-root rd.lvm.lv=fedora/root rw"
remove_cmdline_param $cmdline "root"
cmdline="root=nfs4:192.168.122.9:/ ip=ens3:dhcp rw"
remove_cmdline_param $cmdline "root"
The current implementation will get the wrong results:
"rd.lvm.lv=fedora/ rw"
":/ ip=ens3:dhcp rw"
After this patch we can get the correct results:
"rd.lvm.lv=fedora/root rw"
"ip=ens3:dhcp rw"
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by:Dave Young <dyoung@redhat.com>
When kexec_crash_loaded does not exist, means kdump was not enabled in
kernel we get
$ kdumpctl status
cat: /sys/kernel/kexec_crash_loaded: No such file or directory
/usr/bin/kdumpctl: line 879: [: ==: unary operator expected
Kdump is not operational
After this patch:
$ kdumpctl status
Perhaps CONFIG_CRASH_DUMP is not enabled in kernel
Kdump is not operational
Signed-off-by: Pratyush Anand <panand@redhat.com>
Reviewed-by: Bhupesh Sharma <bhsharma@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We met a problem that the kdump emergency service failed to
start when the target dump timeout(we passed "rd.timeout=30"
to kdump), it reported "Transaction is destructive" messages:
[ TIME ] Timed out waiting for device dev-mapper-fedora\x2droot.device.
[DEPEND] Dependency failed for Initrd Root Device.
[ SKIP ] Ordering cycle found, skipping System Initialization
[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[ SKIP ] Ordering cycle found, skipping System Initialization
[ SKIP ] Ordering cycle found, skipping Initrd Default Target
[DEPEND] Dependency failed for File System Check on /dev/mapper/fedora-root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Stopped dracut pre-udev hook.
[ OK ] Stopped dracut cmdline hook.
Starting Setup Virtual Console...
Starting Kdump Emergency...
[ OK ] Reached target Initrd Default Target.
[ OK ] Stopped dracut initqueue hook.
Failed to start kdump-error-handler.service: Transaction is destructive.
See system logs and 'systemctl status kdump-error-handler.service' for details.
[FAILED] Failed to start Kdump Emergency.
See 'systemctl status emergency.service' for details.
[DEPEND] Dependency failed for Emergency Mode.
This is because in case of root failure, initrd-root-fs.target
will trigger systemd emergency target which requires the systemd
emergency service actually is kdump-emergency.service, then our
kdump-emergency.service starts kdump-error-handler.service with
"systemctl isolate"(see 99kdumpbase/kdump-emergency.service, we
replace systemd's with this one under kdump).
This will lead to systemd two contradictable jobs queued as an
atomic transaction:
job 1) the emergency service gets started by initrd-root-fs.target
job 2) the emergency service gets stopped due to "systemctl isolate"
thereby throwing "Transaction is destructive".
In order to solve it, we can utilize "IgnoreOnIsolate=yes" for both
kdump-emergency.service and kdump-emergency.target. Unit with attribute
"IgnoreOnIsolate=yes" won't be stopped when isolating another unit,
they can keep going as expected in case be triggered by any failure.
We add kdump-emergency.target dedicated to kdump the similar way
as did for kdump-emergency.service(i.e. will replace systemd's
emergency.target with kdump-emergency.target under kdump), and
adds "IgnoreOnIsolate=yes" into both of them.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
[bhe: improve the patch log about IgnoreOnIsolate="]
We replace "reserved_memory = XXXX"(default value is 8192) with
"reserved_memory = 1024" in /etc/lvm/lvm.conf used by "lvm2", it
can save 7MB peak memory consumption, so lower the possibility of
OOM under kdump.
For kdump, we don't have too many lvm targets, lvm2 locates in the
RAM(rootfs), so don't need that much memory, as discussed with lvm
people, they agreed that we use 1MB under kdump as long as there
are not that many lvm targets invloved.
We modify /etc/lvm/lvm.conf when "99kdumpbase" install() is executed,
because it is parsed after "90lvm" by dracut.
We add the code unconditionally with &>/dev/null to ignore errors, it
doesn't matter in case of "lvm" not included(i.e. there is no lvm.conf).
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Patches have been taken from kexec-tools and makedumpfile to fix issue
with `makedumpfile --mem-usage /proc/kcore`.
Two of the patches is from kexec-tools and rest are from makedumpfile.
All the patches have been acked upstream and applies without conflict.
Kexec-tools patches:
(kexec-tools-2.0.14-x86-x86_64-Fix-format-warning-with-die.patch), which
fixes koji build issue.
kexec-tools-2.0.14-build_mem_phdrs-check-if-p_paddr-is-invalid.patch fixes
the regresssion caused by kernel /proc/kcore fix to use -1 as default value
of p_paddr for pt_loads. Without his patch kexec -p will fail with latest
kernel.
Other makedumpfile patches are backported to support --mem-usage while
kernel kaslr being enabled. Details please see the patch log of the individual
patches.
All the patches are backport of upstream commits.
Patches has been tested with kernel 4.11.0-0.rc1.git0.1.fc26.x86_64.
# makedumpfile --mem-usage /proc/kcore -f
The kernel version is not supported.
The makedumpfile operation may be incomplete.
TYPE PAGES EXCLUDABLE DESCRIPTION
----------------------------------------------------------------------
ZERO 1960 yes Pages filled
with zero
NON_PRI_CACHE 22850 yes Cache pages
without private flag
PRI_CACHE 1517 yes Cache pages with
private flag
USER 32522 yes User process
pages
FREE 1898981 yes Free pages
KERN_DATA 78721 no Dumpable kernel
data
page size: 4096
Total pages on system: 2036551
Total size on system: 8341712896 Byte
We won't need to pass -f once fedora kernel is rebased with v4.12.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
For -q option, as man grep says: Exit immediately with zero status if
any match is found, even if an error was detected.
So when matching, the read side of pipe is closed by "grep -q", while
the write side still try to write more data, which cause SIGPIPE to the
process, and the shell can not exit with 0. It depends on the kernel's
implementation of pipe to decide how much data written by the producer
can trigger the bug.
Bash test script:
#!/bin/sh
set -o pipefail
dd if=/dev/zero of=text.file bs=1M count=1
sed -i '1s/^/keyword /' text.file
cat text.file | grep -q keyword
echo $?
Notice the "set -o pipefail" is set by dracut, so
mkdumprd -> dracut -> dracut-module-setup.sh -> is_pcs_fence_kdump()
trigger the bug.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
The file fadump-howto.txt has an incorrect link for further information
about sysrq usage. Fix it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1399436
Since currently crashkernel= will be handled in kdump anaconda addon
we can safely remove rhcrashkernel-param callback.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Check the number of cpus for x86_64 kdump kernel to boot with.
We met an issue on x86_64: kdump runs out of vectors with the
default "nr_cpus=1", when requesting tons of irqs.
This patch detects such situation and warns users about the risk.
The total number of vectors percpu is 256 defined by x86 architecture.
The available vectors can be allocated to io devices percpu starts
from FIRST_EXTERNAL_VECTOR(see kernel code), and some high-numbered
ones are consumed by some system interrupts. As a result, the vectors
for io device are within [FIRST_EXTERNAL_VECTOR, FIRST_SYSTEM_VECTOR),
with one known exception, 0x80 within the range is reserved specially
as the syscall vector.
FIRST_EXTERNAL_VECTOR is invariably 32, while FIRST_SYSTEM_VECTOR can
vary between different kernel versions. E.g. FIRST_SYSTEM_VECTOR gets
0xef(with CONFIG_X86_LOCAL_APIC on)for linux-4.10, that is 17 vectors
reserved, considering it may increase in the future and the special
vectors, we use a flexible variance and assume there are 32 reserved
from FIRST_EXTERNAL_VECTOR. Then the max vectors for device interrupts
percpu is: (256-32)-32=192, we acquire the number N of device interrupts
from /proc/irq/, then the number of minimal cpus required is calculated:
(N + 192 - 1) / 192
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since KDUMP_COMMANDLINE is a global variable, prepare_cmdline can
modify it directly instead of echoing back the result. This change
enables it to output messages.
Changed some coding styles.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Resolves: bz1411240
Use --hostonly-i18n to force dracut to install only needed
keyboard and font files according to host's configuration, which
reduced initramfs's size by 2M on F25 x86_64.
Signed-off-by: Tong Li <tonli@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1403658
commit d3e2e0ebcb2f9ba803576e48e7b9c752cc7e9f1a
Author: Tong Li <tonli@redhat.com>
Date: Mon Dec 12 13:10:08 2016 +0800
Fix 'an unknown error has occurred' issue when selecting languages using non-latin characters
Now when kdump_anaconda_addon is enabled and languages which use non-latin
characters are selected in anaconda, e.g. Chinese and Japanese, it will
raise an error and unable to continue to finish the installation process.
This is because 'gettext.ldgettext' will return a byte object when
translation includes non-latin character, while anaconda's core code
requires a string. To fix this, we apply the mothod used by pyanaconda,
which is invoking gettext after getting a translation instance. This can
make sure that a str object will be returned.
Signed-off-by: Dave Young <dyoung@redhat.com>
