We met a problem that the kdump emergency service failed to
start when the target dump timeout(we passed "rd.timeout=30"
to kdump), it reported "Transaction is destructive" messages:
[ TIME ] Timed out waiting for device dev-mapper-fedora\x2droot.device.
[DEPEND] Dependency failed for Initrd Root Device.
[ SKIP ] Ordering cycle found, skipping System Initialization
[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[ SKIP ] Ordering cycle found, skipping System Initialization
[ SKIP ] Ordering cycle found, skipping Initrd Default Target
[DEPEND] Dependency failed for File System Check on /dev/mapper/fedora-root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Stopped dracut pre-udev hook.
[ OK ] Stopped dracut cmdline hook.
Starting Setup Virtual Console...
Starting Kdump Emergency...
[ OK ] Reached target Initrd Default Target.
[ OK ] Stopped dracut initqueue hook.
Failed to start kdump-error-handler.service: Transaction is destructive.
See system logs and 'systemctl status kdump-error-handler.service' for details.
[FAILED] Failed to start Kdump Emergency.
See 'systemctl status emergency.service' for details.
[DEPEND] Dependency failed for Emergency Mode.
This is because in case of root failure, initrd-root-fs.target
will trigger systemd emergency target which requires the systemd
emergency service actually is kdump-emergency.service, then our
kdump-emergency.service starts kdump-error-handler.service with
"systemctl isolate"(see 99kdumpbase/kdump-emergency.service, we
replace systemd's with this one under kdump).
This will lead to systemd two contradictable jobs queued as an
atomic transaction:
job 1) the emergency service gets started by initrd-root-fs.target
job 2) the emergency service gets stopped due to "systemctl isolate"
thereby throwing "Transaction is destructive".
In order to solve it, we can utilize "IgnoreOnIsolate=yes" for both
kdump-emergency.service and kdump-emergency.target. Unit with attribute
"IgnoreOnIsolate=yes" won't be stopped when isolating another unit,
they can keep going as expected in case be triggered by any failure.
We add kdump-emergency.target dedicated to kdump the similar way
as did for kdump-emergency.service(i.e. will replace systemd's
emergency.target with kdump-emergency.target under kdump), and
adds "IgnoreOnIsolate=yes" into both of them.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
[bhe: improve the patch log about IgnoreOnIsolate="]
We replace "reserved_memory = XXXX"(default value is 8192) with
"reserved_memory = 1024" in /etc/lvm/lvm.conf used by "lvm2", it
can save 7MB peak memory consumption, so lower the possibility of
OOM under kdump.
For kdump, we don't have too many lvm targets, lvm2 locates in the
RAM(rootfs), so don't need that much memory, as discussed with lvm
people, they agreed that we use 1MB under kdump as long as there
are not that many lvm targets invloved.
We modify /etc/lvm/lvm.conf when "99kdumpbase" install() is executed,
because it is parsed after "90lvm" by dracut.
We add the code unconditionally with &>/dev/null to ignore errors, it
doesn't matter in case of "lvm" not included(i.e. there is no lvm.conf).
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Patches have been taken from kexec-tools and makedumpfile to fix issue
with `makedumpfile --mem-usage /proc/kcore`.
Two of the patches is from kexec-tools and rest are from makedumpfile.
All the patches have been acked upstream and applies without conflict.
Kexec-tools patches:
(kexec-tools-2.0.14-x86-x86_64-Fix-format-warning-with-die.patch), which
fixes koji build issue.
kexec-tools-2.0.14-build_mem_phdrs-check-if-p_paddr-is-invalid.patch fixes
the regresssion caused by kernel /proc/kcore fix to use -1 as default value
of p_paddr for pt_loads. Without his patch kexec -p will fail with latest
kernel.
Other makedumpfile patches are backported to support --mem-usage while
kernel kaslr being enabled. Details please see the patch log of the individual
patches.
All the patches are backport of upstream commits.
Patches has been tested with kernel 4.11.0-0.rc1.git0.1.fc26.x86_64.
# makedumpfile --mem-usage /proc/kcore -f
The kernel version is not supported.
The makedumpfile operation may be incomplete.
TYPE PAGES EXCLUDABLE DESCRIPTION
----------------------------------------------------------------------
ZERO 1960 yes Pages filled
with zero
NON_PRI_CACHE 22850 yes Cache pages
without private flag
PRI_CACHE 1517 yes Cache pages with
private flag
USER 32522 yes User process
pages
FREE 1898981 yes Free pages
KERN_DATA 78721 no Dumpable kernel
data
page size: 4096
Total pages on system: 2036551
Total size on system: 8341712896 Byte
We won't need to pass -f once fedora kernel is rebased with v4.12.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
For -q option, as man grep says: Exit immediately with zero status if
any match is found, even if an error was detected.
So when matching, the read side of pipe is closed by "grep -q", while
the write side still try to write more data, which cause SIGPIPE to the
process, and the shell can not exit with 0. It depends on the kernel's
implementation of pipe to decide how much data written by the producer
can trigger the bug.
Bash test script:
#!/bin/sh
set -o pipefail
dd if=/dev/zero of=text.file bs=1M count=1
sed -i '1s/^/keyword /' text.file
cat text.file | grep -q keyword
echo $?
Notice the "set -o pipefail" is set by dracut, so
mkdumprd -> dracut -> dracut-module-setup.sh -> is_pcs_fence_kdump()
trigger the bug.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
The file fadump-howto.txt has an incorrect link for further information
about sysrq usage. Fix it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1399436
Since currently crashkernel= will be handled in kdump anaconda addon
we can safely remove rhcrashkernel-param callback.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Check the number of cpus for x86_64 kdump kernel to boot with.
We met an issue on x86_64: kdump runs out of vectors with the
default "nr_cpus=1", when requesting tons of irqs.
This patch detects such situation and warns users about the risk.
The total number of vectors percpu is 256 defined by x86 architecture.
The available vectors can be allocated to io devices percpu starts
from FIRST_EXTERNAL_VECTOR(see kernel code), and some high-numbered
ones are consumed by some system interrupts. As a result, the vectors
for io device are within [FIRST_EXTERNAL_VECTOR, FIRST_SYSTEM_VECTOR),
with one known exception, 0x80 within the range is reserved specially
as the syscall vector.
FIRST_EXTERNAL_VECTOR is invariably 32, while FIRST_SYSTEM_VECTOR can
vary between different kernel versions. E.g. FIRST_SYSTEM_VECTOR gets
0xef(with CONFIG_X86_LOCAL_APIC on)for linux-4.10, that is 17 vectors
reserved, considering it may increase in the future and the special
vectors, we use a flexible variance and assume there are 32 reserved
from FIRST_EXTERNAL_VECTOR. Then the max vectors for device interrupts
percpu is: (256-32)-32=192, we acquire the number N of device interrupts
from /proc/irq/, then the number of minimal cpus required is calculated:
(N + 192 - 1) / 192
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since KDUMP_COMMANDLINE is a global variable, prepare_cmdline can
modify it directly instead of echoing back the result. This change
enables it to output messages.
Changed some coding styles.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Resolves: bz1411240
Use --hostonly-i18n to force dracut to install only needed
keyboard and font files according to host's configuration, which
reduced initramfs's size by 2M on F25 x86_64.
Signed-off-by: Tong Li <tonli@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1403658
commit d3e2e0ebcb2f9ba803576e48e7b9c752cc7e9f1a
Author: Tong Li <tonli@redhat.com>
Date: Mon Dec 12 13:10:08 2016 +0800
Fix 'an unknown error has occurred' issue when selecting languages using non-latin characters
Now when kdump_anaconda_addon is enabled and languages which use non-latin
characters are selected in anaconda, e.g. Chinese and Japanese, it will
raise an error and unable to continue to finish the installation process.
This is because 'gettext.ldgettext' will return a byte object when
translation includes non-latin character, while anaconda's core code
requires a string. To fix this, we apply the mothod used by pyanaconda,
which is invoking gettext after getting a translation instance. This can
make sure that a str object will be returned.
Signed-off-by: Dave Young <dyoung@redhat.com>
new fedpkg need sha512 sum, see below error:
$ fedpkg upload kexec-tools-2.0.14.tar.xz
Could not execute upload: Can not upload a new source file with a sha512 hash, as the "/home/dyoung/fedpkg/kexec-tools/sources" file contains at least one line with a md5 hash.
Please redo the whole "/home/dyoung/fedpkg/kexec-tools/sources" file using:
`/bin/fedpkg new-sources file1 file2 ...`
Thus update the sources file here.
Signed-off-by: Dave Young <dyoung@redhat.com>
kdump_to_udev_name function name cause confusion to people, change it to
kdump_get_persistent_dev which sounds better.
Signed-off-by: Dave Young <dyoung@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Although we use by-id in mkdumprd as persistent policy for the dump target
checking, finally it is not used in kdump 2nd kernel because we call dracut
function in module-setup.sh without persistent policy specified that means
kdump will copy default "by-uuid" dev name.
Though by-uuid usually works and it is still better to fix it as raw disk
uuid make no sense.
Also do not need to call bind mount adjust function for raw dump, here add
another switch case for raw dump and cleanup the functions with short
variable names to keep code shorter.
Signed-off-by: Dave Young <dyoung@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
We maintained several kdump specific functions which are duplicate with the
similar versions in dracut, Dracut upstream splitted dracut init stuff from
dracut-functions.sh so that we can source it now.
Notes about kdump_get_presistent_dev:
Dracut now has a persistent_policy feature, for kdump when we dump to
raw disks we do not care the filesystem uuid and labels so we prefer to
search disk id instead. For raw disk set the persistent_policy before calling
get_persistent_dev ensure kdump logic still work.
Tested filesystem and raw dump in kvm guests.
[Xunlei: drop other functions other than get_persistent_dev.]
Signed-off-by: Dave Young <dyoung@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Backport upstream kexec-tools commit for correct kaslr page_offset value
commit 9f62cbddddfc93d78d9aafbddf3e1208cb242f7b
Author: Thomas Garnier <thgarnie@google.com>
Date: Tue Sep 13 15:10:05 2016 +0800
kexec/arch/i386: Add support for KASLR memory randomization
Multiple changes were made on KASLR (right now in linux-next). One of
them is randomizing the virtual address of the physical mapping, vmalloc
and vmemmap memory sections. It breaks kdump ability to read physical
memory.
This change identifies if KASLR memories randomization is used by
checking if the page_offset_base variable exists. It search for the
correct PAGE_OFFSET value by looking at the loaded memory section and
find the lowest aligned on PUD (the randomization level).
Related commits on linux-next:
- 0483e1fa6e09d4948272680f691dccb1edb9677f: Base for randomization
- 021182e52fe01c1f7b126f97fd6ba048dc4234fd: Enable for PAGE_OFFSET
Signed-off-by: Thomas Garnier <thgarnie@google.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When fadump mode is enabled, the default initrd is rebuilt with kdump
dracut module. As the default initrd is altered, the original default
initrd is backed up. But we are not restoring it when fadump mode is
disabled. This patch tries to restore the backed up default initrd on
disabling fadump mode.
Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
When a remote dump target is specified, kdump dracut module prefixes
'kdump-' to network interface name (ifname) as kernel assigned names
are not persistent. In fadump mode, kdump dracut module is added to
the default initrd, which adds the 'kdump-' prefix to the ifname of
the prodcution kernel itself. If fadump mode is disabled after this,
kdump dracut module picks the ifname that is already prefixed with
'kdump-' in the production kernel and adds another 'kdump-' to it,
making the ifname something like kdump-kdump-eth0 for kdump kernel.
Eventually, kdump kernel fails with below traces:
dracut-initqueue[246]: RTNETLINK answers: Network is unreachable
dracut-initqueue[246]: arping: Device kdump-kdump-eth0 not available.
The ip command shows the below:
kdump:/# ip addr show kdump-kdump-eth0
2: kdump-kdump-eth: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 \
qdisc pfifo_fast state UNKNOWN qlen 1000
link/ether 22:82:87:7b:98:02 brd ff:ff:ff:ff:ff:ff
inet6 2002:903:15f:550:2082:87ff:fe7b:9802/64 scope global \
mngtmpaddr dynamic
valid_lft 2591890sec preferred_lft 604690sec
inet6 fe80::2082:87ff:fe7b:9802/64 scope link
valid_lft forever preferred_lft forever
kdump:/#
The trailing 0 from kdump-kdump-eth0 is missing in the ifname, probably
truncated owing to ifname length limit, while setting.
This patch fixes this by avoiding addition of the prefix 'kdump-' when
such prefix is already present in the ifname.
Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
Kernels of live images are booted with a kernel parameter which looks
like "root=live:CDLABEL=Fedora-WS-Live-25_A-2". This argument can't be
recognized by dracut during kdump process and will cause failure
of kdump if users didn't set KUDMP_COMMANDLINE in /etc/sysconfig/kdump.
So we should filter out 'root' when we find such a parameter in
/proc/cmdline to make kdump work correctly in live images.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This is a short document about how to setup kdump on live images. All
steps were tested on Fedora 25 Alpha LiveCD and saved vmcore captured
by kdump to a USB stick successfully.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
V1 of the KASLR patch which was backported in 2.0.13-6 had wrong
calculation for page_offset. This patch takes the correct difference.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Fix Bug 925630 - kexec-tools: support for arm64
https://bugzilla.redhat.com/show_bug.cgi?id=925630
involves three things:
1. back porting upstream code to enable the function of kexec-tools on arm64
patchset backported from upstream:
commit abdfe97736f89d9bc73662b9134604b0229a599e
commit 522df5f7217fda01ece3f6ac3e9987b0320c2bb0
commit 217bcc00c9309416a6c6cd0584196559d28a9259
2. fix the arm related building issue by using autoreconf in spec file
3. patches to fix the issue of higher version gcc used by koji (not upstrem yet,
and the corresponding fix in kernel side is in other package)
kexec-tools-2.0.13-arm64-Add-support-for-additional-relocations-in-the-kexec-purgatory-code.patch
kexec-tools-2.0.13-arm64-Add-support-of-R_AARCH64_PREL32-relocation-in-.patch
[panand: apply patches in 3 of above paragraph]
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Since in Fedora 25 kernel kaslr is enabled (x86) but makedumpfile can not save
a correct vmcore, so it means kdump default setup will not work.
Pratyush posted a patch series to upstream which can fix the issue. Let's merge them in F25, will get the normal fixes after it being merged in upstream, we hopefully can rebase soon in rawhide.
This is an urgent fix for F25 since F25 freeze is this week.
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
We get following error on the systems that have everything built-in and no
initrd is used.
Kernel dev name of /dev/root is not found.
Dump target /dev/root is probably not mounted.
It happens because `df $path` gets /dev/root from /proc/self/mountinfo.
Fix this by identifying real target device when `df $path` returns
Filesystem as /dev/root.
Reported-and-tested-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since we also check for mount point of $_target after if/else loop, so
there is no need to do the same thing specifically in else loop as well.
Remove those duplicate statement from else loop.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
On a diskless client /etc/fstab does not exist. Therefore check
modification time of this file for rebuild only if it exists.
Also use --fstab option with findmnt only when /etc/fstab exists.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
commit "28e8c4b5ac89 kdumpctl: Move file modification check logic in
check_system_modified()" copied file modification check logic instead of
moving.
Kill the duplicate logic from original calling function check_rebuild().
Signed-off-by: Pratyush Anand <panand@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Fedora dracut-network version >= 044-117 must be installed to work with
this set of kexec-tools patches. Therefore adding dependency for them.
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Kdump explicitly adds "nfs" dracut module in case of nfs dumping,
actually in case of nfs dump, nfs is a mount target, and will be
added into host_fs_types[], thus dracut will add it automatically,
according to 95nfs/module-setup.sh check().
So, we can safely remove all the add_dracut_module "nfs".
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Update "kexec-kdump-howto" to illustrate the usage of special mount
information via "dracut_args".
Reviewed-by: Pratyush Anand <panand@redhat.com>
Suggested-by: Dave Young <dyoung@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
