Previously, kdump will restart / reload for many times on hotplug
event, especially memory hotplug events. Hotplugged memory may
generate many udev event as memory are managed and hotplugged in
small chunks by the kernel.
This results in unnecessary system workload and an actually longer
delay of kdump reload and the hotplug event, as udev will either
get blocked or kdumpctl will be waiting for other triggered operation.
To fix this, introduce a kdump-udev-throttler as an agent which will
be called by udev and merge concurrent kdump restart requests. Tested
with a Hyper-V VM which is failing due to udev timeout previously,
no new issues found.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
In dracut-049, a new squash module is introduced, it can reduce the
memory usage of kdump initramfs in the capture kernel, this helps a lot
on lowering the risk of OOM failure.
Tested with latest rawhide with NFS, SSH and local dump.
Signed-off-by: Kairui Song <kasong@redhat.com>
Resolves: bz1619122
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1619122
This patch fixes the "Unhandled rela relocation: R_X86_64_PLT32" error
that we are seeing with Fedora 29 (and newer kernels > 4.18) which
trying to run kexec/kdump on x86_64 machines.
The patch is being discussed upstream and has been ACK'ed by Baoquan and
myself (see <https://www.spinics.net/lists/kexec/msg21255.html>) and I
have also tested the same on Fedora 29/rawhide x86_64 machine as well:
Before the patch:
----------------
[root@hp-bl480c-01 ~]# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
Unhandled rela relocation: R_X86_64_PLT32
kexec: failed to load kdump kernel
Starting kdump: [FAILED]
After the patch:
---------------
[root@hp-bl480c-01 ~]# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
kexec: loaded kdump kernel
Starting kdump: [OK]
Suggested Upstream Fix:
In response to a change in binutils, commit b21ebf2fb4c
(x86: Treat R_X86_64_PLT32 as R_X86_64_PC32) was applied to
the linux kernel during the 4.16 development cycle and has
since been backported to earlier stable kernel series. The
change results in the failure message in $SUBJECT when
rebooting via kexec.
Fix this by replicating the change in kexec.
Signed-off-by: Chris Clayton <chris2553@googlemail.com>
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Kdump anaconda has been included as a subpackage for a long time, which
is not a good practice, as the anaconda plugin should be built as
noarch and it does not belong to kexec-tools. We have created a new
package 'kdump-anaconda-addon', so remove it here.
The release version should be bumped later so that kdump-anaconda-addon
could mark previous versions as obsoleted.
Signed-off-by: Kairui Song <kasong@redhat.com>
armv7hl build failed because no makedumpfile* built but the latest commit
tries to install them.
Exclude armv7hl in the code chunk.
Signed-off-by: Dave Young <dyoung@redhat.com>
kexec_test seems to be no longer used upstream, so we had introduced
the 'kexec-tools-2.0.3-disable-kexec-test.patch' earlier to disable the
same from fedora kexec-tools as well.
However an earlier patch "Remove obsolete kdump tool" now explicitly
installs needed files via appropriate logic in .spec file, so we can
drop this patch now to reduce the maintenance burden.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1441677
Testing: On x86_64 Fedora machine. After this patch kdump utility and related
man page cannot be found on this machine:
[root@tyan-gt24-09 ~]# which kdump
/usr/bin/which: no kdump in
(/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)
[root@tyan-gt24-09 ~]# man kdump
No manual entry for kdump
Update the fedora 'kexec-tools.spec' to not install the obsolete
kdump tool.
I have submitted an upstream patch to obsolete the kdump tool from
upstream kexec-tools (which has been accepted), but after an internal
discussion we decided not to backport the upstream 'kexec-tools' patch
(which does the same) for fedora, as we would prefer to manage the
changes directly in the .spec file itself.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Kdump always use _proto=dhcp for both ipv4 and ipv6. But for ipv6
the dhcp address assignment is not like ipv4, there are different ways
for it, stateless and stateful, see below document:
https://fedoraproject.org/wiki/IPv6Guide
In case stateless, kernel can do the address assignment, dracut use
_proto=auto6; for stateful case, dracut use _proto=dhcp6.
But it is hard to decide whether stateless or stateful takes effect,
hence, dracut introduces ip=either6 option, which can try both of these
method automatically for us. For detail, refer to dracut:
commit 67354ee 40network: introduce ip=either6 option
We do not see bug reports before because for the most auto6 cases
kernel assign ip address before dhclient, kdump just happened to work.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Kdump service starts too late, so early crashes will have no chance
to get kdump kernel booting, this will cause crash information to be
lost. It is necessary to add a dracut module in order to load crash
kernel and initramfs as early as possible. You can provide "rd.early
kdump" in grub commandline to enable, then the early kdump will load
those files like the normal kdump, which is disabled by default.
For the normal kdump service, it can check whether the early kdump
has loaded the crash kernel and initramfs. It has no conflict with
the early kdump.
If you rebuild the new initramfs for early kdump, the new initramfs
size will become large, because it will put the vmlinuz and kdump
initramfs into the new initramfs.
In addition, early kdump doesn't support fadump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Reviewed-by: Kazuhito Hagio <khagio@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut has "--hostonly-cmdline" which can generate cmdlines(if any)
regarding the dump target, it's an existing way for us to use to
simplify the code. E.g. We already removed generate_lvm_cmdlines(),
to use "--hostonly-cmdline".
But "--hostonly-cmdline" has other issues(e.g. BZ1451717), it adds
needless devices for kdump like root device.
Now dracut supports "--no-hostonly-default-device" which enables
us to only add the kdump target, which can avoid needless devices
being recognized under kdump. Thus "--hostonly-cmdline" side effects
can be avoided with the help of "--no-hostonly-default-device".
This patch applies dracut's "--hostonly-cmdline" together with
"--no-hostonly-default-device" to achieve above-mentioned purpose.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Added patch from panand which was accepted by upstream but not merged in upstream yet.
kexec-tools-2.0.15-makedumpfile-fix-SECTION_MAP_MASK-for-kernel-bigger-than-4.13.patch
https://bugzilla.redhat.com/show_bug.cgi?id=1474706
Makedumpfile failed with below error messages, which is caused by kernel changes 65ade2f872b474fa8a04c2d397783350326634e6:
Buffer size for the cyclic mode: 95992
vtop4_x86_64: Can't get the symbol of init_level4_pgt.
readmem: Can't convert a virtual address(ffffffff8fe18284) to physical address.
readmem: type_addr: 0, addr:ffffffff8fe18284, size:390
check_release: Can't get the address of system_utsname.
Pull in Pratyush's fix in upstream makedumpfile (not merged yet but acked by
maintainer)
Signed-off-by: Dave Young <dyoung@redhat.com>
This patch fixes the whitespace errors reported by
'rpmlint' or 'fedpkg lint' when they are run on kexec-tools srpm:
kexec-tools.spec:242: W: mixed-use-of-spaces-and-tabs (spaces: line 107,
tab: line 242)
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We met a problem that the kdump emergency service failed to
start when the target dump timeout(we passed "rd.timeout=30"
to kdump), it reported "Transaction is destructive" messages:
[ TIME ] Timed out waiting for device dev-mapper-fedora\x2droot.device.
[DEPEND] Dependency failed for Initrd Root Device.
[ SKIP ] Ordering cycle found, skipping System Initialization
[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[ SKIP ] Ordering cycle found, skipping System Initialization
[ SKIP ] Ordering cycle found, skipping Initrd Default Target
[DEPEND] Dependency failed for File System Check on /dev/mapper/fedora-root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Stopped dracut pre-udev hook.
[ OK ] Stopped dracut cmdline hook.
Starting Setup Virtual Console...
Starting Kdump Emergency...
[ OK ] Reached target Initrd Default Target.
[ OK ] Stopped dracut initqueue hook.
Failed to start kdump-error-handler.service: Transaction is destructive.
See system logs and 'systemctl status kdump-error-handler.service' for details.
[FAILED] Failed to start Kdump Emergency.
See 'systemctl status emergency.service' for details.
[DEPEND] Dependency failed for Emergency Mode.
This is because in case of root failure, initrd-root-fs.target
will trigger systemd emergency target which requires the systemd
emergency service actually is kdump-emergency.service, then our
kdump-emergency.service starts kdump-error-handler.service with
"systemctl isolate"(see 99kdumpbase/kdump-emergency.service, we
replace systemd's with this one under kdump).
This will lead to systemd two contradictable jobs queued as an
atomic transaction:
job 1) the emergency service gets started by initrd-root-fs.target
job 2) the emergency service gets stopped due to "systemctl isolate"
thereby throwing "Transaction is destructive".
In order to solve it, we can utilize "IgnoreOnIsolate=yes" for both
kdump-emergency.service and kdump-emergency.target. Unit with attribute
"IgnoreOnIsolate=yes" won't be stopped when isolating another unit,
they can keep going as expected in case be triggered by any failure.
We add kdump-emergency.target dedicated to kdump the similar way
as did for kdump-emergency.service(i.e. will replace systemd's
emergency.target with kdump-emergency.target under kdump), and
adds "IgnoreOnIsolate=yes" into both of them.
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
[bhe: improve the patch log about IgnoreOnIsolate="]
Patches have been taken from kexec-tools and makedumpfile to fix issue
with `makedumpfile --mem-usage /proc/kcore`.
Two of the patches is from kexec-tools and rest are from makedumpfile.
All the patches have been acked upstream and applies without conflict.
Kexec-tools patches:
(kexec-tools-2.0.14-x86-x86_64-Fix-format-warning-with-die.patch), which
fixes koji build issue.
kexec-tools-2.0.14-build_mem_phdrs-check-if-p_paddr-is-invalid.patch fixes
the regresssion caused by kernel /proc/kcore fix to use -1 as default value
of p_paddr for pt_loads. Without his patch kexec -p will fail with latest
kernel.
Other makedumpfile patches are backported to support --mem-usage while
kernel kaslr being enabled. Details please see the patch log of the individual
patches.
All the patches are backport of upstream commits.
Patches has been tested with kernel 4.11.0-0.rc1.git0.1.fc26.x86_64.
# makedumpfile --mem-usage /proc/kcore -f
The kernel version is not supported.
The makedumpfile operation may be incomplete.
TYPE PAGES EXCLUDABLE DESCRIPTION
----------------------------------------------------------------------
ZERO 1960 yes Pages filled
with zero
NON_PRI_CACHE 22850 yes Cache pages
without private flag
PRI_CACHE 1517 yes Cache pages with
private flag
USER 32522 yes User process
pages
FREE 1898981 yes Free pages
KERN_DATA 78721 no Dumpable kernel
data
page size: 4096
Total pages on system: 2036551
Total size on system: 8341712896 Byte
We won't need to pass -f once fedora kernel is rebased with v4.12.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1399436
Since currently crashkernel= will be handled in kdump anaconda addon
we can safely remove rhcrashkernel-param callback.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Backport upstream kexec-tools commit for correct kaslr page_offset value
commit 9f62cbddddfc93d78d9aafbddf3e1208cb242f7b
Author: Thomas Garnier <thgarnie@google.com>
Date: Tue Sep 13 15:10:05 2016 +0800
kexec/arch/i386: Add support for KASLR memory randomization
Multiple changes were made on KASLR (right now in linux-next). One of
them is randomizing the virtual address of the physical mapping, vmalloc
and vmemmap memory sections. It breaks kdump ability to read physical
memory.
This change identifies if KASLR memories randomization is used by
checking if the page_offset_base variable exists. It search for the
correct PAGE_OFFSET value by looking at the loaded memory section and
find the lowest aligned on PUD (the randomization level).
Related commits on linux-next:
- 0483e1fa6e09d4948272680f691dccb1edb9677f: Base for randomization
- 021182e52fe01c1f7b126f97fd6ba048dc4234fd: Enable for PAGE_OFFSET
Signed-off-by: Thomas Garnier <thgarnie@google.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
This is a short document about how to setup kdump on live images. All
steps were tested on Fedora 25 Alpha LiveCD and saved vmcore captured
by kdump to a USB stick successfully.
Signed-off-by: Tong Li <tonli@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
V1 of the KASLR patch which was backported in 2.0.13-6 had wrong
calculation for page_offset. This patch takes the correct difference.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Fix Bug 925630 - kexec-tools: support for arm64
https://bugzilla.redhat.com/show_bug.cgi?id=925630
involves three things:
1. back porting upstream code to enable the function of kexec-tools on arm64
patchset backported from upstream:
commit abdfe97736f89d9bc73662b9134604b0229a599e
commit 522df5f7217fda01ece3f6ac3e9987b0320c2bb0
commit 217bcc00c9309416a6c6cd0584196559d28a9259
2. fix the arm related building issue by using autoreconf in spec file
3. patches to fix the issue of higher version gcc used by koji (not upstrem yet,
and the corresponding fix in kernel side is in other package)
kexec-tools-2.0.13-arm64-Add-support-for-additional-relocations-in-the-kexec-purgatory-code.patch
kexec-tools-2.0.13-arm64-Add-support-of-R_AARCH64_PREL32-relocation-in-.patch
[panand: apply patches in 3 of above paragraph]
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Since in Fedora 25 kernel kaslr is enabled (x86) but makedumpfile can not save
a correct vmcore, so it means kdump default setup will not work.
Pratyush posted a patch series to upstream which can fix the issue. Let's merge them in F25, will get the normal fixes after it being merged in upstream, we hopefully can rebase soon in rawhide.
This is an urgent fix for F25 since F25 freeze is this week.
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Fedora dracut-network version >= 044-117 must be installed to work with
this set of kexec-tools patches. Therefore adding dependency for them.
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Fedora dracut version >= 044-75 must be installed to work with this set of
kexec-tools patches. Therefore adding dependency for them.
Signed-off-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Use our own code to get system total memory from /proc/meminfo because the
callback to pyanaconda sometimes cause installation hangs.
Signed-off-by: Dave Young <dyoung@redhat.com>
