Currently, if saving vmcore failed, the final failure information won't
be saved to the kexec-dmesg.log, because the action of saving the log
occurs before the final log is printed, it has no chance to save the
log(marked it with the '^^^' below) to the log file(kexec-dmesg.log).
For example:
[1] console log:
[ 3.589967] kdump[453]: saving vmcore-dmesg.txt to /sysroot//var/crash/127.0.0.1-2020-11-26-14:19:17/
[ 3.627261] kdump[458]: saving vmcore-dmesg.txt complete
[ 3.633923] kdump[460]: saving vmcore
[ 3.661020] kdump[465]: saving vmcore failed
^^^^^^^^^^^^^^^^^^^^
[2] kexec-dmesg.log:
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[453]: saving vmcore-dmesg.txt to /sysroot//var/crash/127.0.0.1-2020-11-26-14:19:17/
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[458]: saving vmcore-dmesg.txt complete
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[460]: saving vmcore
Let's improve it in order to avoid the loss of important information.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
commit 44b073b7ec467aee0d7de381d455b8ace1199184
Author: John Ogness <john.ogness@linutronix.de>
Date: Wed Nov 25 10:10:31 2020 +0106
[PATCH 2/2] printk: use committed/finalized state values
* Required for kernel 5.10
The ringbuffer entries use 2 state values (committed and finalized)
rather than a single flag to represent being available for reading.
Copy the definitions and state lookup function directly from the
kernel source and use the new states.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Signed-off-by: Kairui Song <kasong@redhat.com>
Backports:
commit c617ec63339222f3a44d73e36677a9acc8954ccd
Author: John Ogness <john.ogness@linutronix.de>
Date: Thu Nov 19 02:41:21 2020 +0000
[PATCH 1/2] printk: add support for lockless ringbuffer
* Required for kernel 5.10
Linux 5.10 introduces a new lockless ringbuffer. The new ringbuffer
is structured completely different to the previous iterations.
Add support for retrieving the ringbuffer from debug information
and/or using vmcoreinfo. The new ringbuffer is detected based on
the availability of the "prb" symbol.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
systemctl -q --root "$initdir" add-wants X.target X.service is the
recommanded way to add service dependency, and it covers more corner
cases.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently, when generating a kdump initramfs, mkdumprd will determine
how much disk space is available, if the dump target's available space
is not greater than the total system memory, mkdumprd will print a
warning to remind that there might not be enough space to save a vmcore.
Some users are complaining that mkdumprd overestimates the needed size.
But actually, the warning covers extreme scenarios such as the slab
explodes with non-zero data or a full vmcore, etc. Therefore, need to
prevent users from having minimum disk space for crash dump.
In view of this, add some descriptions to clarify it in mkdumprd man
page.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If dracut-initqueue failed in kdump kernel and failure action
is set to dump_to_rootfs, there is no point try again to start the
initqueue. It will also slow down the dump process, and the initqueue
will most like still not work if first attemp failed.
So just try to start sysroot.mount, if it failed, there is no luck.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
The parameter either6 is introduced to dracut by
commit 67354eebbcd4c358b8194ba5fd1ab1cf7dbd42aa
Author: Pingfan Liu <piliu@redhat.com>
Date: Tue Apr 24 16:41:21 2018 +0800
40network: introduce ip=either6 option
But it turns out needless.
On a sensible ipv6 network environment, DHCPv6 can not work properly alone,
because DHCPv6 protocol has no info about the gateway.
An reasonalbe process of ipv6 address set up should look like
host send: Router Solicitation
router reply: Router Advertisements
"Router Advertisements" carries many info like gateway, and if it has
other-config flag set, it carries DNS info etc. As for DHCPv6 address
allocation, it will only start if "Router Advertisements" has the 'managed'
flag set, which directs the host to start a stateful address allocation
from DHCPv6 server.
For more info:
rfc4861: Neighbor Discovery for IP version 6 (IPv6)
rfc5175: IPv6 Router Advertisement Flags Option
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Along with 'on' option, 'fadump=' kernel parameter also supports
'nocma' & 'off' options. Update about these missing options in the
fadump-howto.txt document.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Dracut has switch network-legacy to network-manager by default, which makes
vlan on team easy. So it can be enabled.
Testing network topology with two VMs.
VM1
ens2-\ /----> VLAN8 (192.168.120.50)
---> team0
ens3-/ (192.168.122.10)
VM2
ens2-\ /----> VLAN8 (192.168.120.100)
---> team0
ens3-/ (192.168.122.20)
Both of ens2/ens3 in VM1/VM2 are connected to virbr0.
During test, dump target is set as root@192.168.120.100:/var/crash
then crashing in VM1
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Currently get_bind_mount_source will not work on btrfs, that's because
this function relies on findmnt to detect bind mount.
For a bind mount, findmnt will return different value with "-v" option.
For example, we have /dev/sdc mounted on /mnt/source, and then bind
mount /mnt/source/sub/path to /mnt/bind:
$ findmnt /mnt/bind
TARGET SOURCE FSTYPE OPTIONS
/mnt/bind /dev/sdc[/sub/path] ext4 rw,relatime,seclabel
$ findmnt -v /mnt/bind
TARGET SOURCE FSTYPE OPTIONS
/mnt/bind /dev/sdc ext4 rw,relatime,seclabel
But findmnt also return similiar result for btrfs, on a fresh installed
Fedora 33:
$ findmnt /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/sdb7[/root] btrfs rw,relatime,seclabel,ssd,space_cache,subvolid=256,subvol=/root
$ findmnt -v /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/sdb7 btrfs rw,relatime,seclabel,ssd,space_cache,subvolid=256,subvol=/root
The [...] indicator will contain the subvol of btrfs as well. And if
it's bind mounted under btrfs, it will contain a mixup of btrfs subvol
and the actuall fsroot.
And also, if the bind mount source device is not mounted on /,
get_bind_mount_source will also not work.
So rewrite the get_bind_mount_source function, make it work in every
cases.
Tested with:
- Silverblue's bind mount
- Bind mount with source device mounted not under /
- Btrfs
- Bind mount and source device is Btrfs
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Remove the --real when calling findmnt.
The option is only useful in capture kernel, to avoid
`findmnt` returning the pseudo 'rootfs' for non mounted path.
example, when /kdumproot/mnt/ is not mounted:
kdump:/# findmnt --target /kdumproot/mnt
TARGET SOURCE FSTYPE OPTIONS
/ rootfs rootfs rw,size=61368k,nr_inodes=15342
kdump:/# findmnt --target /kdumproot/mnt
<return 1 and empty output>
But this function will make findmnt also return empty value for bind
mount. So remove it and add an extra if statement for second kernel.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Most watchdogs have a parameter pretimeout, if set to non-zero, it means
before the watchdog really reset the system, it will try to panic the
kernel first, so kdump could kick in, or, just print a panic stacktrace
and then kernel should reset it self.
If we are already in kdump kernel, this is not really helpful, only
increase kernel hanging chance. And it also make thing become complex
as some watchdog triggers the kernel panic in NMI context, which
could also hang the kernel in strange ways, and fail the watchdog it
self. So just disable this parameter.
Also for hpwdt, it have another parameter kdumptimeout, which is
just designed for first kernel. The default behaviour is the watchdog
will simply stop working if timeouted, trigger a panic, and leave the
kernel to kdump. Again, if we are already in kdump this is not helpful.
So also disable that.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Currently the watchdog detection code is broken already, it
get the list of active watchdog drivers, then check if they are
set in the /etc/cmdline.d/* as preload module. But after we
switched to use squash module, /etc/cmdline.d/* is not directly visible.
So just detect whether current needed driver is installed.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
In check_fs_modified, is_nfs_dump_target is already called, the dump
target can't be nfs. No need to check here.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
The driver detection have nothing to do with fs detection, and currently
if the dump target is raw, the block driver detection is skipped which
is wrong. Just split it out and run the block driver detection when dump
target is fs or raw.
Also simplfied the code a bit.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
- ssh-copy-id is bugged and not working, use a more robust way to sync
ssh keys
- systemd-resolvd will bind on port 53 so DHCP server won't work,
disable systemd-resolvd's builtin DNS server
Signed-off-by: Kairui Song <kasong@redhat.com>
Let's remove some redundant descriptions in the usage documentation
of the logger, and make it clear.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Some unused log levels have been removed, and kdump has used the
different options to control the log levels for the first kernel
and the second kernel. Therefore, let's update the kdump sysconfig
accordingly.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In the /etc/sysconfig/kdump, we usually use the uppercase configuration
name for all options. So let's use the same method to handle this.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's add the rd.kdumploglvl option to control log level in the second
kernel, which can make us avoid rebuilding the kdump initramfs after we
change the log level in /etc/sysconfig/kdump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The kdump-logger will be used by the system service(daemons), so let's
appropriately convert the logger numeric level to syslog level with the
facility(daemon). The number is constructed by multiplying the facility
by 8 and then adding the level.
About The Syslog Protocol, please refer to the RFC5424 for more details.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Previously, the range of log level is from 1 to 6, and the TRACE
level and FATAL level are not used, therefore, let's remove these
unused log levels.
Now it has only four log levels: error(1), warn(2), info(3)
and debug(4). We have to remap the numeric log level to the logger
priority or syslog log level, which is finished in kdump-logger.sh
module, it is invisible for user.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's add sanity checks for the log levels in order to avoid
passing illegal log levels to the logger.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
depend() in module-setup.sh is a better place to setup dracut module
dependency, it will do early check, and fail early if needed module is
missing. Also remove a unneeded helper add_dracut_module.
Also remove the unnecessary return in depend() function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Let's add some code comments to help better understanding, and
no code changes.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When using ssh dump target, scp is always used, correct the comment.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut only check if a module failed installtion if the module is listed
in --add params. Without this param, if kdumpbase failed to install due
to any reason, dracut will still build the initramfs only print a
warning. Add this param to ensure it fail early.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The dracut module is opportunistic about using the built-in squashfs
support only when available, but the spec file hard requires it. Demote
it to a weak dep to truly make it optional.
This caters to environments which strive to stay minimal, like FCOS and
RHCOS. See https://github.com/coreos/fedora-coreos-config/pull/708 for
details.
Because otherwise, `kdumpctl start` will fail anyway. This makes it
easier to enable kdump by simply adding the mandatory karg and leaving
the service enabled.
This reverts commit fa8aa52d94.
For the s390x, the vmlinuz image has only single signature according
to the kernel.spec. The dual signature issue doesn't happens on s390x,
therefore, let's restore it in order to enable the file load on s390x
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, the makedumpfile option '--message-level' is set to 1 when
dumping the vmcore, it only displays the progress indicator message,
but there are no common message and error message, it is important to
report some additional messages, especially for the error message,
which is very useful for the debugging.
In view of this, let's change the message level to 7 by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Commit 08276e9 wrongly raise this warning message to error level, fix
this.
Fixes: 08276e9 ('Rework check_config and warn on any duplicated option')
Signed-off-by: Kairui Song <kasong@redhat.com>
Previously journalctl logs are directly dropped to save memory, but this
make journalctl unusable in kdump kernel and diffcult to debug. So
instead just don't let it read kmsg but keep other logs stored as volatile.
Kernel message are already stored in the kernel log ring buffer,
no need to let journalctl make a copy, especially when in kdump
kernel, ususlly there won't be too much kernel log overlapping
the old ring buffer.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Because the logger is introduced to output the kdump logs, need to
add a documentation for this change and describe how to use it.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's use the logger in the second kernel and collect the kernel ring
buffer(dmesg) of the second kernel.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
