Commit Graph

129 Commits

Author SHA1 Message Date
Pingfan Liu
680c0d3414 kdumpctl: distinguish the failed reason of ssh
On a host with ipaddr not ready before kdump service, ssh return errno 255.
While if no ssh-key, ssh also return errno 255. For both of cases, the
current kdump code promote user to run 'kdumpctl propagate'. This confuses
user who already installs ssh-key.

In order to tell these two cases from each other, the ssh warning message
should be involved, and parsed.

For the no ssh-key case , warning message is "Permission denied" or "No
such file or directory". For the other, warning message is "Network
Unreachable"

This patch also does a slight change to enlarge the timeout from 60s to
180s. This value can meet test at the time being

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2019-09-02 17:06:21 +08:00
Pingfan Liu
c1a06343df kdumpctl: wait a while for network ready if dump target is ssh
If dump target is ipv6 address, a host should have ipv6 address ready
before starting kdump service. Otherwise, kdump service fails to start due
to the failure "ssh dump_server_ip mkdir -p $SAVE_PATH".
And user can see message like:
 "Could not create root@2620:52:0:10da:46a8:42ff:fe23:3272/var/crash"

I observe a long period (about 30s) on some machine before they got ipv6
address dynamiclly, which is never seen on ipv4 host.

Hence kdump service has a dependency on ipv6 address. But there is no good
way to resolve it. One way is asking user to run the cmd "nmcli connection
modify eth0 ipv6.may-fail false". But this will block systemd until ipv6
address is ready. Despite doing so, kdump can try its best (wait 1 minutes
after it starts up) before failure.

How to implement the wait is arguable. It will involve too many technique
details if explicitly waiting on ipv6 address, instead, just lean on 'ssh'
return value to see the availability of network.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2019-08-12 16:13:08 +08:00
Kairui Song
f0fa5c8e91 kdumpctl: check for ssh path availability when rebuild
Currently kdumpctl rebuild will simply rebuild the initramfs, and
only perform basic config syntax check. But it should also check if the
target path is available when using SSH target, else kdump may fail.
is second kernel. kdumpctl rebuild should cover this case, and create
the path if it doesn't exist.

This patch make rebuild and restart behaves the same, rebuild is
now equal to restart, except it won't check config change or reload
kdump resource.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-27 16:13:29 +08:00
Kairui Song
43c26b7312 kdumpctl: Check kdump.conf for error when rebuild is called
Although "kdumpctl rebuild" is introduced to help user rebuild the
initramfs without modifying the kdump.conf, if the kdump.conf is
modified and "kdumpctl rebuild" is called, a initramfs with a faulty
kdump.conf will be built.

Kdump will refuse to load the initramfs when restarted, but kdumpctl
reload may load the faulty initramfs. So need to make sure the faulty
build won't be generate in the first place.

Check for kdump.conf error before building the initramfs to ensure such
failure won't happen.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-27 13:57:55 +08:00
Kairui Song
2efc0f1854 kdumpctl: don't always rebuild when extra_modules is set
We don't necessarily have to always rebuild the initramfs when
extra_modules is set. Instead, just detect if any module is updated,
and only rebuild initramfs if found any updated kernel module.

Tested with in-tree kernel modules, out-of-tree kernel modules, weak
modules, all worked as expected.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-20 17:01:25 +08:00
Kairui Song
30913fd667 kdumpctl: follow symlink when checking for modified files
Previously only the symlink's timestamp is used for checking if file are
modified, this will not trigger a rebuild if the symlink target it
modified.

So check both symlink timestamp and symlink target timestamp, rebuild
the initramfs on both symlink changed and target changed.

Also give a proper error message if the file doesn't exist.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-20 16:56:31 +08:00
Kairui Song
75d9132417 Get rid of duplicated strip_comments when reading config
When reading kdump configs, a single parsing should be enough and this
saves a lot of duplicated striping call which speed up the total load
speed.

Speed up about 2 second when building and 0.1 second for reload in my
tests.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-20 16:56:28 +08:00
Lianbo Jiang
9529191d95 earlykdump: provide a prompt message after the rebuilding of kdump initramfs.
Early kdump inherits the settings of normal kdump, so any changes that
caused normal kdump rebuilding also require rebuilding the system initramfs
to make sure that the changes take effect for early kdump.

Therefore, when the early kdump is enabled, provide a prompt message after
the rebuilding of kdump initramfs is completed.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-20 16:56:19 +08:00
Kairui Song
1c1159a586 kdumpctl: Detect block device driver change for initramfs rebuild
Previous we rebuild the initramfs when kenrel load module list changed,
but this is not very stable as some async services may load/unload
kernel modules, and cause unnecessary initramfs rebuild.

Instead, it's better to just check if the module required to dump to
the dump target is loaded or not, and rebuild if not loaded. This
avoids most false-positives, and ensure local target change is always
covered.

Currently only local fs dump target is covered, because this check
requires the dump target to be mounted when building the initramfs,
this guarantee that the module is in the loaded kernel module list,
else we may still get some false positive.

dracut-install could be leveraged to combine the modalias list with
kernel loaded module list as a more stable module list in the initramfs,
but upstream dracut change need to be done first.

Passed test on a KVM VM, changing the storage between SATA/USB/VirtIO
will trigger initramfs rebuild and didn't notice any false-positive.
Also passed test on my laptop with no false-positive.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-08 17:51:18 +08:00
Kairui Song
09f50350d9 Revert "kdumpctl: Rebuild initramfs if loaded kernel modules changed"
This reverts commit 6b479b6572.

Check initramfs rebuild by looking at if there is any change of load
kernel modules list is not very stable after all. Previously we are
counting on udev to settle before kdump is started to ensure all modules
is ready, but actually any service may cause a kernel module load, even
after udev is settled.

The previous commit is trying to workaround an issue that VM created
with disk snapshot may fail in the kdump initramfs. The better fix is to
not include the kdump initramfs in the disk snapshot at all, as the
kdump initramfs is not generated for a generic use. And With new added
"kdumpctl reload" command, admins could rebuild the image easily, and
should rebuild the initramfs on hardware change manually.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-05-06 17:54:26 +08:00
Kairui Song
594ac119c5 kdumpctl: add rebuild support
Use "kdumpctl rebuild" to rebuild the image directly. This could help
admins to rebuild kdump image directly.

Also merge fadump related initramfs backup/restore into setup_initrd,
and do permission only when actually trying to rebuild the image.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-04-05 02:02:43 +08:00
Hari Bathini
689fca5af3 fadump: leverage kernel support to re-regisgter FADump
With kernel commit 0823c68b054b ("powerpc/fadump: re-register firmware-
assisted dump if already registered") support is enabled to re-register
when FADump is alredy registered. Leverage that option in kdump scripts.

Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
2019-03-07 13:33:17 +08:00
Hari Bathini
da6b75f59b fadump: use the original initrd to rebuild fadump initrdfrom
The idea behind adding support for dracut '--rebuild' option was to
ensure the initrd built for fadump takes into consideration all the
build parameters passed to original initrd. Pass original initrd
instead of current default initrd for rebuild as current initrd
might already have build parameters from original initrd along
with parameters from previous fadump intird build making the
build parameters look like this after a few iterations:

-H --persistent-policy 'by-uuid' -f --quiet --hostonly --hostonly-
cmdline --hostonly-i18n --hostonly-mode 'strict' -o 'plymouth dash
resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home /kdumproot
//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --quiet
--hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict'
-o 'plymouth dash resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home
/kdumproot//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --quiet
--hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict'
-o 'plymouth dash resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home
/kdumproot//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs'
--

Since it is not desirable to build initrd with stale and/or  duplicate
build parameters, use original initrd (backed up) to rebuild fadump
initrd, instead of current default initrd.

Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
2019-03-07 13:32:47 +08:00
Kazuhito Hagio
242da37c58 Add final_action option to kdump.conf
If a crash occurs repeatedly after enabling kdump, the system goes
into a crash loop and the dump target may get filled up by vmcores.
This is likely especially with early kdump.

This patch introduces 'final_action' option to kdump.conf, in order
for users to be able to power off the system even after capturing
a vmcore successfully.

Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
2019-01-22 17:58:24 +08:00
Kazuhito Hagio
cc95f0a744 Add failure_action as alias of default and make default obsolete
In preparation for adding 'final_action' option, since it's confusing
to have the 'final_action' and 'default' options at the same time,
this patch introduces 'failure_action' as an alias of the 'default'
option to /etc/kdump.conf, and makes 'default' obsolete to be removed
in the future.

Also, the "default action" term is renamed to "failure action".

Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
2019-01-22 17:57:53 +08:00
Kairui Song
32fc6070a6 Add missing usage info
In commit b34ce3a reload support was added to kdumpctl but the usage
info is not updated. Now add reload to usage output to let user aware
of the new command.

Signed-off-by: Kairui Song <kasong@redhat.com>
2018-11-09 11:17:00 +08:00
Kairui Song
b34ce3a7b4 kdumpctl: Add reload support
Add reload support to kdumpctl, reload will simply unload current
loaded kexec crash kernel and initramfs, and load it again.

Changes in /etc/sysconfig/kdump will take effect with kdumpctl
reload, but reloading will not check the content of
/etc/kdump.conf and won't rebuild anything. reload is fast, the only
time-consuming part of kdumpctl reload is loading kernel and initramfs
with kexec which is always necessary.

Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2018-11-01 22:31:20 +08:00
Kenneth Dsouza
5b385cbd0c kdumpctl: Print warning in case the raw device is formatted and contains filesystem.
Currently the kdumpctl script doesn't check if the raw device is
formatted which might destroy existing data at the time of dump
capture.

This patch addresses this issue, by ensuring kdumpctl prints
a warning in case it finds the raw device to be formatted.

Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2018-10-15 10:47:08 +08:00
Kenneth Dsouza
d92b9364ae kdumpctl: Error out if path is set more than once.
Currently the kdumpctl script doesn't check if the path option is
set more than once due to which a vmcore is not captured.

This patch addresses this issue by ensuring that only one path
is specified in /etc/kdump.conf file.

Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
2018-08-22 15:23:32 +08:00
Kairui Song
6b479b6572 kdumpctl: Rebuild initramfs if loaded kernel modules changed
Currently, we only rebuilt kdump initramfs on config file change,
fs change, or watchdog related change. This will not cover the case
that hardware changed but fs layout and other configurations still
stays the same, and kdump may fail.

To cover such case, we can detect and compare loaded kernel modules,
if a hardware change requires the image to be rebuilt, loaded kernel
modules must have changed.

Starting from commit 7047294 dracut will record loaded kernel modules
when the image is built if hostonly mode is enabled.  With this patch,
kdumpctl will compare the recorded value with currently loaded kernel
modules, and rebuild the image on change.

"kdumpctl start" will be a bit slower, as we have to call lsinitrd one
more time to get the loaded kernel modules list. I measure the time
consumption and we have an overall 0.2s increased loading time.

Time consumption of command "kdumpctl restart":

Before:
real    0m0.587s
user    0m0.481s
sys     0m0.102s

After:
real    0m0.731s
user    0m0.591s
sys     0m0.133s

Time comsumption of command "kdumpctl restart" with image rebuild:

Before (force rebuild):
real    0m10.972s
user    0m8.966s
sys     0m1.318s

After (inserted ~100 new modules):
real    0m11.220s
user    0m9.387s
sys     0m1.337s

Signed-off-by: Kairui Song <kasong@redhat.com>
2018-07-26 19:25:09 +08:00
Lianbo Jiang
b1fbeebd08 move some common functions from kdumpctl to kdump-lib.sh
we move some common functions from kdumpctl to kdump-lib.sh, the
functions could be used in other modules, such as early kdump.
It has no bad effect.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Reviewed-by: Kazuhito Hagio <khagio@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2018-05-29 10:18:40 +08:00
Dave Young
3578c54ff2 Fix kdumpctl showmem
showmem function mistakenly added some noise character before the
real code, it could be some copy-paste error.

Fixes: 1a6cb43a19
2018-05-24 13:27:02 +08:00
Bhupesh Sharma
5221d4b90c kdumpctl: Remove 'netroot' and 'iscsi initiator' entries from kdump cmdline
In a iSCSI multipath environment (which uses iSCSI software initiator
and target environment) when the vmcore file is saved on the target,
kdump always fails to establish a iSCSI session and also fails to
collect dump due to duplicate entries for 'netroot' and
'iscsi initiator' in the kdump bootargs:

   # echo c > /proc/sysrq-trigger

   [83471.842707] SysRq : Trigger a crash
   [83471.843233] BUG: unable to handle kernel NULL pointer dereference at           (null)
   [83471.844155] IP: [<ffffffffac82ed16>] sysrq_handle_crash+0x16/0x20
   [83471.844931] PGD 800000023f710067 PUD 229fd6067 PMD 0
   [83471.845655] Oops: 0002 [#1] SMP

   <snip..>

   [83471.861889] Call Trace:
   [83471.862162]  [<ffffffffac82f53d>] __handle_sysrq+0x10d/0x170
   [83471.862771]  [<ffffffffac82f9af>] write_sysrq_trigger+0x2f/0x40
   [83471.863405]  [<ffffffffac690630>] proc_reg_write+0x40/0x80
   [83471.863984]  [<ffffffffac61acd0>] vfs_write+0xc0/0x1f0
   [83471.864536]  [<ffffffffac61baff>] SyS_write+0x7f/0xf0
   [83471.865075]  [<ffffffffacb1f7d5>] system_call_fastpath+0x1c/0x21
   [83471.865714] Code: eb 9b 45 01 f4 45 39 65 34 75 e5 4c 89 ef e8 e2 f7 ff ff eb db 0f 1f 44 00 00 55 48 89 e5 c7 05 41 47 81 00 01 00 00 00 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 0f 1f 44 00 00 55 31 c0 c7 05 be
   [83471.868888] RIP  [<ffffffffac82ed16>] sysrq_handle_crash+0x16/0x20
   [83471.869700]  RSP <ffff9e7fe77b7e58>
   [83471.870074] CR2: 0000000000000000

   <snip..>

            Starting Login iSCSI Target iqn.2014-08.com.example:t1...
   [  OK  ] Stopped Login iSCSI Target iqn.2014-08.com.example:t1.
            Starting Login iSCSI Target iqn.2014-08.com.example:t1...
   [    6.607051] scsi host2: iSCSI Initiator over TCP/IP
   [FAILED] Failed to start Login iSCSI Target iqn.2014-08.com.example:t1.
   See 'systemctl status "iscsistart_\\x40...com.example:t1.service"' for details.
   [  126.572911] dracut-initqueue[243]: Warning: dracut-initqueue timeout - starting timeout scripts
            Stopping Open-iSCSI...
   [  OK  ] Stopped Open-iSCSI.
            Starting Open-iSCSI...
   [  OK  ] Started Open-iSCSI.
            Starting Login iSCSI Target iqn.2014-08.com.example:t1...
   [  OK  ] Stopped Login iSCSI Target iqn.2014-08.com.example:t1.
            Starting Login iSCSI Target iqn.2014-08.com.example:t1...
   [  131.095897] scsi host3: iSCSI Initiator over TCP/IP
   [FAILED] Failed to start Login iSCSI Target iqn.2014-08.com.example:t1.
   See 'systemctl status "iscsistart_\\x40...com.example:t1.service"' for details.
   [  251.085029] dracut-initqueue[243]: Warning: dracut-initqueue timeout - starting timeout scripts
   [  251.594554] dracut-initqueue[243]: Warning: dracut-initqueue timeout - starting timeout scripts

   <snip..>

This patch fixes the same by removing the 'netroot',
'rd.iscsi.initiator' and 'iscsi_initiator' entries from the kdump boot
cmdline.

One reason why this is safe is our kdump target setup does not
depend on 1st kernel inherited cmdline params now since the work
we dropped root dependency.

Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2018-05-21 14:09:17 +08:00
Pingfan Liu
1a6cb43a19 kdumpctl: add showmem cmd
port from rhel, original patch is contributed by Minfei Huang:

Using /sys to determines crashkernel actual size is confusing since
there is no unit of measure.

Add a new command "kdumpctl showmem" to show the reserved memory kindly.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Signed-off-by: Minfei Huang <mhuang@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2018-05-21 14:06:30 +08:00
Lianbo Jiang
dbe8214586 kdumpctl: Check the modification time of core_collector
When core_collector is changed, the kdump initramfs needs to
be rebuilt before it is loaded.

Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2018-03-22 16:10:58 +08:00
Pingfan Liu
cde5944f93 kdumpctl: skip selinux-relabel for dracut_args --mount dump target
When using "dracut_args --mount" to specify dump target, e.g. nfs like:
    path /
    core_collector makedumpfile -d 31
    dracut_args --mount "host:/path /var/crash nfs defaults"
kdump service should neither guarantees the correctness, nor relabels it.

For current code, since dracut_args dump targets are likely not mounted
so kdump service mistakenly relabel the rootfs, which is meanless and
takes very long time.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
2017-12-04 12:51:15 +08:00
Baoquan He
85156bfc66 Revert "kdumpctl: sanity check of nr_cpus for x86_64 in case running out of vectors"
This reverts commit 2040103bd7.

Reason is it's based on the environment of 1st kernel where all
present devices could be active and initialized during bootup.
Then all pci devices will request irqs. While kdump only brings
up those devices which are necessary for vmcore dumping. So this
commit is not meaningful and helpless to very large extent. And
it will print out 'Warning' when calculated result is larger than
1 cpu, actually it's a false positive report most of the time.

So revert the commit, and can check the git history for later
reference.

[dyoung]: on some machine this warning message shows up but
later we found the irq numbers with and without nr_cpus=1 is
quite different so this need more investigation since
the formula is not accurate.

Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-12-04 12:50:50 +08:00
Bhupesh Sharma
cb3d1c1c3f kdumpctl: Error out in case there are white spaces before an option name
Resolves: BZ1484945
https://bugzilla.redhat.com/show_bug.cgi?id=1484945

Currently the kdumpctl script doesn't handle
whitespaces (including TABs) which might be there before
an option name in the kdump.conf

This patch addresses this issue, by ensuring that the
kdumpctl errors out in case it finds any stray space(s)
or tab(s) before a option name.

Reported-by: Kenneth D'souza <kdsouza@redhat.com>
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-10-11 09:57:31 +08:00
Hari Bathini
601766a3d9 fadump: rebuild default initrd with dump capture capability
As default initrd is used for booting fadump capture kernel, it must be
rebuilt with dump capture capability when dump mode is fadump. Check if
default initrd is already fadump capable and rebuild, if necessary.

Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
2017-09-06 15:42:13 +08:00
Xunlei Pang
2c9a863fd3 kdumpctl: remove some cmdline inheritage from 1st kernel
Now with the help of "--hostonly-cmdline", dracut will generate
the needed cmdlines for the dump target, so we can avoid the
corresponding duplicate or unnecessary inheritage.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-09-06 15:40:15 +08:00
Xunlei Pang
31dc60ad20 Change dump_to_rootfs to use "--mount" instead of "root=X"
Currently, we kept "root=X" for the dump_to_rootfs case, this
patch consolidates to use "--mount" for all the kdump mounts.

One advantage of this way is that dracut can correctly mark root
(in case of dump_to_rootfs is specified) as the host device when
"--no-hostonly-default-device" is added in the following patch.

Changed the code style in passing, as shellcheck tool reported:
Use $(..) instead of deprecated `..`

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-09-06 15:39:41 +08:00
Xunlei Pang
d5fe9022d0 kdumpctl: move is_fadump_capable() to kdump-lib.sh
Make is_fadump_capable() a library function, as we will need
it in mkdumprd.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-09-06 15:39:23 +08:00
Xunlei Pang
1bd757bc96 Revert "kdumpctl: use generated rd.lvm.lv=X"
This reverts commit cb38b32dfc.

We are going to add "--hostonly-cmdline" dracut argument in
the following patch.

With the help of "--hostonly-cmdline", dracut will generate
"rd.lvm.lv=X" for us, no need to implement here again.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-09-06 15:38:39 +08:00
Xunlei Pang
8250f23c10 Revert "mkdumprd: omit crypt when there is no crypt kdump target"
This reverts commit 54a5bcc4ee.

We are going to add "--no-hostonly-default-device" dracut argument
in the following patch.

With the help of "--no-hostonly-default-device", dracut only
adds the dump target as host devices, which naturally guarantees
only required dracut modules being selected.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-09-06 15:38:09 +08:00
Xunlei Pang
9df0cbbeed kdumpctl: use "apicid" other than "initial apicid"
We met a problem on AMD machines, when using "nr_cpus=4" for
kdump, and crash happens on cpus other than cpu0, kdump kernel
will fail to boot and eventually reset.

After some debugging, we found that it stuck at the kernel path
do_boot_cpu()-> ... ->wakeup_secondary_cpu_via_init():
 apic_icr_write(APIC_INT_LEVELTRIG|APIC_INT_ASSERT|APIC_DM_INIT,
            phys_apicid);
that is, it stuck at sending INIT from AP to BP and reset, which
is actually what "disable_cpu_apicid=X" tries to solve. Printing
the value of @phys_apicid showed that it was the value of "apicid"
other that of "initial apicid" showed by /proc/cpuinfo.

As described in x86 specification:
"In MP systems, the local APIC ID is also used as a processor ID by the
BIOS and the operating system. Some processors permit software to modify
the APIC ID. However, the ability of software to modify the APIC ID is
processor model specific. Because of this, operating system software
should avoid writing to the local APIC ID register. The value returned by
bits 31-24 of the EBX register (when the CPUID instruction is executed with a
source operand value of 1 in the EAX register) is always the Initial APIC ID
(determined by the platform initialization). This is true even if software
has changed the value in the Local APIC ID register."

From kernel commit 151e0c7de("x86, apic, kexec: Add disable_cpu_apicid
kernel parameter"), we can see in generic_processor_info(), it uses
a)read_apic_id() and b)@apicid to compare with @disabled_cpu_apicid.

a)@apicid which is actually @phys_apicid above-mentioned is from the
  following calltrace(on the problematic AMD machine):
    generic_processor_info+0x37/0x300
    acpi_register_lapic+0x30/0x90
    acpi_parse_lapic+0x40/0x50
    acpi_table_parse_entries_array+0x171/0x1de
    acpi_boot_init+0xed/0x50f
  The value of @apicid(from acpi MADT) is equal to the value of "apicid"
  showed by /proc/cpuinfo as proved by our debug printk.
b)read_apic_id() gets the value from LAPIC ID register which is "apicid"
  as well.

While the value of "initial apicid" is from cpuid instruction.

One example of "apicid" and "initial apicid" of cpu0 from /proc/cpuinfo
on AMD machine:
  apicid          : 32
  initial apicid  : 0

Therefore, we should assign /proc/cpuifo "apicid" to "disable_cpu_apicid=X".

We've never met such issue before, because we usually tested "nr_cpus=1",
and mostly on Intel machines, and "apicid" and "initial apicid" have the
same value in most cases on Intel machines.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-07-19 09:50:04 +08:00
Xunlei Pang
54a5bcc4ee mkdumprd: omit crypt when there is no crypt kdump target
Resolves: bz1451717
https://bugzilla.redhat.com/1451717

When there is no crypt related kdump target, we can safely
omit "crypt" dracut module, this can avoid the pop asking
disk password during kdump boot in some cases.

This patch introduces omit_dracut_modules() before calling
dracut, we can omit more modules to reduce initrd size in
the future.

We don't want to omit any module for fadump, thus we move
is_fadump_capable() into kdump-lib.sh as a helper to use.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-07-14 14:54:31 +08:00
Xunlei Pang
cb38b32dfc kdumpctl: use generated rd.lvm.lv=X
Resolves: bz1451717
https://bugzilla.redhat.com/1451717

When there is any "rd.lvm.lv=X", "lvm" dracut module will
try to recognize all the lvm volumes which is unnecessary
and probably cause trouble for us.

See https://bugzilla.redhat.com/show_bug.cgi?id=1451717#c2

Remove all the rd.lvm.lv=X inherited from the kernel cmdline,
and generate the corresponding cmdline as needed for kdump.
Because prepare_cmdline() is only used by kdump, we don't need
to add any fadump judgement(also remove the existing judgement
in passing).

Currently, we don't handle "rd.lvm.vg=X", we can add it in
when there is some bug reported in the future.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-07-14 14:54:23 +08:00
Xunlei Pang
3ee00cd384 kdump-lib.sh: introduce get_kdump_targets()
Resolves: bz1451717
https://bugzilla.redhat.com/1451717

We need to know all the kdump targets including the dump
target and root in case of "dump_to_rootfs".

This is useful for us to do some extra work related to the
type of different targets.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-07-14 14:54:06 +08:00
Xunlei Pang
1bc78e025f kdump-lib.sh: fix improper get_block_dump_target()
Resolves: bz1451717
https://bugzilla.redhat.com/1451717

This patch improves get_block_dump_target as follows:
-Consider block device in the special "--dracut-args --mount ..."
 in get_user_configured_dump_disk().
-Consider save path instead of root fs in get_block_dump_target(),
 and move it into kdump-lib.sh because we will have another user
 in the following patch.
-For nfs/ssh dumping, there is no need to check the root device.
-Move get_save_path into kdump-lib.sh.

After this patch, get_block_dump_target() can always return the
correct block dump target specified.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-07-14 14:53:43 +08:00
Ziyue Yang
0933f89f65 kdumpctl: fix infinite loop caused by running under bash
Description of problem
(https://bugzilla.redhat.com/show_bug.cgi?id=1465735):
Run `kdumpctl status` as normal user, get below error messages:

Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
...

The bug is caused by behavior difference between bash
and sh (bash in posix).

In the function single_instance_lock in kdumpctl script,
there is

exec 9>/var/lock/kdump

which will fail in user mode. However, this fail will cause
script exiting under bash but not exiting under sh, causing
infinite loop because the flock will always fail.

According to the 16th item in
ftp://ftp.gnu.org/old-gnu/Manuals/bash-2.02/html_node/bashref_66.html

If a POSIX.2 special builtin returns an error status, a non-
interactive shell exits.

And according to
https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html

exec is one of the POSIX.2 special builtin's.

This patch fixes the bug by checking exec return value.

Fixes: 9fb2996d05 ("kdumpctl: change the shebang header to use /bin/bash")
Signed-off-by: Ziyue Yang <ziyang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
2017-07-14 14:47:37 +08:00
Pingfan Liu
1ca4c9f009 kdumpctl: for fence_kdump, the ipaddr of this node should be excluded from list
kdump should not send fence_kdump notifications to local host, because
the role of the falied node (i.e local host) is to send fence_kdump
notifications to other nodes to tell them I'm kdumping, tell to itself is
nonsense. And we have excluded hostname of local host but when one use ip
address we also need exclude it.

Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-18 16:44:42 +08:00
Xunlei Pang
9fb2996d05 kdumpctl: change the shebang header to use /bin/bash
We met one issue that when changing softlink of "/usr/bin/sh"
to point to "ksh" instead of the default "bash", kdumpctl will
not work and go wrong.

kdumpctl is expected to run under bash like dracut, we should
change its shebang header from "#!/bin/sh" to "#!/bin/bash".

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-12 09:58:51 +08:00
Xunlei Pang
2b4b7a6374 kdumpctl: call strip_comments only when necessary to speedup
The "time kdumpctl start" command shows that strip_comments()
consumes lots of cpu time. By only calling it when necessary,
it saves us nearly half second.

Tested on my Fedora kvm machine.
Before this patch:
$ time kdumpctl start
kexec: loaded kdump kernel
Starting kdump: [OK]

real	0m1.849s
user	0m1.497s
sys	0m0.462s

After this patch:
$ time kdumpctl start
kexec: loaded kdump kernel
Starting kdump: [OK]

real	0m1.344s
user	0m1.195s
sys	0m0.195s

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-12 09:57:29 +08:00
Xunlei Pang
7f725ef13a Revert "kdumpctl: improve "while read" time for /etc/kdump.conf"
Resolves: bz1449801

"cat $KDUMP_CONFIG_FILE|grep -v "^#"|while read ..." use pipes
to invoke subshells, as a result we met "the dreaded inaccessible
variables within a subshell problem" as described in the book
"Advanced Bash-Scripting Guide".

It cause regressions, so revert it.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-12 09:57:29 +08:00
Xunlei Pang
38c0b4aa3a kdumpctl: improve "while read" time for /etc/kdump.conf
I found using "cat $KDUMP_CONFIG_FILE|grep -v "^#"|while read ..."
instead of "while read ... do ...; done < $KDUMP_CONFIG_FILE" will
make the script run faster, it saves us nearly half second.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-05 16:14:33 +08:00
Xunlei Pang
88a3385e96 kdumpctl: update check_dump_fs_modified() to use "lsinitrd -f"
We use faster "lsinitrd XXX -f usr/lib/dracut/build-parameter.txt"
instead of "lsinitrd XXX | grep "^Arguments:" | head -1", this can
save us around one second.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-05 16:14:33 +08:00
Xunlei Pang
2a5f362521 kdumpctl: improve check_wdt_modified()
Use the logic of dracut 04watchdog/module-setup.sh to check,
then we only need to compare the content of 00-watchdog.conf,
so we can save one operation of lsinitrd.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-05 16:14:33 +08:00
Xunlei Pang
c63c0a1084 kdumpctl: remove is_mode_switched()
handle_mode_switch() can ensure the correct logic, so remove
the needless is_mode_switched(). This helps to save one slow
lsinitrd operation for each boot.

Improved backup_default_initrd() to judge DEFAULT_INITRD.

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-05 16:14:33 +08:00
Xunlei Pang
071ea2a277 kdumpctl: bail out earlier in case of no reserved memory
Some cloud people complained that VM boot speed is slower than
Ubuntu and other distributions, after some debugging, we found
that one of the causes is kdump service starts too slow(7seconds
according to the test result on the test VM), actually there is
no "crashkernel=X" specified. Although kdump service is parallel,
it affects the speed more or less especially on VMs with few cpus,
which is unacceptable. It is even worse when kdump initramfs is
built out in case of no reserved memory at first boot.

Commit afa4a35d3 ("kdumpctrl: kdump feasibility should fail if no
crash memory") can actually solve this issue.

This patch is a supplement of above-mentioned commit, we bail out
start() even earlier in case of no reserved memory.

Also made some cosmatic changes for check_crash_mem_reserved().

1) Before this patch
$ time kdumpctl start
No memory reserved for crash kernel.
Starting kdump: [FAILED]

real    0m0.282s
user    0m0.184s
sys     0m0.146s

2) After this patch
$ time kdumpctl start
No memory reserved for crash kernel
Starting kdump: [FAILED]

real    0m0.010s
user    0m0.008s
sys     0m0.001s

Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-05-02 17:23:57 +08:00
Bhupesh Sharma
a284fa9005 kdump: Introduce 'force_no_rebuild' option
This patch introduces the 'force_no_rebuild' option
inside the 'kdump.conf' and its handling inside the 'kdumpctl'
script.

There might be several use cases, where a system admin
decides that he doesn't need to rebuild the kdump initrd
and wants to use an existing version of the same. In such cases,
he can set the 'force_no_rebuild' option inside 'kdump.conf'
to 1, to force the 'kdumpctl' script not to rebuild the kdump
initrd.

Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Xunlei Pang <xlpang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-04-27 13:59:49 +08:00