In 2.0.4, Cliff from HP posted 2 patches:
e35aa29 kexec: include reserved e820 sections in crash kernel
4932034 kexec: lengthen the kernel command line image
However, with both of them kdump kernel may fail to boot, and
are useless because of restriction in kernel side. In upstream,
they have been reverted. Now back port these 2 revert commits.
Also since the commit 1a4e90b has dependency, back port commit
dc607e4 which is depended on by commit 1a4e90b too.
1a4e90b Revert "kexec: include reserved e820 sections in crash kernel"
dc607e4 kexec: i386: Add cmdline_add_memmap_internal() to reduce the code duplication
8274916 Revert: "kexec: lengthen the kernel command line image"
Currently we have two issues against mounting filesystems by systemd.
1. If any failure in sysroot.mount, initrd.target won't be reached.
2. If any failure in mounting /etc/fstab, initrd.target won't be reached
Our kdump.sh is in dracut-pre-pivot hook which is ordered after
initrd.target. That means if systemd doesn't reach initrd.target,
pre-pivot service will not run.
Based on above, we can conclude that in order to run kdump.sh,
initrd.target must be reached.
To fix issue 1), we can add rootflags=nofail to 2nd kernel cmdline, so
that initrd.target will not require sysroot.mount. initrd.target
wouldn't care about the failures in sysroot.mount. That means
initrd.target can always be reached whether or not sysroot.mount fails.
So when initrd.target is reached, kdump.sh can be run.
To fix issue 2), we can append "nofail" mount options to every entry in
/etc/fstab. It has almost the same affects as to sysroot.mount.
initrd.target can be reached whether or not mount /etc/fstab fails. So
when initrd.target is reached, kdump.sh can be run.
If the mount failures block kdump from working properly (for example,
the dump target isn't mounted), the error handling will be done by
"default" action specified in /etc/kdump.conf. Otherwise kdump will
ignore the mount failures and dump as expected.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
From: Wade Mealing <wmealing@redhat.com>
The RHEL 5 release of mkdumprd allowed for comments in the kdump config
file as shown below:
net 192.168.1.1 # this is the comment part
This patch strips them out during processing, but leaves the configuration
file in original condition.
Signed-off-by: Wade Mealing <wmealing@redhat.com>
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Add function strip_comments into kdump-lib.sh, since it's used by
several files.
Signed-off-by: Wade Mealing <wmealing@redhat.com>
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Currently in the whole kdump framework, we have some common functions
used across not only mkdumprd context and dracut context, but also 1st
kernel and 2nd kernel. We defined these functions at each script, which
is obviously not decent.
So let's introduce kdump-lib.sh for the shared functions and put it
to /lib/kdump/kdump-lib.sh.
It starts small, as you can see, only 3 functions are extracted. But in
the future more and more common functions can be added.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Now kdump.service runs "After" network.target. But network.target
doesn't mean network is setup and online[1]. We should use
network-online.target instead for ssh/nfs dump.
And also because nfs dump requires a mounted nfs when rebuilding kdump
initrd, kdump.service should also run "After" remote-fs.target (this
means all remote fs configured in /etc/fstab is mounted).
The downside of this patch is we always need to wait for network-online.target
even when dump target is a local disk. If network fails to come up,
kdump.service have to be stuck until network-online.target timeout (90
seconds by default).
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
In kdump_setup_bridge/bond/team(), we use _dev as a global variable.
That causes following issues when network is br0 over bond0:
-> kdump_setup_bridge br0: _dev to be "bond0" as a brif
-> kdump_setup_bond bond0: _dev is modified to be eth0 as a bond slave
-> (jump back) kdump_setup_bridge br0: we really need _dev is
"bond0" not "eth0".
_dev must be a local variable because it has been used multiple places.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
There will be a race condition if multiple kdumpctl instances are
running at the same time.
By introducing a global mutex lock, only one instance can acquire this
lock and run, others will be waiting for the lock in queue. Now each
kdump instance will be run in serial order and there won't any race
condition.
This is a patch backported from RHEL6.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Chaowang measured the selinux load_policy memory usage, it need ~50M
It's too much under kdump 2nd kernel, it cause more OOM then before.
Here is the findings from Vivek:
- If we don't load policy or don't do restorecon, kernel automatically
uses a label for file as specified by file
/sys/fs/selinux/initial_contexts/file
On my system this value is "system_u:object_r:file_t:s0". Kernel
enforces this label on a file if it is not labeled. That's the reason
that you see above label on vmcore file when selinux policy was not
loaded in second kernel or restorecon was not done.
Note: I did some testing with rhel6 and there also I see file_t context.
Not sure why that's the case.
- Relabeling of root file system over boot happens if there is a file
/.autorelabel present. This file is touched by systemd service
fedora-autorelabel-mark.service. And this file comes from initscritps
package.
So if this service thinks that system was booted with selinux disabled
it will put this file on root and when next time system boots with
selinux enabled, relabeling is enforced by fedora-autorelabel.service
service.
- In our case relabeling is not happening after saving vmcore because
there does not seem be any fedora-autorelabel-mark.service running
from initramfs context. Looks like this service runs after switching
to real root.
Aug 08 10:44:13 vm9-f19 systemd[1]: Started Mark the need to relabel after reboot.
- selinux poicy is now loaded by systemd after root switch has taken
place.
Aug 08 10:44:10 vm9-f19 systemd[1]: Successfully loaded SELinux policy in 357.693ms.
So now we know that why selinux relabeling is not taking place. Reason
being that systemd service which marks the file system for autorelabeling
does not run from initramfs context.
And it might not make to run this service from initramfs context before
switch root. In general it makes sense to first switch to root, load
selinux policy if needed and then check whether to mark this filesystem
for relabel or not. Ideally root is mourted read only before that. It is
just that we break this rule for kdump. So as long as we make sure we
relabel files created by kdump after booting back, things should be fine.
Since we will relabel the vmcore dir after reboot so let's remove
the selinux dracut module dependency to avoid load_policy in 2nd kernel.
If in the future load_policy memory usage shrinks to an acceptable level
or there's a better solution we can add selinux load_policy back later.
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
do_dump() takes care of dump procedure. It'll error out if failing to
save vmcore.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Currently in initrd, hardware clock is always considered to use UTC time
format and system time zone is also UTC. Thus system time isn't correct
if hw clock is localtime or we're using other time zone in real root.
To fix this, install /etc/adjtime and /etc/localtime to initrd.
Previously, this functionality was implemented in dracut base module:
commit 77364fd
Author: WANG Chao <chaowang@redhat.com>
base: setup correct system time and time zone in initrd
But some people complains about a normal boot initrd needs to rebuild
every time if time zone is changed. So let's fix it on our side.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
2nd kernel has very limited memory. Allocating huge pages will probably
trigger OOM. So let's remove hugepages and hugepagesz kernel parameters
for 2nd kernel when 1st kernel are using them.
If user wants huge pages cmdline in 2nd kernel, he/she can still specify
it through KERNEL_COMMANDLINE_APPEND in /etc/sysconfig/kdump.
This patch adds a new function remove_cmdline_param(). It takes a list
of kernel parameters as its arguments and remove them from given kernel
cmdline.
update:
1. Add description of remove_cmdline_param() per Vivek.
2. Remove_cmdline_param() will take kernel cmdline as $1, then strip it
and print the result.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Currently when action_on_fail is enabled, the emergency_shell won't be called
either. In kdump even though user specify the default action as emergency_shell,
dracut still skip it. Now change the implementation of action_on_fail to depend
on a file which is created by kdump when making kdump initrd, then remove it
at the beginning of kdump. This can solve the explicit emergency_shell problem.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
This reverts commit 9e964ff4c6.
Currently, because of dracut implementation, in kdump 'default shell' will
call emergency_shell of dracut. If action_on_fail is enabled, emergency_shell
is skipped. Then 'default shell' won't work either.
Here revert the old commit 9e964ff4 so that take other implementation.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
Currently in kdump.sh, we redirect stdout to stderr, because dracut
pre-pivot service (which kdump.sh is running within) only output stderr
to console. That behavior is defined in dracut-pre-pivot.service:
[Service]
...
StandardInput=null
StandardOutput=syslog
StandardError=syslog+console
...
But during testing, it has been observed that systemd will cache stderr
buffer, and first record to syslog (and it's own journal), then copy the
logs to /dev/console. And this practice is somehow unexpected in our
kdump script. We may have suppressed stdout/stderr that hasn't been
write to /dev/console before we run a force reboot.
With this change of redirecting stdout/stderr to /dev/console, kdump.sh
will output everything immediately to console, not cached/hidden by
systemd.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Build on ppc/ppc64 failed after makedumpfile-1.5.4 is pulled, since the
variable vmap_area_list is not defined. Back port below commit from
upstream to add it.
commit 150b58eb299066c65ef7713a93effc35c00be03a
Author: Baoquan He <bhe@redhat.com>
Date: Mon Jul 15 20:37:14 2013 +0800
[PATCH] Add vmap_area_list definition for ppc/ppc64.
vmap_area_list is added to get vmalloc_start for ppc/ppc64, but its
definition is missing, now add them.
Signed-off-by: Baoquan He <bhe@redhat.com>
Currently some functions are used in subshell to assign string to a
variable. For example:
_mnt=$(to_mount "$1")
In this case if we call perror_exit in the subshell, subshell will exit
1, but the parent process (mkdumprd) won't exit.
So we should handle the exit code of a subshell if the subshell calls
perror_exit over a failure.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
kvm virtio-blk device, for example /dev/vdb, doesn't have serial id by
default. So there's no persistent device node under /dev/disk/ for
/dev/vdb.
In case no persistent dev for dump target, we should use the original
device name directly, not failing the mkdumprd.
v2: update warn message from Vivek.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
dump_to_rootfs is a special handling of dump_fs. It's better we merge them
together to cleaup code.
Now dump_fs() function takes two types of $1, a mount point like
/sysroot or a dump target device like /dev/mapper/vg-lv_kdump.
v2: remove -F option in makedumpfile case from Vivek
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
After dump the vmcore, explicitly commit changed cache to disk in case
umount fail or chances we'll have an incomplete vmcore.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Percent signs in .spec files get expanded as macros. Currently in kexec-tools.spec,
'%{dist}' are appended to changelog item. This older changelog is not correctly for
rhel7 with this. Let's remove it to make it clearer.
stdout if line buffered, thus even it's redirect to stderr, it will not show
on console automaticly. Because monitor_dd_progress is only for rawdump
currently, so I think we can just use "echo" instead of "echo -n".
Another problem is sometimes CURRENT_SIZE does not get value when it's used
in $(($CURRENT_SIZE / 1048576)), fix this issue as well.
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
When using makedumpfile as core_collector, makedumpfile will show its
own progress bar, it will mix with the monitor_dd_progress and cause confusion.
In this patch just call monitor_dd_progress when core_collector is not
makedumpfile
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
Fedora 18 changes the way how to work with services in spec files.
It introduces new macros - %systemd_post, %systemd_preun and
%systemd_postun. These macros are functionally equivalent to the
manual scriptlets used in older versions of Fedora.
By using the new unified RPM macros the .spec file code is
simplified a lot.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
Currently umount fs happens right after saving vmcore. Therefore vmcore
isn't directly accessible in kdump_post script or shell. This patch moves
the umount fs operation down to the very last part, right before kdump
exits.
The patch adds a global variable MOUNTS to keep track which filesystem
is used. And umount these filesystems at do_default_action() and
do_final_action().
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Certain dracut module will mount fs under real root(/sysroot/ or
$NEWROOT/). Thus root fs can not be umounted by `umount /sysroot/`.
We should use `umount -R /sysroot/` to recursively umount root and
its submounts.
v2: do the same for dump_fs() from Baoquan
Signed-off-by: WANG Chao <chaowang@redhat.com>
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Replace $1/$2 with local variable names in dump_raw() and dump_ssh()
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
When makedumpfile failed, it could still generate a invalid vmcore. It's
better to suffix these invalid vmcore files with "-incomplete", as we do
in RHEL6.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
set -x is removed and we'll have little output about the dumping
progress. So it's best to output some messages on the top level to let
user know what's going on.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Add a section for debugging tips. I had to spend some time to figure out
how to drop into a shell from kdump_pre/kdump_post scripts. So thought of
documenting that so that somebdoy else can save time.
Also add a method to log serial consoles for virtual machines. I was
loosing some of the messages over reboot. Logging to file is fine. This
can be helful in debugging.
v2: Add method to log serial console for virtual machines
v3: Fix serial console logging instructions.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When directory is empty, echo * will output *, not empty string. That's
not intended.
Also it looks a little bit nicer now.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
kernel has exported mac address for each interface, we can get it
directly instead of parsing the output from ip address show.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Currently we use runtime mac addr to 2nd kernel to setup bonding
interface. But Bonding master will modify its slaves' mac addr and
incorrect mac addr is passed to 2nd kernel. Thus dracut in 2nd kernel
can't find expected slaves and bonding will fail.
Fix this issue by using perm address.
Tested in Fedora 19 KVM guest configured bonding with two slaves.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Currently "set -x" is specified in dracut-kdump.sh and I see the script
execution commands by default on console while testing with F19. That's
not right. This should be done only if user asked for it. Remove it.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Acked-by: WANG Chao <chaowang@redhat.com>
When 1st kernel is using KMS and crash, 2nd kernel can't reset to
nomodeset and the screen is black. In this case, user can't observe the
boot/dump progress and run commands in shell.
So let's pull in drm dracut module to fix this.
Signed-off-by: WANG Chao <chaowang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When ssh dump, if user doesn't have write permission on save path
of server, the crash kernel can be loaded successfully, but finally
kdump will fail because write is not allowed.
Let's check it in the service start phase, if no write permission
print error message and exit.
For differentiation, change the name of old function mkdir_save_path
to mkdir_save_path_fs.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
