* Sat Feb 03 2024 Jon Maloy <jmaloy@redhat.com> - 6.2.0-47

- kvm-s390x-pci-avoid-double-enable-disable-of-aif.patch [RHEL-22411]
- kvm-s390x-pci-refresh-fh-before-disabling-aif.patch [RHEL-22411]
- kvm-s390x-pci-drive-ISM-reset-from-subsystem-reset.patch [RHEL-22411]
- Resolves: RHEL-22411
  ([s390x] VM fails to start with ISM passed through)
This commit is contained in:
Jon Maloy 2024-02-03 16:57:17 -05:00
parent 6bbe5c7492
commit 248e8acca4
4 changed files with 328 additions and 1 deletions

View File

@ -0,0 +1,106 @@
From 52ad0cc8a82f7a4c3581146fb4d2046898163c4e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>
Date: Tue, 23 Jan 2024 13:59:24 +0100
Subject: [PATCH 1/3] s390x/pci: avoid double enable/disable of aif
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
RH-Author: Cédric Le Goater <clg@redhat.com>
RH-MergeRequest: 349: s390x: Fix reset ordering of passthrough ISM devices
RH-Jira: RHEL-22411
RH-Acked-by: Thomas Huth <thuth@redhat.com>
RH-Acked-by: Cornelia Huck <cohuck@redhat.com>
RH-Commit: [1/3] 450e4ca607d801bce93415994250374d70fb72f6
JIRA: https://issues.redhat.com/browse/RHEL-22411
commit 07b2c8e034d80ff92e202405c494d2ff80fcf848
Author: Matthew Rosato <mjrosato@linux.ibm.com>
Date: Thu Jan 18 13:51:49 2024 -0500
s390x/pci: avoid double enable/disable of aif
Use a flag to keep track of whether AIF is currently enabled. This can be
used to avoid enabling/disabling AIF multiple times as well as to determine
whether or not it should be disabled during reset processing.
Fixes: d0bc7091c2 ("s390x/pci: enable adapter event notification for interpreted devices")
Reported-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20240118185151.265329-2-mjrosato@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>
---
hw/s390x/s390-pci-kvm.c | 25 +++++++++++++++++++++++--
include/hw/s390x/s390-pci-bus.h | 1 +
2 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/hw/s390x/s390-pci-kvm.c b/hw/s390x/s390-pci-kvm.c
index ff41e4106d..1ee510436c 100644
--- a/hw/s390x/s390-pci-kvm.c
+++ b/hw/s390x/s390-pci-kvm.c
@@ -27,6 +27,7 @@ bool s390_pci_kvm_interp_allowed(void)
int s390_pci_kvm_aif_enable(S390PCIBusDevice *pbdev, ZpciFib *fib, bool assist)
{
+ int rc;
struct kvm_s390_zpci_op args = {
.fh = pbdev->fh,
.op = KVM_S390_ZPCIOP_REG_AEN,
@@ -38,15 +39,35 @@ int s390_pci_kvm_aif_enable(S390PCIBusDevice *pbdev, ZpciFib *fib, bool assist)
.u.reg_aen.flags = (assist) ? 0 : KVM_S390_ZPCIOP_REGAEN_HOST
};
- return kvm_vm_ioctl(kvm_state, KVM_S390_ZPCI_OP, &args);
+ if (pbdev->aif) {
+ return -EINVAL;
+ }
+
+ rc = kvm_vm_ioctl(kvm_state, KVM_S390_ZPCI_OP, &args);
+ if (rc == 0) {
+ pbdev->aif = true;
+ }
+
+ return rc;
}
int s390_pci_kvm_aif_disable(S390PCIBusDevice *pbdev)
{
+ int rc;
+
struct kvm_s390_zpci_op args = {
.fh = pbdev->fh,
.op = KVM_S390_ZPCIOP_DEREG_AEN
};
- return kvm_vm_ioctl(kvm_state, KVM_S390_ZPCI_OP, &args);
+ if (!pbdev->aif) {
+ return -EINVAL;
+ }
+
+ rc = kvm_vm_ioctl(kvm_state, KVM_S390_ZPCI_OP, &args);
+ if (rc == 0) {
+ pbdev->aif = false;
+ }
+
+ return rc;
}
diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h
index e0a9f9385b..7a658f5e30 100644
--- a/include/hw/s390x/s390-pci-bus.h
+++ b/include/hw/s390x/s390-pci-bus.h
@@ -361,6 +361,7 @@ struct S390PCIBusDevice {
bool unplug_requested;
bool interp;
bool forwarding_assist;
+ bool aif;
QTAILQ_ENTRY(S390PCIBusDevice) link;
};
--
2.41.0

View File

@ -0,0 +1,137 @@
From dda71c431be22772f3241af45b62737c988e85d4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>
Date: Tue, 23 Jan 2024 13:59:24 +0100
Subject: [PATCH 3/3] s390x/pci: drive ISM reset from subsystem reset
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
RH-Author: Cédric Le Goater <clg@redhat.com>
RH-MergeRequest: 349: s390x: Fix reset ordering of passthrough ISM devices
RH-Jira: RHEL-22411
RH-Acked-by: Thomas Huth <thuth@redhat.com>
RH-Acked-by: Cornelia Huck <cohuck@redhat.com>
RH-Commit: [3/3] 42e89595dd5e24538a2d3f075391b4534497eece
JIRA: https://issues.redhat.com/browse/RHEL-22411
commit 68c691ca99a2538d6a53a70ce8a9ce06ee307ff1
Author: Matthew Rosato <mjrosato@linux.ibm.com>
Date: Thu Jan 18 13:51:51 2024 -0500
s390x/pci: drive ISM reset from subsystem reset
ISM devices are sensitive to manipulation of the IOMMU, so the ISM device
needs to be reset before the vfio-pci device is reset (triggering a full
UNMAP). In order to ensure this occurs, trigger ISM device resets from
subsystem_reset before triggering the PCI bus reset (which will also
trigger vfio-pci reset). This only needs to be done for ISM devices
which were enabled for use by the guest.
Further, ensure that AIF is disabled as part of the reset event.
Fixes: ef1535901a ("s390x: do a subsystem reset before the unprotect on reboot")
Fixes: 03451953c7 ("s390x/pci: reset ISM passthrough devices on shutdown and system reset")
Reported-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20240118185151.265329-4-mjrosato@linux.ibm.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>
---
hw/s390x/s390-pci-bus.c | 26 +++++++++++++++++---------
hw/s390x/s390-virtio-ccw.c | 8 ++++++++
include/hw/s390x/s390-pci-bus.h | 1 +
3 files changed, 26 insertions(+), 9 deletions(-)
diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
index 2d92848b0f..a8953693b9 100644
--- a/hw/s390x/s390-pci-bus.c
+++ b/hw/s390x/s390-pci-bus.c
@@ -160,20 +160,12 @@ static void s390_pci_shutdown_notifier(Notifier *n, void *opaque)
pci_device_reset(pbdev->pdev);
}
-static void s390_pci_reset_cb(void *opaque)
-{
- S390PCIBusDevice *pbdev = opaque;
-
- pci_device_reset(pbdev->pdev);
-}
-
static void s390_pci_perform_unplug(S390PCIBusDevice *pbdev)
{
HotplugHandler *hotplug_ctrl;
if (pbdev->pft == ZPCI_PFT_ISM) {
notifier_remove(&pbdev->shutdown_notifier);
- qemu_unregister_reset(s390_pci_reset_cb, pbdev);
}
/* Unplug the PCI device */
@@ -1137,7 +1129,6 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
if (pbdev->pft == ZPCI_PFT_ISM) {
pbdev->shutdown_notifier.notify = s390_pci_shutdown_notifier;
qemu_register_shutdown_notifier(&pbdev->shutdown_notifier);
- qemu_register_reset(s390_pci_reset_cb, pbdev);
}
} else {
pbdev->fh |= FH_SHM_EMUL;
@@ -1284,6 +1275,23 @@ static void s390_pci_enumerate_bridge(PCIBus *bus, PCIDevice *pdev,
pci_default_write_config(pdev, PCI_SUBORDINATE_BUS, s->bus_no, 1);
}
+void s390_pci_ism_reset(void)
+{
+ S390pciState *s = s390_get_phb();
+
+ S390PCIBusDevice *pbdev, *next;
+
+ /* Trigger reset event for each passthrough ISM device currently in-use */
+ QTAILQ_FOREACH_SAFE(pbdev, &s->zpci_devs, link, next) {
+ if (pbdev->interp && pbdev->pft == ZPCI_PFT_ISM &&
+ pbdev->fh & FH_MASK_ENABLE) {
+ s390_pci_kvm_aif_disable(pbdev);
+
+ pci_device_reset(pbdev->pdev);
+ }
+ }
+}
+
static void s390_pcihost_reset(DeviceState *dev)
{
S390pciState *s = S390_PCI_HOST_BRIDGE(dev);
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 94434c3bb1..51e5b39888 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -108,6 +108,14 @@ static void subsystem_reset(void)
DeviceState *dev;
int i;
+ /*
+ * ISM firmware is sensitive to unexpected changes to the IOMMU, which can
+ * occur during reset of the vfio-pci device (unmap of entire aperture).
+ * Ensure any passthrough ISM devices are reset now, while CPUs are paused
+ * but before vfio-pci cleanup occurs.
+ */
+ s390_pci_ism_reset();
+
for (i = 0; i < ARRAY_SIZE(reset_dev_types); i++) {
dev = DEVICE(object_resolve_path_type("", reset_dev_types[i], NULL));
if (dev) {
diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h
index 7a658f5e30..2bfad5563a 100644
--- a/include/hw/s390x/s390-pci-bus.h
+++ b/include/hw/s390x/s390-pci-bus.h
@@ -401,5 +401,6 @@ S390PCIBusDevice *s390_pci_find_dev_by_target(S390pciState *s,
const char *target);
S390PCIBusDevice *s390_pci_find_next_avail_dev(S390pciState *s,
S390PCIBusDevice *pbdev);
+void s390_pci_ism_reset(void);
#endif
--
2.41.0

View File

@ -0,0 +1,71 @@
From fe70e87ef8d2f7e538867052e06012051919083f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>
Date: Tue, 23 Jan 2024 13:59:24 +0100
Subject: [PATCH 2/3] s390x/pci: refresh fh before disabling aif
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
RH-Author: Cédric Le Goater <clg@redhat.com>
RH-MergeRequest: 349: s390x: Fix reset ordering of passthrough ISM devices
RH-Jira: RHEL-22411
RH-Acked-by: Thomas Huth <thuth@redhat.com>
RH-Acked-by: Cornelia Huck <cohuck@redhat.com>
RH-Commit: [2/3] 4a7d3fccdac508253bd7e5765973a08482022edb
JIRA: https://issues.redhat.com/browse/RHEL-22411
commit 30e35258e25c75c9d799c34fd89afcafffb37084
Author: Matthew Rosato <mjrosato@linux.ibm.com>
Date: Thu Jan 18 13:51:50 2024 -0500
s390x/pci: refresh fh before disabling aif
Typically we refresh the host fh during CLP enable, however it's possible
that the device goes through multiple reset events before the guest
performs another CLP enable. Let's handle this for now by refreshing the
host handle from vfio before disabling aif.
Fixes: 03451953c7 ("s390x/pci: reset ISM passthrough devices on shutdown and system reset")
Reported-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Eric Farman <farman@linux.ibm.com>
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Message-ID: <20240118185151.265329-3-mjrosato@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>
---
hw/s390x/s390-pci-kvm.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/hw/s390x/s390-pci-kvm.c b/hw/s390x/s390-pci-kvm.c
index 1ee510436c..9eef4fc3ec 100644
--- a/hw/s390x/s390-pci-kvm.c
+++ b/hw/s390x/s390-pci-kvm.c
@@ -18,6 +18,7 @@
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/s390-pci-kvm.h"
#include "hw/s390x/s390-pci-inst.h"
+#include "hw/s390x/s390-pci-vfio.h"
#include "cpu_models.h"
bool s390_pci_kvm_interp_allowed(void)
@@ -64,6 +65,14 @@ int s390_pci_kvm_aif_disable(S390PCIBusDevice *pbdev)
return -EINVAL;
}
+ /*
+ * The device may have already been reset but we still want to relinquish
+ * the guest ISC, so always be sure to use an up-to-date host fh.
+ */
+ if (!s390_pci_get_host_fh(pbdev, &args.fh)) {
+ return -EPERM;
+ }
+
rc = kvm_vm_ioctl(kvm_state, KVM_S390_ZPCI_OP, &args);
if (rc == 0) {
pbdev->aif = false;
--
2.41.0

View File

@ -83,7 +83,7 @@ Obsoletes: %1-rhev <= %{epoch}:%{version}-%{release}
Summary: QEMU is a machine emulator and virtualizer
Name: qemu-kvm
Version: 6.2.0
Release: 46%{?rcrel}%{?dist}
Release: 47%{?rcrel}%{?dist}
# Epoch because we pushed a qemu-1.0 package. AIUI this can't ever be dropped
Epoch: 15
License: GPLv2 and GPLv2+ and CC-BY
@ -827,6 +827,12 @@ Patch332: kvm-hw-s390x-pv-Restrict-Protected-Virtualization-to-sys.patch
Patch333: kvm-hw-s390x-Move-KVM-specific-PV-from-hw-to-target-s390.patch
# For RHEL-18214 - [RHEL8][Secure-execution][s390x] The error message is not clear when boot up a SE guest with wrong encryption
Patch334: kvm-target-s390x-kvm-pv-Provide-some-more-useful-informa.patch
# For RHEL-22411 - [s390x] VM fails to start with ISM passed through
Patch335: kvm-s390x-pci-avoid-double-enable-disable-of-aif.patch
# For RHEL-22411 - [s390x] VM fails to start with ISM passed through
Patch336: kvm-s390x-pci-refresh-fh-before-disabling-aif.patch
# For RHEL-22411 - [s390x] VM fails to start with ISM passed through
Patch337: kvm-s390x-pci-drive-ISM-reset-from-subsystem-reset.patch
BuildRequires: wget
BuildRequires: rpm-build
@ -1996,6 +2002,13 @@ sh %{_sysconfdir}/sysconfig/modules/kvm.modules &> /dev/null || :
%changelog
* Sat Feb 03 2024 Jon Maloy <jmaloy@redhat.com> - 6.2.0-47
- kvm-s390x-pci-avoid-double-enable-disable-of-aif.patch [RHEL-22411]
- kvm-s390x-pci-refresh-fh-before-disabling-aif.patch [RHEL-22411]
- kvm-s390x-pci-drive-ISM-reset-from-subsystem-reset.patch [RHEL-22411]
- Resolves: RHEL-22411
([s390x] VM fails to start with ISM passed through)
* Wed Jan 17 2024 Jon Maloy <jmaloy@redhat.com> - 6.2.0-46
- kvm-MAINTAINERS-split-out-s390x-sections.patch [RHEL-18214]
- kvm-s390x-pv-remove-semicolon-from-macro-definition.patch [RHEL-18214]