- Add DMABUF support [VOYAGER-19 VOYAGER-53] - Accelerated SMMU device for GH GPU passthrough [VOYAGER-5 VOYAGER-16 VOYAGER-17 VOYAGER-48] - Resolves: VOYAGER-5 (Backport CMDQV support) - Resolves: VOYAGER-16 (Backport HW accelerated nesting support for arm SMMUv3) - Resolves: VOYAGER-17 (Backport vEVENTQ support for smmuv3) - Resolves: VOYAGER-19 (Backport vfio: Add DMABUF support for PCI BAR regions - qemu-kvm) - Resolves: VOYAGER-48 (qemu-kvm coredump when using traditional smmuv3 device without any GPU device) - Resolves: VOYAGER-53 (qemu-kvm coredump when hotunplug NIC VF interface without smmuv3)
152 lines
5.6 KiB
Diff
152 lines
5.6 KiB
Diff
From 1e87a05f4a9a52b9f265059918ba0fd10c71000a Mon Sep 17 00:00:00 2001
|
|
From: Steve Sistare <steven.sistare@oracle.com>
|
|
Date: Wed, 1 Oct 2025 08:33:59 -0700
|
|
Subject: [PATCH] migration: cpr-exec docs
|
|
MIME-Version: 1.0
|
|
Content-Type: text/plain; charset=UTF-8
|
|
Content-Transfer-Encoding: 8bit
|
|
|
|
RH-Author: Cédric Le Goater <clg@redhat.com>
|
|
RH-MergeRequest: 458: Add DMABUF support
|
|
RH-Jira: VOYAGER-19 VOYAGER-53
|
|
RH-Acked-by: Eric Auger <eric.auger@redhat.com>
|
|
RH-Commit: [1/101] 188b93fdb7 (clegoate/qemu-kvm-centos)
|
|
|
|
Update developer documentation for cpr-exec mode.
|
|
|
|
Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
|
|
Reviewed-by: Fabiano Rosas <farosas@suse.de>
|
|
Link: https://lore.kernel.org/r/1759332851-370353-8-git-send-email-steven.sistare@oracle.com
|
|
Signed-off-by: Peter Xu <peterx@redhat.com>
|
|
---
|
|
docs/devel/migration/CPR.rst | 112 ++++++++++++++++++++++++++++++++++-
|
|
1 file changed, 111 insertions(+), 1 deletion(-)
|
|
|
|
diff --git a/docs/devel/migration/CPR.rst b/docs/devel/migration/CPR.rst
|
|
index 0a0fd4f6dc..b6178568a8 100644
|
|
--- a/docs/devel/migration/CPR.rst
|
|
+++ b/docs/devel/migration/CPR.rst
|
|
@@ -5,7 +5,7 @@ CPR is the umbrella name for a set of migration modes in which the
|
|
VM is migrated to a new QEMU instance on the same host. It is
|
|
intended for use when the goal is to update host software components
|
|
that run the VM, such as QEMU or even the host kernel. At this time,
|
|
-the cpr-reboot and cpr-transfer modes are available.
|
|
+the cpr-reboot, cpr-transfer, and cpr-exec modes are available.
|
|
|
|
Because QEMU is restarted on the same host, with access to the same
|
|
local devices, CPR is allowed in certain cases where normal migration
|
|
@@ -324,3 +324,113 @@ descriptors from old to new QEMU. In the future, descriptors for
|
|
vhost, and char devices could be transferred,
|
|
preserving those devices and their kernel state without interruption,
|
|
even if they do not explicitly support live migration.
|
|
+
|
|
+cpr-exec mode
|
|
+-------------
|
|
+
|
|
+In this mode, QEMU stops the VM, writes VM state to the migration
|
|
+URI, and directly exec's a new version of QEMU on the same host,
|
|
+replacing the original process while retaining its PID. Guest RAM is
|
|
+preserved in place, albeit with new virtual addresses. The user
|
|
+completes the migration by specifying the ``-incoming`` option, and
|
|
+by issuing the ``migrate-incoming`` command if necessary; see details
|
|
+below.
|
|
+
|
|
+This mode supports VFIO/IOMMUFD devices by preserving device
|
|
+descriptors and hence kernel state across the exec, even for devices
|
|
+that do not support live migration.
|
|
+
|
|
+Because the old and new QEMU instances are not active concurrently,
|
|
+the URI cannot be a type that streams data from one instance to the
|
|
+other.
|
|
+
|
|
+This mode does not require a channel of type ``cpr``. The information
|
|
+that is passed over that channel for cpr-transfer mode is instead
|
|
+serialized to a memfd, the number of the fd is saved in the
|
|
+QEMU_CPR_EXEC_STATE environment variable during the exec of new QEMU.
|
|
+and new QEMU mmaps the memfd.
|
|
+
|
|
+Usage
|
|
+^^^^^
|
|
+
|
|
+Arguments for the new QEMU process are taken from the
|
|
+@cpr-exec-command parameter. The first argument should be the
|
|
+path of a new QEMU binary, or a prefix command that exec's the
|
|
+new QEMU binary, and the arguments should include the ''-incoming''
|
|
+option.
|
|
+
|
|
+Memory backend objects must have the ``share=on`` attribute.
|
|
+The VM must be started with the ``-machine aux-ram-share=on`` option.
|
|
+
|
|
+Outgoing:
|
|
+ * Set the migration mode parameter to ``cpr-exec``.
|
|
+ * Set the ``cpr-exec-command`` parameter.
|
|
+ * Issue the ``migrate`` command. It is recommended that the URI be
|
|
+ a ``file`` type, but one can use other types such as ``exec``,
|
|
+ provided the command captures all the data from the outgoing side,
|
|
+ and provides all the data to the incoming side.
|
|
+
|
|
+Incoming:
|
|
+ * You do not need to explicitly start new QEMU. It is started as
|
|
+ a side effect of the migrate command above.
|
|
+ * If the VM was running when the outgoing ``migrate`` command was
|
|
+ issued, then QEMU automatically resumes VM execution.
|
|
+
|
|
+Example 1: incoming URI
|
|
+^^^^^^^^^^^^^^^^^^^^^^^
|
|
+
|
|
+In these examples, we simply restart the same version of QEMU, but in
|
|
+a real scenario one would set a new QEMU binary path in
|
|
+cpr-exec-command.
|
|
+
|
|
+::
|
|
+
|
|
+ # qemu-kvm -monitor stdio
|
|
+ -object memory-backend-memfd,id=ram0,size=4G
|
|
+ -machine memory-backend=ram0
|
|
+ -machine aux-ram-share=on
|
|
+ ...
|
|
+
|
|
+ QEMU 10.2.50 monitor - type 'help' for more information
|
|
+ (qemu) info status
|
|
+ VM status: running
|
|
+ (qemu) migrate_set_parameter mode cpr-exec
|
|
+ (qemu) migrate_set_parameter cpr-exec-command qemu-kvm ... -incoming file:vm.state
|
|
+ (qemu) migrate -d file:vm.state
|
|
+ (qemu) QEMU 10.2.50 monitor - type 'help' for more information
|
|
+ (qemu) info status
|
|
+ VM status: running
|
|
+
|
|
+Example 2: incoming defer
|
|
+^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
+::
|
|
+
|
|
+ # qemu-kvm -monitor stdio
|
|
+ -object memory-backend-memfd,id=ram0,size=4G
|
|
+ -machine memory-backend=ram0
|
|
+ -machine aux-ram-share=on
|
|
+ ...
|
|
+
|
|
+ QEMU 10.2.50 monitor - type 'help' for more information
|
|
+ (qemu) info status
|
|
+ VM status: running
|
|
+ (qemu) migrate_set_parameter mode cpr-exec
|
|
+ (qemu) migrate_set_parameter cpr-exec-command qemu-kvm ... -incoming defer
|
|
+ (qemu) migrate -d file:vm.state
|
|
+ (qemu) QEMU 10.2.50 monitor - type 'help' for more information
|
|
+ (qemu) info status
|
|
+ status: paused (inmigrate)
|
|
+ (qemu) migrate_incoming file:vm.state
|
|
+ (qemu) info status
|
|
+ VM status: running
|
|
+
|
|
+Caveats
|
|
+^^^^^^^
|
|
+
|
|
+cpr-exec mode may not be used with postcopy, background-snapshot,
|
|
+or COLO.
|
|
+
|
|
+cpr-exec mode requires permission to use the exec system call, which
|
|
+is denied by certain sandbox options, such as spawn.
|
|
+
|
|
+The guest pause time increases for large guest RAM backed by small pages.
|