Don't use nbdkit-file-plugin cache=none when writing

Further performance fix for https://bugzilla.redhat.com/2011713

related: rhbz#2011713
This commit is contained in:
Richard W.M. Jones 2021-12-07 13:48:05 +00:00
parent a6ee991e8e
commit 94cf9be0f8
11 changed files with 79 additions and 24 deletions

View File

@ -0,0 +1,57 @@
From ac59d3b2310511b1537d408b675b19ec9a5d384e Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 7 Dec 2021 12:53:41 +0000
Subject: [PATCH] output: Don't use nbdkit-file-plugin cache=none when writing
nbdkit-file-plugin flag cache=none is meant to ensure that the file
plugin does not pollute the page cache. However it doesn't yet work
very well, especially for writing. As you can see here:
https://gitlab.com/nbdkit/nbdkit/-/blob/048d5b9818c88355e596824355269773e5c4f6ad/plugins/file/file.c#L594
it is currently implemented by flushing the whole file (including
parallel requests), and then evicting the data from the page cache.
This implementation is naive. (See nbdcopy copy/file-ops.c for a much
better implementation.)
This causes quite a considerable slow down when writing to a local
file (eg -os local). The penalty varies between machines and possibly
kernels. On my Fedora Rawhide AMD server with 12 cores and SSDs, the
slow down is under 5%. But on my RHEL 9 Intel NUC server with 2 cores
and a hard disk, it makes a really huge difference, nearly doubling
the total time taken.
I left a note in the code that we should fix this in nbdkit and re-add
this option or similar when it is working.
I also removed the fadvise=sequential option. This is associated with
cache=none in the nbdkit documentation, and anyway we are not doing
sequential writes. (This makes no measurable difference in my testing)
---
output/output.ml | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/output/output.ml b/output/output.ml
index 7e976508..f829fe43 100644
--- a/output/output.ml
+++ b/output/output.ml
@@ -98,8 +98,14 @@ let output_to_local_file ?(changeuid = fun f -> f ())
let cmd = Nbdkit.add_arg cmd "file" filename in
let cmd =
if Nbdkit.version nbdkit_config >= (1, 22, 0) then (
- let cmd = Nbdkit.add_arg cmd "fadvise" "sequential" in
- let cmd = Nbdkit.add_arg cmd "cache" "none" in
+ (* nbdkit 1.28 has a very naive implementation of
+ * page cache eviction. We need to copy the one from
+ * nbdcopy copy/file-ops.c. Until then do not use
+ * this flag because it causes a large slow down on
+ * some machines. XXX
+ *)
+ (*let cmd = Nbdkit.add_arg cmd "fadvise" "sequential" in
+ let cmd = Nbdkit.add_arg cmd "cache" "none" in*)
cmd
)
else cmd in
--
2.31.1

View File

@ -1,4 +1,4 @@
From 950985cd43569d085b1198a15d9ffe0185e954a5 Mon Sep 17 00:00:00 2001
From a0cc1fcb8f577802908887ed6cf98913f4260481 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sun, 28 Sep 2014 19:14:43 +0100
Subject: [PATCH] RHEL: v2v: Select correct qemu binary for -o qemu mode

View File

@ -1,4 +1,4 @@
From 30dd7bf52be72eb2d4d638db89924a701a8290c3 Mon Sep 17 00:00:00 2001
From 46ae00f83c2f4459fcdbf090d02a3f5cb4421fe9 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 30 Sep 2014 10:50:27 +0100
Subject: [PATCH] RHEL: v2v: Disable the --qemu-boot / -oo qemu-boot option

View File

@ -1,4 +1,4 @@
From 8d76ce9a924775b20f5d7a92e2709393e80a3312 Mon Sep 17 00:00:00 2001
From b12adf807a16e71764601f5912f98ead7839e25a Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Fri, 24 Apr 2015 09:45:41 -0400
Subject: [PATCH] RHEL: Fix list of supported sound cards to match RHEL qemu

View File

@ -1,4 +1,4 @@
From 225d717b0bfc70d4eaacee8bb09dc4d8cfc3219e Mon Sep 17 00:00:00 2001
From 836076de3059b12d54f098853bf5b35087885eeb Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sun, 30 Aug 2015 03:21:57 -0400
Subject: [PATCH] RHEL: Fixes for libguestfs-winsupport.

View File

@ -1,4 +1,4 @@
From 105f5aa22cd03e480b63f61b56e7459db69d41c8 Mon Sep 17 00:00:00 2001
From 9b570f662ac6e72abeec7271856a08c759a3bca7 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Thu, 2 Mar 2017 14:21:37 +0100
Subject: [PATCH] RHEL: v2v: -i disk: force VNC as display (RHBZ#1372671)

View File

@ -1,4 +1,4 @@
From 8d8dcce022f27e5c82f0bb72fddf5b9329373761 Mon Sep 17 00:00:00 2001
From 2a96ca52ae36874d333b199caea11594ad82638f Mon Sep 17 00:00:00 2001
From: Pino Toscano <ptoscano@redhat.com>
Date: Wed, 8 Mar 2017 11:03:40 +0100
Subject: [PATCH] RHEL: v2v: do not mention SUSE Xen hosts (RHBZ#1430203)

View File

@ -1,4 +1,4 @@
From 98e2603f7ff0a2cd9f23f80dd1d22f908a1a869d Mon Sep 17 00:00:00 2001
From d3536cfb1a741b296cd904db303cb002cae4fbfd Mon Sep 17 00:00:00 2001
From: Pino Toscano <ptoscano@redhat.com>
Date: Tue, 26 Mar 2019 09:42:25 +0100
Subject: [PATCH] RHEL: point to KB for supported v2v hypervisors/guests

View File

@ -1,4 +1,4 @@
From e1f82d3ca83499a4fce6347664a277679c04db93 Mon Sep 17 00:00:00 2001
From 608b966f2f41ccd3e500a16496b9ff1b456d99e0 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Wed, 30 Jun 2021 11:15:52 +0100
Subject: [PATCH] RHEL: Disable -o glance

View File

@ -1,4 +1,4 @@
From c7de0b56ada81868aecd1c719bc3621109675b60 Mon Sep 17 00:00:00 2001
From c4e64b06d8e92cf82fc1fee881876ffe45add057 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Thu, 2 Dec 2021 11:56:05 +0000
Subject: [PATCH] RHEL: Remove the --in-place option

View File

@ -15,7 +15,7 @@
Name: virt-v2v
Epoch: 1
Version: 1.45.94
Release: 1%{?dist}
Release: 2%{?dist}
Summary: Convert a virtual machine to run on KVM
License: GPLv2+
@ -51,15 +51,16 @@ ExclusiveArch: x86_64
# Downstream (RHEL-only) patches.
%if 0%{?rhel}
# Patches.
Patch0001: 0001-RHEL-v2v-Select-correct-qemu-binary-for-o-qemu-mode-.patch
Patch0002: 0002-RHEL-v2v-Disable-the-qemu-boot-oo-qemu-boot-option-R.patch
Patch0003: 0003-RHEL-Fix-list-of-supported-sound-cards-to-match-RHEL.patch
Patch0004: 0004-RHEL-Fixes-for-libguestfs-winsupport.patch
Patch0005: 0005-RHEL-v2v-i-disk-force-VNC-as-display-RHBZ-1372671.patch
Patch0006: 0006-RHEL-v2v-do-not-mention-SUSE-Xen-hosts-RHBZ-1430203.patch
Patch0007: 0007-RHEL-point-to-KB-for-supported-v2v-hypervisors-guest.patch
Patch0008: 0008-RHEL-Disable-o-glance.patch
Patch0009: 0009-RHEL-Remove-the-in-place-option.patch
Patch0001: 0001-output-Don-t-use-nbdkit-file-plugin-cache-none-when-.patch
Patch0002: 0002-RHEL-v2v-Select-correct-qemu-binary-for-o-qemu-mode-.patch
Patch0003: 0003-RHEL-v2v-Disable-the-qemu-boot-oo-qemu-boot-option-R.patch
Patch0004: 0004-RHEL-Fix-list-of-supported-sound-cards-to-match-RHEL.patch
Patch0005: 0005-RHEL-Fixes-for-libguestfs-winsupport.patch
Patch0006: 0006-RHEL-v2v-i-disk-force-VNC-as-display-RHBZ-1372671.patch
Patch0007: 0007-RHEL-v2v-do-not-mention-SUSE-Xen-hosts-RHBZ-1430203.patch
Patch0008: 0008-RHEL-point-to-KB-for-supported-v2v-hypervisors-guest.patch
Patch0009: 0009-RHEL-Disable-o-glance.patch
Patch0010: 0010-RHEL-Remove-the-in-place-option.patch
%endif
%if 0%{patches_touch_autotools}
@ -300,11 +301,8 @@ popd
%changelog
* Tue Dec 07 2021 Richard W.M. Jones <rjones@redhat.com> - 1:1.45.94-1
- New upstream development version 1.45.94 (preview of 2.0)
* Fri Dec 03 2021 Richard W.M. Jones <rjones@redhat.com> - 1:1.45.93-1
- Rebase to upstream 1.45.93.
* Tue Dec 07 2021 Richard W.M. Jones <rjones@redhat.com> - 1:1.45.94-2
- Rebase to upstream 1.45.94.
- Change video type to VGA (instead of QXL).
- Remove --in-place support properly.
- Remove -o glance support properly.