rpms/nbdkit
7
0
Fork 0
Code Issues Pull Requests Releases Activity

import nbdkit-1.28.3-2.el9

Browse Source
This commit is contained in:
CentOS Sources 2022-01-11 12:50:44 -05:00 committed by Stepan Oksanichenko
parent 2193c3b856
commit af70f16f18
34 changed files with 495 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1,2 +1,2 @@
SOURCES/libguestfs.keyring
SOURCES/nbdkit-1.28.2.tar.gz
SOURCES/nbdkit-1.28.3.tar.gz

2
.nbdkit.metadata
View File

@ -1,2 +1,2 @@
cc1b37b9cfafa515aab3eefd345ecc59aac2ce7b SOURCES/libguestfs.keyring
ee726a0b63e9a52b787de1462b11e1af34ca51ab SOURCES/nbdkit-1.28.2.tar.gz
0ec6c759802dd658a7e78b657886c1af11fd16e8 SOURCES/nbdkit-1.28.3.tar.gz

2
SOURCES/0001-vddk-Refactor-how-D-vddk.stats-1-is-collected.patch
View File

@ -1,4 +1,4 @@
From 96ee8f6f2844bceb8e27ffb442359a2b7521c950 Mon Sep 17 00:00:00 2001
From 40016b4cf27cd68c55fc505e97b658b4fc3643c9 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Thu, 21 Oct 2021 14:49:52 +0100
Subject: [PATCH] vddk: Refactor how -D vddk.stats=1 is collected

2
SOURCES/0002-vddk-Extend-D-vddk.stats-1-to-show-number-of-calls-a.patch
View File

@ -1,4 +1,4 @@
From f388c9b6c983d395ced0d4f467980b182d0a1b84 Mon Sep 17 00:00:00 2001
From 262f7fcd22dd57ae30739703c94bc56f32a8ceec Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Thu, 21 Oct 2021 15:10:00 +0100
Subject: [PATCH] vddk: Extend -D vddk.stats=1 to show number of calls and

2
SOURCES/0003-vddk-Simplify-and-consolidate-VDDK_CALL_START-END-ma.patch
View File

@ -1,4 +1,4 @@
From cc1c3b4ab57a1662bf87766161167fac40a78c0e Mon Sep 17 00:00:00 2001
From ea29f00332300e5a5808931ae48920c715c59182 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Thu, 21 Oct 2021 22:55:17 +0100
Subject: [PATCH] vddk: Simplify and consolidate VDDK_CALL_START/END macros

2
SOURCES/0004-vddk-Document-troubleshooting-performance-problems.patch
View File

@ -1,4 +1,4 @@
From 4bd9926c0e506fdb04976d348b1c7614865c8b06 Mon Sep 17 00:00:00 2001
From 7433c58a81d156fe078a8d5d39d75c8cfde458de Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Fri, 22 Oct 2021 18:00:27 +0100
Subject: [PATCH] vddk: Document troubleshooting performance problems

2
SOURCES/0005-vddk-Include-VDDK-major-library-version-in-dump-plug.patch
View File

@ -1,4 +1,4 @@
From eb6ccb03d0ca12ef19e5705cd96f81824910087b Mon Sep 17 00:00:00 2001
From e1b498bef17d1be8a19a5d29f34bc99c21b54e12 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 23 Oct 2021 16:16:39 +0100
Subject: [PATCH] vddk: Include VDDK major library version in --dump-plugin

2
SOURCES/0006-vddk-Add-logical-and-physical-sector-size-to-D-vddk..patch
View File

@ -1,4 +1,4 @@
From 0139f1815e9259fa789d84d2f32d30ee59bd728c Mon Sep 17 00:00:00 2001
From 7cd9b2bbbdae01513baa215abfacecbf19af2b1d Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 23 Oct 2021 16:24:27 +0100
Subject: [PATCH] vddk: Add logical and physical sector size to -D

2
SOURCES/0007-vddk-Fix-typo-in-debug-message.patch
View File

@ -1,4 +1,4 @@
From a5f73cbcbb6891d2e3c2cb541d47b44a236785ce Mon Sep 17 00:00:00 2001
From f00a059f71372a38703a2393b791703dfcb4f143 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 23 Oct 2021 19:41:07 +0100
Subject: [PATCH] vddk: Fix typo in debug message

2
SOURCES/0008-vddk-Only-print-vddk_library_version-when-we-managed.patch
View File

@ -1,4 +1,4 @@
From 1cb810a416e1bdd78a8e5df886a3185d3cfa54d0 Mon Sep 17 00:00:00 2001
From 27926a76f087cc81fc07f5a72da50384aab785c6 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 23 Oct 2021 19:50:52 +0100
Subject: [PATCH] vddk: Only print vddk_library_version when we managed to load

2
SOURCES/0009-vddk-Print-one-line-in-dump-plugin-output-for-each-V.patch
View File

@ -1,4 +1,4 @@
From 8780009ec092d9cc5a408b7597d88aa54db13639 Mon Sep 17 00:00:00 2001
From 7b43137763287b3d79874e27146ea2fecc5f9a17 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Mon, 25 Oct 2021 08:36:53 +0100
Subject: [PATCH] vddk: Print one line in --dump-plugin output for each VDDK

2
SOURCES/0010-vddk-Move-minimum-version-to-VDDK-6.5.patch
View File

@ -1,4 +1,4 @@
From e34016cbba4340b25f9a52c98db918aa72b38a7c Mon Sep 17 00:00:00 2001
From d4e1acf2e9b607a05b08ceb49d2879db02016cfe Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 26 Oct 2021 19:46:32 +0100
Subject: [PATCH] vddk: Move minimum version to VDDK 6.5

2
SOURCES/0011-vddk-Add-read-write-and-wait-asynchronous-functions.patch
View File

@ -1,4 +1,4 @@
From 69b989b37c8e33f52d928c7202146e9e11a2a93c Mon Sep 17 00:00:00 2001
From 53622e928682d08acafb27ce973a654ef7814c4d Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Wed, 27 Oct 2021 11:57:35 +0100
Subject: [PATCH] vddk: Add read, write and wait asynchronous functions

2
SOURCES/0012-vddk-Start-to-split-VDDK-over-several-files.patch
View File

@ -1,4 +1,4 @@
From 98a499c0e9d08f208474759012ec3ed823ce2335 Mon Sep 17 00:00:00 2001
From 35436075eb0df59c6480b79adaa755a2c6138833 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Wed, 27 Oct 2021 12:20:31 +0100
Subject: [PATCH] vddk: Start to split VDDK over several files

2
SOURCES/0013-vddk-Refactor-D-vddk.stats-1-into-a-new-file.patch
View File

@ -1,4 +1,4 @@
From d602150dbb5ebacea42c25a0f6c8c26c45766a49 Mon Sep 17 00:00:00 2001
From 27b7a2b4b58a99eb79386412a2b3d5387d28f6bc Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Wed, 27 Oct 2021 12:30:41 +0100
Subject: [PATCH] vddk: Refactor -D vddk.stats=1 into a new file

2
SOURCES/0014-vddk-Implement-parallel-thread-model.patch
View File

@ -1,4 +1,4 @@
From 5744b0000addaa0d50b6e0ee8e4540349623be0a Mon Sep 17 00:00:00 2001
From d12a244fa6b5b582a7790a525a621b4573133749 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Wed, 27 Oct 2021 10:17:22 +0100
Subject: [PATCH] vddk: Implement parallel thread model

2
SOURCES/0015-vddk-Assume-that-VixDiskLib_Flush-is-available.patch
View File

@ -1,4 +1,4 @@
From eda9dd7f5e610fd4e17019813c5a045f0b3603df Mon Sep 17 00:00:00 2001
From dd2ae58503f3c17e8aee2f78f97e0f669ce464c6 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Fri, 29 Oct 2021 20:56:55 +0100
Subject: [PATCH] vddk: Assume that VixDiskLib_Flush is available

2
SOURCES/0016-vddk-Simplify-detection-of-VDDK-symbols-and-baseline.patch
View File

@ -1,4 +1,4 @@
From 1b2b386c9a254808a25fbfce3640c96bdb8cf9be Mon Sep 17 00:00:00 2001
From 10bbf28ad230a887ec05a2b1d25d95fd0d742072 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Fri, 29 Oct 2021 21:02:54 +0100
Subject: [PATCH] vddk: Simplify detection of VDDK symbols and baseline 6.5

2
SOURCES/0017-vddk-Remove-some-whitespace-from-a-couple-of-functio.patch
View File

@ -1,4 +1,4 @@
From 2363e76ab34a2e11b57970d82161f73453a4a8ec Mon Sep 17 00:00:00 2001
From c0d6f863ecd47535fe55bae12ce5bf292e356dbd Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 30 Oct 2021 08:34:28 +0100
Subject: [PATCH] vddk: Remove some whitespace from a couple of functions

2
SOURCES/0018-vddk-Move-config-debug-error-and-utility-functions-a.patch
View File

@ -1,4 +1,4 @@
From 6c0034cf8802d466b170135fec0d6a97d1eb2f2a Mon Sep 17 00:00:00 2001
From c55eb650440d9cd713db74622ddc2afa3c1ce740 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 30 Oct 2021 08:27:39 +0100
Subject: [PATCH] vddk: Move config, debug/error and utility functions around

2
SOURCES/0019-common-utils-test-vector.c-Add-vector-benchmarks.patch
View File

@ -1,4 +1,4 @@
From 6459704cc66f5fa0a2e6fc1e199458b77327fe52 Mon Sep 17 00:00:00 2001
From 4cc4e2cfc6202d522b3c4282bbaedf3d47de16bb Mon Sep 17 00:00:00 2001
From: Nir Soffer <nsoffer@redhat.com>
Date: Fri, 5 Nov 2021 20:36:42 +0200
Subject: [PATCH] common/utils/test-vector.c: Add vector benchmarks

2
SOURCES/0020-common-urils-vector.c-Optimize-vector-append.patch
View File

@ -1,4 +1,4 @@
From 5454ced7c8cfc2ba278c2635eecb9a5e4841e613 Mon Sep 17 00:00:00 2001
From 20029d23e2992f929a3d75b5dea64832f8684d20 Mon Sep 17 00:00:00 2001
From: Nir Soffer <nsoffer@redhat.com>
Date: Fri, 5 Nov 2021 22:16:26 +0200
Subject: [PATCH] common/urils/vector.c: Optimize vector append

2
SOURCES/0021-common-utils-vector-Rename-alloc-to-cap.patch
View File

@ -1,4 +1,4 @@
From 304f180b61fa28421b9901d2173a280e633b55c2 Mon Sep 17 00:00:00 2001
From a329cd90b350b17eac3227c0f30794015b710366 Mon Sep 17 00:00:00 2001
From: Nir Soffer <nsoffer@redhat.com>
Date: Fri, 5 Nov 2021 22:59:38 +0200
Subject: [PATCH] common/utils/vector: Rename `alloc` to `cap`

4
SOURCES/0022-common-utils-vector-Rename-size-to-len.patch
View File

@ -1,4 +1,4 @@
From a53a2234147543b04ee483aff7b9895c0d5082b5 Mon Sep 17 00:00:00 2001
From 932679668fc44839727ae350a1c2e94b26bcfc0a Mon Sep 17 00:00:00 2001
From: Nir Soffer <nsoffer@redhat.com>
Date: Sat, 6 Nov 2021 00:03:11 +0200
Subject: [PATCH] common/utils/vector: Rename `size` to `len`
@ -139,7 +139,7 @@ index 81fe4ed0..1675d21c 100644
return NULL;
}
diff --git a/common/regions/regions.h b/common/regions/regions.h
index 13fc41e2..34a398cd 100644
index 6dfd5d88..3d562316 100644
--- a/common/regions/regions.h
+++ b/common/regions/regions.h
@@ -84,17 +84,17 @@ extern void free_regions (regions *regions)

2
SOURCES/0023-podwrapper.pl.in-Use-short-commit-date.patch
View File

@ -1,4 +1,4 @@
From ece6d7e1a5827de17e86a20f7dae5f6f853d419b Mon Sep 17 00:00:00 2001
From 83f7325a317b4a3d670f1288715af8854627b0f6 Mon Sep 17 00:00:00 2001
From: Nir Soffer <nsoffer@redhat.com>
Date: Mon, 8 Nov 2021 19:47:57 +0200
Subject: [PATCH] podwrapper.pl.in: Use short commit date

2
SOURCES/0024-ocaml-Replace-noalloc-with-noalloc-annotation.patch
View File

@ -1,4 +1,4 @@
From 2955179919fc6233427b82d27ae61755b2b5e3d7 Mon Sep 17 00:00:00 2001
From 8e5248fda608adbce5b86d938855294001276906 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 9 Nov 2021 09:07:42 +0000
Subject: [PATCH] ocaml: Replace "noalloc" with [@@noalloc] annotation

39
SOURCES/0025-vddk-Drop-obsolete-documentation-related-to-thread-m.patch Normal file
View File

@ -0,0 +1,39 @@
From 6e85906311377b845f299d3a570e23db2809529c Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 27 Nov 2021 16:44:41 +0000
Subject: [PATCH] vddk: Drop obsolete documentation related to thread model
Since commit 1eecf15fc3 ("vddk: Implement parallel thread model") we
have implemented a parallel thread model in this plugin, and thread
handling is believed to be safe and in conformity with the VDDK
documentation. Remove obsolete documentation contradicting this.
Reported-by: Ming Xie
Fixes: commit 1eecf15fc3d8ea253ccec4f5883fdbb9aa6f8c2b
(cherry picked from commit 370ecb711c23f9143c933e13468e11d688d0d651)
---
plugins/vddk/nbdkit-vddk-plugin.pod | 8 --------
1 file changed, 8 deletions(-)
diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod b/plugins/vddk/nbdkit-vddk-plugin.pod
index ce82a734..acec0bd2 100644
--- a/plugins/vddk/nbdkit-vddk-plugin.pod
+++ b/plugins/vddk/nbdkit-vddk-plugin.pod
@@ -452,14 +452,6 @@ sector boundaries. This is because the VDDK Read and Write APIs only
take sector numbers. If your client needs finer granularity, you can
use L<nbdkit-blocksize-filter(1)> with the setting C<minblock=512>.
-=head2 Threads
-
-Handling threads in the VDDK API is complex and does not map well to
-any of the thread models offered by nbdkit (see
-L<nbdkit-plugin(3)/THREADS>). The plugin uses the nbdkit
-C<SERIALIZE_REQUESTS> model, but technically even this is not
-completely safe. This is a subject of future work.
-
=head2 Out of memory errors
In the verbose log you may see errors like:
--
2.31.1

32
SOURCES/0026-Revert-podwrapper.pl.in-Use-short-commit-date.patch Normal file
View File

@ -0,0 +1,32 @@
From b235076a7df96253aa41a7859c3ba54386bd3fef Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Sat, 20 Nov 2021 17:50:25 +0000
Subject: [PATCH] Revert "podwrapper.pl.in: Use short commit date"
This commit breaks man page output because there is an extra newline
after the date which wasn't being removed.
This reverts commit 7a1e79c6b5ca4adcef47fc0929d25d54610fc417.
(cherry picked from commit 750ad5972bb082d188f17f8f71ef1ec0c616c676)
---
podwrapper.pl.in | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/podwrapper.pl.in b/podwrapper.pl.in
index 63c1025a..abad578d 100755
--- a/podwrapper.pl.in
+++ b/podwrapper.pl.in
@@ -233,7 +233,8 @@ my $date;
my $filename = "$abs_top_srcdir/.git";
if (!$date && -d $filename) {
local $ENV{GIT_DIR} = $filename;
- $date = `git show -O/dev/null -s --format=%cs`;
+ $_ = `git show -O/dev/null -s --format=%ci`;
+ $date = $1 if /^(\d+-\d+-\d+)\s/;
}
if (!$date) {
my ($day, $month, $year) = (gmtime($ENV{SOURCE_DATE_EPOCH} || time))[3,4,5];
--
2.31.1

31
SOURCES/0027-Fix-podwrapper.pl.in-Use-short-commit-date.patch Normal file
View File

@ -0,0 +1,31 @@
From fb7749b3497a7aaf1b00e975af166af029209c01 Mon Sep 17 00:00:00 2001
From: Eric Blake <eblake@redhat.com>
Date: Tue, 30 Nov 2021 12:42:01 -0600
Subject: [PATCH] Fix "podwrapper.pl.in: Use short commit date"
This reverts commit 750ad5972bb082d188f17f8f71ef1ec0c616c676, then
fixes the broken newline as suggested in the thread at
https://listman.redhat.com/archives/libguestfs/2021-November/msg00275.html.
(cherry picked from commit 80036dbb0b8f9e0aab5994d80de6321c2a55c669)
---
podwrapper.pl.in | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/podwrapper.pl.in b/podwrapper.pl.in
index abad578d..6f256ba8 100755
--- a/podwrapper.pl.in
+++ b/podwrapper.pl.in
@@ -233,8 +233,7 @@ my $date;
my $filename = "$abs_top_srcdir/.git";
if (!$date && -d $filename) {
local $ENV{GIT_DIR} = $filename;
- $_ = `git show -O/dev/null -s --format=%ci`;
- $date = $1 if /^(\d+-\d+-\d+)\s/;
+ $date = `git show -O/dev/null -s --format=format:%cs`;
}
if (!$date) {
my ($day, $month, $year) = (gmtime($ENV{SOURCE_DATE_EPOCH} || time))[3,4,5];
--
2.31.1

154
SOURCES/0028-scripts-Add-simple-script-for-automating-VDDK-disk-c.patch Normal file
View File

@ -0,0 +1,154 @@
From 599f64f5885cf546f85e6c28d98b0444c18ad1fa Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 30 Nov 2021 17:56:02 +0000
Subject: [PATCH] scripts: Add simple script for automating VDDK disk
connections
It's tedious to work out how to do this by hand every time. Include a
developer script to make connecting to a guest disk easy.
(cherry picked from commit 44ee90ee01677032a14d5b71118b7af0651db3d5)
---
.gitignore | 1 +
Makefile.am | 2 +-
configure.ac | 2 +
scripts/vddk-open.sh.in | 89 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 93 insertions(+), 1 deletion(-)
create mode 100755 scripts/vddk-open.sh.in
diff --git a/.gitignore b/.gitignore
index 847b72dd..6565600f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -89,6 +89,7 @@ plugins/*/*.3
/plugins/S3/nbdkit-S3-plugin
/plugins/tmpdisk/default-command.c
/podwrapper.pl
+/scripts/vddk-open.sh
/server/libnbdkit.a
/server/local/nbdkit.pc
/server/nbdkit
diff --git a/Makefile.am b/Makefile.am
index 49f5d91c..6df5eba0 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -46,7 +46,7 @@ EXTRA_DIST = \
SECURITY \
$(NULL)
-CLEANFILES += html/*.html
+CLEANFILES += html/*.html scripts/*~
if !ENABLE_LIBFUZZER
# NB: This is not the real nbdkit binary. It's a wrapper that allows
diff --git a/configure.ac b/configure.ac
index 19514ef6..8a86326e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1253,6 +1253,8 @@ dnl Produce output files.
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([podwrapper.pl],
[chmod +x,-w podwrapper.pl])
+AC_CONFIG_FILES([scripts/vddk-open.sh],
+ [chmod +x,-w scripts/vddk-open.sh])
AC_CONFIG_FILES([common/protocol/generate-protostrings.sh],
[chmod +x,-w common/protocol/generate-protostrings.sh])
AC_CONFIG_FILES([Makefile
diff --git a/scripts/vddk-open.sh.in b/scripts/vddk-open.sh.in
new file mode 100755
index 00000000..218bc93c
--- /dev/null
+++ b/scripts/vddk-open.sh.in
@@ -0,0 +1,89 @@
+#!/bin/bash -
+# @configure_input@
+# Copyright (C) 2013-2021 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+# Open an nbdkit-vddk-plugin connection to the first disk of a guest
+# on a VMware ESXi server. This script automates the tedious bits of
+# getting the disk name, moref, etc. However please read the
+# nbdkit-vddk-plugin documentation as well.
+#
+# Usage:
+# scripts/vddk-open.sh SERVER GUEST -r -f -v libdir=/path/to/vmware-vix-disklib-distrib [...]
+#
+# where SERVER is the hostname or IP address of the ESXi server and
+# GUEST is the name of the guest.
+#
+# These two required parameters are followed by any extra nbdkit
+# parameters you want to use, such as VDDK libdir, flags, filters etc.
+#
+# Note that the script runs ./nbdkit (ie. the wrapper in the top build
+# directory).
+
+nbdkit="@abs_top_builddir@/nbdkit"
+
+server="$1"
+guest="$2"
+shift 2
+
+# Get the libvirt XML, filename and moref.
+echo -n "root password? "
+xml="$( virsh -c "esx://root@$server/?no_verify=1" dumpxml "$guest" )"
+echo
+
+file="$( echo "$xml" | grep '<source file=' | head -1 |
+ sed -e "s/.*'\(.*\)'.*/\1/" )"
+moref="$( echo "$xml" | grep '<vmware:moref' |
+ sed -e 's,.*>\(.*\)<.*,\1,' )"
+
+#echo file="$file"
+#echo moref="$moref"
+
+# Get the thumbprint.
+thumbprint="$( openssl s_client -connect "$server:443" </dev/null 2>/dev/null |
+ openssl x509 -in /dev/stdin -fingerprint -sha1 -noout 2>/dev/null |
+ grep '^sha1 Fingerprint=' |
+ sed 's/.*Fingerprint=\([A-F0-9:]\+\)/\1/' )"
+
+#echo thumbprint="$thumbprint"
+
+# Construct the nbdkit command line.
+declare -a args
+
+args[${#args[@]}]="$nbdkit"
+args[${#args[@]}]="vddk"
+args[${#args[@]}]="file=$file"
+args[${#args[@]}]="vm=moref=$moref"
+args[${#args[@]}]="server=$server"
+args[${#args[@]}]="thumbprint=$thumbprint"
+args[${#args[@]}]="user=root"
+
+echo "${args[@]}" "$@"
+"${args[@]}" "$@"
--
2.31.1

181
SOURCES/0029-file-Fix-implementation-of-cache-none-for-writes.patch Normal file
View File

@ -0,0 +1,181 @@
From 86b2baa7cd4a32246d447cbcedaaef3a15212ea4 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 7 Dec 2021 21:08:26 +0000
Subject: [PATCH] file: Fix implementation of cache=none for writes
When testing virt-v2v we found that cache=none had very pessimal
performance in its current implementation when writing. See:
https://github.com/libguestfs/virt-v2v/commit/ac59d3b2310511b1537d408b675b19ec9a5d384e
However we know of a much better implementation - the one in nbdcopy.
This commit copies that implementation (for writes only).
A simple test is to do:
$ ./nbdkit file out.img cache=none --run 'nbdcopy fedora-33.img $uri'
and then check the cache usage of the output file, which should be
around 0% (using https://github.com/Feh/nocache):
$ cachestats out.img
pages in cache: 409/1572864 (0.0%) [filesize=6291456.0K, pagesize=4K]
For modular virt-v2v doing a local disk to local disk conversion:
- before this change, without cache=none
virt-v2v took 93.7 seconds, 19.1% pages cached in output file
- before this change, enabling cache=none
virt-v2v took 125.4 seconds, 0.0% pages cached in output file
^^^ this is the bad case which caused the investigation
- after this change, without cache=none
virt-v2v took 93.2 seconds, 19.1% pages cached in output file
- after this change, enabling cache=none
virt-v2v took 97.9 seconds, 0.1% pages cached in output file
I tried to adjust NR_WINDOWS to find an optimum. Increasing it made
no difference in performance but predictably caused a slight increase
in cached pages. Reducing it slowed performance slightly. So I
conclude that 8 is about right, but it probably depends on the
hardware.
(cherry picked from commit a956e2e75d6c88eeefecd967505667c9f176e3af)
---
plugins/file/file.c | 79 +++++++++++++++++++++++++----
plugins/file/nbdkit-file-plugin.pod | 3 ++
2 files changed, 72 insertions(+), 10 deletions(-)
diff --git a/plugins/file/file.c b/plugins/file/file.c
index 35270a24..caf24b2c 100644
--- a/plugins/file/file.c
+++ b/plugins/file/file.c
@@ -85,6 +85,69 @@ static int fadvise_mode =
/* cache mode */
static enum { cache_default, cache_none } cache_mode = cache_default;
+/* Define EVICT_WRITES if we are going to evict the page cache
+ * (cache=none) after writing. This is only known to work on Linux.
+ */
+#ifdef __linux__
+#define EVICT_WRITES 1
+#endif
+
+#ifdef EVICT_WRITES
+/* Queue writes so they will be evicted from the cache. See
+ * libnbd.git copy/file-ops.c for the rationale behind this.
+ */
+#define NR_WINDOWS 8
+
+struct write_window {
+ int fd;
+ uint64_t offset;
+ size_t len;
+};
+
+static pthread_mutex_t window_lock = PTHREAD_MUTEX_INITIALIZER;
+static struct write_window window[NR_WINDOWS];
+
+static void
+evict_writes (int fd, uint64_t offset, size_t len)
+{
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&window_lock);
+
+ /* Evict the oldest window from the page cache. */
+ if (window[0].len > 0) {
+ sync_file_range (window[0].fd, window[0].offset, window[0].len,
+ SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE|
+ SYNC_FILE_RANGE_WAIT_AFTER);
+ posix_fadvise (window[0].fd, window[0].offset, window[0].len,
+ POSIX_FADV_DONTNEED);
+ }
+
+ /* Move the Nth window to N-1. */
+ memmove (&window[0], &window[1], sizeof window[0] * (NR_WINDOWS-1));
+
+ /* Set up the current window and tell Linux to start writing it out
+ * to disk (asynchronously).
+ */
+ sync_file_range (fd, offset, len, SYNC_FILE_RANGE_WRITE);
+ window[NR_WINDOWS-1].fd = fd;
+ window[NR_WINDOWS-1].offset = offset;
+ window[NR_WINDOWS-1].len = len;
+}
+
+/* When we close the handle we must remove any windows which are still
+ * associated. They missed the boat, oh well :-(
+ */
+static void
+remove_fd_from_window (int fd)
+{
+ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&window_lock);
+ size_t i;
+
+ for (i = 0; i < NR_WINDOWS; ++i)
+ if (window[i].len > 0 && window[i].fd == fd)
+ window[i].len = 0;
+}
+#endif /* EVICT_WRITES */
+
/* Any callbacks using lseek must be protected by this lock. */
static pthread_mutex_t lseek_lock = PTHREAD_MUTEX_INITIALIZER;
@@ -431,6 +494,9 @@ file_close (void *handle)
{
struct handle *h = handle;
+#ifdef EVICT_WRITES
+ remove_fd_from_window (h->fd);
+#endif
close (h->fd);
free (h);
}
@@ -583,15 +649,9 @@ file_pwrite (void *handle, const void *buf, uint32_t count, uint64_t offset,
{
struct handle *h = handle;
-#if defined (HAVE_POSIX_FADVISE) && defined (POSIX_FADV_DONTNEED)
+#if EVICT_WRITES
uint32_t orig_count = count;
uint64_t orig_offset = offset;
-
- /* If cache=none we want to force pages we have just written to the
- * file to be flushed to disk so we can immediately evict them from
- * the page cache.
- */
- if (cache_mode == cache_none) flags |= NBDKIT_FLAG_FUA;
#endif
while (count > 0) {
@@ -608,10 +668,9 @@ file_pwrite (void *handle, const void *buf, uint32_t count, uint64_t offset,
if ((flags & NBDKIT_FLAG_FUA) && file_flush (handle, 0) == -1)
return -1;
-#if defined (HAVE_POSIX_FADVISE) && defined (POSIX_FADV_DONTNEED)
- /* On Linux this will evict the pages we just wrote from the page cache. */
+#if EVICT_WRITES
if (cache_mode == cache_none)
- posix_fadvise (h->fd, orig_offset, orig_count, POSIX_FADV_DONTNEED);
+ evict_writes (h->fd, orig_offset, orig_count);
#endif
return 0;
diff --git a/plugins/file/nbdkit-file-plugin.pod b/plugins/file/nbdkit-file-plugin.pod
index 0ac0ee53..f8f0e198 100644
--- a/plugins/file/nbdkit-file-plugin.pod
+++ b/plugins/file/nbdkit-file-plugin.pod
@@ -117,6 +117,9 @@ cache:
nbdkit file disk.img fadvise=sequential cache=none
+Only use fadvise=sequential if reading, and the reads are mainly
+sequential.
+
=head2 Files on tmpfs
If you want to expose a file that resides on a file system known to
--
2.31.1

17
SOURCES/nbdkit-1.28.2.tar.gz.sig
View File

@ -1,17 +0,0 @@
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEE93dPsa0HSn6Mh2fqkXOPc+G3aKAFAmGKYx0RHHJpY2hAYW5u
ZXhpYS5vcmcACgkQkXOPc+G3aKAw7g//Q/42avSvu3DZQXPCl772S9K7erYzw1LK
PzffbDKWwsR4YN+m4w94gFjb6n2NZOnSBVpDYofkt5mzHM9dwYTlCJdS+J4SxV8e
s23+RQjgX0fQMDv4qfGE7CUNQXX6CLRWUlXSu4dMMbsxbF8y18VHl+eLIN7U15Ec
8E2lwcthQ8ipwz0xuxKE1GUNXzWsJ1SHBaCmo8HH1x1Rs+mWu0QjwFllhTc4+gq9
5yrdZuViv96DxIsk3u7RByqrTJYZi2tK9dU75dVeXI6o74b1WUrXkwqW8uqIR4m5
hZry1+vXPzWIO4G5o1gika+C4xvYuyIShNl/c/aWJsA0ku4fDtUxAkjC/k78D4yR
0FQbNfztiCFgvQNS3UH6M0d63S4GE76m7HgwI7SkUtczlnPl2AhzSjjfXZ5DrHH9
S8SgmgwP4qfwszY6Xs7ISFofRV5YBgMhSIiJc/5yYRLtZZCcRWNvv/OWYNTGdVBp
rRYQRvnhe2VbaaVexOn1Fkzcv/FNeD+96bsTp1B1SailUUFrTOpRZ0yIOxfZZZO/
JcroGuFvq0BywP1U6Rq2PI1IHD4I6x3QLYIfNBP076Rc17BhcXyn2Ei18XkRo+xd
iFTCDpe0zlSCl3tLL7Hvp2EtetYkpiQKHA59g7RoSVQDX/RAml3VFDvwZ2rNBeCL
yegMoVPEmc0=
=c3GR
-----END PGP SIGNATURE-----

17
SOURCES/nbdkit-1.28.3.tar.gz.sig Normal file
View File

@ -0,0 +1,17 @@
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEE93dPsa0HSn6Mh2fqkXOPc+G3aKAFAmGvLUARHHJpY2hAYW5u
ZXhpYS5vcmcACgkQkXOPc+G3aKCUVA//RZwRNUPITvdw9CG8gaLAyvkQfhmHWR+X
AlEHW3MmpG8esuX8Q3DEGwgub9q029J5kvnbKtZsKuGF78Fi1XPvbzZPCLa3M+VF
kUABAOIht3Hg97h8dxURVUjv8QbG871buEQy3sENaGPAzI0sJHiO7VnykG0N/5S8
YWbEh7i75KRMy3OqHQ9mNb7kq8OKM2h7LmJzK4QItvPX1/b5UkxC6w9h4N4fEbc1
oJb9FXcS0TP/vYIZZwhO4FTmChaTGXRYFmfqe5b5zzeacqec3ufHpTevFCwxuVSc
zHoVxYN7zGdLheYeUG2MMbbs8JR7Ia8hBHoP8tG707Y+IJ2yFYubs8SupbA4oJKj
RfZD8jOIErdn56Qh3r9V0ExuGWWkhFYdjUBaEACMRm366d5GdPkvw5VEW1/Tf1Id
JtRroke50G2hNrt+nrnGO2lU9G/xdJ1gz9+aMBqXMtk1MIBedtvDnDthEfEHNUwY
dJGhoFwKBuWUI1ha2DX9bk7pb4jS29+eiIFbMv9Gw748Etfmr2pcNQEGIcauGz9e
6rAtg5Afk8Z4vQUK0IN5Ex1+JnF1OIjTHFQSuCTVVyYhhVn7nWQWKWlwgXT/A4F4
ciu6WlBnNgD86Kq0HJ9fSjWE6K3kKkUUHnDYYtnmt3CqzNeoKCZMt4VHMFMOMfR7
LPKn+zwrGb8=
=SDiY
-----END PGP SIGNATURE-----

19
SPECS/nbdkit.spec
View File

@ -50,8 +50,8 @@ ExclusiveArch: x86_64
%global source_directory 1.28-stable
Name: nbdkit
Version: 1.28.2
Release: 1%{?dist}
Version: 1.28.3
Release: 2%{?dist}
Summary: NBD server
License: BSD
@ -100,6 +100,11 @@ Patch0021: 0021-common-utils-vector-Rename-alloc-to-cap.patch
Patch0022: 0022-common-utils-vector-Rename-size-to-len.patch
Patch0023: 0023-podwrapper.pl.in-Use-short-commit-date.patch
Patch0024: 0024-ocaml-Replace-noalloc-with-noalloc-annotation.patch
Patch0025: 0025-vddk-Drop-obsolete-documentation-related-to-thread-m.patch
Patch0026: 0026-Revert-podwrapper.pl.in-Use-short-commit-date.patch
Patch0027: 0027-Fix-podwrapper.pl.in-Use-short-commit-date.patch
Patch0028: 0028-scripts-Add-simple-script-for-automating-VDDK-disk-c.patch
Patch0029: 0029-file-Fix-implementation-of-cache-none-for-writes.patch
BuildRequires: make
%if 0%{patches_touch_autotools}
@ -1213,15 +1218,19 @@ export LIBGUESTFS_TRACE=1
%changelog
* Tue Nov 09 2021 Richard W.M. Jones <rjones@redhat.com> - 1.28.2-1
* Wed Dec 08 2021 Richard W.M. Jones <rjones@redhat.com> - 1.28.3-2
- Rebase to new stable branch version 1.28.3
resolves: rhbz#2011709
- Move nbdkit-null-plugin to nbdkit-server package
resolves: rhbz#2021154
- Add asynchronous support in nbdkit-vddk-plugin
- Drop obsolete documentation related to thread model in vddk plugin
resolves: rhbz#2018463
- Rebase to new stable branch version 1.28.2
resolves: rhbz#2011709
- Switch to xorriso (instead of genisoimage)
- Distribute README.VDDK in nbdkit-vddk-plugin subpackage
- Fix nbdkit-cow-filter cow-block-size=4096
- file: Fix implementation of cache=none for writes
resolves: rhbz#2029751
* Thu Aug 19 2021 Richard W.M. Jones <rjones@redhat.com> - 1.26.5-1
- Rebase along stable branch to 1.26.5