From 65a9c8ed09f4cd04ae2176b48c4c9c1f69b08399 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" Date: Tue, 29 Apr 2025 10:42:46 +0100 Subject: [PATCH] input: Document my findings with nbdkit-noextents-filter This just adds a comment, so makes no change. (cherry picked from commit 29fae7985eda1d1cf3e176f123a16b60cac2db53) --- input/nbdkit_vddk.ml | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml index b79c28cc..3ba00d55 100644 --- a/input/nbdkit_vddk.ml +++ b/input/nbdkit_vddk.ml @@ -140,6 +140,38 @@ See also the virt-v2v-input-vmware(1) manual.") libNN *) Nbdkit.add_filter_if_available cmd "retry"; + (* VDDK's QueryAllocatedBlocks API is infamously slow. It appears + * to block all other requests while it is running. This API is + * also only called during the copy phase, not during conversion + * (or if it is, extremely rarely). + * + * If fstrim was successful, then trimmed blocks are stored in + * the COW filter (see below), and so requests for extents stop + * at that layer. However for areas of the disk that fstrim + * thinks contain data, we still have to go through to VDDK to + * fetch extents. + * + * We could therefore add nbdkit-noextents-filter here (below COW, + * above VDDK plugin) which stops extents requests from going + * to VDDK, which would stop QueryAllocatedBlocks ever being + * called. In my testing this is a moderate performance win. + * + * However ... in the case where fstrim failed, or for filesystems + * or partitions on the disk that we don't understand, doing this + * would mean that those are copied completely, as there would be + * no extent data (nbdcopy will still sparsify them on the target, + * but we'd have to copy all the bits from VMware). Because + * here we don't know if this is the case, be conservative and + * actually don't use this filter. + * + * If used, this filter should be close to the plugin and MUST + * be below the COW filter. + * + * XXX Add some kind of debugging flag so we can test how this + * works in production. + *) + (*Nbdkit.add_filter_if_available cmd "noextents";*) + (* Split very large requests to avoid out of memory errors on the * server. Since we're using this filter, also add minblock=512 * although it will make no difference.