121 lines
5.8 KiB
Diff
121 lines
5.8 KiB
Diff
From: Jonathan Wright <jonathan@almalinux.org>
|
|
Date: Wed, 5 Aug 2026 00:00:00 +0000
|
|
Subject: gve: harden modifiable ring size support for the EL9 gve architecture
|
|
|
|
Follow-up hardening for the modifiable-ring-size backport (patches 1107
|
|
and 1108), addressing three ways the older EL9 gve architecture differs
|
|
from the EL10/upstream driver the feature was designed against. Found in
|
|
review; none affect the DQO resize paths already validated on GCP.
|
|
|
|
1) Do not enable modifiable ring sizes for GQI queue formats.
|
|
In this kernel's gve, the GQI branches of
|
|
gve_adminq_create_{tx,rx}_queue carry no ring size (only the DQO
|
|
branches send one), so after a resize the device would keep using its
|
|
probe-time default ring size against rings the driver re-sized:
|
|
device and driver would index the same DMA memory modulo different
|
|
sizes. Shrinking the RX ring would additionally let the device DMA
|
|
past the smaller allocation, and the GQI datapath sizes its data ring
|
|
and mask from the fixed rx_data_slot_cnt (with a NAPI livelock
|
|
possible on GQI-RDA once rx_desc_cnt >= 2 * data slots). Gate the
|
|
feature on !gve_is_gqi(): GQI keeps the pre-feature fixed-ring
|
|
behavior and ethtool -G returns -EOPNOTSUPP.
|
|
|
|
2) Key gve_adjust_ring_sizes() on netif_running(), not
|
|
netif_carrier_ok(), matching the EL10 implementation. The rings are
|
|
allocated whenever the interface is administratively up, even while
|
|
the link is down; taking the "set for next up" shortcut in that state
|
|
would change priv->{tx,rx}_desc_cnt under allocated rings, and the
|
|
GQI RX free path sizes dma_free_coherent() from priv->rx_desc_cnt.
|
|
|
|
3) Clamp the advertised ring size range for DQO-RDA. In this kernel the
|
|
DQO-RDA TX completion ring and RX buffer ring keep the fixed sizes
|
|
from the DQO-RDA device option and do not scale with the descriptor
|
|
rings (upstream removed that coupling before modifiable ring sizes
|
|
were introduced). Growing the TX ring beyond tx_comp_ring_entries
|
|
could overrun the TX completion queue with report-event completions
|
|
(num_pending_packets reserves complq_size/32 slots for REs, but REs
|
|
scale with posted descriptors on the grown ring) or have queue
|
|
creation rejected; shrinking the RX descriptor/completion ring below
|
|
rx_buff_ring_entries could overflow the RX completion queue. Clamp
|
|
max_tx_desc_cnt and min_rx_desc_cnt so such sizes cannot be
|
|
requested. DQO-QPL is unaffected (its completion/buffer rings track
|
|
the descriptor ring size).
|
|
Signed-off-by: Jonathan Wright <jonathan@almalinux.org>
|
|
---
|
|
diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
|
|
index b7f86bc..eef7cca 100644
|
|
--- a/drivers/net/ethernet/google/gve/gve_adminq.c
|
|
+++ b/drivers/net/ethernet/google/gve/gve_adminq.c
|
|
@@ -807,8 +807,18 @@ static void gve_enable_supported_features(struct gve_priv *priv,
|
|
priv->dev->max_mtu = be16_to_cpu(dev_op_jumbo_frames->max_mtu);
|
|
}
|
|
|
|
- /* Read and store ring size ranges given by device */
|
|
- if (dev_op_modify_ring &&
|
|
+ /* Read and store ring size ranges given by device.
|
|
+ *
|
|
+ * Modifiable ring sizes are supported for DQO queue formats only.
|
|
+ * In this kernel's gve, the GQI branches of
|
|
+ * gve_adminq_create_{tx,rx}_queue do not carry a ring size, so after
|
|
+ * a resize the device would keep using its default ring size against
|
|
+ * rings the driver re-sized, desynchronizing device and driver views
|
|
+ * of the rings. The GQI datapath (rx_data_slot_cnt data ring, TX
|
|
+ * FIFO) also assumes the probe-time fixed ring sizes. Keep GQI at
|
|
+ * fixed ring sizes.
|
|
+ */
|
|
+ if (dev_op_modify_ring && !gve_is_gqi(priv) &&
|
|
(supported_features_mask & GVE_SUP_MODIFY_RING_MASK)) {
|
|
priv->modify_ring_size_enabled = true;
|
|
|
|
@@ -823,6 +833,27 @@ static void gve_enable_supported_features(struct gve_priv *priv,
|
|
priv->min_rx_desc_cnt = be16_to_cpu(dev_op_modify_ring->min_rx_ring_size);
|
|
priv->min_tx_desc_cnt = be16_to_cpu(dev_op_modify_ring->min_tx_ring_size);
|
|
}
|
|
+
|
|
+ /* In this kernel's gve, the DQO-RDA TX completion ring and RX
|
|
+ * buffer ring keep the fixed sizes provided in the DQO-RDA
|
|
+ * device option; they do not scale with the descriptor rings
|
|
+ * (upstream removed that coupling before modifiable ring
|
|
+ * sizes were introduced). Clamp the advertised range so a
|
|
+ * resize cannot create a descriptor ring the fixed rings
|
|
+ * cannot absorb:
|
|
+ * - growing the TX ring beyond tx_comp_ring_entries could
|
|
+ * overrun the TX completion queue with report-event
|
|
+ * completions (or be rejected at queue creation);
|
|
+ * - shrinking the RX ring (the RX completion queue) below
|
|
+ * rx_buff_ring_entries could overflow the RX completion
|
|
+ * queue.
|
|
+ */
|
|
+ if (priv->queue_format == GVE_DQO_RDA_FORMAT) {
|
|
+ priv->max_tx_desc_cnt = min(priv->max_tx_desc_cnt,
|
|
+ priv->options_dqo_rda.tx_comp_ring_entries);
|
|
+ priv->min_rx_desc_cnt = max(priv->min_rx_desc_cnt,
|
|
+ priv->options_dqo_rda.rx_buff_ring_entries);
|
|
+ }
|
|
}
|
|
}
|
|
|
|
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
|
|
index e939750..516f222 100644
|
|
--- a/drivers/net/ethernet/google/gve/gve_main.c
|
|
+++ b/drivers/net/ethernet/google/gve/gve_main.c
|
|
@@ -1786,7 +1786,14 @@ int gve_adjust_ring_sizes(struct gve_priv *priv,
|
|
{
|
|
int err;
|
|
|
|
- if (netif_carrier_ok(priv->dev)) {
|
|
+ /* Key on the administrative state (netif_running), not the link
|
|
+ * state: the rings are allocated whenever the interface is up, even
|
|
+ * while the carrier is down. Changing the descriptor counts without
|
|
+ * reallocating the rings would desynchronize them from
|
|
+ * priv->{tx,rx}_desc_cnt, which the GQI free path uses to size
|
|
+ * dma_free_coherent().
|
|
+ */
|
|
+ if (netif_running(priv->dev)) {
|
|
/* To make this process as simple as possible we teardown the
|
|
* device, set the new ring sizes, and then bring the device
|
|
* up again.
|
|
--
|
|
2.43.0
|
|
|