kernel/1401-drm-gem-fix-inconsistent-plane-dimension-calculation-in-drm.patch

64 lines
2.9 KiB
Diff

From 97b8798760568e666030571c7fff4b2d93ce1f7b Mon Sep 17 00:00:00 2001
From: CKI Backport Bot <cki-ci-bot+cki-gitlab-backport-bot@redhat.com>
Date: Thu, 28 May 2026 13:34:21 +0000
Subject: [PATCH] drm/gem: Fix inconsistent plane dimension calculation in
drm_gem_fb_init_with_funcs()
JIRA: https://redhat.atlassian.net/browse/RHEL-179911
CVE: CVE-2026-46209
Backported from tree(s): linux
commit 3d4c2268bd7243c3780fe32bf24ff876da272acf
Author: Ashutosh Desai <ashutoshdesai993@gmail.com>
Date: Mon Apr 20 01:36:37 2026 +0000
drm/gem: Fix inconsistent plane dimension calculation in drm_gem_fb_init_with_funcs()
drm_gem_fb_init_with_funcs() computes sub-sampled plane dimensions
using plain integer division:
unsigned int width = mode_cmd->width / (i ? info->hsub : 1);
unsigned int height = mode_cmd->height / (i ? info->vsub : 1);
However, the ioctl-level framebuffer_check() in drm_framebuffer.c uses
drm_format_info_plane_width/height() which round up dimensions via
DIV_ROUND_UP(). This inconsistency corrupts the subsequent GEM object
size check for certain pixel format and dimension combinations.
For example, with NV12 (vsub=2) and a 1-pixel-tall framebuffer the
GEM size validation path sees height=0 instead of height=1. The
expression (height - 1) then wraps to UINT_MAX as an unsigned int,
causing min_size to overflow and wrap back to a small value. A tiny
GEM object therefore passes the size guard, yet when the GPU accesses
the chroma plane it will read or write memory beyond the object's
bounds.
Fix by replacing the open-coded divisions with drm_format_info_plane_width()
and drm_format_info_plane_height(), which use DIV_ROUND_UP() and match
the calculation already used in framebuffer_check().
Fixes: 4c3dbb2c312c ("drm: Add GEM backed framebuffer library")
Cc: stable@vger.kernel.org # v4.14+
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Ashutosh Desai <ashutoshdesai993@gmail.com>
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patch.msgid.link/20260420013637.457751-1-ashutoshdesai993@gmail.com
Signed-off-by: CKI Backport Bot <cki-ci-bot+cki-gitlab-backport-bot@redhat.com>
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index 4bc89d3..daa5471 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -171,8 +171,8 @@ int drm_gem_fb_init_with_funcs(struct drm_device *dev,
}
for (i = 0; i < info->num_planes; i++) {
- unsigned int width = mode_cmd->width / (i ? info->hsub : 1);
- unsigned int height = mode_cmd->height / (i ? info->vsub : 1);
+ unsigned int width = drm_format_info_plane_width(info, mode_cmd->width, i);
+ unsigned int height = drm_format_info_plane_height(info, mode_cmd->height, i);
unsigned int min_size;
objs[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]);