CVE-2024-22211: Check codec resolution for overflow

Resolves: RHEL-4290
Resolves: RHEL-4292
Resolves: RHEL-4296
Resolves: RHEL-4298
Resolves: RHEL-4300
Resolves: RHEL-4302
Resolves: RHEL-4304
Resolves: RHEL-4306
Resolves: RHEL-4308
Resolves: RHEL-4310
Resolves: RHEL-4312
Resolves: RHEL-10060

aeac3040cc99eeaff1e1171a822114c857b9dca9.patch

@@ -0,0 +1,32 @@
+From aeac3040cc99eeaff1e1171a822114c857b9dca9 Mon Sep 17 00:00:00 2001
+From: Armin Novak <anovak@thincast.com>
+Date: Sat, 13 Jan 2024 21:01:55 +0100
+Subject: [PATCH] [codec,planar] check resolution for overflow
+
+If the codec resolution is too large return an error as the internal
+buffers would otherwise overflow.
+
+(cherry picked from commit 44edab1deae4f8c901c00a00683f888cef36d853)
+---
+ libfreerdp/codec/planar.c | 8 +++++++-
+ 1 file changed, 7 insertions(+), 1 deletion(-)
+
+diff --git a/libfreerdp/codec/planar.c b/libfreerdp/codec/planar.c
+index b4815a632309..0a5ec581c6cc 100644
+--- a/libfreerdp/codec/planar.c
++++ b/libfreerdp/codec/planar.c
+@@ -1496,7 +1496,13 @@ BOOL freerdp_bitmap_planar_context_reset(BITMAP_PLANAR_CONTEXT* context, UINT32
+ 	context->bgr = FALSE;
+ 	context->maxWidth = PLANAR_ALIGN(width, 4);
+ 	context->maxHeight = PLANAR_ALIGN(height, 4);
+-	context->maxPlaneSize = context->maxWidth * context->maxHeight;
++	const UINT64 tmp = (UINT64)context->maxWidth * context->maxHeight;
++	if (tmp > UINT32_MAX)
++		return FALSE;
++	context->maxPlaneSize = tmp;
++
++	if (context->maxWidth > UINT32_MAX / 4)
++		return FALSE;
+ 	context->nTempStep = context->maxWidth * 4;
+ 	free(context->planesBuffer);
+ 	free(context->pTempData);

freerdp.spec

@@ -27,7 +27,7 @@
 
 Name:           freerdp
 Version:        2.11.2
-Release:        1%{?dist}
+Release:        2%{?dist}
 Epoch:          2
 Summary:        Free implementation of the Remote Desktop Protocol (RDP)
 License:        ASL 2.0
@@ -35,6 +35,9 @@ URL:            http://www.freerdp.com/
 
 Source0:        https://github.com/FreeRDP/FreeRDP/archive/%{version}/FreeRDP-%{version}.tar.gz
 
+# https://issues.redhat.com/browse/RHEL-22244
+Patch:          aeac3040cc99eeaff1e1171a822114c857b9dca9.patch
+
 BuildRequires:  gcc
 BuildRequires:  gcc-c++
 BuildRequires:  alsa-lib-devel
@@ -292,6 +295,9 @@ find %{buildroot} -name "*.a" -delete
 %{_libdir}/pkgconfig/winpr-tools2.pc
 
 %changelog
+* Tue Mar 12 2024 Ondrej Holy <oholy@redhat.com> - 2:2.11.2-2
+- CVE-2024-22211: Check codec resolution for overflow (RHEL-22244)
+
 * Fri Nov 10 2023 Ondrej Holy <oholy@redhat.com> - 2:2.11.2-1
 - Update to 2.11.2 (RHEL-4290, RHEL-4292, RHEL-4296, RHEL-4298, RHEL-4300,
   RHEL-4302, RHEL-4304, RHEL-4306, RHEL-4308, RHEL-4310, RHEL-4312,