Backport fix for CVE-2022-44638

pixman < 0.42.2 is affected by an out-of-bounds write error in the
`rasterize_edges_8()` function due to an integer overflow in the
`pixman_sample_floor_y()` function.

For more information please check the upstream bug report [1].

This patch backports commit a1f88e842e02 ("Avoid integer overflow
leading to out-of-bounds write") [2] to fix CVE-2022-44638.

In order to test and validate the fix, a reproducer can be found in the
original bug report [3] and compiled with the following command:

    $ gcc -o poc poc.c -ldl -fsanitize=address \
      $(pkg-config --cflags --libs pixman-1)

[1] https://gitlab.freedesktop.org/pixman/pixman/-/issues/63
[2] a1f88e842e
[3] https://gitlab.freedesktop.org/pixman/pixman/uploads/a55795e36afc03445ed838b0fda786f9/poc.c
Resolves: https://issues.redhat.com/browse/RHEL-11645

0001-Avoid-integer-overflow-leading-to-out-of-bounds-writ.patch

@@ -0,0 +1,29 @@
+From a1f88e842e0216a5b4df1ab023caebe33c101395 Mon Sep 17 00:00:00 2001
+From: Matt Turner <mattst88@gmail.com>
+Date: Wed, 2 Nov 2022 12:07:32 -0400
+Subject: [PATCH] Avoid integer overflow leading to out-of-bounds write
+
+Thanks to Maddie Stone and Google's Project Zero for discovering this
+issue, providing a proof-of-concept, and a great analysis.
+
+Closes: https://gitlab.freedesktop.org/pixman/pixman/-/issues/63
+---
+ pixman/pixman-trap.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/pixman/pixman-trap.c b/pixman/pixman-trap.c
+index 91766fd..7560405 100644
+--- a/pixman/pixman-trap.c
++++ b/pixman/pixman-trap.c
+@@ -74,7 +74,7 @@ pixman_sample_floor_y (pixman_fixed_t y,
+ 
+     if (f < Y_FRAC_FIRST (n))
+     {
+-	if (pixman_fixed_to_int (i) == 0x8000)
++	if (pixman_fixed_to_int (i) == 0xffff8000)
+ 	{
+ 	    f = 0; /* saturate */
+ 	}
+-- 
+2.41.0
+

pixman.spec

@@ -3,7 +3,7 @@
 
 Name:           pixman
 Version:        0.40.0
-Release:        5%{?dist}
+Release:        6%{?dist}
 Summary:        Pixel manipulation library
 
 License:        MIT
@@ -15,6 +15,8 @@ URL:            https://gitlab.freedesktop.org/pixman/pixman
 Source0:        https://xorg.freedesktop.org/archive/individual/lib/%{name}-%{version}.tar.xz
 Source1:        make-pixman-snapshot.sh
 
+Patch0: 0001-Avoid-integer-overflow-leading-to-out-of-bounds-writ.patch
+
 BuildRequires:  gcc
 BuildRequires:  meson
 
@@ -64,6 +66,9 @@ sed -i 's/120/600/' test/meson.build
 %{_libdir}/pkgconfig/pixman-1.pc
 
 %changelog
+* Wed Oct 04 2023 José Expósito <jexposit@redhat.com> - 0.40.0-6
+- Backport fix for CVE-2022-44638
+
 * Mon Aug 09 2021 Mohan Boddu <mboddu@redhat.com> - 0.40.0-5
 - Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
   Related: rhbz#1991688