rpms/libvpx
6
0
Fork 0
Code Issues Pull Requests Releases Activity

Add patch for double free

Browse Source 
Resolves: RHEL-93910
This commit is contained in:
Wim Taymans 2025-06-03 17:15:42 +02:00
parent 7c4098a055
commit 1620d84356
2 changed files with 70 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
0001-vpx_codec_enc_init_multi-fix-double-free-on-init-fai.patch Normal file
View File

@ -0,0 +1,63 @@
From 1f96cfd558b7780c87b4709b3e589b66850a0b54 Mon Sep 17 00:00:00 2001
From: James Zern <jzern@google.com>
Date: Wed, 30 Apr 2025 19:28:48 -0700
Subject: [PATCH] vpx_codec_enc_init_multi: fix double free on init failure
In `vp8e_init()`, the encoder would take ownership of
`mr_cfg.mr_low_res_mode_info` even if `vp8_create_compressor()` failed.
This caused confusion at the call site as other failures in
`vp8e_init()` did not result in ownership transfer and the caller would
free the memory. In the case of `vp8_create_compressor()` failure both
the caller and `vpx_codec_destroy()` would free the memory, causing a
crash. `mr_*` related variables are now cleared on failure to prevent
this situation.
Bug: webm:413411335
Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1962421
Change-Id: Ie951d42b9029a586bf9059b650bd8863db9f9ffc
(cherry picked from commit 1c758781c428c0e895645b95b8ff1512b6bdcecb)
---
vp8/vp8_cx_iface.c | 12 +++++++++++-
vpx/src/vpx_encoder.c | 3 +++
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/vp8/vp8_cx_iface.c b/vp8/vp8_cx_iface.c
index 1160f51d6..c1f866ae3 100644
--- a/vp8/vp8_cx_iface.c
+++ b/vp8/vp8_cx_iface.c
@@ -674,7 +674,17 @@ static vpx_codec_err_t vp8e_init(vpx_codec_ctx_t *ctx,
set_vp8e_config(&priv->oxcf, priv->cfg, priv->vp8_cfg, mr_cfg);
priv->cpi = vp8_create_compressor(&priv->oxcf);
- if (!priv->cpi) res = VPX_CODEC_MEM_ERROR;
+ if (!priv->cpi) {
+#if CONFIG_MULTI_RES_ENCODING
+ // Release ownership of mr_cfg->mr_low_res_mode_info on failure. This
+ // prevents ownership confusion with the caller and avoids a double
+ // free when vpx_codec_destroy() is called on this instance.
+ priv->oxcf.mr_total_resolutions = 0;
+ priv->oxcf.mr_encoder_id = 0;
+ priv->oxcf.mr_low_res_mode_info = NULL;
+#endif
+ res = VPX_CODEC_MEM_ERROR;
+ }
}
}
diff --git a/vpx/src/vpx_encoder.c b/vpx/src/vpx_encoder.c
index f636b54a3..2e359d865 100644
--- a/vpx/src/vpx_encoder.c
+++ b/vpx/src/vpx_encoder.c
@@ -109,6 +109,9 @@ vpx_codec_err_t vpx_codec_enc_init_multi_ver(
ctx->priv = NULL;
ctx->init_flags = flags;
ctx->config.enc = cfg;
+ // ctx takes ownership of mr_cfg.mr_low_res_mode_info if and only if
+ // this call succeeds. The first ctx entry in the array is
+ // responsible for freeing the memory.
res = ctx->iface->init(ctx, &mr_cfg);
}
--
2.49.0

8
libvpx.spec
View File

@ -6,7 +6,7 @@
Name: libvpx
Summary: VP8/VP9 Video Codec SDK
Version: 1.9.0
Release: 8%{?dist}
Release: 9%{?dist}
License: BSD
#Source0: http://downloads.webmproject.org/releases/webm/%{name}-%{version}.tar.bz2
Source0: https://github.com/webmproject/libvpx/archive/v%{version}.tar.gz
@ -28,6 +28,7 @@ Patch2: 0001-VP8-disallow-thread-count-changes.patch
Patch3: 0001-Fix-integer-overflows-in-calc-of-stride_in_bytes.patch
Patch4: 0002-Apply-stride_align-to-byte-count-not-pixel-count.patch
Patch5: 0003-Fix-a-bug-in-alloc_size-for-high-bit-depths.patch
Patch6: 0001-vpx_codec_enc_init_multi-fix-double-free-on-init-fai.patch
%description
libvpx provides the VP8/VP9 SDK, which allows you to integrate your applications
@ -58,6 +59,7 @@ and decoder.
%patch3 -p1 -b .0003
%patch4 -p1 -b .0004
%patch5 -p1 -b .0005
%patch6 -p1 -b .0006
%build
# This package fails to build with LTO due to undefined symbols. LTO
@ -248,6 +250,10 @@ rm -rf %{buildroot}%{_prefix}/src
%{_bindir}/*
%changelog
* Tue Jun 03 2025 Wim Taymans <wtaymans@redhat.com> - 1.9.0-9
- Add patch for double free
Resolves: RHEL-93910
* Fri Jul 5 2024 Wim Taymans <wtaymans@redhat.com> - 1.9.0-8
- Add patch to fix integer overflows.
- Disable LTO to fix build