Compare commits

...

No commits in common. "imports/c8s/python3.11-3.11.1-2.el8" and "c8-beta" have entirely different histories.

14 changed files with 2613 additions and 297 deletions

3
.gitignore vendored
View File

@ -1,2 +1 @@
SOURCES/Python-3.11.1.tar.xz
SOURCES/pgp_keys.asc
SOURCES/Python-3.11.7.tar.xz

View File

@ -1,2 +1 @@
89ee31611b73dc0c32c178d15aa208734b462c5a SOURCES/Python-3.11.1.tar.xz
befe131eceffa73877ba3adceca3b7b1da1d2319 SOURCES/pgp_keys.asc
f2534d591121f3845388fbdd6a121b96dfe305a6 SOURCES/Python-3.11.7.tar.xz

1189
SOURCES/00329-fips.patch Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,66 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: "Miss Islington (bot)"
<31488909+miss-islington@users.noreply.github.com>
Date: Wed, 21 Dec 2022 02:24:19 -0800
Subject: [PATCH] 00395: GH-100133: fix `asyncio` subprocess losing `stderr`
and `stdout` output
(cherry picked from commit a7715ccfba5b86ab09f86ec56ac3755c93b46b48)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
---
Lib/asyncio/base_subprocess.py | 3 ---
Lib/test/test_asyncio/test_subprocess.py | 17 +++++++++++++++++
...22-12-10-08-36-07.gh-issue-100133.g-zQlp.rst | 1 +
3 files changed, 18 insertions(+), 3 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2022-12-10-08-36-07.gh-issue-100133.g-zQlp.rst
diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py
index e15bb4141f..4c9b0dd565 100644
--- a/Lib/asyncio/base_subprocess.py
+++ b/Lib/asyncio/base_subprocess.py
@@ -215,9 +215,6 @@ def _process_exited(self, returncode):
# object. On Python 3.6, it is required to avoid a ResourceWarning.
self._proc.returncode = returncode
self._call(self._protocol.process_exited)
- for p in self._pipes.values():
- if p is not None:
- p.pipe.close()
self._try_finish()
diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py
index f71ad72f99..bea2314a52 100644
--- a/Lib/test/test_asyncio/test_subprocess.py
+++ b/Lib/test/test_asyncio/test_subprocess.py
@@ -684,6 +684,23 @@ async def execute():
self.assertIsNone(self.loop.run_until_complete(execute()))
+ def test_subprocess_communicate_stdout(self):
+ # See https://github.com/python/cpython/issues/100133
+ async def get_command_stdout(cmd, *args):
+ proc = await asyncio.create_subprocess_exec(
+ cmd, *args, stdout=asyncio.subprocess.PIPE,
+ )
+ stdout, _ = await proc.communicate()
+ return stdout.decode().strip()
+
+ async def main():
+ outputs = [f'foo{i}' for i in range(10)]
+ res = await asyncio.gather(*[get_command_stdout(sys.executable, '-c',
+ f'print({out!r})') for out in outputs])
+ self.assertEqual(res, outputs)
+
+ self.loop.run_until_complete(main())
+
if sys.platform != 'win32':
# Unix
diff --git a/Misc/NEWS.d/next/Library/2022-12-10-08-36-07.gh-issue-100133.g-zQlp.rst b/Misc/NEWS.d/next/Library/2022-12-10-08-36-07.gh-issue-100133.g-zQlp.rst
new file mode 100644
index 0000000000..881e6ed80f
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2022-12-10-08-36-07.gh-issue-100133.g-zQlp.rst
@@ -0,0 +1 @@
+Fix regression in :mod:`asyncio` where a subprocess would sometimes lose data received from pipe.

View File

@ -1,171 +0,0 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Serhiy Storchaka <storchaka@gmail.com>
Date: Tue, 10 Jan 2023 22:20:09 +0200
Subject: [PATCH] 00396: gh-100160: Remove any deprecation warnings in
asyncio.get_event_loop()
Some deprecation warnings will reappear (in a slightly different form) in 3.12.
Co-authored-by: Guido van Rossum <guido@python.org>
---
Doc/library/asyncio-eventloop.rst | 14 +++++++-------
Doc/library/asyncio-policy.rst | 9 +++++----
Doc/whatsnew/3.10.rst | 13 -------------
Lib/asyncio/events.py | 15 ---------------
Lib/test/test_asyncio/test_events.py | 12 +++---------
...2022-12-21-18-29-24.gh-issue-100160.isBmL5.rst | 2 ++
6 files changed, 17 insertions(+), 48 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2022-12-21-18-29-24.gh-issue-100160.isBmL5.rst
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
index 28b7a90058..886399e7ae 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -48,7 +48,7 @@ an event loop:
running event loop.
If there is no running event loop set, the function will return
- the result of ``get_event_loop_policy().get_event_loop()`` call.
+ the result of the ``get_event_loop_policy().get_event_loop()`` call.
Because this function has rather complex behavior (especially
when custom event loop policies are in use), using the
@@ -59,15 +59,15 @@ an event loop:
instead of using these lower level functions to manually create and close an
event loop.
- .. deprecated:: 3.10
- Deprecation warning is emitted if there is no current event loop.
- In Python 3.12 it will be an error.
-
.. note::
In Python versions 3.10.0--3.10.8 and 3.11.0 this function
- (and other functions which used it implicitly) emitted a
+ (and other functions which use it implicitly) emitted a
:exc:`DeprecationWarning` if there was no running event loop, even if
- the current loop was set.
+ the current loop was set on the policy.
+ In Python versions 3.10.9, 3.11.1 and 3.12 they emit a
+ :exc:`DeprecationWarning` if there is no running event loop and no
+ current loop is set.
+ In some future Python release this will become an error.
.. function:: set_event_loop(loop)
diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst
index d0af45febd..eb043b3e5e 100644
--- a/Doc/library/asyncio-policy.rst
+++ b/Doc/library/asyncio-policy.rst
@@ -112,10 +112,11 @@ asyncio ships with the following built-in policies:
On Windows, :class:`ProactorEventLoop` is now used by default.
- .. deprecated:: 3.11.1
- :meth:`get_event_loop` now emits a :exc:`DeprecationWarning` if there
- is no current event loop set and a new event loop has been implicitly
- created. In Python 3.12 it will be an error.
+ .. note::
+ In Python versions 3.10.9, 3.11.1 and 3.12 this function emits a
+ :exc:`DeprecationWarning` if there is no running event loop and no
+ current loop is set.
+ In some future Python release this will become an error.
.. class:: WindowsSelectorEventLoopPolicy
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index d0b436664a..38b30deff7 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -1710,19 +1710,6 @@ Deprecated
scheduled for removal in Python 3.12.
(Contributed by Erlend E. Aasland in :issue:`42264`.)
-* :func:`asyncio.get_event_loop` now emits a deprecation warning if there is
- no running event loop. In the future it will be an alias of
- :func:`~asyncio.get_running_loop`.
- :mod:`asyncio` functions which implicitly create :class:`~asyncio.Future`
- or :class:`~asyncio.Task` objects now emit
- a deprecation warning if there is no running event loop and no explicit
- *loop* argument is passed: :func:`~asyncio.ensure_future`,
- :func:`~asyncio.wrap_future`, :func:`~asyncio.gather`,
- :func:`~asyncio.shield`, :func:`~asyncio.as_completed` and constructors of
- :class:`~asyncio.Future`, :class:`~asyncio.Task`,
- :class:`~asyncio.StreamReader`, :class:`~asyncio.StreamReaderProtocol`.
- (Contributed by Serhiy Storchaka in :issue:`39529`.)
-
* The undocumented built-in function ``sqlite3.enable_shared_cache`` is now
deprecated, scheduled for removal in Python 3.12. Its use is strongly
discouraged by the SQLite3 documentation. See `the SQLite3 docs
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
index af3f9e970b..b1799320ea 100644
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -671,21 +671,6 @@ def get_event_loop(self):
if (self._local._loop is None and
not self._local._set_called and
threading.current_thread() is threading.main_thread()):
- stacklevel = 2
- try:
- f = sys._getframe(1)
- except AttributeError:
- pass
- else:
- while f:
- module = f.f_globals.get('__name__')
- if not (module == 'asyncio' or module.startswith('asyncio.')):
- break
- f = f.f_back
- stacklevel += 1
- import warnings
- warnings.warn('There is no current event loop',
- DeprecationWarning, stacklevel=stacklevel)
self.set_event_loop(self.new_event_loop())
if self._local._loop is None:
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
index c431fea401..18c4fd15d9 100644
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2547,9 +2547,7 @@ def test_event_loop_policy(self):
def test_get_event_loop(self):
policy = asyncio.DefaultEventLoopPolicy()
self.assertIsNone(policy._local._loop)
- with self.assertWarns(DeprecationWarning) as cm:
- loop = policy.get_event_loop()
- self.assertEqual(cm.filename, __file__)
+ loop = policy.get_event_loop()
self.assertIsInstance(loop, asyncio.AbstractEventLoop)
self.assertIs(policy._local._loop, loop)
@@ -2563,10 +2561,8 @@ def test_get_event_loop_calls_set_event_loop(self):
policy, "set_event_loop",
wraps=policy.set_event_loop) as m_set_event_loop:
- with self.assertWarns(DeprecationWarning) as cm:
- loop = policy.get_event_loop()
+ loop = policy.get_event_loop()
self.addCleanup(loop.close)
- self.assertEqual(cm.filename, __file__)
# policy._local._loop must be set through .set_event_loop()
# (the unix DefaultEventLoopPolicy needs this call to attach
@@ -2755,10 +2751,8 @@ def test_get_event_loop_returns_running_loop2(self):
loop = asyncio.new_event_loop()
self.addCleanup(loop.close)
- with self.assertWarns(DeprecationWarning) as cm:
- loop2 = asyncio.get_event_loop()
+ loop2 = asyncio.get_event_loop()
self.addCleanup(loop2.close)
- self.assertEqual(cm.filename, __file__)
asyncio.set_event_loop(None)
with self.assertRaisesRegex(RuntimeError, 'no current'):
asyncio.get_event_loop()
diff --git a/Misc/NEWS.d/next/Library/2022-12-21-18-29-24.gh-issue-100160.isBmL5.rst b/Misc/NEWS.d/next/Library/2022-12-21-18-29-24.gh-issue-100160.isBmL5.rst
new file mode 100644
index 0000000000..c3b518ca85
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2022-12-21-18-29-24.gh-issue-100160.isBmL5.rst
@@ -0,0 +1,2 @@
+Remove any deprecation warnings in :func:`asyncio.get_event_loop`. They are
+deferred to Python 3.12.

View File

@ -0,0 +1,251 @@
From 8b70605b594b3831331a9340ba764ff751871612 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <encukou@gmail.com>
Date: Mon, 6 Mar 2023 17:24:24 +0100
Subject: [PATCH] CVE-2007-4559, PEP-706: Add filters for tarfile extraction
(downstream)
Add and test RHEL-specific ways of configuring the default behavior: environment
variable and config file.
---
Lib/tarfile.py | 42 +++++++++++++
Lib/test/test_shutil.py | 3 +-
Lib/test/test_tarfile.py | 128 ++++++++++++++++++++++++++++++++++++++-
3 files changed, 169 insertions(+), 4 deletions(-)
diff --git a/Lib/tarfile.py b/Lib/tarfile.py
index 130b5e0..3b7d8d5 100755
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -72,6 +72,13 @@ __all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError", "ReadError",
"ENCODING", "USTAR_FORMAT", "GNU_FORMAT", "PAX_FORMAT",
"DEFAULT_FORMAT", "open"]
+# If true, use the safer (but backwards-incompatible) 'tar' extraction filter,
+# rather than 'fully_trusted', by default.
+# The emitted warning is changed to match.
+_RH_SAFER_DEFAULT = True
+
+# System-wide configuration file
+_CONFIG_FILENAME = '/etc/python/tarfile.cfg'
#---------------------------------------------------------
# tar constants
@@ -2211,6 +2218,41 @@ class TarFile(object):
if filter is None:
filter = self.extraction_filter
if filter is None:
+ name = os.environ.get('PYTHON_TARFILE_EXTRACTION_FILTER')
+ if name is None:
+ try:
+ file = bltn_open(_CONFIG_FILENAME)
+ except FileNotFoundError:
+ pass
+ else:
+ import configparser
+ conf = configparser.ConfigParser(
+ interpolation=None,
+ comment_prefixes=('#', ),
+ )
+ with file:
+ conf.read_file(file)
+ name = conf.get('tarfile',
+ 'PYTHON_TARFILE_EXTRACTION_FILTER',
+ fallback='')
+ if name:
+ try:
+ filter = _NAMED_FILTERS[name]
+ except KeyError:
+ raise ValueError(f"filter {filter!r} not found") from None
+ self.extraction_filter = filter
+ return filter
+ if _RH_SAFER_DEFAULT:
+ warnings.warn(
+ 'The default behavior of tarfile extraction has been '
+ + 'changed to disallow common exploits '
+ + '(including CVE-2007-4559). '
+ + 'By default, absolute/parent paths are disallowed '
+ + 'and some mode bits are cleared. '
+ + 'See https://access.redhat.com/articles/7004769 '
+ + 'for more details.',
+ RuntimeWarning)
+ return tar_filter
return fully_trusted_filter
if isinstance(filter, str):
raise TypeError(
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
index 9bf4145..f247b82 100644
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -1665,7 +1665,8 @@ class TestArchives(BaseTest, unittest.TestCase):
def check_unpack_tarball(self, format):
self.check_unpack_archive(format, filter='fully_trusted')
self.check_unpack_archive(format, filter='data')
- with warnings_helper.check_no_warnings(self):
+ with warnings_helper.check_warnings(
+ ('.*CVE-2007-4559', RuntimeWarning)):
self.check_unpack_archive(format)
def test_unpack_archive_tar(self):
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
index cdea033..4724285 100644
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -2,7 +2,7 @@ import sys
import os
import io
from hashlib import sha256
-from contextlib import contextmanager
+from contextlib import contextmanager, ExitStack
from random import Random
import pathlib
import shutil
@@ -2999,7 +2999,11 @@ class NoneInfoExtractTests(ReadTest):
tar = tarfile.open(tarname, mode='r', encoding="iso8859-1")
cls.control_dir = pathlib.Path(TEMPDIR) / "extractall_ctrl"
tar.errorlevel = 0
- tar.extractall(cls.control_dir, filter=cls.extraction_filter)
+ with ExitStack() as cm:
+ if cls.extraction_filter is None:
+ cm.enter_context(warnings.catch_warnings())
+ warnings.simplefilter(action="ignore", category=RuntimeWarning)
+ tar.extractall(cls.control_dir, filter=cls.extraction_filter)
tar.close()
cls.control_paths = set(
p.relative_to(cls.control_dir)
@@ -3674,7 +3678,8 @@ class TestExtractionFilters(unittest.TestCase):
"""Ensure the default filter does not warn (like in 3.12)"""
with ArchiveMaker() as arc:
arc.add('foo')
- with warnings_helper.check_no_warnings(self):
+ with warnings_helper.check_warnings(
+ ('.*CVE-2007-4559', RuntimeWarning)):
with self.check_context(arc.open(), None):
self.expect_file('foo')
@@ -3844,6 +3849,123 @@ class TestExtractionFilters(unittest.TestCase):
self.expect_exception(TypeError) # errorlevel is not int
+ @contextmanager
+ def rh_config_context(self, config_lines=None):
+ """Set up for testing various ways of overriding the default filter
+
+ return a triple with:
+ - temporary directory
+ - EnvironmentVarGuard()
+ - a test archive for use with check_* methods below
+
+ If config_lines is given, write them to the config file. Otherwise
+ the config file is missing.
+ """
+ tempdir = pathlib.Path(TEMPDIR) / 'tmp'
+ configfile = tempdir / 'tarfile.cfg'
+ with ArchiveMaker() as arc:
+ arc.add('good')
+ arc.add('ugly', symlink_to='/etc/passwd')
+ arc.add('../bad')
+ with (
+ os_helper.temp_dir(tempdir),
+ support.swap_attr(tarfile, '_CONFIG_FILENAME', str(configfile)),
+ os_helper.EnvironmentVarGuard() as env,
+ arc.open() as tar,
+ ):
+ if config_lines is not None:
+ with configfile.open('w') as f:
+ for line in config_lines:
+ print(line, file=f)
+ yield tempdir, env, tar
+
+ def check_rh_default_behavior(self, tar, tempdir):
+ """Check RH default: warn and refuse to extract dangerous files."""
+ with (
+ warnings_helper.check_warnings(
+ ('.*CVE-2007-4559', RuntimeWarning)),
+ self.assertRaises(tarfile.OutsideDestinationError),
+ ):
+ tar.extractall(tempdir / 'outdir')
+
+ def check_trusted_default(self, tar, tempdir):
+ """Check 'fully_trusted' is configured as the default filter."""
+ with (
+ warnings_helper.check_no_warnings(self),
+ ):
+ tar.extractall(tempdir / 'outdir')
+ self.assertTrue((tempdir / 'outdir/good').exists())
+ self.assertEqual((tempdir / 'outdir/ugly').readlink(),
+ pathlib.Path('/etc/passwd'))
+ self.assertTrue((tempdir / 'bad').exists())
+
+ def test_rh_default_no_conf(self):
+ with self.rh_config_context() as (tempdir, env, tar):
+ self.check_rh_default_behavior(tar, tempdir)
+
+ def test_rh_default_from_file(self):
+ lines = ['[tarfile]', 'PYTHON_TARFILE_EXTRACTION_FILTER=fully_trusted']
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ self.check_trusted_default(tar, tempdir)
+
+ def test_rh_empty_config_file(self):
+ """Empty config file -> default behavior"""
+ lines = []
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ self.check_rh_default_behavior(tar, tempdir)
+
+ def test_empty_config_section(self):
+ """Empty section in config file -> default behavior"""
+ lines = ['[tarfile]']
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ self.check_rh_default_behavior(tar, tempdir)
+
+ def test_rh_default_empty_config_option(self):
+ """Empty option value in config file -> default behavior"""
+ lines = ['[tarfile]', 'PYTHON_TARFILE_EXTRACTION_FILTER=']
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ self.check_rh_default_behavior(tar, tempdir)
+
+ def test_bad_config_option(self):
+ """Bad option value in config file -> ValueError"""
+ lines = ['[tarfile]', 'PYTHON_TARFILE_EXTRACTION_FILTER=unknown!']
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ with self.assertRaises(ValueError):
+ tar.extractall(tempdir / 'outdir')
+
+ def test_default_from_envvar(self):
+ with self.rh_config_context() as (tempdir, env, tar):
+ env['PYTHON_TARFILE_EXTRACTION_FILTER'] = 'fully_trusted'
+ self.check_trusted_default(tar, tempdir)
+
+ def test_empty_envvar(self):
+ """Empty env variable -> default behavior"""
+ with self.rh_config_context() as (tempdir, env, tar):
+ env['PYTHON_TARFILE_EXTRACTION_FILTER'] = ''
+ self.check_rh_default_behavior(tar, tempdir)
+
+ def test_bad_envvar(self):
+ with self.rh_config_context() as (tempdir, env, tar):
+ env['PYTHON_TARFILE_EXTRACTION_FILTER'] = 'unknown!'
+ with self.assertRaises(ValueError):
+ tar.extractall(tempdir / 'outdir')
+
+ def test_envvar_overrides_file(self):
+ lines = ['[tarfile]', 'PYTHON_TARFILE_EXTRACTION_FILTER=data']
+ with self.rh_config_context(lines) as (tempdir, env, tar):
+ env['PYTHON_TARFILE_EXTRACTION_FILTER'] = 'fully_trusted'
+ self.check_trusted_default(tar, tempdir)
+
+ def test_monkeypatch_overrides_envvar(self):
+ with self.rh_config_context(None) as (tempdir, env, tar):
+ env['PYTHON_TARFILE_EXTRACTION_FILTER'] = 'data'
+ with support.swap_attr(
+ tarfile.TarFile, 'extraction_filter',
+ staticmethod(tarfile.fully_trusted_filter)
+ ):
+ self.check_trusted_default(tar, tempdir)
+
+
def setUpModule():
os_helper.unlink(TEMPDIR)
os.makedirs(TEMPDIR)
--
2.41.0

View File

@ -0,0 +1,755 @@
From d8b0fafb202bf884135a3f7f0ce0b086217a2da2 Mon Sep 17 00:00:00 2001
From: Victor Stinner <vstinner@python.org>
Date: Fri, 15 Dec 2023 16:10:40 +0100
Subject: [PATCH 1/2] 00415: [CVE-2023-27043] gh-102988: Reject malformed
addresses in email.parseaddr() (#111116)
Detect email address parsing errors and return empty tuple to
indicate the parsing error (old API). Add an optional 'strict'
parameter to getaddresses() and parseaddr() functions. Patch by
Thomas Dwyer.
Co-Authored-By: Thomas Dwyer <github@tomd.tel>
---
Doc/library/email.utils.rst | 19 +-
Lib/email/utils.py | 151 ++++++++++++-
Lib/test/test_email/test_email.py | 204 +++++++++++++++++-
...-10-20-15-28-08.gh-issue-102988.dStNO7.rst | 8 +
4 files changed, 361 insertions(+), 21 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2023-10-20-15-28-08.gh-issue-102988.dStNO7.rst
diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst
index 0e266b6..6723dc4 100644
--- a/Doc/library/email.utils.rst
+++ b/Doc/library/email.utils.rst
@@ -60,13 +60,18 @@ of the new API.
begins with angle brackets, they are stripped off.
-.. function:: parseaddr(address)
+.. function:: parseaddr(address, *, strict=True)
Parse address -- which should be the value of some address-containing field such
as :mailheader:`To` or :mailheader:`Cc` -- into its constituent *realname* and
*email address* parts. Returns a tuple of that information, unless the parse
fails, in which case a 2-tuple of ``('', '')`` is returned.
+ If *strict* is true, use a strict parser which rejects malformed inputs.
+
+ .. versionchanged:: 3.13
+ Add *strict* optional parameter and reject malformed inputs by default.
+
.. function:: formataddr(pair, charset='utf-8')
@@ -84,12 +89,15 @@ of the new API.
Added the *charset* option.
-.. function:: getaddresses(fieldvalues)
+.. function:: getaddresses(fieldvalues, *, strict=True)
This method returns a list of 2-tuples of the form returned by ``parseaddr()``.
*fieldvalues* is a sequence of header field values as might be returned by
- :meth:`Message.get_all <email.message.Message.get_all>`. Here's a simple
- example that gets all the recipients of a message::
+ :meth:`Message.get_all <email.message.Message.get_all>`.
+
+ If *strict* is true, use a strict parser which rejects malformed inputs.
+
+ Here's a simple example that gets all the recipients of a message::
from email.utils import getaddresses
@@ -99,6 +107,9 @@ of the new API.
resent_ccs = msg.get_all('resent-cc', [])
all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs)
+ .. versionchanged:: 3.13
+ Add *strict* optional parameter and reject malformed inputs by default.
+
.. function:: parsedate(date)
diff --git a/Lib/email/utils.py b/Lib/email/utils.py
index cfdfeb3..9522341 100644
--- a/Lib/email/utils.py
+++ b/Lib/email/utils.py
@@ -48,6 +48,7 @@ TICK = "'"
specialsre = re.compile(r'[][\\()<>@,:;".]')
escapesre = re.compile(r'[\\"]')
+
def _has_surrogates(s):
"""Return True if s contains surrogate-escaped binary data."""
# This check is based on the fact that unless there are surrogates, utf8
@@ -106,12 +107,127 @@ def formataddr(pair, charset='utf-8'):
return address
+def _iter_escaped_chars(addr):
+ pos = 0
+ escape = False
+ for pos, ch in enumerate(addr):
+ if escape:
+ yield (pos, '\\' + ch)
+ escape = False
+ elif ch == '\\':
+ escape = True
+ else:
+ yield (pos, ch)
+ if escape:
+ yield (pos, '\\')
+
+
+def _strip_quoted_realnames(addr):
+ """Strip real names between quotes."""
+ if '"' not in addr:
+ # Fast path
+ return addr
+
+ start = 0
+ open_pos = None
+ result = []
+ for pos, ch in _iter_escaped_chars(addr):
+ if ch == '"':
+ if open_pos is None:
+ open_pos = pos
+ else:
+ if start != open_pos:
+ result.append(addr[start:open_pos])
+ start = pos + 1
+ open_pos = None
+
+ if start < len(addr):
+ result.append(addr[start:])
+
+ return ''.join(result)
-def getaddresses(fieldvalues):
- """Return a list of (REALNAME, EMAIL) for each fieldvalue."""
- all = COMMASPACE.join(str(v) for v in fieldvalues)
- a = _AddressList(all)
- return a.addresslist
+
+supports_strict_parsing = True
+
+def getaddresses(fieldvalues, *, strict=True):
+ """Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue.
+
+ When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in
+ its place.
+
+ If strict is true, use a strict parser which rejects malformed inputs.
+ """
+
+ # If strict is true, if the resulting list of parsed addresses is greater
+ # than the number of fieldvalues in the input list, a parsing error has
+ # occurred and consequently a list containing a single empty 2-tuple [('',
+ # '')] is returned in its place. This is done to avoid invalid output.
+ #
+ # Malformed input: getaddresses(['alice@example.com <bob@example.com>'])
+ # Invalid output: [('', 'alice@example.com'), ('', 'bob@example.com')]
+ # Safe output: [('', '')]
+
+ if not strict:
+ all = COMMASPACE.join(str(v) for v in fieldvalues)
+ a = _AddressList(all)
+ return a.addresslist
+
+ fieldvalues = [str(v) for v in fieldvalues]
+ fieldvalues = _pre_parse_validation(fieldvalues)
+ addr = COMMASPACE.join(fieldvalues)
+ a = _AddressList(addr)
+ result = _post_parse_validation(a.addresslist)
+
+ # Treat output as invalid if the number of addresses is not equal to the
+ # expected number of addresses.
+ n = 0
+ for v in fieldvalues:
+ # When a comma is used in the Real Name part it is not a deliminator.
+ # So strip those out before counting the commas.
+ v = _strip_quoted_realnames(v)
+ # Expected number of addresses: 1 + number of commas
+ n += 1 + v.count(',')
+ if len(result) != n:
+ return [('', '')]
+
+ return result
+
+
+def _check_parenthesis(addr):
+ # Ignore parenthesis in quoted real names.
+ addr = _strip_quoted_realnames(addr)
+
+ opens = 0
+ for pos, ch in _iter_escaped_chars(addr):
+ if ch == '(':
+ opens += 1
+ elif ch == ')':
+ opens -= 1
+ if opens < 0:
+ return False
+ return (opens == 0)
+
+
+def _pre_parse_validation(email_header_fields):
+ accepted_values = []
+ for v in email_header_fields:
+ if not _check_parenthesis(v):
+ v = "('', '')"
+ accepted_values.append(v)
+
+ return accepted_values
+
+
+def _post_parse_validation(parsed_email_header_tuples):
+ accepted_values = []
+ # The parser would have parsed a correctly formatted domain-literal
+ # The existence of an [ after parsing indicates a parsing failure
+ for v in parsed_email_header_tuples:
+ if '[' in v[1]:
+ v = ('', '')
+ accepted_values.append(v)
+
+ return accepted_values
def _format_timetuple_and_zone(timetuple, zone):
@@ -205,16 +321,33 @@ def parsedate_to_datetime(data):
tzinfo=datetime.timezone(datetime.timedelta(seconds=tz)))
-def parseaddr(addr):
+def parseaddr(addr, *, strict=True):
"""
Parse addr into its constituent realname and email address parts.
Return a tuple of realname and email address, unless the parse fails, in
which case return a 2-tuple of ('', '').
+
+ If strict is True, use a strict parser which rejects malformed inputs.
"""
- addrs = _AddressList(addr).addresslist
- if not addrs:
- return '', ''
+ if not strict:
+ addrs = _AddressList(addr).addresslist
+ if not addrs:
+ return ('', '')
+ return addrs[0]
+
+ if isinstance(addr, list):
+ addr = addr[0]
+
+ if not isinstance(addr, str):
+ return ('', '')
+
+ addr = _pre_parse_validation([addr])[0]
+ addrs = _post_parse_validation(_AddressList(addr).addresslist)
+
+ if not addrs or len(addrs) > 1:
+ return ('', '')
+
return addrs[0]
diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py
index 677f209..20b6779 100644
--- a/Lib/test/test_email/test_email.py
+++ b/Lib/test/test_email/test_email.py
@@ -17,6 +17,7 @@ from unittest.mock import patch
import email
import email.policy
+import email.utils
from email.charset import Charset
from email.generator import Generator, DecodedGenerator, BytesGenerator
@@ -3321,15 +3322,154 @@ Foo
[('Al Person', 'aperson@dom.ain'),
('Bud Person', 'bperson@dom.ain')])
+ def test_getaddresses_comma_in_name(self):
+ """GH-106669 regression test."""
+ self.assertEqual(
+ utils.getaddresses(
+ [
+ '"Bud, Person" <bperson@dom.ain>',
+ 'aperson@dom.ain (Al Person)',
+ '"Mariusz Felisiak" <to@example.com>',
+ ]
+ ),
+ [
+ ('Bud, Person', 'bperson@dom.ain'),
+ ('Al Person', 'aperson@dom.ain'),
+ ('Mariusz Felisiak', 'to@example.com'),
+ ],
+ )
+
+ def test_parsing_errors(self):
+ """Test for parsing errors from CVE-2023-27043 and CVE-2019-16056"""
+ alice = 'alice@example.org'
+ bob = 'bob@example.com'
+ empty = ('', '')
+
+ # Test utils.getaddresses() and utils.parseaddr() on malformed email
+ # addresses: default behavior (strict=True) rejects malformed address,
+ # and strict=False which tolerates malformed address.
+ for invalid_separator, expected_non_strict in (
+ ('(', [(f'<{bob}>', alice)]),
+ (')', [('', alice), empty, ('', bob)]),
+ ('<', [('', alice), empty, ('', bob), empty]),
+ ('>', [('', alice), empty, ('', bob)]),
+ ('[', [('', f'{alice}[<{bob}>]')]),
+ (']', [('', alice), empty, ('', bob)]),
+ ('@', [empty, empty, ('', bob)]),
+ (';', [('', alice), empty, ('', bob)]),
+ (':', [('', alice), ('', bob)]),
+ ('.', [('', alice + '.'), ('', bob)]),
+ ('"', [('', alice), ('', f'<{bob}>')]),
+ ):
+ address = f'{alice}{invalid_separator}<{bob}>'
+ with self.subTest(address=address):
+ self.assertEqual(utils.getaddresses([address]),
+ [empty])
+ self.assertEqual(utils.getaddresses([address], strict=False),
+ expected_non_strict)
+
+ self.assertEqual(utils.parseaddr([address]),
+ empty)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Comma (',') is treated differently depending on strict parameter.
+ # Comma without quotes.
+ address = f'{alice},<{bob}>'
+ self.assertEqual(utils.getaddresses([address]),
+ [('', alice), ('', bob)])
+ self.assertEqual(utils.getaddresses([address], strict=False),
+ [('', alice), ('', bob)])
+ self.assertEqual(utils.parseaddr([address]),
+ empty)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Real name between quotes containing comma.
+ address = '"Alice, alice@example.org" <bob@example.com>'
+ expected_strict = ('Alice, alice@example.org', 'bob@example.com')
+ self.assertEqual(utils.getaddresses([address]), [expected_strict])
+ self.assertEqual(utils.getaddresses([address], strict=False), [expected_strict])
+ self.assertEqual(utils.parseaddr([address]), expected_strict)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Valid parenthesis in comments.
+ address = 'alice@example.org (Alice)'
+ expected_strict = ('Alice', 'alice@example.org')
+ self.assertEqual(utils.getaddresses([address]), [expected_strict])
+ self.assertEqual(utils.getaddresses([address], strict=False), [expected_strict])
+ self.assertEqual(utils.parseaddr([address]), expected_strict)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Invalid parenthesis in comments.
+ address = 'alice@example.org )Alice('
+ self.assertEqual(utils.getaddresses([address]), [empty])
+ self.assertEqual(utils.getaddresses([address], strict=False),
+ [('', 'alice@example.org'), ('', ''), ('', 'Alice')])
+ self.assertEqual(utils.parseaddr([address]), empty)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Two addresses with quotes separated by comma.
+ address = '"Jane Doe" <jane@example.net>, "John Doe" <john@example.net>'
+ self.assertEqual(utils.getaddresses([address]),
+ [('Jane Doe', 'jane@example.net'),
+ ('John Doe', 'john@example.net')])
+ self.assertEqual(utils.getaddresses([address], strict=False),
+ [('Jane Doe', 'jane@example.net'),
+ ('John Doe', 'john@example.net')])
+ self.assertEqual(utils.parseaddr([address]), empty)
+ self.assertEqual(utils.parseaddr([address], strict=False),
+ ('', address))
+
+ # Test email.utils.supports_strict_parsing attribute
+ self.assertEqual(email.utils.supports_strict_parsing, True)
+
def test_getaddresses_nasty(self):
- eq = self.assertEqual
- eq(utils.getaddresses(['foo: ;']), [('', '')])
- eq(utils.getaddresses(
- ['[]*-- =~$']),
- [('', ''), ('', ''), ('', '*--')])
- eq(utils.getaddresses(
- ['foo: ;', '"Jason R. Mastaler" <jason@dom.ain>']),
- [('', ''), ('Jason R. Mastaler', 'jason@dom.ain')])
+ for addresses, expected in (
+ (['"Sürname, Firstname" <to@example.com>'],
+ [('Sürname, Firstname', 'to@example.com')]),
+
+ (['foo: ;'],
+ [('', '')]),
+
+ (['foo: ;', '"Jason R. Mastaler" <jason@dom.ain>'],
+ [('', ''), ('Jason R. Mastaler', 'jason@dom.ain')]),
+
+ ([r'Pete(A nice \) chap) <pete(his account)@silly.test(his host)>'],
+ [('Pete (A nice ) chap his account his host)', 'pete@silly.test')]),
+
+ (['(Empty list)(start)Undisclosed recipients :(nobody(I know))'],
+ [('', '')]),
+
+ (['Mary <@machine.tld:mary@example.net>, , jdoe@test . example'],
+ [('Mary', 'mary@example.net'), ('', ''), ('', 'jdoe@test.example')]),
+
+ (['John Doe <jdoe@machine(comment). example>'],
+ [('John Doe (comment)', 'jdoe@machine.example')]),
+
+ (['"Mary Smith: Personal Account" <smith@home.example>'],
+ [('Mary Smith: Personal Account', 'smith@home.example')]),
+
+ (['Undisclosed recipients:;'],
+ [('', '')]),
+
+ ([r'<boss@nil.test>, "Giant; \"Big\" Box" <bob@example.net>'],
+ [('', 'boss@nil.test'), ('Giant; "Big" Box', 'bob@example.net')]),
+ ):
+ with self.subTest(addresses=addresses):
+ self.assertEqual(utils.getaddresses(addresses),
+ expected)
+ self.assertEqual(utils.getaddresses(addresses, strict=False),
+ expected)
+
+ addresses = ['[]*-- =~$']
+ self.assertEqual(utils.getaddresses(addresses),
+ [('', '')])
+ self.assertEqual(utils.getaddresses(addresses, strict=False),
+ [('', ''), ('', ''), ('', '*--')])
def test_getaddresses_embedded_comment(self):
"""Test proper handling of a nested comment"""
@@ -3520,6 +3660,54 @@ multipart/report
m = cls(*constructor, policy=email.policy.default)
self.assertIs(m.policy, email.policy.default)
+ def test_iter_escaped_chars(self):
+ self.assertEqual(list(utils._iter_escaped_chars(r'a\\b\"c\\"d')),
+ [(0, 'a'),
+ (2, '\\\\'),
+ (3, 'b'),
+ (5, '\\"'),
+ (6, 'c'),
+ (8, '\\\\'),
+ (9, '"'),
+ (10, 'd')])
+ self.assertEqual(list(utils._iter_escaped_chars('a\\')),
+ [(0, 'a'), (1, '\\')])
+
+ def test_strip_quoted_realnames(self):
+ def check(addr, expected):
+ self.assertEqual(utils._strip_quoted_realnames(addr), expected)
+
+ check('"Jane Doe" <jane@example.net>, "John Doe" <john@example.net>',
+ ' <jane@example.net>, <john@example.net>')
+ check(r'"Jane \"Doe\"." <jane@example.net>',
+ ' <jane@example.net>')
+
+ # special cases
+ check(r'before"name"after', 'beforeafter')
+ check(r'before"name"', 'before')
+ check(r'b"name"', 'b') # single char
+ check(r'"name"after', 'after')
+ check(r'"name"a', 'a') # single char
+ check(r'"name"', '')
+
+ # no change
+ for addr in (
+ 'Jane Doe <jane@example.net>, John Doe <john@example.net>',
+ 'lone " quote',
+ ):
+ self.assertEqual(utils._strip_quoted_realnames(addr), addr)
+
+
+ def test_check_parenthesis(self):
+ addr = 'alice@example.net'
+ self.assertTrue(utils._check_parenthesis(f'{addr} (Alice)'))
+ self.assertFalse(utils._check_parenthesis(f'{addr} )Alice('))
+ self.assertFalse(utils._check_parenthesis(f'{addr} (Alice))'))
+ self.assertFalse(utils._check_parenthesis(f'{addr} ((Alice)'))
+
+ # Ignore real name between quotes
+ self.assertTrue(utils._check_parenthesis(f'")Alice((" {addr}'))
+
# Test the iterator/generators
class TestIterators(TestEmailBase):
diff --git a/Misc/NEWS.d/next/Library/2023-10-20-15-28-08.gh-issue-102988.dStNO7.rst b/Misc/NEWS.d/next/Library/2023-10-20-15-28-08.gh-issue-102988.dStNO7.rst
new file mode 100644
index 0000000..3d0e9e4
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2023-10-20-15-28-08.gh-issue-102988.dStNO7.rst
@@ -0,0 +1,8 @@
+:func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now
+return ``('', '')`` 2-tuples in more situations where invalid email
+addresses are encountered instead of potentially inaccurate values. Add
+optional *strict* parameter to these two functions: use ``strict=False`` to
+get the old behavior, accept malformed inputs.
+``getattr(email.utils, 'supports_strict_parsing', False)`` can be use to check
+if the *strict* paramater is available. Patch by Thomas Dwyer and Victor
+Stinner to improve the CVE-2023-27043 fix.
--
2.43.0
From 6c34f5b95da90bd494e29776c0e807af44689fae Mon Sep 17 00:00:00 2001
From: Lumir Balhar <lbalhar@redhat.com>
Date: Wed, 10 Jan 2024 08:53:53 +0100
Subject: [PATCH 2/2] Make it possible to disable strict parsing in email
module
---
Doc/library/email.utils.rst | 26 +++++++++++
Lib/email/utils.py | 54 +++++++++++++++++++++-
Lib/test/test_email/test_email.py | 74 ++++++++++++++++++++++++++++++-
3 files changed, 150 insertions(+), 4 deletions(-)
diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst
index 6723dc4..c89602d 100644
--- a/Doc/library/email.utils.rst
+++ b/Doc/library/email.utils.rst
@@ -69,6 +69,19 @@ of the new API.
If *strict* is true, use a strict parser which rejects malformed inputs.
+ The default setting for *strict* is set to ``True``, but you can override
+ it by setting the environment variable ``PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING``
+ to non-empty string.
+
+ Additionally, you can permanently set the default value for *strict* to
+ ``False`` by creating the configuration file ``/etc/python/email.cfg``
+ with the following content:
+
+ .. code-block:: ini
+
+ [email_addr_parsing]
+ PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING = true
+
.. versionchanged:: 3.13
Add *strict* optional parameter and reject malformed inputs by default.
@@ -97,6 +110,19 @@ of the new API.
If *strict* is true, use a strict parser which rejects malformed inputs.
+ The default setting for *strict* is set to ``True``, but you can override
+ it by setting the environment variable ``PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING``
+ to non-empty string.
+
+ Additionally, you can permanently set the default value for *strict* to
+ ``False`` by creating the configuration file ``/etc/python/email.cfg``
+ with the following content:
+
+ .. code-block:: ini
+
+ [email_addr_parsing]
+ PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING = true
+
Here's a simple example that gets all the recipients of a message::
from email.utils import getaddresses
diff --git a/Lib/email/utils.py b/Lib/email/utils.py
index 9522341..2e30e09 100644
--- a/Lib/email/utils.py
+++ b/Lib/email/utils.py
@@ -48,6 +48,46 @@ TICK = "'"
specialsre = re.compile(r'[][\\()<>@,:;".]')
escapesre = re.compile(r'[\\"]')
+_EMAIL_CONFIG_FILE = "/etc/python/email.cfg"
+_cached_strict_addr_parsing = None
+
+
+def _use_strict_email_parsing():
+ """"Cache implementation for _cached_strict_addr_parsing"""
+ global _cached_strict_addr_parsing
+ if _cached_strict_addr_parsing is None:
+ _cached_strict_addr_parsing = _use_strict_email_parsing_impl()
+ return _cached_strict_addr_parsing
+
+
+def _use_strict_email_parsing_impl():
+ """Returns True if strict email parsing is not disabled by
+ config file or env variable.
+ """
+ disabled = bool(os.environ.get("PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING"))
+ if disabled:
+ return False
+
+ try:
+ file = open(_EMAIL_CONFIG_FILE)
+ except FileNotFoundError:
+ pass
+ else:
+ with file:
+ import configparser
+ config = configparser.ConfigParser(
+ interpolation=None,
+ comment_prefixes=('#', ),
+
+ )
+ config.read_file(file)
+ disabled = config.getboolean('email_addr_parsing', "PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING", fallback=None)
+
+ if disabled:
+ return False
+
+ return True
+
def _has_surrogates(s):
"""Return True if s contains surrogate-escaped binary data."""
@@ -149,7 +189,7 @@ def _strip_quoted_realnames(addr):
supports_strict_parsing = True
-def getaddresses(fieldvalues, *, strict=True):
+def getaddresses(fieldvalues, *, strict=None):
"""Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue.
When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in
@@ -158,6 +198,11 @@ def getaddresses(fieldvalues, *, strict=True):
If strict is true, use a strict parser which rejects malformed inputs.
"""
+ # If default is used, it's True unless disabled
+ # by env variable or config file.
+ if strict == None:
+ strict = _use_strict_email_parsing()
+
# If strict is true, if the resulting list of parsed addresses is greater
# than the number of fieldvalues in the input list, a parsing error has
# occurred and consequently a list containing a single empty 2-tuple [('',
@@ -321,7 +366,7 @@ def parsedate_to_datetime(data):
tzinfo=datetime.timezone(datetime.timedelta(seconds=tz)))
-def parseaddr(addr, *, strict=True):
+def parseaddr(addr, *, strict=None):
"""
Parse addr into its constituent realname and email address parts.
@@ -330,6 +375,11 @@ def parseaddr(addr, *, strict=True):
If strict is True, use a strict parser which rejects malformed inputs.
"""
+ # If default is used, it's True unless disabled
+ # by env variable or config file.
+ if strict == None:
+ strict = _use_strict_email_parsing()
+
if not strict:
addrs = _AddressList(addr).addresslist
if not addrs:
diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py
index 20b6779..d7d99f0 100644
--- a/Lib/test/test_email/test_email.py
+++ b/Lib/test/test_email/test_email.py
@@ -8,6 +8,9 @@ import base64
import unittest
import textwrap
import warnings
+import contextlib
+import tempfile
+import os
from io import StringIO, BytesIO
from itertools import chain
@@ -41,8 +44,8 @@ from email import quoprimime
from email import utils
from test import support
-from test.support import threading_helper
-from test.support.os_helper import unlink
+from test.support import threading_helper, swap_attr
+from test.support.os_helper import unlink, EnvironmentVarGuard
from test.test_email import openfile, TestEmailBase
# These imports are documented to work, but we are testing them using a
@@ -3427,6 +3430,73 @@ Foo
# Test email.utils.supports_strict_parsing attribute
self.assertEqual(email.utils.supports_strict_parsing, True)
+ def test_parsing_errors_strict_set_via_env_var(self):
+ address = 'alice@example.org )Alice('
+ empty = ('', '')
+
+ # Reset cached default value to make the function
+ # reload the config file provided below.
+ utils._cached_strict_addr_parsing = None
+
+ # Strict disabled via env variable, old behavior expected
+ with EnvironmentVarGuard() as environ:
+ environ["PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING"] = "1"
+
+ self.assertEqual(utils.getaddresses([address]),
+ [('', 'alice@example.org'), ('', ''), ('', 'Alice')])
+ self.assertEqual(utils.parseaddr([address]), ('', address))
+
+ # Clear cache again
+ utils._cached_strict_addr_parsing = None
+
+ # Default strict=True, empty result expected
+ self.assertEqual(utils.getaddresses([address]), [empty])
+ self.assertEqual(utils.parseaddr([address]), empty)
+
+ # Clear cache again
+ utils._cached_strict_addr_parsing = None
+
+ # Empty string in env variable = strict parsing enabled (default)
+ with EnvironmentVarGuard() as environ:
+ environ["PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING"] = ""
+
+ # Default strict=True, empty result expected
+ self.assertEqual(utils.getaddresses([address]), [empty])
+ self.assertEqual(utils.parseaddr([address]), empty)
+
+ @contextlib.contextmanager
+ def _email_strict_parsing_conf(self):
+ """Context for the given email strict parsing configured in config file"""
+ with tempfile.TemporaryDirectory() as tmpdirname:
+ filename = os.path.join(tmpdirname, 'conf.cfg')
+ with swap_attr(utils, "_EMAIL_CONFIG_FILE", filename):
+ with open(filename, 'w') as file:
+ file.write('[email_addr_parsing]\n')
+ file.write('PYTHON_EMAIL_DISABLE_STRICT_ADDR_PARSING = true')
+ utils._EMAIL_CONFIG_FILE = filename
+ yield
+
+ def test_parsing_errors_strict_disabled_via_config_file(self):
+ address = 'alice@example.org )Alice('
+ empty = ('', '')
+
+ # Reset cached default value to make the function
+ # reload the config file provided below.
+ utils._cached_strict_addr_parsing = None
+
+ # Strict disabled via config file, old results expected
+ with self._email_strict_parsing_conf():
+ self.assertEqual(utils.getaddresses([address]),
+ [('', 'alice@example.org'), ('', ''), ('', 'Alice')])
+ self.assertEqual(utils.parseaddr([address]), ('', address))
+
+ # Clear cache again
+ utils._cached_strict_addr_parsing = None
+
+ # Default strict=True, empty result expected
+ self.assertEqual(utils.getaddresses([address]), [empty])
+ self.assertEqual(utils.parseaddr([address]), empty)
+
def test_getaddresses_nasty(self):
for addresses, expected in (
(['"Sürname, Firstname" <to@example.com>'],
--
2.43.0

View File

@ -1,16 +0,0 @@
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEz9yiRbEEPPKl+Xhl/+h0BBaL2EcFAmOPlhsACgkQ/+h0BBaL
2EdY5w/+KKZY3ghMcjuxxF4o9CylFvhHGI7LP6FKZE5xnGtSZ2cXjcad+FwFMnFS
JE5fLpPD3xmkRoCIOwVKIos4l/chfAIE8gNlTBFOAwUYP0uVpA+SYNDOciT64Apj
32jELwHJJVgjG21Lubx35kOtmQa884hBB9T8RsovL35PhFvspSvTx8U+YfGKIZzG
liWwj/gBMMGd3p6pvz9UQsnqBLAfw50M6BDDQrQtoIDnw2R5s8oBqYa7uiRBzQch
dUGUm/gt9lBTI0fT3ZgCMD3Zu2et252nsbzMYgBuPSg6SlT63wHktzq1aewQ2lL2
VcBBbIf4hpkL5QnPgzKuiHcU7tBeRngTaWhw0Nc8kfGuz56HsEJJyhaHtD5mlCx9
0treI/NPAeA8KcrpnkufTpMCee7/R7CfH/dNp29yJlhbC+WYMbr6s600jJISf6zn
s0C40/MGLvVwIgT6HBkXkDL0Lii8vxc3w5smLiQ4xvQSHSS/fkP2qIDUhrX0eUlq
atacso0j7XAKYWBRHT70ZeXIN4UJuQ+dfK7xAC+bmyo9X9jcpUeozws8OvczYBRq
2qk4hCFFP/WgZ/MBiVoe2xmC6+ak2gH6xX6w2bB0/4Dc6KBMxWyUmRPuBVvx/cCp
AwXvH94gZl9wj/tmvOoZNqaMFG3tWuWo7+YzosWOBHAoUk8ILNM=
=ZuYB
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,16 @@
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEz9yiRbEEPPKl+Xhl/+h0BBaL2EcFAmVuFigACgkQ/+h0BBaL
2EeHPg/+LU5xs2ZDrQogDcH+A1v8RyursiggypdM5hXTrsFsTCIk4iekcI9xkhG1
ltNX4UuCe5PUEbTgtaWP0ncXARrUnPCoQaQ1sHVDTYoHegancsk+sXZc1JM7qr0p
Y4Ig6mKjuHFMXCInQSI2GaH4t5r4Z1jGk/PGrecIHOPJgqfA/6Z3TBF5N+y3jEvS
2QazMB298q4RDhh9m3REe8LwFPHDlfw9eRohv0MB8xygg9KtxhLZrN7gLBQZvKGD
ihNw6EgJj5OZ0dvwKCCXnlZuwknuJW7vAOPHhYeenPdVdYCGoRSyN7JdD07L+5AG
O14l2rqZrz5Eu28by+kAUrcPYAfAXekw1PmtT3HSd9U/nqnUiTkkJcjyGG/e3cjJ
sUDKMNCSBq0G7j5DB3bB6VHkZjVuz+T+iR5QdfJ4kI2pYSuE/rUj1rhkUXApYsHl
7Wff0QbOW6QT1wCtQcMpJSzkTDVJVYxiqrko/ihlOhphDHYLdOIGOrxWAUwc06x/
BhJD6tM1kEVZvifoJp1OsNwDzZ/Ku6CUs05E1vWxdeNVeANyKAgCZ5hOVmhnv866
11zfgo/znRsMzMIyJuy0bhO0C6omVLzzfhipAbZM2jDorn37xxV0v/I0pceNtLrp
YR7Tjs7+Ihe6/oItjW53j9T7ANdgQ1RVDg98lKlPFNL+hxfctwY=
=0Pkd
-----END PGP SIGNATURE-----

View File

@ -16,9 +16,10 @@ LEVELS = (None, 1, 2)
# list of globs of test and other files that we expect not to have bytecode
not_compiled = [
'/usr/bin/*',
'*/test/bad_coding.py',
'*/test/bad_coding2.py',
'*/test/badsyntax_*.py',
'/usr/lib/rpm/redhat/*',
'*/test/*/bad_coding.py',
'*/test/*/bad_coding2.py',
'*/test/*/badsyntax_*.py',
'*/lib2to3/tests/data/bom.py',
'*/lib2to3/tests/data/crlf.py',
'*/lib2to3/tests/data/different_encoding.py',

View File

@ -0,0 +1,171 @@
'''Script to perform import of each module given to %%py_check_import
'''
import argparse
import importlib
import fnmatch
import os
import re
import site
import sys
from contextlib import contextmanager
from pathlib import Path
def read_modules_files(file_paths):
'''Read module names from the files (modules must be newline separated).
Return the module names list or, if no files were provided, an empty list.
'''
if not file_paths:
return []
modules = []
for file in file_paths:
file_contents = file.read_text()
modules.extend(file_contents.split())
return modules
def read_modules_from_cli(argv):
'''Read module names from command-line arguments (space or comma separated).
Return the module names list.
'''
if not argv:
return []
# %%py3_check_import allows to separate module list with comma or whitespace,
# we need to unify the output to a list of particular elements
modules_as_str = ' '.join(argv)
modules = re.split(r'[\s,]+', modules_as_str)
# Because of shell expansion in some less typical cases it may happen
# that a trailing space will occur at the end of the list.
# Remove the empty items from the list before passing it further
modules = [m for m in modules if m]
return modules
def filter_top_level_modules_only(modules):
'''Filter out entries with nested modules (containing dot) ie. 'foo.bar'.
Return the list of top-level modules.
'''
return [module for module in modules if '.' not in module]
def any_match(text, globs):
'''Return True if any of given globs fnmatchcase's the given text.'''
return any(fnmatch.fnmatchcase(text, g) for g in globs)
def exclude_unwanted_module_globs(globs, modules):
'''Filter out entries which match the either of the globs given as argv.
Return the list of filtered modules.
'''
return [m for m in modules if not any_match(m, globs)]
def read_modules_from_all_args(args):
'''Return a joined list of modules from all given command-line arguments.
'''
modules = read_modules_files(args.filename)
modules.extend(read_modules_from_cli(args.modules))
if args.exclude:
modules = exclude_unwanted_module_globs(args.exclude, modules)
if args.top_level:
modules = filter_top_level_modules_only(modules)
# Error when someone accidentally managed to filter out everything
if len(modules) == 0:
raise ValueError('No modules to check were left')
return modules
def import_modules(modules):
'''Procedure to perform import check for each module name from the given list of modules.
'''
for module in modules:
print('Check import:', module, file=sys.stderr)
importlib.import_module(module)
def argparser():
parser = argparse.ArgumentParser(
description='Generate list of all importable modules for import check.'
)
parser.add_argument(
'modules', nargs='*',
help=('Add modules to check the import (space or comma separated).'),
)
parser.add_argument(
'-f', '--filename', action='append', type=Path,
help='Add importable module names list from file.',
)
parser.add_argument(
'-t', '--top-level', action='store_true',
help='Check only top-level modules.',
)
parser.add_argument(
'-e', '--exclude', action='append',
help='Provide modules globs to be excluded from the check.',
)
return parser
@contextmanager
def remove_unwanteds_from_sys_path():
'''Remove cwd and this script's parent from sys.path for the import test.
Bring the original contents back after import is done (or failed)
'''
cwd_absolute = Path.cwd().absolute()
this_file_parent = Path(__file__).parent.absolute()
old_sys_path = list(sys.path)
for path in old_sys_path:
if Path(path).absolute() in (cwd_absolute, this_file_parent):
sys.path.remove(path)
try:
yield
finally:
sys.path = old_sys_path
def addsitedirs_from_environ():
'''Load directories from the _PYTHONSITE environment variable (separated by :)
and load the ones already present in sys.path via site.addsitedir()
to handle .pth files in them.
This is needed to properly import old-style namespace packages with nspkg.pth files.
See https://bugzilla.redhat.com/2018551 for a more detailed rationale.'''
for path in os.getenv('_PYTHONSITE', '').split(':'):
if path in sys.path:
site.addsitedir(path)
def main(argv=None):
cli_args = argparser().parse_args(argv)
if not cli_args.modules and not cli_args.filename:
raise ValueError('No modules to check were provided')
modules = read_modules_from_all_args(cli_args)
with remove_unwanteds_from_sys_path():
addsitedirs_from_environ()
import_modules(modules)
if __name__ == '__main__':
main()

View File

@ -61,7 +61,7 @@
if rpm.expand("%{?py3_shebang_flags}") ~= "" then
command = command .. "-%{py3_shebang_flags}"
end
command = command .. " %{_rpmconfigdir}/redhat/import_all_modules.py "
command = command .. " %{_rpmconfigdir}/redhat/import_all_modules_py3_11.py "
-- handle multiline arguments correctly, see https://bugzilla.redhat.com/2018809
local args=rpm.expand('%{?**}'):gsub("[%s\\\\]*%s+", " ")
print(command .. args)

109
SOURCES/pgp_keys.asc Normal file
View File

@ -0,0 +1,109 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFq+ToQBEADRYvIVtbK6owynD3j3nxwpW2KEk/p+aDvtXmc2SR2dBcZ8sFW2
R5vEsG8d3/D3wgv5pcL3KfNNXQYUnXVbobrFUUWQYc79qIsE3MgiPf5NVOtwKPUR
i5g9YJgKvpBxkQfqp3LYGm9ZBtwo3DVLA3yn7KsazCmAgTNFJYw7ku1XxgmIzY6K
5J30DfbJiqDqj4f9GslCCCCH3qiPnuLG/HUyVLHMpbWlaiy9NI0GcaLxjJewHj9w
W2D2lydkxe5JGo7egUkV3ILcuLVSVKA35SKY27dYqfuyqp9tAzaRbjDYjsYdHA6G
BqrNrKBn/GwlFDPrVdcvN3ZSY2wMLTxWE3Axc/FweuHxFnou/80FwX7F3JD+oEQ6
rofmcxOBCC7J98I7HZAhP9jBn88XIS2hztbLq8d6rZJZRtcz0k61VR0ddO+TrFmf
9rMYCPgCckRtVxeFIVIabrN1IzKynLFeo040h8hSGswd6YKDOVwjJY6Oa6EmVefZ
a8QSt4+M65RSzH6SEPY008F3nJUAK6MEkzTak+tFltZNrVWu8p2xd1j9nmxAwEhZ
/lgbxLqzYgaUWmfyHeZ8yVA0MhHzdiAL8nVUEdG3KecIq0RWCJLGLWWIjd6KAJl1
yAmhRYKK/sjPDsL3elHsFACfZbyx3o5GGQNlas1FYoPLWbaNGaJtgFTF2QARAQAB
tCtQYWJsbyBHYWxpbmRvIFNhbGdhZG8gPHBhYmxvZ3NhbEBnbWFpbC5jb20+iQJO
BBMBCgA4FiEEoDXIwZIZuoIezqhrZOYo+NaEaW0FAlq+ToQCGwMFCwkIBwMFFQoJ
CAsFFgIDAQACHgECF4AACgkQZOYo+NaEaW2bmA/+PXIap2udLoUVOHxnsIBdqYwp
sv1Aj5lfIJmNhmxPbHShwp1Jg+w4urxe+2Dj5ofKVlIo1i83bQkvnKJMDXDVuc/K
P6zqhBJ3rT4Q3qx2mzX8bIfQoJ2JHuH4lkP+I7doDcHHRyeNASyk72VdQmU4twNw
Ibn8nSNV6ThKHdoPYzVnO2rZUFcGIqH5HNsvR+B7cc1MBCHsgURYwSVhSePIFGlZ
iasdBD6QQkDSe4QWi7AcJFWFElw4kbOKJWxAWsrEk+tMXJVGRjnmL289EmPCx/vx
BqKy7Mse0yWCSRR3vB+O6TB1S5SgEyEgqlYsfGNv1qf/rfRD4KkyCbNU3LhY1Aim
vJP4pDW+KFxTk2Ks8vrx8gOSd2aFqPeO/pFDrpsF7PD62XwsfoXu4xc5V0Giw7r1
Nai0nax7kOrldNF8TbbtRjW0jmoC7wLIDujAkwDIOroZ0CXA3N4HVHdSbrHm/urX
nyxJXupXAQNwGx64JCBcbF2fp3Kvu1VAXBEFnd01KaopthHcbG5pA50Kl2Vhe+98
OdezUX42fHkQpQkB7HgtXfm6W1bw6YRBamrNvs1OoHBYmUjlECpe566IIu25Hc8s
x3qA+6eca7iqizyLG+WyMT8ZIYTWGAS59jxwR4esqGczbbZPSAPHFwLbGv7Wr0Rd
TPu5B0FcKpDkTd4IxQW5Ag0EWr5O2gEQAMjLe4CtbSfofmJrz5wfNkMVsZ81Gbqe
MoYd3dtkJnQYERUj8flzBj3ucaxGJ+Cuf7ybh3naPopKvEI1q0vkcgCDqrEgXK//
jKJbP28uPSMGhOG28q4PbamG55gy5FtM3ezzAxPWWKe9qBpV65GMmFy7eBQx2iJs
yiDIOOQQ4kraS+cTqNFimEXAGLCOQRNLcwIZzwAAHoW7HEpNUfVwaBD9kMlbo1ND
I60IKcNrNcmcmRxhJqfxjj8YBMwcKHO6GBE3AVpaE/+UO9zyr4TH+0YuQUgxKlPW
Dkg5XlkDo0S1GyLY5e9ckIDIlkTdDa2pOkoE2yB5MQCEga3YiHrKUVTTWaxn9XVJ
6x5ZjUF6bgSWGkrG5dUqSYoO1iDMuNVjtiujNyf/rvfj5cNxS7/lgxchhQKZHZXL
WVqxlneeVJ6s0P4+ROVG9ga2Sve7aUJ6wXIewZwulBcV2sE/W/DgxHgLBi53CUQt
vEzFzKvo48GnDqL5VYjA7l0HMYHd4GksCLi8E8U6Cgj+imXiM8voL7pHRZfs8mY8
udR+UT4e1Scl2MYP2qBJ9/17B/X52B3s1EZdqI/r+hfOyqrhPs+dbAN0mtMPn68+
nrvY1+nscvrSYEP6ZBlc9Hp2mgJdb6IcTvINXBEeLRjgc3pjViva443pkiFp9Axm
ecOckMKP3uSlABEBAAGJBGwEGAEKACAWIQSgNcjBkhm6gh7OqGtk5ij41oRpbQUC
Wr5O2gIbAgJACRBk5ij41oRpbcF0IAQZAQoAHRYhBM/cokWxBDzypfl4Zf/odAQW
i9hHBQJavk7aAAoJEP/odAQWi9hHr7YP/RCLre1CmOoWYpAtoa1yVCeYMDV6eQgL
B488/BEZHQE1zbrYy16XkhORob3JF/kUMjmJW7XaFF8FrWvRcdj/xaUGbOOEulKg
v+8zWfswYQRiZ4/JlwER4vRLi6fTE89MVER6Fkj2ASD4D2cifY+EztD4flV3sq3s
vIogGFaN9IvdrdeptOVGXs1RmAyoTsiS2mKQ6xsGh8B9ZAm55W8fBOGiSzLX21Xk
Ofdw53BrFQxn3cu/JgIKpdeZxgukcvEAI62B6X+YL6Na4j0eqEGLzsNtU1+xeJlo
WtVvmRwnRHGSxF6fzIZ3mk/p/aFiXAEq/xITCTY6tDv7x7pFE/RpdlJZyNJ+R5Y4
SQiuDsylxNCa/4G5EB6q+7iVYtbEQ9MnZg2phowEE42tlj0rz8/rvDK3LH3xibot
KHIodCWKlWByxH99u2PuHUQ0c1oCVBUE1KkruMpvI236DpU/dvdq4JLSg/fWrys/
VIjqLZgsIE5g/KO9XqngWHkLcBLh4CNAmHJ8Iia+s+/rfgsejQWB5uJb6eYg2JjB
4WP1EI0rULM6fdrCNB+MJ36wE2Lnb4bfT0phOMgjjH5/Ki7ZCbkxkOsBs4SRjiS+
weCsmpAtMqodWY/Cnw9pWSA/qLSRD5/mKeb9SO6OZ/OPfAatwnGHsvZ2sAueC6rR
04W5BfXZWrnJUXQP/id/EKE1Ksp5fKoxSCbkKTCig+Sf5Afwe36yFN+niZBqzn5b
BgL/HIKaZM97oDHersPPANeEgS+JVlBf95iKIYnQbZP43FLVbvOuaINhBIVtFO54
2Y7EYwl41kP7ILDElVy36KAmdQyBAfrjnZiRA70xShOxApLug1L0lxhR3YfmLwNi
RJ0V6KnYDKf0pfdhO9VFyFFWUojX1usn2SmSsXNizsNtvRqHXzPnX0rbJzZ9+N4O
9k1nxygYFG/2R/jGonVmTjRzcAHrAkNJETMWXMA7/8wRMDwluz8j+cCldey9x8Vk
JwgLGnZSbQtVpcFAnm5r/36Gt+9wc1VWMyrUrVr6Z679aqAbG7PMaeR5h5ygMj1k
VqRTYAUPSk1f8bZKRssQkQwEbp9dVIjm9SsR8VT7/tB+UuB85dABxgHfv3psJRT+
tL8g9V7kSZqQfcLNGmvEVvr2Zl9NtxwXtsFM2OBprxCenwb+e9Ppm1LjfJG/NE72
mAnOERfDaiLt4bqNo36Ei5sGCJ4Fx61phzNBXzkdRNM47i8J5UZRKFkE91c99BVM
HKUaY61NRK24fR0zP98ftDU82YFw0VRFJpTeBrO5ivN1MlQxUPzUWxKxMxO+20wa
UOXroEw11Tb4SRLGOla1pCl6lCUPJRy9IzadPDgTr/OTMkob/snt/XLdnV5/uQIN
BFq+TvoBEAC8Oy1g6pPWBbrCMhIq7VWY2fjylJ1fwg5BPXkOKVK1dsGYO4QD7oW9
L0aSqcFSNFGF9Cl0Ri4TFXZC3hnG4HeSXUWApuKdBLn21H3jba36Ay1oGcGfdm0v
Zght4c6BlMVBpGCw2wIkJbUNEy6InMM+O8CCbbaH3iJkJ4141P7pODHignx5AmZI
conMui4YOhC+IXQXynVEv1Juk7erB1Nh1RcRvsA4lb44HWx49lIwe85ejOmoZ0O3
6f9NJRer6bV0+rHWmg4IV5Q9h/Gn4IhEDZxA0DZl1RQI7dMgaMbIFbXGq7Kgzstz
EUnOoy29hXodxVmwIsMrAiQUYtwJ9hW+ESsw47+W2iPHVgviGWl7r/SgcgMYmf6m
5kiTBtwU7BQPS9G3zwwP2Rm3AA/6g39Q+tQKjOwi1I8+GZsY2On44Zly7BreBNg5
4gJgdAGcMOYU9etr050clH3UpTYcAEtX++ahtOKhJgLIPNcIAQNlnifqvU0VYpgw
R4YpZ7hgg+AVDzC73PIM0lFI0XiDuqChbxE+K1jmLXWe5iJF0dzgVTwP+PmsifNZ
Wg3+YxSsS+hDMPQ2xPiQN49gT4JJDHcDuyhHyCGYgyMiVJCsku9KrkubbfVRivyN
ZF2Zfo3f+nbrRxsftz0yjAq8byCvb0V0XOpt4pJ/ddlug9ytRxALNwARAQABiQI2
BBgBCgAgFiEEoDXIwZIZuoIezqhrZOYo+NaEaW0FAlq+TvoCGwwACgkQZOYo+NaE
aW3urA//UQ/cKQ7HvWjcLphzQOZc+6m5YL0wxvZkSjemU7mqjZdpacteIvRAoers
EqXHc208liIBtNfRzoreXdcXNzie65xXkrRnWoHVH/fTWy4lOnHr2CMXLeHjUgg/
M6PYi8+sARm05YFB8nsYhlhx3IdLhcfeVVbJedQKO0yL3CK1okT30DUVq5Lq6X/K
DC6AxuJR3D6UMSoT0WLaoX8qbhAp88qLynInfBVL18d97h916WPLTPeP0eHwhwND
bYtKDCMDuKQ9XX5+QsNH0RmbxlX274LHrUMMvkLKxcfCBvP+iuqrBeIuoeVzXYJZ
j7ZJtEH79bW44eecl/CY/STFYgSQ2XGTp2BI2q60wAmtKlNhwxY5ena0FgyFl6Tm
5OBHW/Pwo+ndQJGfbrCyWkTgRay9c8er3gl3GQYIBH6X0kCiG7h/Epj0b5CHOPU5
hCw0kEB8MB4poTIjeiY+Q01472/lQ68CL3DX158hR5d3XaPSIxAN+qFsfB1o316p
yjxhfK1MD/IfrOgjlggPPnc/KmLkCzpgdwKcZwLCdZq9hYBvF1Zs34HbaVMYbWTK
uxLowtXGU43vatCXXqmPOvl4/g4tZD6rysJDgOrHQnEHzT+Napn07s0BRC0IbbNn
FynUrkr5KMSuRz7Hg7xMApENOrb0nqdHSUJ914ZpuMIS6RhJgGu5Ag0EWr5PIAEQ
ALfh9vPD2B+miHDTMADI8aRZ7g9tnzynZYkk3+2sCiiusetsQQ+HIPJ/ASEJB7On
ane9dyT/LTRhrK9qaxgVMimk2COXB/xyh7Mnw7nJgFU0aRSbtX0vbvQz2suSzrQ6
9mPKzan28JGoClqB0bw1vwf3VjjxHV2dgD57CmqFPv7kAC/2a56dE+etzXattZAL
+2JWTpmfQ0ePRRadtBm0VahQhnU8x0+jvAVrEawqpVW83ozYFyW/0WInM2J7jHgQ
16OosY4lj5L/DxpVxaArhRFoRfWPXfC37iE8Mou/I95isvPQIhp1wTo4jG0KM02B
oIVbp/QRNBQ6WtpOzvJs1gqQiJJTfqbKJXQ3NDEY9crpVS83HJ+Zv99PNsyNkFjG
QpU84U3ZhsI4ygjdY45mpZueqI1RVcRQdu8Hgvoo/78Q/Sir6gMGop3mVdVo2guI
kFcJrXh0Xk3ech4aVqrmKx/mPXGwOAQU0DAul4RW3fKg1QxQE7Tlw3+95Ee/+q5j
HARL0uDbCJpRO8Sl8NDEuL32n/2Ot6kQeCSHrU7KJRYAkTxkKvr8zNow7hFhHFPE
SnHvTnskI6noh0VY6NwMhmLvhm0wKkRxZPzUNc3sgLvbK1NymIZ9aKCZamzhZrmG
vnblEz/OSLwGUua465H3hM1vvBQiartj7+6ZqWIkSmBPABEBAAGJAjYEGAEKACAW
IQSgNcjBkhm6gh7OqGtk5ij41oRpbQUCWr5PIAIbIAAKCRBk5ij41oRpbWmeEACG
+axtDC8UoNp9ORiYwEWLzZWDuugE+ah7DYYGD4Vs633FXVZW3SgM/bFtJ/0Lg8CF
74jI4LMHyIjDzEjcoItwnhBLix+kUoJTvrY58GPydwekLuw1p4KXLqtRs4fsZbNQ
YTknl4jYtRWoxO98x7tun7Gq2gqmJkIB2uj630fKz5cBk6p6oDFKjzyrHe+V7BiK
3okQPaD4x7hq8OnTy7lOy92ZZAqztS4tNEb4DkYW1MpuwsJ7hbBZitc1siI+FVVb
GjVVGZz6ssXoW67Tz8+VxdWJxNLXlv27eMcj4sme5S0th/YYNA5fRRv6zuzqZAru
YNGLpYYU7JLvZJ+3lCwa5j5ycOGBF0GvsGs6gj6h+CHkjR/BgzAgWC+GgUgslt6q
aH04rWtV6rVz+Y91LcrX5P6OM4anmXD3Gp3kl35AypXb4KyASF19+11RUziD4Z7q
wQEWfbwOltNyZv2lD8s2jPr7P02axWRQUbZAEhxRmvOQev/FZPyCF6gqUo/HxRbQ
y3bzmnipyHSv1DlXNfCFCHvN8kGyZnRWARqIKRg+j9ediJgOUqlLhg6KmrTVxd5v
3Dfv52PW2UODDTM20s3cQGuX/UswzMRwPI/+P44iCMwEKdm7duM/5oisZT9Vhy7g
P15MreFZLcZvUVgjqgy0u57cstyGK1Bo9e2sFcK2fA==
=6Zb4
-----END PGP PUBLIC KEY BLOCK-----

View File

@ -16,11 +16,11 @@ URL: https://www.python.org/
# WARNING When rebasing to a new Python version,
# remember to update the python3-docs package as well
%global general_version %{pybasever}.1
%global general_version %{pybasever}.7
#global prerel ...
%global upstream_version %{general_version}%{?prerel}
Version: %{general_version}%{?prerel:~%{prerel}}
Release: 2%{?dist}
Release: 1%{?dist}
License: Python
@ -55,15 +55,15 @@ License: Python
# IMPORTANT: When bootstrapping, it's very likely the wheels for pip and
# setuptools are not available. Turn off the rpmwheels bcond until
# the two packages are built with wheels to get around the issue.
%bcond_without bootstrap
%bcond_with bootstrap
# Whether to use RPM build wheels from the python-{pip,setuptools}-wheel package
# Uses upstream bundled prebuilt wheels otherwise
%bcond_with rpmwheels
%bcond_without rpmwheels
# If the rpmwheels condition is disabled, we use the bundled wheel packages
# from Python with the versions below.
# This needs to be manually updated when we update Python.
%global pip_version 22.3.1
%global pip_version 23.2.1
%global setuptools_version 65.5.0
# Expensive optimizations (mainly, profile-guided optimizations)
@ -252,7 +252,10 @@ Source0: %{url}ftp/python/%{general_version}/Python-%{upstream_version}.tar.xz
Source1: %{url}ftp/python/%{general_version}/Python-%{upstream_version}.tar.xz.asc
# The release manager for Python 3.11 is pablogsal
Source2: https://keybase.io/pablogsal/pgp_keys.asc
# Sources for the python3.11-rpm-macros
Source3: macros.python3.11
Source4: import_all_modules_py3_11.py
# A simple script to check timestamps of bytecode files
# Run in check section with Python that is currently being built
@ -307,6 +310,21 @@ Patch251: 00251-change-user-install-location.patch
# Ideally, we should talk to upstream and explain why we don't want this
Patch328: 00328-pyc-timestamp-invalidation-mode.patch
# 00329 #
# Support OpenSSL FIPS mode
# - In FIPS mode, OpenSSL wrappers are always used in hashlib
# - The "usedforsecurity" keyword argument can be used to the various digest
# algorithms in hashlib so that you can whitelist a callsite with
# "usedforsecurity=False"
# - OpenSSL wrappers for the hashes blake2{b512,s256},
# - In FIPS mode, the blake2 hashes use OpenSSL wrappers
# and do not offer extended functionality (keys, tree hashing, custom digest size)
#
# - In FIPS mode, hmac.HMAC can only be instantiated with an OpenSSL wrapper
# or a string with OpenSSL hash name as the "digestmod" argument.
# The argument must be specified (instead of defaulting to md5).
Patch329: 00329-fips.patch
# 00371 # c1754d9c2750f89cb702e1b63a99201f5f7cff00
# Revert "bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549) (GH-28589)"
#
@ -343,15 +361,29 @@ Patch371: 00371-revert-bpo-1596321-fix-threading-_shutdown-for-the-main-thread-g
# Upstream: https://bugs.python.org/issue46811
Patch378: 00378-support-expat-2-4-5.patch
# 00395 # 18ff37a92c507144edf32274b356dd1dd734cf07
# GH-100133: fix `asyncio` subprocess losing `stderr` and `stdout` output
Patch395: 00395-gh-100133-fix-asyncio-subprocess-losing-stderr-and-stdout-output.patch
# 00397 #
# Filters for tarfile extraction (CVE-2007-4559, PEP-706)
# First patch fixes determination of symlink targets, which were treated
# as relative to the root of the archive,
# rather than the directory containing the symlink.
# Not yet upstream as of this writing.
# The second patch is Red Hat configuration, see KB for documentation:
# - https://access.redhat.com/articles/7004769
Patch397: 00397-tarfile-filter.patch
# 00396 # 4c775fbed65016fec5dfd66316559024d2af9135
# gh-100160: Remove any deprecation warnings in asyncio.get_event_loop()
# 00415 #
# [CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() (#111116)
#
# Some deprecation warnings will reappear (in a slightly different form) in 3.12.
Patch396: 00396-gh-100160-remove-any-deprecation-warnings-in-asyncio-get_event_loop.patch
# Detect email address parsing errors and return empty tuple to
# indicate the parsing error (old API). Add an optional 'strict'
# parameter to getaddresses() and parseaddr() functions. Patch by
# Thomas Dwyer.
#
# Upstream PR: https://github.com/python/cpython/pull/111116
#
# Second patch implmenets the possibility to restore the old behavior via
# config file or environment variable.
Patch415: 00415-cve-2023-27043-gh-102988-reject-malformed-addresses-in-email-parseaddr-111116.patch
# (New patches go here ^^^)
#
@ -371,10 +403,10 @@ Patch396: 00396-gh-100160-remove-any-deprecation-warnings-in-asyncio-get_event_l
# Descriptions, and metadata for subpackages
# ==========================================
# Require alternatives version that implements the --keep-foreign flag
Requires: alternatives >= 1.19.1-1
Requires(post): alternatives >= 1.19.1-1
Requires(postun): alternatives >= 1.19.1-1
# Require alternatives version that implements the --keep-foreign flag and fixes rhbz#2203820
Requires: alternatives >= 1.19.2-1
Requires(post): alternatives >= 1.19.2-1
Requires(postun): alternatives >= 1.19.2-1
# When the user tries to `yum install python`, yum will list this package among
# the possible alternatives
@ -521,11 +553,9 @@ Requires: %{pkgname}-libs%{?_isa} = %{version}-%{release}
# But we want them when packages BuildRequire python3-devel
Requires: (python-rpm-macros if rpm-build)
Requires: (python3-rpm-macros if rpm-build)
Requires: (python-modular-rpm-macros if rpm-build)
Suggests: python3.11-rpm-macros
# Require alternatives version that implements the --keep-foreign flag
Requires(postun): alternatives >= 1.19.1-1
# Require alternatives version that implements the --keep-foreign flag and fixes rhbz#2203820
Requires(postun): alternatives >= 1.19.2-1
# python3.11 installs the alternatives master symlink to which we attach a slave
Requires(post): %{pkgname}
@ -578,8 +608,8 @@ Provides: idle = %{version}-%{release}
Provides: %{pkgname}-tools = %{version}-%{release}
Provides: %{pkgname}-tools%{?_isa} = %{version}-%{release}
# Require alternatives version that implements the --keep-foreign flag
Requires(postun): alternatives >= 1.19.1-1
# Require alternatives version that implements the --keep-foreign flag and fixes rhbz#2203820
Requires(postun): alternatives >= 1.19.2-1
# python3.11 installs the alternatives master symlink to which we attach a slave
Requires(post): %{pkgname}
@ -644,8 +674,8 @@ Requires: %{pkgname}-idle%{?_isa} = %{version}-%{release}
%unversioned_obsoletes_of_python3_X_if_main debug
# Require alternatives version that implements the --keep-foreign flag
Requires(postun): alternatives >= 1.19.1-1
# Require alternatives version that implements the --keep-foreign flag and fixes rhbz#2203820
Requires(postun): alternatives >= 1.19.2-1
# python3.11 installs the alternatives master symlink to which we attach a slave
Requires(post): %{pkgname}
@ -796,6 +826,7 @@ BuildPython() {
--with-dtrace \
--with-lto \
--with-ssl-default-suites=openssl \
--with-builtin-hashlib-hashes=blake2 \
--without-static-libpython \
%if %{with rpmwheels}
--with-wheel-pkg-dir=%{python_wheel_dir} \
@ -992,6 +1023,10 @@ for tool in pygettext msgfmt; do
ln -s ${tool}%{pybasever}.py %{buildroot}%{_bindir}/${tool}3.py
done
# Install missing test data
# Fixed upstream: https://github.com/python/cpython/pull/112784
cp -rp Lib/test/regrtestdata/ %{buildroot}%{pylibdir}/test/
# Switch all shebangs to refer to the specific Python version.
# This currently only covers files matching ^[a-zA-Z0-9_]+\.py$,
# so handle files named using other naming scheme separately.
@ -1094,6 +1129,10 @@ mkdir -p %{buildroot}%{rpmmacrodir}/
install -m 644 %{SOURCE3} \
%{buildroot}/%{rpmmacrodir}/
# Add a script that is being used by python3.11-rpm-macros
mkdir -p %{buildroot}%{_rpmconfigdir}/redhat
install -m 644 %{SOURCE4} %{buildroot}%{_rpmconfigdir}/redhat/
# All ghost files controlled by alternatives need to exist for the files
# section check to succeed
# - Don't list /usr/bin/python as a ghost file so `yum install /usr/bin/python`
@ -1168,10 +1207,14 @@ CheckPython() {
# test_freeze_simple_script is skipped, because it fails when bundled wheels
# are removed in Fedora.
# upstream report: https://bugs.python.org/issue45783
# test_check_probes is failing since it was introduced in 3.11.5,
# the test is skipped until it is fixed in upstream.
# see: https://github.com/python/cpython/issues/104280#issuecomment-1669249980
LD_LIBRARY_PATH=$ConfDir $ConfDir/python -m test.regrtest \
-wW --slowest -j0 --timeout=1800 \
-i test_freeze_simple_script \
-i test_check_probes \
%if %{with bootstrap}
-x test_distutils \
%endif
@ -1274,7 +1317,7 @@ if [ $1 -eq 0 ]; then
fi
%post idle
alternatives --keep-foreign --add-slave python3 %{_bindir}/python3.11 \
alternatives --add-slave python3 %{_bindir}/python3.11 \
%{_bindir}/idle3 \
idle3 \
%{_bindir}/idle3.11
@ -1282,7 +1325,7 @@ alternatives --keep-foreign --add-slave python3 %{_bindir}/python3.11 \
%postun idle
# Do this only during uninstall process (not during update)
if [ $1 -eq 0 ]; then
alternatives --remove-slave python3 %{_bindir}/python3.11 \
alternatives --keep-foreign --remove-slave python3 %{_bindir}/python3.11 \
idle3
fi
@ -1292,6 +1335,7 @@ fi
%files -n %{pkgname}-rpm-macros
%{rpmmacrodir}/macros.python%{pybasever}
%{_rpmconfigdir}/redhat/import_all_modules_py3_11.py
%files -n %{pkgname}
%doc README.rst
@ -1378,11 +1422,6 @@ fi
%{pylibdir}/pydoc_data
%{dynload_dir}/_blake2.%{SOABI_optimized}.so
%{dynload_dir}/_md5.%{SOABI_optimized}.so
%{dynload_dir}/_sha1.%{SOABI_optimized}.so
%{dynload_dir}/_sha256.%{SOABI_optimized}.so
%{dynload_dir}/_sha3.%{SOABI_optimized}.so
%{dynload_dir}/_sha512.%{SOABI_optimized}.so
%{dynload_dir}/_asyncio.%{SOABI_optimized}.so
%{dynload_dir}/_bisect.%{SOABI_optimized}.so
@ -1645,6 +1684,7 @@ fi
%{dynload_dir}/_ctypes_test.%{SOABI_optimized}.so
%{dynload_dir}/_testbuffer.%{SOABI_optimized}.so
%{dynload_dir}/_testcapi.%{SOABI_optimized}.so
%{dynload_dir}/_testclinic.%{SOABI_optimized}.so
%{dynload_dir}/_testimportmultiple.%{SOABI_optimized}.so
%{dynload_dir}/_testinternalcapi.%{SOABI_optimized}.so
%{dynload_dir}/_testmultiphase.%{SOABI_optimized}.so
@ -1673,11 +1713,6 @@ fi
# ...with debug builds of the built-in "extension" modules:
%{dynload_dir}/_blake2.%{SOABI_debug}.so
%{dynload_dir}/_md5.%{SOABI_debug}.so
%{dynload_dir}/_sha1.%{SOABI_debug}.so
%{dynload_dir}/_sha256.%{SOABI_debug}.so
%{dynload_dir}/_sha3.%{SOABI_debug}.so
%{dynload_dir}/_sha512.%{SOABI_debug}.so
%{dynload_dir}/_asyncio.%{SOABI_debug}.so
%{dynload_dir}/_bisect.%{SOABI_debug}.so
@ -1773,6 +1808,7 @@ fi
%{dynload_dir}/_ctypes_test.%{SOABI_debug}.so
%{dynload_dir}/_testbuffer.%{SOABI_debug}.so
%{dynload_dir}/_testcapi.%{SOABI_debug}.so
%{dynload_dir}/_testclinic.%{SOABI_debug}.so
%{dynload_dir}/_testimportmultiple.%{SOABI_debug}.so
%{dynload_dir}/_testinternalcapi.%{SOABI_debug}.so
%{dynload_dir}/_testmultiphase.%{SOABI_debug}.so
@ -1803,6 +1839,49 @@ fi
# ======================================================
%changelog
* Mon Jan 22 2024 Charalampos Stratakis <cstratak@redhat.com> - 3.11.7-1
- Rebase to 3.11.7
Resolves: RHEL-21915
* Tue Jan 09 2024 Lumír Balhar <lbalhar@redhat.com> - 3.11.5-2
- Security fix for CVE-2023-27043
Resolves: RHEL-7842
* Thu Sep 07 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.5-1
- Rebase to 3.11.5
- Security fixes for CVE-2023-40217 and CVE-2023-41105
Resolves: RHEL-3047, RHEL-3267
* Thu Aug 10 2023 Tomas Orsava <torsava@redhat.com> - 3.11.4-4
- Add the import_all_modules_py3_11.py file for the python3.11-rpm-macros subpackage
Resolves: rhbz#2207631
* Wed Aug 09 2023 Petr Viktorin <pviktori@redhat.com> - 3.11.4-3
- Fix symlink handling in the fix for CVE-2023-24329
Resolves: rhbz#263261
* Fri Jun 30 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.4-2
- Security fix for CVE-2007-4559
Resolves: rhbz#263261
* Mon Jun 26 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.4-1
- Update to 3.11.4
- Security fix for CVE-2023-24329
Resolves: rhbz#2173917
* Thu Feb 16 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.2-2
- Support OpenSSL FIPS mode
* Thu Feb 09 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.2-1
- Update to 3.11.2
* Tue Jan 31 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.1-4
- Disable the builtin hashlib hashes except blake2
* Mon Jan 30 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.1-3
- Disable bootstrap
- Revert python3.11-rpm-macros requirement
* Mon Jan 30 2023 Charalampos Stratakis <cstratak@redhat.com> - 3.11.1-2
- Fix macros requirements