That way, accidentally unpackaged files within are reported as errors.
Currently, when %pyproject_extras_subpkg is used, the dist-info directory
is packaged as %ghost. When the main package does not have it,
the RPM build would succeed. The extras packages would have the python3dist()
requires and provides, but the main package would not.
By adding %dir after %ghost, we only package the directory
(which is enough for python3-rpm-generators to process it),
but the files in the directory are not included.
When not packaged in the main package, the RPM build fails.
This is a safeguard against packaging mistakes.
The visible difference is that rpm -ql/repoquery -l would only return the metadata directory.
See also https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/195
For packages with a lot of files, it took a really long time to run.
Profiling the code revealed most of the time is spent in PurePath.__eq__.
The code was using lists but checked
if to-be-added paths were not already in them.
Considering the order is insignificant
(the generated %files list is sorted at the end anyway),
the lists were essentially working as (very slow) sets.
Using sets instead of lists makes %pyproject_save_files over 20 times faster
(51.66 -> 2.39 seconds) for the ansible package (~62k files).
Checking if an item is in a list is O(N),
checking every added item is O(N**2).
Checking if an item is in a set is O(1),
checking every added item is O(N).
Additionally, with set, it is unnecessary to check for presence before addition,
so the code is easier.
(Commit message and removal of the check by Miro.)
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
- Fixes: rhbz#2391290
Consider code like this:
%generate_buildrequires
mv setup.py{,.ignore}
%pyproject_buildrequires
mv setup.py{.ignore,}
When %pyproject_buildrequires exits, the second mv is never executed.
The next round of %generate_buildrequires will then fail.
In fact, there is probably not a good reason to call exit from %pyproject_buildrequires.
From https://src.fedoraproject.org/rpms/libcpuid/pull-request/7
See https://packaging.python.org/en/latest/specifications/core-metadata/#requires
That field is deprecated and should include importable module names, not distribution packages.
We have no RPM Provides for importable names.
Treating this like python3dist() Requires is wrong and may result in stuff like:
No match for argument: python3dist(pkg-resources)
For packages using python-distutils-extra.
See https://bugzilla.redhat.com/show_bug.cgi?id=2378463#c2
This bug existed from the very beginning of the %pyproject_buildrequires,
but the field is almost unused in real packages, so it was not noticed until
we asked all Python packages to be ported to the new macros.
---
I considered flattening the structure returned from requires_from_parsed_metadata_file,
but then we would need to hardcode "Requires-Dist" in various source= declarations,
so I kept the structure as is.
Use double quotes around a shell variable that may contain spaces.
There were errors like:
+ '[' -z markupsafe==2.0.1 tldr==0.4.4 ']'
/var/tmp/rpm-tmp.v6rA4u: line 55: [: markupsafe==2.0.1: binary operator expected
Or:
+ '[' -z nemo_audio_tab==6.4.0 nemo_compare==6.4.0 nemo_emblems==6.4.0 nemo_pastebin==6.4.0 nemo_terminal==6.4.0 ']'
/var/tmp/rpm-tmp.BD4qxp: line 53: [: too many arguments
But considering the exit code of [ was non-zero, the built continued.
We got:
DEBUG: File "/builddir/build/BUILD/python-pytest-7.2.0-build/BUILDROOT/usr/lib/python3.14/site-packages/_pytest/assertion/rewrite.py", line 670, in run
DEBUG: and isinstance(item.value, ast.Str)
DEBUG: ^^^^^^^
DEBUG: AttributeError: module 'ast' has no attribute 'Str'
Version 8.0.2 builds in Fedora 43, 42, 41, EPEL 10 (still without tests).
The setup.py manipulation was moved to pluggy,
as pytest 8 no longer has setup.py.
PEP 517 says that the argument should be named config_settings but
pip/pyproject-hooks and other build frontends just pass it as a
positional argument, so some build backends name this argument other
things. Even though those build backends are wrong, it still makes sense
to align pyproject-rpm-macros with what the other frontends do.
Ref: https://github.com/PyO3/maturin/pull/2612
Ref: https://github.com/Rogdham/pyzstd/pull/2
Since tox 4, tox does not fail without configuration
(tox.ini, or tox section in setup.cfg/pyproject.toml).
As a result, packages that use %pyproject_buildrequires with -t or -e
without having a tox confuration
only generate additional BuildRequires on tox & tox-current-env itself.
More dangerously, %tox without tox configuration does nothing (and succeeds).
This behavior is dangerous and warrants an announced breakage.
Packagers of ~100 affected Fedora packages were informed about the problem earlier in
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/ZSHSHZKVA4XJQBJD7FMMCALKN4UP5SAJ/
There will be a further announcement and warning about this change.
EPEL 9 packages are not affected,
EPEL 9 has tox 3 which fails without config by default.
The change used a newly added option for tox-current-env: --assert-config.
This was added in tox-current-env 0.0.16:
https://github.com/fedora-python/tox-current-env/pull/89
When updating setuptools from 74 to 76, order of extras has changed.
The order of the output is not considered stable and does not need to be asserted.
- The flag can be used to indicate no Python modules should be saved
The change wrt users using '*' was necessary,
as `glob` was undefined when `module_globs` was empty.
This way, when we add new options to the actual macro in pyproject-rpm-macros,
spec files that use them are parsable with the old pyproject-srpm-macros package.
Last time, when we added the -g option in 1.16.0,
it took at least a week for Fedora CI to be able to parse a spec file with it,
as pyproject-srpm-macros 1.16.0+ needed to actually be installed on the CI systems.
Next time this happens, the new option will be parsable with older versions of pyproject-rpm-macros
Note that the (-) syntax is not supported yet on RPM 4.16 in RHEL 9.
I'd use an expression to keep the literal set of flags for older RPM versions:
%pyproject_buildrequires(%[v"0%{?rpmversion}" >= v"4.19" ? "-" : "rRxtNwpe:g:C:"])
But macro options are not processed as macros:
https://github.com/rpm-software-management/rpm/issues/3440
As a result, this is not compatible with RHEL 9 and cannot be backported to it.
Unless the (-) syntax is backported:
https://issues.redhat.com/browse/RHEL-67161https://gitlab.com/redhat/centos-stream/rpms/rpm/-/merge_requests/60
Change the test source location to support evolving downstream testing
requirements. This is needed both for downstream certification
activities and changes to test development for internal infrastructure
differences.
Cherry-picked from https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_requests/25
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
This test was previously skipped because we didn't have new enough tox.
That's why it was never fixed for setuptools 70+.
This is a fixup for 20b7ac63f3
Keep the information about the requirement extras by storing the
Requirement instances in the list of the ignored requirements, rather
than the strings in the form they were initially read from metadata.
The requirements strings read from pyproject.toml don't contain the
extra information, we insert the extra marker only after converting them to
Requirement instances. When stored as the text, the information about
the extra went missing in the course of the script.
This adds a new flag, -p, to %pyproject_buildrequires.
When set, the runtime dependencies are read from the pyproject.toml's
[project] table.
See: https://bugzilla.redhat.com/2261939
pyproject_buildrequires.py already had a short `-p` option for
--python3_pkgversion (hidden from the macro users).
This change removes the one-letter option and leaves the long-one.
`-p` is now reused for reading dependencies from pyproject.toml
and made visible to the macro users.
