This adds a new -D/--dist-name option for:
- %pyproject_save_files
- %pyproject_files
- %pyproject_check_import
- %pyproject_extras_subpkg
And allows the use of all of those in multi-package spec files.
See README.md changes for usage.
Assisted-By: Claude Opus 4.6
Many useful suggestions by Maxwell G <maxwell@gtmx.me> and Karolina Surma <ksurma@redhat.com>.
Python 3.15 changed importlib.metadata.PathDistribution.metadata
to raise MetadataNotFound instead of returning None when no METADATA
file exists (https://github.com/python/cpython/issues/143387).
Add minimal METADATA entries for the 8 test packages that were
missing them, so the test fixtures create METADATA files on disk.
Assisted-By: Claude Opus 4.6
argparse.FileType is deprecated since Python 3.14 and triggers
(Pending)DeprecationWarning. Use pathlib.Path as the argument type
instead, and read file contents via read_text() at the call site.
Note that this brings two little changes, as pointed out by Maxwell G during review:
> This code is theoretically less performant than the previous code since it
> reads the whole file to a str and then converts it into a list instead
> of just iterating through the file stream like it did before.
This is acceptable to me. I don't think the possibility of requirements.txt file
so large that it would not fit into memory is worth considering.
Such file would likely cause memory exhaustion even later when we generate the deps.
> ...the list created by splitlines does not have the \ns preserved ...,
That's why pyproject_requirements_txt.py removed the rstrip('\n') call.
Assisted-By: Claude Opus 4.6
Previously, the log messages used the original requirement string
(before dependency overrides were applied), which was confusing.
Add a regression test that checks stderr contains the overridden form.
Assisted-By: Claude Opus 4.6
The override check in self.dependency_overrides was done twice: once at
the call site and once inside _apply_dependency_overrides. By moving the
full logic into the method and having it accept a Requirement directly,
the redundant check is eliminated.
Also remove _base_package_name, since Requirement.name already strips
extras and the only remaining need is canonicalize_name.
Assisted-By: Claude Opus 4.6
Downloads rpm-specs-latest.tar.xz, finds all specs using %pyproject
macros, and runs rpm --specfile on each to verify they parse without
errors. Only runs on rawhide as the spec files are rawhide only.
Assisted-By: Claude Opus 4.6
When macro arguments span across multiple lines with backslash-escaped newlines, e.g.:
%{pyproject_buildrequires \
-x pathops \
-x autohint \
-x json \
-x repacker}
rpm.define() for positional args would fail with:
Macro %__pyproject_positional_args has empty body
error: lua script failed: .../pyproject_getopt.lua:252: error defining macro
Because the "-x pathops" would be consumed by value parsing
and all the positional arguments would be just backslash-escaped newlines.
Defining the %__pyproject_positional_args macro to whitespace is not possible,
unless properly quoted.
Filter out line-continuation artifacts (tokens containing only whitespace,
backslashes, and newlines) before parsing, so they cannot be misinterpreted
as option values or positional args. Extract a shared quote_value() function
for safe embedding of values in rpm.define() calls.
Additionally, RPM may embed newlines within tokens when macros use
%{expand:} with literal newlines. Such tokens are split on newlines
before parsing.
The regression impacted parsing of 14 Fedora Rawhide spec files:
- dcm2niix
- fontmake
- fonttools
- libarrow
- python-django5
- python-django6
- python-pydantic-extra-types
- python-pydantic-settings
- python-pylero
- python-pynetdicom
- python-tablib
- python-trimesh
- python-urllib3
- scapy
And all are now fixed.
No other Fedora Rawhide spec files fail to parse due to the long option support
after this fix.
Assisted-By: Claude Opus 4.6
When python3 and pyproject_dependency_overrides.py are available,
the macro now calls parse_override_string at %prep time,
catching invalid overrides immediately instead of in %pyproject_buildrequires.
When only pyproject-srpm-macros is installed, validation is skipped gracefully.
Assisted-By: Claude Opus 4.6
All public parametric macros now accept long options in addition
to short options. For example, %pyproject_buildrequires --no-runtime
is equivalent to %pyproject_buildrequires -R.
This is implemented via a Lua option parser (pyproject_getopt.lua)
installed to %{_rpmluadir}/fedora/rpm/. All macros use (-) to opt out
of RPM native getopt and call the Lua parser for validation, mutual
exclusion checks, and value extraction.
Other changes:
- Toxenv construction inlined in Lua, replacing pyproject_construct_toxenv.py
- Added --config-settings and --directory long forms to Python argparse
- Renamed --read-pyproject-dependencies to --pyproject-dependencies
(this was not part of API and now it is, so I picked a shorter name)
- Added test_getopt_consistency.py to assert option mappings match
across macros.pyproject, Python scripts, and README.md
- Converted ~half of test specs to use long options
- Documented all short/long mappings in README.md
- All Python scripts expose their argparser() functions fer easier consistency asserts
Co-Authored-By: Maxwell G <maxwell@gtmx.me>
Assisted-By: Claude Opus 4.6
Validate whether extras passed into %pyproject_buildrequires -x exist
in upstream metadata, otherwise ValueError is raised on Fedora >=45
and RHEL >=11. It will emit a warning instead on older releases.
Both user-specified extras and metadata extras are normalized following PEP 685 conventions.
Fixes: rhbz#2360766
- Make `pyproject-rpm-macros` depend on `pyproject-srpm-macros`
- Resolves: rhbz#2463187
The idea here is that `%pyproject_patch_dependency` should normally be
used in `%prep`, before `%pyproject_buildrequires`. This does not work
unless there is an explicit, manual dependency on `python3-devel` (in
Fedora) or `pyproject-rpm-macros`.
Thanks to Miro Hrončok for noting that the implementation of
`%pyproject_patch_dependency` is compact enough to include in the srpm
macros.
Overrides are declared in %prep and automatically apply to both
BuildRequires (via %pyproject_buildrequires) and runtime Requires
(by patching installed .dist-info/METADATA during %pyproject_install).
Supported actions: drop_upper, drop_lower, drop_constraints,
set_upper, set_lower, and ignore. For == and ~=, drop_upper/drop_lower
decompose into half-bounds following PEP 440 semantics.
The :br_only suffix restricts an override to BuildRequires only,
for multi-wheel specs where a sibling package must be kept as a
runtime dependency.
Resolves: rhbz#2386906
Assisted-by: Claude (Anthropic)
Made-with: Cursor
The pipe in double-install made the entire %install section successful
regardless of exit code from %pyproject_install.
Such build failed anyway in %files, but this makes it much nicer.
The pipe in setuptools_scm would pass %check even when tox failed.
- Fixes: rhbz#2371389
This adds a new option for the macros, so spec files that use it will not
be parseable with older version of the package. It will be parseable without them.
Also document the behavior for building multiple wheels at a time,
which turned out to be related to the newly added documentation.
The actual RPM implementation (sans the test and docs) is heavily based on
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/569
which is:
Co-Authored-By: Shrey Naithani <shrey1212@yahoo.com>
Assisted-By: Gemini
LegacyVersion has been removed from Packaging in 2022. If the call
to `parse` function returns LegacyVersion, access to
version._version cannot emit DeprecationWarning.
For example, due to old versions of setuptools or tox,
the expected results of some of the tests might need adjustments.
Note that CentOS Stream mock builds run with EPEL (Next) enabled,
hence we also enable tox tests in %check for better coverage.
Hence, running this makes sense even for actual c10s/c9s backports.
anyio contains this metadata in pyproject.toml:
[project.optional-dependencies]
trio = ["trio >= 0.31.0"]
[dependency-groups]
test = ["anyio[trio]", "pytest"]
Before, pyproject_buildrequires would generate a test dependency group
requiring "anyio[trio]", creating an incorrect cyclic dependency in the
resolved requirements.
With the change, the group is resolved to require "trio >= 0.31.0"; the
self-reference is no longer created.
That way, accidentally unpackaged files within are reported as errors.
Currently, when %pyproject_extras_subpkg is used, the dist-info directory
is packaged as %ghost. When the main package does not have it,
the RPM build would succeed. The extras packages would have the python3dist()
requires and provides, but the main package would not.
By adding %dir after %ghost, we only package the directory
(which is enough for python3-rpm-generators to process it),
but the files in the directory are not included.
When not packaged in the main package, the RPM build fails.
This is a safeguard against packaging mistakes.
The visible difference is that rpm -ql/repoquery -l would only return the metadata directory.
See also https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/195
For packages with a lot of files, it took a really long time to run.
Profiling the code revealed most of the time is spent in PurePath.__eq__.
The code was using lists but checked
if to-be-added paths were not already in them.
Considering the order is insignificant
(the generated %files list is sorted at the end anyway),
the lists were essentially working as (very slow) sets.
Using sets instead of lists makes %pyproject_save_files over 20 times faster
(51.66 -> 2.39 seconds) for the ansible package (~62k files).
Checking if an item is in a list is O(N),
checking every added item is O(N**2).
Checking if an item is in a set is O(1),
checking every added item is O(N).
Additionally, with set, it is unnecessary to check for presence before addition,
so the code is easier.
(Commit message and removal of the check by Miro.)
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
- Fixes: rhbz#2391290
Consider code like this:
%generate_buildrequires
mv setup.py{,.ignore}
%pyproject_buildrequires
mv setup.py{.ignore,}
When %pyproject_buildrequires exits, the second mv is never executed.
The next round of %generate_buildrequires will then fail.
In fact, there is probably not a good reason to call exit from %pyproject_buildrequires.
From https://src.fedoraproject.org/rpms/libcpuid/pull-request/7
See https://packaging.python.org/en/latest/specifications/core-metadata/#requires
That field is deprecated and should include importable module names, not distribution packages.
We have no RPM Provides for importable names.
Treating this like python3dist() Requires is wrong and may result in stuff like:
No match for argument: python3dist(pkg-resources)
For packages using python-distutils-extra.
See https://bugzilla.redhat.com/show_bug.cgi?id=2378463#c2
This bug existed from the very beginning of the %pyproject_buildrequires,
but the field is almost unused in real packages, so it was not noticed until
we asked all Python packages to be ported to the new macros.
---
I considered flattening the structure returned from requires_from_parsed_metadata_file,
but then we would need to hardcode "Requires-Dist" in various source= declarations,
so I kept the structure as is.
