Compare commits

...

3 Commits

Author SHA1 Message Date
AlmaLinux RelEng Bot
e21a067c70 import CS pyproject-rpm-macros-1.23.2-1.el9 2026-09-03 07:53:16 -04:00
AlmaLinux RelEng Bot
cbdf20ad3d import CS pyproject-rpm-macros-1.18.5-1.el9 2026-03-30 10:55:03 -04:00
16234bda52 import CS pyproject-rpm-macros-1.16.2-1.el9 2025-03-11 08:00:58 +00:00
22 changed files with 4745 additions and 466 deletions

View File

@ -74,36 +74,27 @@ The popular buildsystems (setuptools, flit, poetry) do support it.
This behavior can be disabled
(e.g. when the project's build system does not support it)
using the `-R` flag:
using the `-R`/`--no-runtime` flag:
%generate_buildrequires
%pyproject_buildrequires -R
%pyproject_buildrequires --no-runtime
Alternatively, the runtime dependencies can be obtained by building the wheel and reading the metadata from the built wheel.
This can be enabled by using the `-w` flag.
Support for building wheels with `%pyproject_buildrequires -w` is **provisional** and the behavior might change.
Please subscribe to Fedora's [python-devel list] if you use the option.
Alternatively, if the project specifies its dependencies in the pyproject.toml
`[project]` table (as defined in [PEP 621]),
the runtime dependencies can be obtained by reading that metadata.
%generate_buildrequires
%pyproject_buildrequires -w
This can be enabled by using the `-p`/`--pyproject-dependencies` flag.
This flag supports reading both the runtime dependencies, and the selected extras
(see the `-x` flag described below).
When this is used, the wheel is going to be built at least twice,
becasue the `%generate_buildrequires` section runs repeatedly.
To avoid accidentally reusing a wheel leaking from a previous (different) build,
it cannot be reused between `%generate_buildrequires` rounds.
Contrarily to that, rebuilding the wheel again in the `%build` section is redundant
and the packager can omit the `%build` section entirely
to reuse the wheel built from the last round of `%generate_buildrequires`.
Be extra careful when attempting to modify the sources after `%pyproject_buildrequires`,
e.g. when running extra commands in the `%build` section:
%build
cython src/wrong.pyx # this is too late with %%pyproject_buildrequires -w
%pyproject_wheel
Please note that not all build backends which use pyproject.toml support the
`[project]` table scheme.
For example, poetry-core (at least in 1.9.0) defines package metadata in the
custom `[tool.poetry]` table which is not supported by the `%pyproject_buildrequires` macro.
For projects that specify test requirements using an [`extra`
provide](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use),
these can be added using the `-x` flag.
these can be added using the `-x`/`--extras` flag.
Multiple extras can be supplied by repeating the flag or as a comma separated list.
For example, if upstream suggests installing test dependencies with
`pip install mypackage[testing]`, the test deps would be generated by:
@ -111,9 +102,21 @@ For example, if upstream suggests installing test dependencies with
%generate_buildrequires
%pyproject_buildrequires -x testing
The macros verify that requested extras exist in the project's metadata.
If a requested extra is not found after PEP 685 validation, the build fails with a `ValueError`
on Fedora >=45 and RHEL >=11. It will emit a warning instead on older releases.
For projects that specify test requirements using [PEP 735] dependency groups,
these can be added using the `-g`/`--dependency-groups` flag.
Multiple groups can be supplied by repeating the flag or as a comma separated list.
For example, if upstream uses a dependency group called `tests`, the test deps would be generated by:
%generate_buildrequires
%pyproject_buildrequires -g tests
For projects that specify test requirements in their [tox] configuration,
these can be added using the `-t` flag (default tox environment)
or the `-e` flag followed by the tox environment.
these can be added using the `-t`/`--tox` flag (default tox environment)
or the `-e`/`--toxenv` flag followed by the tox environment.
The default tox environment (such as `py37` assuming the Fedora's Python version is 3.7)
is available in the `%{toxenv}` macro.
For example, if upstream suggests running the tests on Python 3.7 with `tox -e py37`,
@ -134,49 +137,92 @@ The `-e` option redefines `%{toxenv}` for further reuse.
Use `%{default_toxenv}` to get the default value.
The `-t`/`-e` option uses [tox-current-env]'s `--print-deps-to-file` behind the scenes.
It generates dependencies listed directly in `deps`,
dependencies defined through `extras`,
and on tox 4.22+ also dependencies defined through `dependency_groups`.
If your package specifies some tox plugins in `tox.requires`,
such plugins will be BuildRequired as well.
Not all plugins are guaranteed to play well with [tox-current-env],
in worst case, patch/sed the requirement out from the tox configuration.
Note that neither `-x` or `-t` can be used with `-R`,
Note that neither `-x`/`--extras` or `-t`/`--tox` can be used with `-R`/`--no-runtime` or `-N`/`--no-use-build-system`,
because runtime dependencies are always required for testing.
You can only use those options if the build backend supports the [prepare-metadata-for-build-wheel hook],
or together with `-w`.
or together with `-p`.
However, using `-g`/`--dependency-groups` with `-R`/`--no-runtime` or `-N`/`--no-use-build-system` is supported because dependency groups don't need to be used for testing
and can be obtained by reading `pyproject.toml` only.
[tox]: https://tox.readthedocs.io/
[tox-current-env]: https://github.com/fedora-python/tox-current-env/
[prepare-metadata-for-build-wheel hook]: https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel
[python-devel list]: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/
Additionally to generated requirements you can supply multiple file names to `%pyproject_buildrequires` macro.
Dependencies will be loaded from them:
%pyproject_buildrequires requirements/tests.in requirements/docs.in requirements/dev.in
For packages not using build system you can use `-N` to entirely skip automatical
For packages not using build system you can use `-N`/`--no-use-build-system` to entirely skip automatical
generation of requirements and install requirements only from manually specified files.
`-N` option implies `-R` and cannot be used in combination with other options mentioned above
(`-w`, `-e`, `-t`, `-x`).
`-N`/`--no-use-build-system` option implies `-R`/`--no-runtime` and cannot be used in combination with other options mentioned above
(`-e`/`--toxenv`, `-t`/`--tox`, `-x`/`--extras`, `-p`/`--pyproject-dependencies`).
The `%pyproject_buildrequires` macro also accepts the `-r` flag for backward compatibility;
The `%pyproject_buildrequires` macro also accepts the `-r`/`--runtime` flag for backward compatibility;
it means "include runtime dependencies" which has been the default since version 0-53.
Building wheels from custom directories
---------------------------------------
The `%pyproject_buildrequires` and `%pyproject_wheel` macros accept a `-d`/`--directory` flag
to specify a working directory.
For example:
%pyproject_wheel --directory bindings/python
Is roughly equivalent to:
pushd bindings/python
%pyproject_wheel
popd
If you use `%pyproject_wheel` to build multiple wheels, for example like this:
%pyproject_wheel --directory directory1/
%pyproject_wheel --directory directory2/
You still only call `%pyproject_install` once and it will install all such wheels.
To use `%pyproject_save_files` (and optionally `%pyproject_check_import`) with multiple wheels,
use the `-D`/`--dist-name` option to select a specific package,
for details, see *Generating the %files section*
(and *Performing an import check on all importable modules*).
If you use `%pyproject_buildrequires -d ...` for multiple co-dependent packages
in one spec file, it will create one or more build time dependencies
on the packages you are currently building.
The only fully supported way to avoid some of such unnecessary BuildRequires
is to use the `-R` flag to avoid generating BuildRequires based on runtime dependencies.
Alternatively, use the **provisional** `%pyproject_patch_dependency` macro
with the `:br_only` suffix to mitigate this problem.
Passing config settings to build backends
-----------------------------------------
The `%pyproject_buildrequires` and `%pyproject_wheel` macros accept a `-C` flag
The `%pyproject_buildrequires` and `%pyproject_wheel` macros accept a `-C`/`--config-settings` flag
to pass [configuration settings][config_settings] to the build backend.
Options take the form of `-C KEY`, `-C KEY=VALUE`, or `-C--option-with-dashes`.
Pass `-C` multiple times to specify multiple options.
Options take the form of `-C KEY`, `-C KEY=VALUE`, `-C--option-with-dashes`,
or `--config-settings KEY=VALUE`.
Pass `-C`/`--config-settings` multiple times to specify multiple options.
This option is equivalent to pip's `--config-settings` flag.
These are passed on to PEP 517 hooks' `config_settings` argument as a Python
dictionary.
The `%pyproject_buildrequires` macro passes these options to the
`get_requires_for_build_wheel` and `prepare_metadata_for_build_wheel` hooks.
Passing `-C` to `%pyproject_buildrequires` is incompatible with `-N` which does
Passing `-C`/`--config-settings` to `%pyproject_buildrequires` is incompatible with `-N`/`--no-use-build-system` which does
not call these hooks at all.
The `%pyproject_wheel` macro passes these options to the `build_wheel` hook.
@ -189,7 +235,7 @@ and other projects may only accept config settings for one of the two steps.
Note that the current implementation of the macros uses `pip` to build wheels.
On some systems (notably on RHEL 9 with Python 3.9),
`pip` is too old to understand `--config-settings`.
Using the `-C` option for `%pyproject_wheel` (or `%pyproject_buildrequires -w`)
Using the `-C`/`--config-settings` option for `%pyproject_wheel` (or `%pyproject_buildrequires -w`/`--wheel`)
is not supported there and will result to an error like:
Usage:
@ -200,6 +246,68 @@ is not supported there and will result to an error like:
[config_settings]: https://peps.python.org/pep-0517/#config-settings
Provisional: Overriding dependency constraints
-----------------------------------------------
Upstream Python projects sometimes pin or constrain dependency versions in ways
that are incompatible with the versions available in the distribution.
The `%pyproject_patch_dependency` macro allows you to override these constraints
for both build-time (BuildRequires) and run-time (Requires) dependencies.
The feature is **provisional** and the behavior might change.
Please subscribe to Fedora's [python-devel list] if you use the feature.
Declare overrides in `%prep`, after `%autosetup`:
%prep
%autosetup -p1
%pyproject_patch_dependency cython:drop_upper
%pyproject_patch_dependency numpy:set_upper:2.0
The overrides are written to a file that is automatically consumed by
`%pyproject_buildrequires` (for BuildRequires) and `%pyproject_install`
(for patching installed `.dist-info/METADATA`, which controls Requires).
The syntax is:
%pyproject_patch_dependency PACKAGE:ACTION[:VALUE][:br_only]
where `PACKAGE` is the Python package name (normalized per [PEP 503]),
`ACTION` is one of the following, `VALUE` is required for `set_upper`
and `set_lower`, and the optional `br_only` suffix restricts the override
to BuildRequires only (the installed METADATA is left untouched).
Available actions:
* `drop_upper` -- remove upper-bound constraints (`<`, `<=`);
decompose `==V` to `>=V` and `~=V` to `>=V` (keeping the lower half)
* `drop_lower` -- remove lower-bound constraints (`>`, `>=`);
decompose `==V` to `<=V` (keeping the upper half); remove `~=` entirely
* `drop_constraints` -- remove all version constraints
* `set_upper` -- replace the upper bound with `< VALUE`
(same decomposition as `drop_upper`, then adds `< VALUE`)
* `set_lower` -- replace the lower bound with `>= VALUE`
(same decomposition as `drop_lower`, then adds `>= VALUE`)
* `ignore` -- remove the dependency entirely
The `==` decomposition follows from the mathematical equivalence
`==V` = `>=V AND <=V`. The `~=` decomposition follows PEP 440 which
defines `~= V.N` as `>= V.N, == V.*`. Exclusions (`!=`) are always
preserved by `drop_upper`/`drop_lower` (but removed by `drop_constraints`).
The `br_only` suffix is useful in multi-wheel spec files where a sibling
package should be ignored in BuildRequires but kept as a runtime dependency:
%pyproject_patch_dependency sibling-package:ignore:br_only
Note that the macro operates on the **output** of the build backend's PEP 517
hooks, not on the source files themselves. If a dependency is enforced
internally by the build backend (e.g. a `setup_requires` entry that is imported
at the top level of `setup.py`), the hook may fail before the macro can
intercept. In such cases, use traditional `sed`/`patch` in `%prep`, or
combine a static `BuildRequires` with `%pyproject_patch_dependency` to ensure
the dependency is installed while still being filtered from dynamic output.
Running tox based tests
-----------------------
@ -216,15 +324,15 @@ The macro:
- Always prepends `$PATH` with `%{buildroot}%{_bindir}`
- If not defined, sets `$PYTHONPATH` to `%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}`
- If not defined, sets `$TOX_TESTENV_PASSENV` to `*`
- Runs `tox` with `-q` (quiet), `--recreate` and `--current-env` (from [tox-current-env]) flags
- Runs `tox` with `-q` (quiet), `--recreate`, `--current-env` (from [tox-current-env]) and `--assert-config` (from [tox-current-env]) flags
- Implicitly uses the tox environment name stored in `%{toxenv}` - as overridden by `%pyproject_buildrequires -e`
By using the `-e` flag, you can use a different tox environment(s):
By using the `-e`/`--toxenv` flag, you can use a different tox environment(s):
%check
%tox
%if %{with integration_test}
%tox -e %{default_toxenv}-integration
%tox --toxenv %{default_toxenv}-integration
%endif
If you wish to provide custom `tox` flags or arguments, add them after `--`:
@ -266,11 +374,11 @@ You can use globs in the module names if listing them explicitly would be too te
In fully automated environments, you can use the `*` glob to include all modules (put it in single quotes to prevent Shell from expanding it). In Fedora however, you should always use a more specific glob to avoid accidentally packaging unwanted files (for example, a top level module named `test`).
Speaking about automated environments, some files cannot be classified with `%pyproject_save_files`, but it is possible to list all unclassified files by adding a special `+auto` argument.
Speaking about automated environments, some files cannot be classified with `%pyproject_save_files`, but it is possible to list all unclassified files by adding `-a`/`--auto` (or the legacy `+auto` argument).
%install
%pyproject_install
%pyproject_save_files '*' +auto
%pyproject_save_files '*' --auto
%files -n python3-requests -f %{pyproject_files}
@ -284,12 +392,19 @@ However, in Fedora packages, always list executables explicitly to avoid uninten
%doc README.rst
%{_bindir}/downloader
If the package has no Python modules in it, you can explicitly use `-M`/`--allow-no-modules` to denote that.
%install
%pyproject_install
%pyproject_save_files -M
Otherwise, at least one module-glob argument is required.
`%pyproject_save_files` can automatically mark license files with `%license` macro
and language (`*.mo`) files with `%lang` macro and appropriate language code.
Only license files declared via [PEP 639] `License-File` field are detected.
[PEP 639] is still a draft and can be changed in the future.
It is possible to use the `-l` flag to declare that a missing license should
terminate the build or `-L` (the default) to explicitly disable this check.
It is possible to use the `-l`/`--assert-license` flag to declare that a missing license should
terminate the build or `-L`/`--no-assert-license` (the default) to explicitly disable this check.
Packagers are encouraged to use the `-l` flag when the `%license` file is not manually listed in `%files`
to avoid accidentally losing the file in a future version.
When the `%license` file is manually listed in `%files`,
@ -301,6 +416,17 @@ If you wish to rename, remove or otherwise change the installed files of a packa
If possible, remove/rename such files in `%prep`.
If not possible, avoid using `%pyproject_save_files` or edit/replace `%{pyproject_files}`.
When processing files from multiple Python packages
(most likely as a result of multiple `%pyproject_wheel` invocations),
use the `-D`/`--dist-name` option to select a specific package
in `%pyproject_save_files` and `%{pyproject_files}`:
%pyproject_save_files --dist-name package_a module_glob_a...
%pyproject_save_files --dist-name package_b module_glob_b...
%files -n python3-package-a -f %{pyproject_files -D package_a}
%files -n python3-package-b -f %{pyproject_files -D package_b}
Performing an import check on all importable modules
----------------------------------------------------
@ -325,7 +451,7 @@ Use the macro in `%check`:
%check
%pyproject_check_import
By using the `-e` flag, you can exclude module names matching the given glob(s) from the import check
By using the `-e`/`--exclude` flag, you can exclude module names matching the given glob(s) from the import check
(put it in single quotes to prevent Shell from expanding it).
The flag can be used repeatedly.
For example, to exclude all submodules ending with `config` and all submodules starting with `test`, you can use:
@ -335,7 +461,7 @@ For example, to exclude all submodules ending with `config` and all submodules s
There must be at least one module left for the import check;
if, as a result of greedy excluding, no modules are left to check, the check fails.
When the `-t` flag is used, only top-level modules are checked,
When the `-t`/`--top-level-only` flag is used, only top-level modules are checked,
qualified module names with a dot (`.`) are excluded.
If the modules detected by `%pyproject_save_files` are `requests`, `requests.models`, and `requests.packages`, this will only perform an import of `requests`:
@ -346,7 +472,16 @@ The reason should be documented in a comment.
The `%pyproject_check_import` macro also accepts positional arguments with
additional qualified module names to check, useful for example if some modules are installed manually.
Note that filtering by `-t`/`-e` also applies to the positional arguments.
Note that filtering by `-t`/`--top-level-only`/`-e`/`--exclude` also applies to the positional arguments.
When listing files from multiple Python packages via `%pyproject_save_files -D`/`--dist-name`,
pass `-D`/`--dist-name` to `%pyproject_check_import` as well.
%pyproject_save_files --dist-name package_a module_glob_a...
%pyproject_save_files --dist-name package_b module_glob_b...
%pyproject_check_import --dist-name package_a
%pyproject_check_import --dist-name package_b
Another macro, `%_pyproject_check_import_allow_no_modules` allows to pass the import check,
even if no Python modules are detected in the package.
@ -384,6 +519,154 @@ These arguments are still required:
* Positional arguments: the extra name(s).
Multiple subpackages are generated when multiple names are provided.
When processing files from multiple Python packages
(see *Generating the %files section* for details),
pass `-D` to `%pyproject_extras_subpkg` as well:
%pyproject_extras_subpkg -n python3-package-a -D package_a extra1
%pyproject_extras_subpkg -n python3-package-b -D package_b extra2
Provisional: Declarative Buildsystem (RPM 4.20+)
------------------------------------------------
It is possible to reduce some of the spec boilerplate by using the provided
pyproject [declarative buildsystem].
This option is only available with RPM 4.20+ (e.g. in Fedora 41+, ELN/CentOS Stream 11+).
The declarative buildsystem is **provisional** and the behavior might change.
Please subscribe to Fedora's [python-devel list] if you use the feature.
To enable the pyproject declarative buildsystem, use the following:
BuildSystem: pyproject
BuildOption(install): <options for %%pyproject_save_files>
That way, RPM will automatically fill-in the `%prep`, `%generate_buildrequires`,
`%build`, `%install`, and `%check` sections the following defaults:
%prep
%autosetup -p1 -C
%generate_buildrequires
%pyproject_buildrequires
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files <options from BuildOption(install)>
%check
%pyproject_check_import
To pass options to the individual macros, use `BuildOption` (see the [documentation of declarative buildsystems][declarative buildsystem]).
# pass options for %%pyproject_save_files (mandatory when not overriding %%install)
BuildOption(install): --assert-license _module --auto
# replace the default options for %%autosetup
BuildOption(prep): -S git_am -C
# pass options to %%pyproject_buildrequires
BuildOption(generate_buildrequires): docs-requirements.txt --tox
# pass options to %%pyproject_wheel
BuildOption(build): -C--global-option=--no-cython-compile
# pass options to %%pyproject_check_import
BuildOption(check): --exclude '*.test*'
Alternatively, you can supply your own sections to override the automatic ones:
BuildOption(generate_buildrequires): --wheel
...
%build
# do nothing, the wheel was built in %%generate_buildrequires
You can append to end of the automatic sections:
%check -a
# run %%pytest after %%pyproject_check_import
%pytest
Or prepend to the beginning of them:
%prep -p
# run %%gpgverify before %%autosetup
%gpgverify -k2 -s1 -d0
[declarative buildsystem]: https://rpm-software-management.github.io/rpm/manual/buildsystem.html
Long option support and options reference
-----------------------------------------
All public macros accept both short and long options.
For example, `%pyproject_buildrequires -R` and `%pyproject_buildrequires --no-runtime`
are equivalent.
Here is a complete reference of all options:
### `%pyproject_buildrequires`
| Short | Long | Description |
|---------------|------------------------------|----------------------------------------------------------|
| `-r` | `--runtime` | Generate run-time requirements (default) |
| `-R` | `--no-runtime` | Don't generate run-time requirements |
| `-x EXTRAS` | `--extras EXTRAS` | Comma-separated list of extras |
| `-t` | `--tox` | Generate test requirements from tox |
| `-e TOXENVS` | `--toxenv TOXENVS` | Specify tox environments |
| `-g GROUPS` | `--dependency-groups GROUPS` | Comma-separated dependency groups (PEP 735) |
| `-p` | `--pyproject-dependencies` | Read dependencies from pyproject.toml `[project]` table |
| `-N` | `--no-use-build-system` | Project does not use a build system |
| `-w` | `--wheel` | Build wheel to get runtime requirements (deprecated) |
| `-C SETTING` | `--config-settings SETTING` | Configuration setting for PEP 517 backend |
| `-d DIR` | `--directory DIR` | Working directory |
### `%pyproject_wheel`
| Short | Long | Description |
|--------------|-----------------------------|-------------------------------------------|
| `-C SETTING` | `--config-settings SETTING` | Configuration setting for PEP 517 backend |
| `-d DIR` | `--directory DIR` | Working directory |
### `%pyproject_save_files`
| Short | Long | Description |
|------------|------------------------|--------------------------------------------------------------|
| `-l` | `--assert-license` | Fail when no License-File (PEP 639) is found |
| `-L` | `--no-assert-license` | Don't fail on missing License-File (default) |
| `-M` | `--allow-no-modules` | Allow no module globs |
| `-D NAME` | `--dist-name NAME` | Save files for a specific distribution package (multi-wheel) |
| `-a` | `--auto` | Include non-module files (same as `+auto`) |
### `%pyproject_check_import`
| Short | Long | Description |
|-----------|-----------------------|---------------------------------------------------|
| `-e GLOB` | `--exclude GLOB` | Exclude module names matching glob |
| `-D NAME` | `--dist-name NAME` | Check imports for a specific distribution package |
| `-t` | `--top-level-only` | Only check top-level modules |
### `%pyproject_extras_subpkg`
| Short | Long | Description |
|------------|---------------------------|-------------------------------------------------------------------------------------------------|
| `-n NAME` | `--name NAME` | Name of the base RPM package (required) |
| `-D NAME` | `--dist-name NAME` | Select %ghost .dist-info for a specific distribution package (mutually exclusive with -i/-f/-F) |
| `-a` | `--noarch` | Insert BuildArch: noarch |
| `-A` | `--no-noarch` | Do not insert BuildArch: noarch (default) |
| `-i PATH` | `--dist-info-path PATH` | Custom buildroot path to the .dist-info metadata folder (optional, advanced usage) |
| `-f FILE` | `--filelist FILE` | Custom path to a filelist (optional, advanced usage) |
| `-F` | `--no-filelist` | Skip %files section entirely (optional, advanced usage) |
### `%tox`
| Short | Long | Description |
|-------------|-------------------|----------------------|
| `-e TOXENV` | `--toxenv TOXENV` | Tox environment to use |
Limitations
-----------
@ -430,10 +713,13 @@ The requirements will be converted to package names without versions, e.g.:
However upstreams usually only use direct URLs for their requirements as workarounds,
so be prepared for problems.
[PEP 503]: https://www.python.org/dev/peps/pep-0503/
[PEP 508]: https://www.python.org/dev/peps/pep-0508/
[PEP 517]: https://www.python.org/dev/peps/pep-0517/
[PEP 518]: https://www.python.org/dev/peps/pep-0518/
[PEP 621]: https://www.python.org/dev/peps/pep-0621/
[PEP 639]: https://www.python.org/dev/peps/pep-0639/
[PEP 735]: https://www.python.org/dev/peps/pep-0735/
[pip's documentation]: https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support
@ -442,6 +728,11 @@ Deprecated
The `%{pyproject_build_lib}` macro is deprecated, don't use it.
The `%pyproject_buildrequires` `-w`/`--wheel` option is deprecated, don't use it.
If the build backend does not support the [prepare-metadata-for-build-wheel hook],
consider using the `-p`/`--pyproject-dependencies` flag to read the metadata from the pyproject.toml
`[project]` table (as defined in [PEP 621]) instead.
Testing the macros
------------------
@ -465,11 +756,11 @@ For each `$PKG.spec` in `tests/`:
- download the sources:
spectool -g -R $PKG.spec
spectool -g $PKG.spec
- build a SRPM:
rpmbuild -bs $PKG.spec
rpmbuild -bs --define '_sourcedir .' $PKG.spec
- build in mock, using the path from the command above as `$SRPM`:

View File

@ -3,5 +3,38 @@
# When this file is installed but macros.pyproject is not
# this macro will cause the package with the real macro to be installed.
# When macros.pyproject is installed, it overrides this macro.
# Note: This needs to maintain the same set of options as the real macro.
%pyproject_buildrequires(rRxtNwe:C:) echo 'pyproject-rpm-macros' && exit 0
# Note: This takes arbitrary options, to ease addition of new options to the real macro.
%pyproject_buildrequires(-) echo 'pyproject-rpm-macros'
# Similarly, we define a dummy (to be overridden) %%pyproject_files, to avoid:
# error: Bad option -D: %%files -n python3-... -f %%{pyproject_files -D ...}
%pyproject_files(-) dummy
# We need to define %%pyproject_patch_dependency here in the srpm macros, along with any path-related macros it uses,
# because this macro is normally used in %%prep, before any generated BR is installed.
# - The backward-compatible %%_pyproject_files_pkgversion suffix is used in all pyproject-rpm-macros directories.
# For the main Python it's empty, for all others it's "-3.X".
# - We prefix all created files with %%_pyproject_files_prefix to make them unique.
# Ideally, we would put them into %%{buildsubdir}, but that value changes during the spec.
# The used value is similar to the one used to define the default %%buildroot.
%_pyproject_files_pkgversion %{expr:"%{python3_pkgversion}" != "3" ? "-%{python3_pkgversion}" : ""}
%_pyproject_files_prefix %{name}-%{version}-%{release}.%{_arch}%{_pyproject_files_pkgversion}
%_pyproject_dep_overrides %{_builddir}/%{_pyproject_files_prefix}-pyproject-dep-overrides
%pyproject_patch_dependency() %{expand:\\\
%{!?1:%{error:%%pyproject_patch_dependency requires an argument}}\\\
%{?2:%{error:%%pyproject_patch_dependency accepts exactly one argument per call}}\\\
if [ -f %{__python3} ] && [ -f %{_rpmconfigdir}/redhat/pyproject_dependency_overrides.py ]; then
%{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_dependency_overrides.py '%1'
fi
echo '%1' >> %{_pyproject_dep_overrides}
}
# Declarative buildsystem, requires RPM 4.20+ to work
# https://rpm-software-management.github.io/rpm/manual/buildsystem.html
# This is the minimal implementation to be in the srpm package,
# as required even before the BuildRequires are installed
%buildsystem_pyproject_conf() %nil
%buildsystem_pyproject_generate_buildrequires() %pyproject_buildrequires %*
%buildsystem_pyproject_build() %nil
%buildsystem_pyproject_install() %nil

View File

@ -1,9 +1,13 @@
# This is a backward-compatible suffix used in all pyproject-rpm-macros directories
# For the main Python it's empty, for all others it's "-3.X"
%_pyproject_files_pkgversion %{expr:"%{python3_pkgversion}" != "3" ? "-%{python3_pkgversion}" : ""}
# %%_pyproject_files_pkgversion defined in srpm macros
# In RPM < 4.20 (4.19.9x is 4.20 alpha), there is no guaranteed, RPM-controlled per-build directory (%%mkbuilddir step).
# Hence we use %%{buildsubdir} if available.
# On newer RPM 4.20+ this is no longer necessary and breaks the declarative buildsystem:
# https://github.com/rpm-software-management/rpm/issues/3890
%_pyproject_buildsubdir_compat %[ v"0%{?rpmversion}" < v"4.19.90" ? "%{?buildsubdir:/%{buildsubdir}}" : ""]
# This is a directory where wheels are stored and installed from, absolute
%_pyproject_wheeldir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/pyproject-wheeldir%{_pyproject_files_pkgversion}
%_pyproject_wheeldir %{_builddir}%{_pyproject_buildsubdir_compat}/pyproject-wheeldir%{_pyproject_files_pkgversion}
# This is a directory used as TMPDIR, where pip copies sources to and builds from, relative to PWD
# For proper debugsource packages, we create TMPDIR within PWD
@ -12,30 +16,64 @@
# This will be used in debugsource package paths (applies to extension modules only)
# NB: pytest collects tests from here if not hidden
# https://docs.pytest.org/en/latest/reference.html#confval-norecursedirs
%_pyproject_builddir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/.pyproject-builddir%{_pyproject_files_pkgversion}
%_pyproject_builddir %{_builddir}%{_pyproject_buildsubdir_compat}/.pyproject-builddir%{_pyproject_files_pkgversion}
# We prefix all created files with this value to make them unique
# Ideally, we would put them into %%{buildsubdir}, but that value changes during the spec
# The used value is similar to the one used to define the default %%buildroot
%_pyproject_files_prefix %{name}-%{version}-%{release}.%{_arch}%{_pyproject_files_pkgversion}
# %%_pyproject_files_prefix defined in srpm macros
# PEP 503 name normalization: lowercase, one or more [-_.] → single -
%__pyproject_normalize_name(-) %{lua:print((rpm.expand("%1"):lower():gsub("[%-_.]+", "-")))}
%__pyproject_getopt_restore %{lua:require("fedora.rpm.pyproject_getopt").restore()}
%_pyproject_files_base %{_builddir}/%{_pyproject_files_prefix}-pyproject-files
%_pyproject_modules_base %{_builddir}/%{_pyproject_files_prefix}-pyproject-modules
%_pyproject_ghost_distinfo_base %{_builddir}/%{_pyproject_files_prefix}-pyproject-ghost-distinfo
%pyproject_files(-) %{lua:
require("fedora.rpm.pyproject_getopt").getopt({
{short="D", long="dist-name", value=true},
})}%{_pyproject_files_base}%{?__pyproject_opt_D:-%{__pyproject_normalize_name %{__pyproject_opt_D}}}%{__pyproject_getopt_restore}
%_pyproject_modules(-) %{lua:
require("fedora.rpm.pyproject_getopt").getopt({
{short="D", long="dist-name", value=true},
})}%{_pyproject_modules_base}%{?__pyproject_opt_D:-%{__pyproject_normalize_name %{__pyproject_opt_D}}}%{__pyproject_getopt_restore}
%_pyproject_ghost_distinfo(-) %{lua:
require("fedora.rpm.pyproject_getopt").getopt({
{short="D", long="dist-name", value=true},
})}%{_pyproject_ghost_distinfo_base}%{?__pyproject_opt_D:-%{__pyproject_normalize_name %{__pyproject_opt_D}}}%{__pyproject_getopt_restore}
%pyproject_files %{_builddir}/%{_pyproject_files_prefix}-pyproject-files
%_pyproject_modules %{_builddir}/%{_pyproject_files_prefix}-pyproject-modules
%_pyproject_ghost_distinfo %{_builddir}/%{_pyproject_files_prefix}-pyproject-ghost-distinfo
%_pyproject_record %{_builddir}/%{_pyproject_files_prefix}-pyproject-record
%_pyproject_buildrequires %{_builddir}/%{_pyproject_files_prefix}-pyproject-buildrequires
# %%_pyproject_dep_overrides defined in srpm macros
# Internal macro, takes %%set_build_flags and strips all the exports
# TODO: Make such a list an actual source of %%set_build_flags (in redhat-rpm-config)
# Cannot use %%gsub directly to preserve EL 9 compatibility
%_pyproject_build_flags %{lua:local exports = rpm.expand('%{set_build_flags} ;'); print((exports:gsub('%s*;+%s+export%s+[%u_]+%s*;+%s*', ' ')))}
# Avoid leaking %%{_pyproject_builddir} to pytest collection
# https://bugzilla.redhat.com/show_bug.cgi?id=1935212
# The value is read and used by the %%pytest and %%tox macros:
%_set_pytest_addopts %global __pytest_addopts --ignore=%{_pyproject_builddir}
%pyproject_wheel(C:) %{expand:\\\
# %%pyproject_patch_dependency defined in srpm macros
%pyproject_wheel(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="C", long="config-settings", value=true, separator=","},
{short="d", long="directory", value=true},
})}\
%{expand:\\\
%_set_pytest_addopts
%{?__pyproject_opt_d:pushd "%{__pyproject_opt_d}"}
mkdir -p "%{_pyproject_builddir}"
CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}" TMPDIR="%{_pyproject_builddir}" \\\
%{_pyproject_build_flags} \\\
TMPDIR="%{_pyproject_builddir}" \\\
%{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_wheel.py %{?**} %{_pyproject_wheeldir}
}
%{?__pyproject_opt_d:popd}
%{__pyproject_getopt_restore}}
%pyproject_build_lib %{!?__pyproject_build_lib_warned:%{warn:The %%{pyproject_build_lib} macro is deprecated.
@ -69,16 +107,17 @@ echo $(IFS=:; echo "${pyproject_build_lib[*]}")
%pyproject_install() %{expand:\\\
specifier=$(ls %{_pyproject_wheeldir}/*.whl | xargs basename --multiple | sed -E 's/([^-]+)-([^-]+)-.+\\\.whl/\\\1==\\\2/')
if [ -z $specifier ]; then
if [ -z "$specifier" ]; then
echo 'ERROR: %%%%pyproject_install found no wheel in %%%%{_pyproject_wheeldir} %{_pyproject_wheeldir}' >&2
exit 1
fi
PIP_CONFIG_FILE="${PIP_CONFIG_FILE:-/dev/null}" \
TMPDIR="%{_pyproject_builddir}" %{__python3} -m pip install --root %{buildroot} --prefix %{_prefix} --no-deps --disable-pip-version-check --progress-bar off --verbose --ignore-installed --no-warn-script-location --no-index --no-cache-dir --find-links %{_pyproject_wheeldir} $specifier
if [ -d %{buildroot}%{_bindir} ]; then
%py3_shebang_fix %{buildroot}%{_bindir}/*
rm -rfv %{buildroot}%{_bindir}/__pycache__
fi
rm -f %{_pyproject_ghost_distinfo}
rm -f %{_pyproject_ghost_distinfo_base}
site_dirs=()
# Process %%{python3_sitelib} if exists
if [ -d %{buildroot}%{python3_sitelib} ]; then
@ -91,36 +130,67 @@ fi
# Process all *.dist-info dirs in sitelib/sitearch
for site_dir in ${site_dirs[@]}; do
for distinfo in %{buildroot}$site_dir/*.dist-info; do
echo "%ghost ${distinfo#%{buildroot}}" >> %{_pyproject_ghost_distinfo}
echo "%ghost %dir ${distinfo#%{buildroot}}" >> %{_pyproject_ghost_distinfo_base}
_normalized=$(PYTHONPATH=%{_rpmconfigdir}/redhat \\
%{__python3} -sBc \\
"import sys; from pyproject_save_files import canonical_name_from_distinfo; print(canonical_name_from_distinfo(sys.argv[1]))" \\
"$(basename ${distinfo})")
echo "%ghost %dir ${distinfo#%{buildroot}}" >> "%{_pyproject_ghost_distinfo_base}-${_normalized}"
sed -i 's/pip/rpm/' ${distinfo}/INSTALLER
PYTHONPATH=%{_rpmconfigdir}/redhat \\
%{__python3} -B %{_rpmconfigdir}/redhat/pyproject_preprocess_record.py \\
--buildroot %{buildroot} --record ${distinfo}/RECORD --output %{_pyproject_record}
rm -fv ${distinfo}/RECORD
rm -fv ${distinfo}/REQUESTED
if [ -f %{_pyproject_dep_overrides} ]; then
%{__python3} -B %{_rpmconfigdir}/redhat/pyproject_patch_metadata.py \\
--overrides %{_pyproject_dep_overrides} \\
--metadata ${distinfo}/METADATA
fi
done
done
lines=$(wc -l %{_pyproject_ghost_distinfo} | cut -f1 -d" ")
lines=$(wc -l %{_pyproject_ghost_distinfo_base} | cut -f1 -d" ")
if [ $lines -ne 1 ]; then
echo -e "\\n\\nWARNING: %%%%pyproject_extras_subpkg won't work without explicit -i or -F, found $lines dist-info directories.\\n\\n" >&2
rm %{_pyproject_ghost_distinfo} # any attempt to use this will fail
echo -e "%%%%pyproject_extras_subpkg will require -D <name> or explicit -i/-F with multiple dist-info directories (found $lines)." >&2
rm %{_pyproject_ghost_distinfo_base} # any attempt to use this will fail
fi
}
# Note: the three times nested questionmarked -i -f -F pattern means: If none of those options was used -- in that case, we inject our own -f
%pyproject_extras_subpkg(n:i:f:F) %{expand:%{?python_extras_subpkg:%{python_extras_subpkg%{?!-i:%{?!-f:%{?!-F: -f %{_pyproject_ghost_distinfo}}}} %**}}}
# Note: the three times nested __pyproject_opt_{i,f,F} pattern means: If none of those options was used -- in that case, we inject our own -f
# -D selects the per-package ghost distinfo file for multi-wheel installs
%pyproject_extras_subpkg(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="n", long="name", value=true},
{short="i", long="dist-info-path", value=true},
{short="f", long="filelist", value=true},
{short="F", long="no-filelist"},
{short="a", long="noarch"},
{short="A", long="no-noarch"},
{short="D", long="dist-name", value=true},
}, {
{"D", {"i", "f", "F"}},
})}\
%{expand:%{?python_extras_subpkg:%{python_extras_subpkg%{!?__pyproject_opt_i:%{!?__pyproject_opt_f:%{!?__pyproject_opt_F: -f %{_pyproject_ghost_distinfo %{?__pyproject_optflag_D}}}}} %{?__pyproject_optflag_n} %{?__pyproject_optflag_i} %{?__pyproject_optflag_f} %{?__pyproject_optflag_F} %{?__pyproject_optflag_a} %{?__pyproject_optflag_A} %{?__pyproject_positional_args}}}%{__pyproject_getopt_restore}}
# Escaping an actual percentage sign in path by 8 signs has been verified in RPM 4.16 and 4.17.
# See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
# Since RPM 4.19, 2 signs are needed instead. 4.18.90+ is a pre-release of RPM 4.19.
# On the CI, we build tests/escape_percentages.spec to verify the assumptions.
%pyproject_save_files(lL) %{expand:\\\
%{expr:v"0%{?rpmversion}" >= v"4.18.90" ? "RPM_PERCENTAGES_COUNT=2" : "RPM_PERCENTAGES_COUNT=8" } \\
# Escaping shell-globs, percentage signs and spaces was reworked in RPM 4.19+
# https://github.com/rpm-software-management/rpm/issues/1749#issuecomment-1020420616
# Since we support both ways, we pass either 4.19 or 4.18 to the script, so it knows which one to use
# Rather than passing the actual version, we let RPM compare the versions, as it is easier done here than in Python
%pyproject_save_files(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="l", long="assert-license"},
{short="L", long="no-assert-license"},
{short="M", long="allow-no-modules"},
{short="a", long="auto"},
{short="D", long="dist-name", value=true},
})}\
%{expand:\\\
%{expr:v"0%{?rpmversion}" >= v"4.18.90" ? "RPM_FILES_ESCAPE=4.19" : "RPM_FILES_ESCAPE=4.18" } \\
%{__python3} %{_rpmconfigdir}/redhat/pyproject_save_files.py \\
--output-files "%{pyproject_files}" \\
--output-modules "%{_pyproject_modules}" \\
--output-files "%{pyproject_files %{?__pyproject_optflag_D}}" \\
--output-modules "%{_pyproject_modules %{?__pyproject_optflag_D}}" \\
--buildroot "%{buildroot}" \\
--sitelib "%{python3_sitelib}" \\
--sitearch "%{python3_sitearch}" \\
@ -128,74 +198,102 @@ fi
--pyproject-record "%{_pyproject_record}" \\
--prefix "%{_prefix}" \\
%{**}
}
%{__pyproject_getopt_restore}}
# -t - Process only top-level modules
# -e - Exclude the module names matching given glob, may be used repeatedly
%pyproject_check_import(e:t) %{expand:\\\
if [ ! -f "%{_pyproject_modules}" ]; then
echo 'ERROR: %%%%pyproject_check_import only works when %%%%pyproject_save_files is used' >&2
%pyproject_check_import(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="e", long="exclude", value=true, separator=" -e "},
{short="D", long="dist-name", value=true},
{short="t", long="top-level-only"},
})}\
%{expand:\\\
if [ ! -f "%{_pyproject_modules %{?__pyproject_optflag_D}}" ]; then
_modules_glob=( %{_pyproject_modules_base}-* )
if [ -e "${_modules_glob[0]}" ]; then
_available=$(printf '%%s\n' "${_modules_glob[@]}" | sed 's|.*-pyproject-modules-||' | tr '\n' ' ')
echo "ERROR: %%%%pyproject_check_import requires -D <name> when %%%%pyproject_save_files was called with -D" >&2
echo "Available -D values: ${_available}" >&2
else
echo 'ERROR: %%%%pyproject_check_import only works when %%%%pyproject_save_files is used' >&2
fi
exit 1
fi
%py3_check_import -f "%{_pyproject_modules}" %{?**}
}
%py3_check_import -f "%{_pyproject_modules %{?__pyproject_optflag_D}}" %{?__pyproject_optflag_e} %{?__pyproject_optflag_t} %{?__pyproject_positional_args}
%{__pyproject_getopt_restore}}
%_pyproject_check_import_allow_no_modules(e:t) \
if [ -z "$(cat %{_pyproject_modules})" ]; then\
%_pyproject_check_import_allow_no_modules(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="e", long="exclude", value=true, separator=" -e "},
{short="D", long="dist-name", value=true},
{short="t", long="top-level-only"},
})}\
if [ -z "$(cat %{_pyproject_modules %{?__pyproject_optflag_D}})" ]; then\
echo "No modules to check found, exiting check"\
else\
%pyproject_check_import %{?**}\
fi
fi\
%{__pyproject_getopt_restore}
%default_toxenv py%{python3_version_nodots}
%toxenv %{default_toxenv}
%_pyproject_tomlidep %["%{python3_pkgversion}" == "3"\
? "echo '(python%{python3_pkgversion}dist(tomli) if python%{python3_pkgversion}-devel < 3.11)'"\
: "%[v"%{python3_pkgversion}" < v"3.11"\
? "echo 'python%{python3_pkgversion}dist(tomli)'"\
: "true # will use tomllib, echo nothing"\
]"\
]
# Note: Keep the options in sync with this macro from macros.aaa-pyproject-srpm
%pyproject_buildrequires(rRxtNwe:C:) %{expand:\\\
%pyproject_buildrequires(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="r", long="runtime"},
{short="R", long="no-runtime"},
{short="x", long="extras", value=true, separator=","},
{short="t", long="tox"},
{short="N", long="no-use-build-system"},
{short="w", long="wheel"},
{short="p", long="pyproject-dependencies"},
{short="e", long="toxenv", value=true, separator=","},
{short="g", long="dependency-groups", value=true, separator=","},
{short="C", long="config-settings", value=true, separator=","},
{short="d", long="directory", value=true},
}, {
{"R", {"r", "x", "e", "t", "w", "p"}},
{"N", {"r", "x", "e", "t", "w", "p", "C"}},
{"w", {"p"}},
})}\
%{expand:\\\
%_set_pytest_addopts
# The _auto_set_build_flags feature does not do this in %%generate_buildrequires section,
# but we want to get an environment consistent with %%build:
%{?_auto_set_build_flags:%set_build_flags}
# The default flags expect the package note file to exist
# see https://bugzilla.redhat.com/show_bug.cgi?id=2097535
%{?_package_note_flags:%_generate_package_note_file}
%{-R:
%{-r:%{error:The -R and -r options are mutually exclusive}}
%{-x:%{error:The -R and -x options are mutually exclusive}}
%{-e:%{error:The -R and -e options are mutually exclusive}}
%{-t:%{error:The -R and -t options are mutually exclusive}}
%{-w:%{error:The -R and -w options are mutually exclusive}}
%{?__pyproject_opt_d:pushd "%{__pyproject_opt_d}" >&2}
%{?__pyproject_opt_w:
%{!?__pyproject_buildrequires_w_warned:%{warn:The %%pyproject_buildrequires -w/--wheel option is deprecated.
It's not efficient to build the wheel several times during the build.
The option is not scheduled for removal, but packagers should use the -p/--pyproject-dependencies option instead.
}%global __pyproject_buildrequires_w_warned 1}
}
%{-N:
%{-r:%{error:The -N and -r options are mutually exclusive}}
%{-x:%{error:The -N and -x options are mutually exclusive}}
%{-e:%{error:The -N and -e options are mutually exclusive}}
%{-t:%{error:The -N and -t options are mutually exclusive}}
%{-w:%{error:The -N and -w options are mutually exclusive}}
%{-C:%{error:The -N and -C options are mutually exclusive}}
%{?__pyproject_opt_N:
%{?__pyproject_opt_g:if [ -f pyproject.toml ]; then
%_pyproject_tomlidep
fi}
}
%{-e:%{expand:%global toxenv %(%{__python3} -s %{_rpmconfigdir}/redhat/pyproject_construct_toxenv.py %{?**})}}
%{?__pyproject_opt_e:%{expand:%global toxenv %{__pyproject_opt_e}}}
echo 'pyproject-rpm-macros' # first stdout line matches the implementation in macros.aaa-pyproject-srpm
echo 'python%{python3_pkgversion}-devel'
echo 'python%{python3_pkgversion}dist(pip) >= 19'
echo 'python%{python3_pkgversion}dist(packaging)'
%{!-N:if [ -f pyproject.toml ]; then
%["%{python3_pkgversion}" == "3"
? "echo '(python%{python3_pkgversion}dist(tomli) if python%{python3_pkgversion}-devel < 3.11)'"
: "%[v"%{python3_pkgversion}" < v"3.11"
? "echo 'python%{python3_pkgversion}dist(tomli)'"
: "true # will use tomllib, echo nothing"
]"
]
%{!?__pyproject_opt_N:echo 'python%{python3_pkgversion}dist(pip) >= 19'
if [ -f pyproject.toml ]; then
%_pyproject_tomlidep
elif [ -f setup.py ]; then
# Note: If the default requirements change, also change them in the script!
echo 'python%{python3_pkgversion}dist(setuptools) >= 40.8'
echo 'python%{python3_pkgversion}dist(wheel)'
else
echo 'ERROR: Neither pyproject.toml nor setup.py found, consider using %%%%pyproject_buildrequires -N <requirements-file> if this is not a Python package.' >&2
echo 'ERROR: Neither pyproject.toml nor setup.py found, consider using %%%%pyproject_buildrequires -N/--no-use-build-system <requirements-file> if this is not a Python package.' >&2
exit 1
fi}
# setuptools assumes no pre-existing dist-info
@ -203,21 +301,37 @@ rm -rfv *.dist-info/ >&2
if [ -f %{__python3} ]; then
mkdir -p "%{_pyproject_builddir}"
echo -n > %{_pyproject_buildrequires}
CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}" TMPDIR="%{_pyproject_builddir}" \\\
RPM_TOXENV="%{toxenv}" HOSTNAME="rpmbuild" %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_buildrequires.py %{?!_python_no_extras_requires:--generate-extras} --python3_pkgversion %{python3_pkgversion} --wheeldir %{_pyproject_wheeldir} --output %{_pyproject_buildrequires} %{?**} >&2
%{_pyproject_build_flags} \\\
TMPDIR="%{_pyproject_builddir}" \\\
RPM_TOXENV="%{toxenv}" FEDORA=%{?fedora} HOSTNAME="rpmbuild" %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_buildrequires.py %{!?_python_no_extras_requires:--generate-extras} --python3_pkgversion %{python3_pkgversion} --wheeldir %{_pyproject_wheeldir} --output %{_pyproject_buildrequires} --dep-overrides-file %{_pyproject_dep_overrides} %{?**} >&2
cat %{_pyproject_buildrequires}
fi
# Incomplete .dist-info dir might confuse importlib.metadata
rm -rfv *.dist-info/ >&2
}
%{?__pyproject_opt_d:popd >&2}
%{__pyproject_getopt_restore}}
%tox(e:) %{expand:\\\
%tox(-) \
%{lua:require("fedora.rpm.pyproject_getopt").getopt({
{short="e", long="toxenv", value=true, separator=","},
})}\
%{expand:\\\
TOX_TESTENV_PASSENV="${TOX_TESTENV_PASSENV:-*}" \\
%{?py3_test_envvars}%{?!py3_test_envvars:PYTHONDONTWRITEBYTECODE=1 \\
%{?py3_test_envvars}%{!?py3_test_envvars:PYTHONDONTWRITEBYTECODE=1 \\
PATH="%{buildroot}%{_bindir}:$PATH" \\
PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}}" \\
%{?__pytest_addopts:PYTEST_ADDOPTS="${PYTEST_ADDOPTS:-} %{__pytest_addopts}"}} \\
HOSTNAME="rpmbuild" \\
%{__python3} -m tox --current-env -q --recreate -e "%{-e:%{-e*}}%{!-e:%{toxenv}}" %{?*}
}
%{__python3} -m tox --current-env --assert-config -q --recreate -e "%{?__pyproject_opt_e}%{!?__pyproject_opt_e:%{toxenv}}" %{?__pyproject_positional_args}
%{__pyproject_getopt_restore}}
# Declarative buildsystem, requires RPM 4.20+ to work
# https://rpm-software-management.github.io/rpm/manual/buildsystem.html
%buildsystem_pyproject_conf() %nil
%buildsystem_pyproject_generate_buildrequires() %pyproject_buildrequires %*
%buildsystem_pyproject_build() %pyproject_wheel %*
%buildsystem_pyproject_install() %["%{shrink:%*}" == "" ? "%{error:BuildOption(install) is mandatory with pyproject BuildSystem.}" : "%pyproject_install \
%pyproject_save_files %*"]
%buildsystem_pyproject_check() %pyproject_check_import %*

View File

@ -10,6 +10,7 @@ import subprocess
import re
import tempfile
import email.parser
import functools
import pathlib
import zipfile
@ -21,6 +22,15 @@ from pyproject_wheel import parse_config_settings_args
# Allow only the forms we know we can handle.
VERSION_RE = re.compile(r'[a-zA-Z0-9.-]+(\.\*)?')
FEDORA = int(os.getenv('FEDORA') or 0)
RHEL = int(os.getenv('RHEL') or 0)
# To avoid breakage on Fedora < 45 and RHEL < 11
# we don't compare extras listed in %pyproject_buildrequires -x
# with upstream metadata.
# Instead we issue warning on old releases.
REJECT_INVALID_EXTRAS = FEDORA >= 45 or RHEL >= 11
class EndPass(Exception):
"""End current pass of generating requirements"""
@ -34,7 +44,9 @@ def print_err(*args, **kwargs):
try:
from packaging.markers import Marker
from packaging.requirements import Requirement, InvalidRequirement
from packaging.specifiers import SpecifierSet
from packaging.utils import canonicalize_name
except ImportError as e:
print_err('Import error:', e)
@ -43,6 +55,9 @@ except ImportError as e:
# uses packaging, needs to be imported after packaging is verified to be present
from pyproject_convert import convert
from pyproject_dependency_overrides import (
parse_override_string, apply_overrides_to_specifiers,
)
def guess_reason_for_invalid_requirement(requirement_str):
@ -68,10 +83,12 @@ def guess_reason_for_invalid_requirement(requirement_str):
class Requirements:
"""Requirement gatherer. The macro will eventually print out output_lines."""
def __init__(self, get_installed_version, extras=None,
generate_extras=False, python3_pkgversion='3', config_settings=None):
generate_extras=False, python3_pkgversion='3', config_settings=None,
dependency_overrides=None):
self.get_installed_version = get_installed_version
self.output_lines = []
self.extras = set()
self.extras_ok_nonexisting = set()
if extras:
for extra in extras:
@ -83,9 +100,56 @@ class Requirements:
self.generate_extras = generate_extras
self.python3_pkgversion = python3_pkgversion
self.config_settings = config_settings
self.dependency_overrides = self._parse_dependency_overrides(dependency_overrides or [])
self.metadata_extras = []
def add_extras(self, *extras):
self.extras |= set(e.strip() for e in extras)
self.package_name = None
def add_extras(self, *extras, error_nonexisting=None):
if error_nonexisting is None:
error_nonexisting = REJECT_INVALID_EXTRAS
new_extras = set(canonicalize_name(e.strip()) for e in extras if e.strip())
self.extras |= new_extras
if not error_nonexisting:
self.extras_ok_nonexisting |= new_extras
return new_extras
def _parse_dependency_overrides(self, overrides):
"""Parse dependency override specifications into a structured format.
Each override is a string of the form: package:action[:value][:br_only]
"""
parsed_overrides = {}
for override in overrides:
# Strip br_only scope suffix (irrelevant for BuildRequires)
parts = override.split(':')
if parts and parts[-1].strip() == 'br_only':
override = ':'.join(parts[:-1])
package, action, value = parse_override_string(override)
parsed_overrides.setdefault(package, []).append(
{'action': action, 'value': value})
return parsed_overrides
def _should_ignore_dependency(self, package_name):
"""Check if a dependency should be completely ignored."""
if package_name not in self.dependency_overrides:
return False
return any(o['action'] == 'ignore' for o in self.dependency_overrides[package_name])
def _apply_dependency_overrides(self, requirement):
"""Apply dependency overrides to a list of specifiers for a given package."""
package_name = canonicalize_name(requirement.name)
if package_name not in self.dependency_overrides:
return requirement
overridden = apply_overrides_to_specifiers(
requirement.specifier, self.dependency_overrides[package_name],
package_name=package_name, log_fn=print_err)
requirement.specifier = SpecifierSet(','.join(str(s) for s in overridden))
return requirement
@property
def marker_envs(self):
@ -99,18 +163,32 @@ class Requirements:
return True
return False
def add(self, requirement_str, *, package_name=None, source=None):
def set_package_name(self, name):
canonical_name = canonicalize_name(name)
if self.package_name is None:
self.package_name = canonical_name
else:
# This really shouldn't happen, but it's better to be safe than sorry
if canonical_name != self.package_name:
raise ValueError(f'Package name mismatch: {canonical_name} != {self.package_name}')
def add(self, requirement, *, source=None, extra=None):
"""Output a Python-style requirement string as RPM dep"""
requirement_str = str(requirement)
print_err(f'Handling {requirement_str} from {source}')
try:
requirement = Requirement(requirement_str)
except InvalidRequirement:
hint = guess_reason_for_invalid_requirement(requirement_str)
message = f'Requirement {requirement_str!r} from {source} is invalid.'
if hint:
message += f' Hint: {hint}'
raise ValueError(message)
# requirements read initially from the metadata are strings
# further on we work with them as Requirement instances
if not isinstance(requirement, Requirement):
try:
requirement = Requirement(requirement)
except InvalidRequirement:
hint = guess_reason_for_invalid_requirement(requirement)
message = f'Requirement {requirement!r} from {source} is invalid.'
if hint:
message += f' Hint: {hint}'
raise ValueError(message)
if requirement.url:
print_err(
@ -118,14 +196,25 @@ class Requirements:
)
name = canonicalize_name(requirement.name)
if self._should_ignore_dependency(name):
print_err(f'Ignoring dependency {name} per dependency override')
return
if extra is not None:
extra_str = f'extra == "{extra}"'
if requirement.marker is not None:
extra_str = f'({requirement.marker}) and {extra_str}'
requirement.marker = Marker(extra_str)
if (requirement.marker is not None and
not self.evaluate_all_environments(requirement)):
print_err(f'Ignoring alien requirement:', requirement_str)
self.ignored_alien_requirements.append(requirement_str)
self.ignored_alien_requirements.append(requirement)
return
# Handle self-referencing requirements
if package_name and canonicalize_name(package_name) == name:
if self.package_name and self.package_name == name:
# Self-referential extras need to be handled specially
if requirement.extras:
if not (requirement.extras <= self.extras): # only handle it if needed
@ -133,11 +222,16 @@ class Requirements:
self.add_extras(*requirement.extras)
# re-add all of the alien requirements ignored in the past
# they might no longer be alien now
self.readd_ignored_alien_requirements(package_name=package_name)
self.readd_ignored_alien_requirements()
else:
print_err(f'Ignoring self-referential requirement without extras:', requirement_str)
return
# Apply dependency overrides before the installed-version check,
# so the check reflects the constraints we will actually output.
requirement = self._apply_dependency_overrides(requirement)
requirement_str = str(requirement)
# We need to always accept pre-releases as satisfying the requirement
# Otherwise e.g. installed cffi version 1.15.0rc2 won't even satisfy the requirement for "cffi"
# https://bugzilla.redhat.com/show_bug.cgi?id=2014639#c3
@ -215,7 +309,8 @@ def toml_load(opened_binary_file):
return tomllib.load(opened_binary_file)
def get_backend(requirements):
@functools.cache
def load_pyproject():
try:
f = open('pyproject.toml', 'rb')
except FileNotFoundError:
@ -223,6 +318,11 @@ def get_backend(requirements):
else:
with f:
pyproject_data = toml_load(f)
return pyproject_data
def get_backend(requirements):
pyproject_data = load_pyproject()
buildsystem_data = pyproject_data.get('build-system', {})
requirements.extend(
@ -248,7 +348,6 @@ def get_backend(requirements):
# with pyproject.toml without a specified build backend.
# If the default requirements change, also change them in the macro!
requirements.add('setuptools >= 40.8', source='default build backend')
requirements.add('wheel', source='default build backend')
requirements.check(source='build backend')
@ -271,7 +370,7 @@ def get_backend(requirements):
def generate_build_requirements(backend, requirements):
get_requires = getattr(backend, 'get_requires_for_build_wheel', None)
if get_requires:
new_reqs = get_requires(config_settings=requirements.config_settings)
new_reqs = get_requires(requirements.config_settings)
requirements.extend(new_reqs, source='get_requires_for_build_wheel')
requirements.check(source='get_requires_for_build_wheel')
@ -281,18 +380,24 @@ def parse_metadata_file(metadata_file):
def requires_from_parsed_metadata_file(message):
return {k: message.get_all(k, ()) for k in ('Requires', 'Requires-Dist')}
return {k: message.get_all(k, ()) for k in ('Requires-Dist',)}
def package_name_from_parsed_metadata_file(message):
return message.get('name')
def package_name_and_requires_from_metadata_file(metadata_file):
def extras_from_parsed_metadata_file(message):
raw_extras = message.get_all('provides-extra') or []
return [canonicalize_name(extra) for extra in raw_extras if extra]
def extract_data_from_metadata_file(metadata_file):
message = parse_metadata_file(metadata_file)
package_name = package_name_from_parsed_metadata_file(message)
extras_names = extras_from_parsed_metadata_file(message)
requires = requires_from_parsed_metadata_file(message)
return package_name, requires
return package_name, requires, extras_names
def generate_run_requirements_hook(backend, requirements):
@ -302,16 +407,18 @@ def generate_run_requirements_hook(backend, requirements):
raise ValueError(
'The build backend cannot provide build metadata '
'(incl. runtime requirements) before build. '
'Use the provisional -w flag to build the wheel and parse the metadata from it, '
'or use the -R flag not to generate runtime dependencies.'
'If the dependencies are specified in the pyproject.toml [project] '
'table, you can use the -p flag to read them. '
'Alternatively, use the -R flag not to generate runtime dependencies.'
)
dir_basename = prepare_metadata('.', config_settings=requirements.config_settings)
dir_basename = prepare_metadata('.', requirements.config_settings)
with open(dir_basename + '/METADATA') as metadata_file:
name, requires = package_name_and_requires_from_metadata_file(metadata_file)
name, requires, metadata_extras = extract_data_from_metadata_file(metadata_file)
requirements.set_package_name(name)
requirements.metadata_extras.extend(metadata_extras)
for key, req in requires.items():
requirements.extend(req,
package_name=name,
source=f'hook generated metadata: {key} ({name})')
source=f'hook generated metadata: {key} ({requirements.package_name})')
def find_built_wheel(wheeldir):
@ -350,18 +457,46 @@ def generate_run_requirements_wheel(backend, requirements, wheeldir):
for name in wheelfile.namelist():
if name.count('/') == 1 and name.endswith('.dist-info/METADATA'):
with io.TextIOWrapper(wheelfile.open(name), encoding='utf-8') as metadata_file:
name, requires = package_name_and_requires_from_metadata_file(metadata_file)
name, requires, metadata_extras = extract_data_from_metadata_file(metadata_file)
requirements.set_package_name(name)
requirements.metadata_extras.extend(metadata_extras)
for key, req in requires.items():
requirements.extend(req,
package_name=name,
source=f'built wheel metadata: {key} ({name})')
break
else:
raise RuntimeError('Could not find *.dist-info/METADATA in built wheel.')
def generate_run_requirements(backend, requirements, *, build_wheel, wheeldir):
if build_wheel:
def generate_run_requirements_pyproject(requirements):
pyproject_data = load_pyproject()
if not (project_table := pyproject_data.get('project', {})):
raise ValueError('Could not find the [project] table in pyproject.toml.')
dynamic_fields = project_table.get('dynamic', [])
if 'dependencies' in dynamic_fields or 'optional-dependencies' in dynamic_fields:
raise ValueError('Could not read the dependencies or optional-dependencies '
'from the [project] table in pyproject.toml, as the field is dynamic.')
dependencies = project_table.get('dependencies', [])
name = project_table.get('name')
requirements.set_package_name(name)
requirements.extend(dependencies,
source=f'pyproject.toml generated metadata: [dependencies] ({name})')
optional_dependencies = project_table.get('optional-dependencies', {})
for extra, dependencies in optional_dependencies.items():
requirements.extend(dependencies,
source=f'pyproject.toml generated metadata: [optional-dependencies] {extra} ({name})',
extra=extra)
requirements.metadata_extras.append(canonicalize_name(extra))
def generate_run_requirements(backend, requirements, *, build_wheel, pyproject_dependencies, wheeldir):
if pyproject_dependencies:
generate_run_requirements_pyproject(requirements)
elif build_wheel:
generate_run_requirements_wheel(backend, requirements, wheeldir)
else:
generate_run_requirements_hook(backend, requirements)
@ -369,7 +504,7 @@ def generate_run_requirements(backend, requirements, *, build_wheel, wheeldir):
def generate_tox_requirements(toxenv, requirements):
toxenv = ','.join(toxenv)
requirements.add('tox-current-env >= 0.0.6', source='tox itself')
requirements.add('tox-current-env >= 0.0.16', source='tox itself')
requirements.check(source='tox itself')
with tempfile.NamedTemporaryFile('r') as deps, \
tempfile.NamedTemporaryFile('r') as extras, \
@ -379,6 +514,7 @@ def generate_tox_requirements(toxenv, requirements):
'--print-deps-to', deps.name,
'--print-extras-to', extras.name,
'--no-provision', provision.name,
'--assert-config',
'-q', '-r', '-e', toxenv],
check=False,
encoding='utf-8',
@ -404,13 +540,114 @@ def generate_tox_requirements(toxenv, requirements):
else:
r.check_returncode()
tox_extras = {e for e in extras.read().splitlines() if e}
if not (tox_extras <= requirements.extras):
requirements.add_extras(*tox_extras, error_nonexisting=False)
requirements.readd_ignored_alien_requirements(source=f'tox added extras: {toxenv}')
deplines = deps.read().splitlines()
packages = convert_requirements_txt(deplines)
requirements.add_extras(*extras.read().splitlines())
requirements.extend(packages,
source=f'tox --print-deps-only: {toxenv}')
def tox_dependency_groups(toxenv):
# We call this command separately instead of folding it into the previous one
# becasue --print-dependency-groups-to only works with tox 4.22+ and tox-current-env 0.0.14+.
# We handle failure gracefully: upstreams using dependency_groups should require tox >= 4.22.
toxenv = ','.join(toxenv)
with tempfile.NamedTemporaryFile('r') as groups:
r = subprocess.run(
[sys.executable, '-m', 'tox',
'--print-dependency-groups-to', groups.name,
'-q', '-e', toxenv],
check=False,
encoding='utf-8',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
if r.returncode == 0:
if r.stdout:
print_err(r.stdout, end='')
if output := groups.read().strip():
return output.splitlines()
return []
def generate_dependency_groups(requested_groups, requirements):
"""Adapted from https://peps.python.org/pep-0735/#reference-implementation (public domain)"""
from collections import defaultdict
def _normalize_name(name: str) -> str:
return re.sub(r"[-_.]+", "-", name).lower()
def _normalize_group_names(dependency_groups: dict) -> dict:
original_names = defaultdict(list)
normalized_groups = {}
for group_name, value in dependency_groups.items():
normed_group_name = _normalize_name(group_name)
original_names[normed_group_name].append(group_name)
normalized_groups[normed_group_name] = value
errors = []
for normed_name, names in original_names.items():
if len(names) > 1:
errors.append(f"{normed_name} ({', '.join(names)})")
if errors:
raise ValueError(f"Duplicate dependency group names: {', '.join(errors)}")
return normalized_groups
def _resolve_dependency_group(
dependency_groups: dict, group: str, past_groups: tuple[str, ...] = ()
) -> list[str]:
if group in past_groups:
raise ValueError(f"Cyclic dependency group include: {group} -> {past_groups}")
if group not in dependency_groups:
raise LookupError(f"Dependency group '{group}' not found")
raw_group = dependency_groups[group]
if not isinstance(raw_group, list):
raise ValueError(f"Dependency group '{group}' is not a list")
realized_group = []
for item in raw_group:
if isinstance(item, str):
realized_group.append(item)
elif isinstance(item, dict):
if tuple(item.keys()) != ("include-group",):
raise ValueError(f"Invalid dependency group item: {item}")
include_group = _normalize_name(next(iter(item.values())))
realized_group.extend(
_resolve_dependency_group(
dependency_groups, include_group, past_groups + (group,)
)
)
else:
raise ValueError(f"Invalid dependency group item: {item}")
return realized_group
def resolve(dependency_groups: dict, group: str) -> list[str]:
if not isinstance(dependency_groups, dict):
raise TypeError("Dependency Groups table is not a dict")
return _resolve_dependency_group(dependency_groups, _normalize_name(group))
pyproject_data = load_pyproject()
dependency_groups_raw = pyproject_data.get("dependency-groups", {})
dependency_groups = _normalize_group_names(dependency_groups_raw)
for group_names in requested_groups:
for group_name in group_names.split(","):
requirements.extend(
resolve(dependency_groups, group_name),
source=f"Dependency group {group_name}",
)
def python3dist(name, op=None, version=None, python3_pkgversion="3"):
prefix = f"python{python3_pkgversion}dist"
@ -423,10 +660,11 @@ def python3dist(name, op=None, version=None, python3_pkgversion="3"):
def generate_requires(
*, include_runtime=False, build_wheel=False, wheeldir=None, toxenv=None, extras=None,
*, include_runtime=False, build_wheel=False, wheeldir=None, toxenv=None, extras=None, dependency_groups=None,
get_installed_version=importlib.metadata.version, # for dep injection
generate_extras=False, python3_pkgversion="3", requirement_files=None, use_build_system=True,
output, config_settings=None,
pyproject_dependencies=False,
output, config_settings=None, dependency_overrides=None,
):
"""Generate the BuildRequires for the project in the current directory
@ -439,33 +677,51 @@ def generate_requires(
generate_extras=generate_extras,
python3_pkgversion=python3_pkgversion,
config_settings=config_settings,
dependency_overrides=dependency_overrides or [],
)
dependency_groups = dependency_groups or []
try:
if (include_runtime or toxenv) and not use_build_system:
raise ValueError('-N option cannot be used in combination with -r, -e, -t, -x options')
if (include_runtime or toxenv or pyproject_dependencies) and not use_build_system:
raise ValueError('-N option cannot be used in combination with -r, -e, -t, -x, -p options')
if requirement_files:
for req_file in requirement_files:
requirements.extend(
convert_requirements_txt(req_file, pathlib.Path(req_file.name)),
source=f'requirements file {req_file.name}'
convert_requirements_txt(req_file.read_text().splitlines(), req_file),
source=f'requirements file {req_file}'
)
requirements.check(source='all requirements files')
if use_build_system:
backend = get_backend(requirements)
generate_build_requirements(backend, requirements)
if include_runtime or toxenv:
generate_run_requirements(backend, requirements, build_wheel=build_wheel,
pyproject_dependencies=pyproject_dependencies, wheeldir=wheeldir)
if toxenv:
include_runtime = True
generate_tox_requirements(toxenv, requirements)
dependency_groups.extend(tox_dependency_groups(toxenv))
if dependency_groups:
generate_dependency_groups(dependency_groups, requirements)
if include_runtime:
generate_run_requirements(backend, requirements, build_wheel=build_wheel, wheeldir=wheeldir)
for extra in requirements.extras:
if extra not in requirements.metadata_extras:
if extra in requirements.extras_ok_nonexisting:
print_err(
f'WARNING: Extra {extra!r} not found in project metadata. '
f'Available extras: {requirements.metadata_extras}. '
)
else:
raise ValueError(
f'Extra {extra} does not exist in upstream metadata. '
f'Available extras: {requirements.metadata_extras}.'
)
except EndPass:
return
finally:
output.write_text(os.linesep.join(requirements.output_lines) + os.linesep)
def main(argv):
def argparser():
parser = argparse.ArgumentParser(
description='Generate BuildRequires for a Python project.',
prog='%pyproject_buildrequires',
@ -485,7 +741,7 @@ def main(argv):
help=argparse.SUPPRESS,
)
parser.add_argument(
'-p', '--python3_pkgversion', metavar='PYTHON3_PKGVERSION',
'--python3_pkgversion', metavar='PYTHON3_PKGVERSION',
default="3", help=argparse.SUPPRESS,
)
parser.add_argument(
@ -500,6 +756,11 @@ def main(argv):
help='comma separated list of "extras" for runtime requirements '
'(e.g. -x testing,feature-x) (implies --runtime, can be repeated)',
)
parser.add_argument(
'-g', '--dependency-groups', metavar='GROUPS', action='append',
help='comma separated list of dependency groups (PEP 735) for requirements '
'(e.g. -g tests,docs) (can be repeated)',
)
parser.add_argument(
'-t', '--tox', action='store_true',
help=('generate test tequirements from tox environment '
@ -513,7 +774,12 @@ def main(argv):
parser.add_argument(
'-w', '--wheel', action='store_true', default=False,
help=('Generate run-time requirements by building the wheel '
'(useful for build backends without the prepare_metadata_for_build_wheel hook)'),
'(useful for build backends without the prepare_metadata_for_build_wheel hook, deprecated)'),
)
parser.add_argument(
'-p', '--pyproject-dependencies', action='store_true', default=False,
help=('Generate dependencies from [project] table of pyproject.toml '
'instead of calling prepare_metadata_for_build_wheel hook)'),
)
parser.add_argument(
'-R', '--no-runtime', action='store_false', dest='runtime',
@ -524,19 +790,32 @@ def main(argv):
action='store_false', help='Use -N to indicate that project does not use any build system',
)
parser.add_argument(
'requirement_files', nargs='*', type=argparse.FileType('r'),
'requirement_files', nargs='*', type=pathlib.Path,
metavar='REQUIREMENTS.TXT',
help=('Add buildrequires from file'),
)
parser.add_argument(
'-C',
'-C', '--config-settings',
dest='config_settings',
action='append',
help='Configuration settings to pass to the PEP 517 backend',
)
parser.add_argument(
'--dep-overrides-file', type=pathlib.Path, default=None,
help=argparse.SUPPRESS,
)
parser.add_argument('-d', '--directory', help=argparse.SUPPRESS) # processed by RPM macro
return parser
def main(argv):
parser = argparser()
args = parser.parse_args(argv)
for req_file in args.requirement_files:
if not req_file.exists():
parser.error(f"can't open '{req_file}': No such file or directory")
if not args.use_build_system:
args.runtime = False
@ -556,6 +835,10 @@ def main(argv):
if args.extras:
args.runtime = True
dependency_overrides = []
if args.dep_overrides_file and args.dep_overrides_file.is_file():
dependency_overrides = args.dep_overrides_file.read_text().split()
try:
generate_requires(
include_runtime=args.runtime,
@ -563,12 +846,15 @@ def main(argv):
wheeldir=args.wheeldir,
toxenv=args.toxenv,
extras=args.extras,
dependency_groups=args.dependency_groups,
generate_extras=args.generate_extras,
python3_pkgversion=args.python3_pkgversion,
requirement_files=args.requirement_files,
use_build_system=args.use_build_system,
pyproject_dependencies=args.pyproject_dependencies,
output=args.output,
config_settings=parse_config_settings_args(args.config_settings),
dependency_overrides=dependency_overrides,
)
except Exception:
# Log the traceback explicitly (it's useful debug info)

File diff suppressed because it is too large Load Diff

View File

@ -1,15 +0,0 @@
import argparse
import sys
def main(argv):
parser = argparse.ArgumentParser(
description='Parse -e arguments instead of RPM getopt.'
)
parser.add_argument('-e', '--toxenv', action='append')
args, _ = parser.parse_known_args(argv)
return ','.join(args.toxenv)
if __name__ == '__main__':
print(main(sys.argv[1:]))

View File

@ -28,14 +28,14 @@ from packaging.version import parse as parse_version
class RpmVersion():
def __init__(self, version_id):
version = parse_version(version_id)
if isinstance(version._version, str):
if version.__class__.__name__ == 'LegacyVersion':
self.version = version._version
else:
self.epoch = version._version.epoch
self.version = list(version._version.release)
self.pre = version._version.pre
self.dev = version._version.dev
self.post = version._version.post
self.epoch = version.epoch
self.version = list(version.release)
self.pre = version.pre
self.dev = version.dev
self.post = version.post
# version.local is ignored as it is not expected to appear
# in public releases
# https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
@ -63,9 +63,9 @@ class RpmVersion():
if self.pre:
rpm_suffix = '~{}'.format(''.join(str(x) for x in self.pre))
elif self.dev:
rpm_suffix = '~~{}'.format(''.join(str(x) for x in self.dev))
rpm_suffix = '~~dev{}'.format(self.dev)
elif self.post:
rpm_suffix = '^post{}'.format(self.post[1])
rpm_suffix = '^post{}'.format(self.post)
else:
rpm_suffix = ''
return '{}{}{}'.format(rpm_epoch, rpm_version, rpm_suffix)

View File

@ -0,0 +1,147 @@
"""Shared logic for dependency override parsing and application."""
from packaging.specifiers import Specifier
from packaging.utils import canonicalize_name
VALID_ACTIONS = ['drop_upper', 'drop_lower', 'set_upper', 'set_lower',
'drop_constraints', 'ignore']
# Operators classified as upper or lower bounds.
# == and ~= impose both bounds and are decomposed by drop_upper/drop_lower
# (see _remove_upper_bounds / _remove_lower_bounds).
# === is removed atomically (no meaningful decomposition).
# != (exclusions) are always preserved.
UPPER_OPS = ['<', '<=', '==', '===', '~=']
LOWER_OPS = ['>', '>=', '==', '===', '~=']
def parse_override_string(override_str):
"""Parse and validate a single dependency override string.
The br_only suffix must be handled by the caller before calling this.
Returns (canonical_package_name, action, value_or_None).
Raises ValueError on invalid input.
"""
parts = override_str.split(':')
if len(parts) < 2:
raise ValueError(
f'Invalid dependency override format: {override_str!r}. '
f'Expected format: package:action[:value][:br_only]')
package = canonicalize_name(parts[0].strip())
action = parts[1].strip()
value = parts[2].strip() if len(parts) > 2 else None
if action not in VALID_ACTIONS:
raise ValueError(
f'Invalid dependency override action: {action!r}. '
f'Valid actions: {VALID_ACTIONS}')
if action.startswith('set_') and not value:
raise ValueError(
f'Action {action!r} requires a value: {override_str!r}')
if action in ('drop_upper', 'drop_lower', 'drop_constraints',
'ignore') and value:
raise ValueError(
f'Action {action!r} does not accept a value: {override_str!r}')
if value:
try:
Specifier(f'<{value}')
except Exception:
raise ValueError(
f'Invalid version in dependency override: {value!r} '
f'from {override_str!r}')
return (package, action, value)
def _remove_upper_bounds(specifiers):
"""Remove upper-bound specifiers, decomposing == and ~= into their lower half.
PEP 440 defines ~= V.N as >= V.N, == V.*, so the lower half is >= V.N.
== V is mathematically >= V AND <= V, so the lower half is >= V.
=== has no meaningful decomposition and is removed entirely.
"""
result = []
for s in specifiers:
if s.operator not in UPPER_OPS:
result.append(s)
elif s.operator == '~=':
result.append(Specifier(f'>={s.version}'))
elif s.operator == '==' and not s.version.endswith('.*'):
result.append(Specifier(f'>={s.version}'))
return result
def _remove_lower_bounds(specifiers):
"""Remove lower-bound specifiers, decomposing == into its upper half.
== V is mathematically >= V AND <= V, so the upper half is <= V.
~= has a complex effective upper bound (prefix match), so it is removed.
=== has no meaningful decomposition and is removed entirely.
"""
result = []
for s in specifiers:
if s.operator not in LOWER_OPS:
result.append(s)
elif s.operator == '==' and not s.version.endswith('.*'):
result.append(Specifier(f'<={s.version}'))
return result
def apply_overrides_to_specifiers(specifiers, overrides, package_name=None,
log_fn=None):
"""Apply a list of override dicts to a list of Specifier objects.
Each override is {'action': str, 'value': str or None}.
Returns a new list of Specifier objects.
The 'ignore' action is skipped (handled separately by callers).
If log_fn is provided, it is called with a message string per action.
"""
new_specifiers = list(specifiers)
for override in overrides:
action = override['action']
value = override['value']
if action == 'ignore':
continue
elif action == 'drop_constraints':
if log_fn:
log_fn(f'Applying override: removing all constraints for {package_name}')
new_specifiers = []
elif action == 'drop_upper':
if log_fn:
log_fn(f'Applying override: removing upper bounds for {package_name}')
new_specifiers = _remove_upper_bounds(new_specifiers)
elif action == 'drop_lower':
if log_fn:
log_fn(f'Applying override: removing lower bounds for {package_name}')
new_specifiers = _remove_lower_bounds(new_specifiers)
elif action == 'set_upper':
if log_fn:
log_fn(f'Applying override: setting upper bound for {package_name} to < {value}')
new_specifiers = _remove_upper_bounds(new_specifiers)
new_specifiers.append(Specifier(f'<{value}'))
elif action == 'set_lower':
if log_fn:
log_fn(f'Applying override: setting lower bound for {package_name} to >= {value}')
new_specifiers = _remove_lower_bounds(new_specifiers)
new_specifiers.append(Specifier(f'>={value}'))
return new_specifiers
if __name__ == '__main__':
import sys
override_str = sys.argv[1].removesuffix(':br_only')
try:
parse_override_string(override_str)
except ValueError as e:
print(e, file=sys.stderr)
sys.exit(1)

View File

@ -0,0 +1,318 @@
local M = {}
local function error_out(macro_name, message)
rpm.expand("%{error:%%" .. macro_name .. ": " .. message .. "}")
end
local function option_repr(spec)
return "-" .. spec.short .. "/--" .. spec.long
end
local function error_mutually_exclusive(macro_name, a, b)
error_out(macro_name,
option_repr(a) .. " and " .. option_repr(b) ..
" are mutually exclusive")
end
-- Append a value to found[short]. Errors if repeated without a separator.
local function record_value(macro_name, spec, label, val, found)
if found[spec.short] then
if not spec.separator then
error_out(macro_name, "option " .. label .. " cannot be repeated")
return false
end
else
found[spec.short] = {}
end
found[spec.short][#found[spec.short] + 1] = val
return true
end
-- Build a token table from RPM macro arguments (%1, %2, ..., %{%#}).
-- This preserves %{quote:...} quoting, unlike splitting %** on whitespace.
-- Can be replaced by the native RPM Lua args table when we no longer support c9s.
function M.rpm_args()
local args = {}
local nargs = tonumber(rpm.expand("%#"))
for n = 1, nargs do
args[n] = rpm.expand("%" .. n)
end
return args
end
-- RPM may embed newlines within tokens (from %{expand:} or line continuations).
-- Split such tokens on newlines, strip trailing backslashes and whitespace
-- (from \-newline continuations inside %{macro:} syntax), and discard
-- whitespace/backslash-only fragments.
-- Intentional empty strings (e.g. from -d "") are preserved.
local function normalize_tokens(tokens)
local filtered = {}
for _, t in ipairs(tokens) do
if t:find("\n") then
for part in (t .. "\n"):gmatch("([^\n]*)\n") do
local stripped = part:gsub("[%s\\]+$", "")
if stripped ~= "" then
filtered[#filtered + 1] = stripped
end
end
elseif not t:find("^[%s\\]+$") then
filtered[#filtered + 1] = t
end
end
return filtered
end
local function build_lookup(opt_spec)
local by_short = {}
local by_long = {}
for _, spec in ipairs(opt_spec) do
by_short[spec.short] = spec
by_long[spec.long] = spec
end
return by_short, by_long
end
-- Store an option's value. If val is nil, consume the next token.
-- Returns the updated token index, or nil on error.
local function consume_value(macro_name, spec, label, val, tokens, i, found)
if val == nil then
i = i + 1
if i > #tokens then
error_out(macro_name, "option " .. label .. " requires a value")
return nil
end
val = tokens[i]
end
if not record_value(macro_name, spec, label, val, found) then
return nil
end
return i
end
-- Process a --long-option token. Handles --name, --name=val, and --name val.
local function handle_long_option(macro_name, token, tokens, i, by_long, found)
local rest = token:sub(3)
local eq = rest:find("=", 1, true)
local name, val
if eq then
name = rest:sub(1, eq - 1)
val = rest:sub(eq + 1)
else
name = rest
end
local spec = by_long[name]
if not spec then
error_out(macro_name, "unknown option: --" .. name)
return nil
end
if spec.value then
return consume_value(macro_name, spec, "--" .. name, val, tokens, i, found)
else
if val ~= nil then
error_out(macro_name, "option --" .. name .. " does not take a value")
return nil
end
found[spec.short] = true
return i
end
end
-- Process a short option token. Handles -x, -xval, -x val, and bundled -xyz.
local function handle_short_options(macro_name, token, tokens, i, by_short, found)
local j = 2
while j <= #token do
local ch = token:sub(j, j)
local spec = by_short[ch]
if not spec then
error_out(macro_name, "unknown option: -" .. ch)
return nil
end
if spec.value then
local val
if j < #token then
val = token:sub(j + 1)
j = #token
end
i = consume_value(macro_name, spec, "-" .. ch, val, tokens, i, found)
if not i then return nil end
else
found[spec.short] = true
end
j = j + 1
end
return i
end
-- Parse a token list into found options and positional arguments.
-- Returns {found={short_char=true_or_values_table}, positional={...}}, or nil on error.
local function parse(macro_name, tokens, by_short, by_long)
local found = {}
local positional = {}
local i = 1
while i <= #tokens do
local token = tokens[i]
if token == "--" then
for j = i + 1, #tokens do
positional[#positional + 1] = tokens[j]
end
break
elseif token:sub(1, 2) == "--" then
i = handle_long_option(macro_name, token, tokens, i, by_long, found)
if not i then return nil end
elseif token:sub(1, 1) == "-" and #token > 1 then
i = handle_short_options(macro_name, token, tokens, i, by_short, found)
if not i then return nil end
else
positional[#positional + 1] = token
end
i = i + 1
end
return {found = found, positional = positional}
end
-- Verify no mutually exclusive options were used together. Returns false on violation.
local function check_exclusions(macro_name, found, by_short, exclusion_rules)
for _, rule in ipairs(exclusion_rules) do
local opt = rule[1]
local conflicts = rule[2]
if found[opt] then
for _, c in ipairs(conflicts) do
if found[c] then
error_mutually_exclusive(macro_name, by_short[opt], by_short[c])
return false
end
end
end
end
return true
end
local function cleanup_macros(opt_spec)
for _, spec in ipairs(opt_spec) do
rpm.undefine("__pyproject_opt_" .. spec.short)
rpm.undefine("__pyproject_optflag_" .. spec.short)
end
rpm.undefine("__pyproject_positional_args")
end
-- On RPM 4.20+, %{quote:} is transparent (no \x1f delimiters in expanded output).
-- On older RPM, %{quote:} leaks \x1f into shell commands, so we skip quoting there.
M._use_quote = rpm.vercmp(rpm.expand("0%{?rpmversion}"), "4.19.90") >= 0
-- Quote a single value for safe embedding in an rpm.define() call.
-- On RPM 4.20+, empty values or values with spaces are wrapped in %{quote:}.
-- On older RPM, empty values are represented as %{nil}, values with spaces are verbatim.
local function quote_value(v)
if M._use_quote and (v == "" or v:find("%s")) then
return "%{quote:" .. v .. "}"
elseif v == "" then
return "%{nil}"
else
return v
end
end
-- Apply quote_value to each element, return a new table.
local function quote_values(values)
local parts = {}
for _, v in ipairs(values) do
parts[#parts + 1] = quote_value(v)
end
return parts
end
-- Define %__pyproject_opt_{short}, %__pyproject_optflag_{short}, and
-- %__pyproject_positional_args from a parse result.
-- Takes a table {found, opt_spec, positional}.
-- opt: Flags get %{nil} (defined but empty), value options get each individual
-- value joined by separator.
-- optflag: Ready-to-forward form: "-X" for flags, "-X value" for value options.
-- See the quote_value function about how values are quoted.
local function define_macros(state)
for _, spec in ipairs(state.opt_spec) do
if state.found[spec.short] then
if spec.value then
local parts = quote_values(state.found[spec.short])
local value = table.concat(parts, spec.separator)
rpm.define("__pyproject_opt_" .. spec.short .. " " .. value)
rpm.define("__pyproject_optflag_" .. spec.short .. " -" .. spec.short .. " " .. value)
else
rpm.define("__pyproject_opt_" .. spec.short .. " %{nil}")
rpm.define("__pyproject_optflag_" .. spec.short .. " -" .. spec.short)
end
end
end
if #state.positional > 0 then
local parts = quote_values(state.positional)
rpm.define("__pyproject_positional_args " .. table.concat(parts, " "))
end
end
-- Save/restore stack for nested getopt calls.
-- rpm.define()/rpm.undefine() are global (not scoped to parametric macros),
-- so an inner getopt cleanup clobbers outer values. We save the structured
-- parse result and re-run define_macros() in restore(), avoiding any
-- rpm.expand() round-trip that would strip %{quote:} wrappers.
local _current = nil -- {found, opt_spec, positional} from the most recent getopt()
local _save_stack = {} -- stack of {found, opt_spec, positional} tables
function M.restore()
if _current then
cleanup_macros(_current.opt_spec)
end
_current = table.remove(_save_stack)
if not _current then return end
define_macros(_current)
end
-- Parse and validate macro options, define %__pyproject_opt_* and %__pyproject_positional_args.
-- opt_spec: list of {short=, long=, value=bool, separator=string} tables.
-- value: option takes an argument.
-- separator: allow repeats and join values with this string. Without it, repeats with values are an error.
-- exclusion_rules: optional list of {short_char, {conflicting_short_chars...}} pairs.
-- tokens and macro_name default to the current RPM macro's arguments and name.
function M.getopt(opt_spec, exclusion_rules, tokens, macro_name)
tokens = normalize_tokens(tokens or M.rpm_args())
macro_name = macro_name or rpm.expand("%0")
local by_short, by_long = build_lookup(opt_spec)
_save_stack[#_save_stack + 1] = _current
cleanup_macros(opt_spec)
local result = parse(macro_name, tokens, by_short, by_long)
if not result then return end
if exclusion_rules then
if not check_exclusions(macro_name, result.found, by_short, exclusion_rules) then
return
end
end
_current = {found = result.found, opt_spec = opt_spec, positional = result.positional}
define_macros(_current)
end
return M

View File

@ -0,0 +1,134 @@
"""Patch .dist-info/METADATA files based on dependency override specifications.
Called from %pyproject_install to apply runtime dependency overrides
to installed wheel metadata, so that pythondistdeps.py generates
correct Requires.
"""
import argparse
import sys
def print_err(*args, **kwargs):
kwargs.setdefault('file', sys.stderr)
print(*args, **kwargs)
try:
from packaging.requirements import Requirement
from packaging.specifiers import SpecifierSet
from packaging.utils import canonicalize_name
except ImportError as e:
print_err('Import error:', e)
sys.exit(1)
# uses packaging, needs to be imported after packaging is verified to be present
from pyproject_dependency_overrides import (
parse_override_string, apply_overrides_to_specifiers,
)
def parse_overrides(override_lines):
"""Parse override lines, skipping br_only entries."""
overrides = {}
for line in override_lines:
parts = line.split(':')
if parts and parts[-1].strip() == 'br_only':
continue
package, action, value = parse_override_string(line)
overrides.setdefault(package, []).append(
{'action': action, 'value': value})
return overrides
def apply_overrides_to_requirement(req, overrides_for_pkg):
"""Apply overrides to a single Requirement object. Returns None if ignored."""
for override in overrides_for_pkg:
if override['action'] == 'ignore':
return None
specifiers = apply_overrides_to_specifiers(
list(req.specifier), overrides_for_pkg)
if specifiers:
req.specifier = SpecifierSet(','.join(str(s) for s in specifiers))
else:
req.specifier = SpecifierSet()
return req
def patch_metadata(metadata_path, overrides):
"""Read METADATA, apply overrides to Requires-Dist lines, and rewrite."""
if not overrides:
return
with open(metadata_path, 'r', encoding='utf-8') as f:
lines = f.readlines()
new_lines = []
patched = False
for line in lines:
if line.startswith('Requires-Dist:'):
req_str = line[len('Requires-Dist:'):].strip()
try:
req = Requirement(req_str)
except Exception:
new_lines.append(line)
continue
name = canonicalize_name(req.name)
if name in overrides:
result = apply_overrides_to_requirement(req, overrides[name])
if result is None:
print_err(f'Removing dependency {name} from METADATA per override')
patched = True
continue
new_line = f'Requires-Dist: {result}\n'
if new_line != line:
print_err(f'Patching dependency {name} in METADATA: {req_str.strip()} -> {result}')
patched = True
new_lines.append(new_line)
else:
new_lines.append(line)
else:
new_lines.append(line)
if patched:
with open(metadata_path, 'w', encoding='utf-8') as f:
f.writelines(new_lines)
print_err(f'Patched {metadata_path}')
def main(argv=None):
parser = argparse.ArgumentParser(
description='Patch .dist-info/METADATA based on dependency overrides')
parser.add_argument(
'--overrides', required=True,
help='Path to the dependency overrides file')
parser.add_argument(
'--metadata', required=True,
help='Path to the .dist-info/METADATA file')
args = parser.parse_args(argv)
try:
with open(args.overrides, 'r', encoding='utf-8') as f:
override_lines = f.read().split()
except FileNotFoundError:
return
try:
overrides = parse_overrides(override_lines)
except ValueError as e:
print_err(f'ERROR: {e}')
sys.exit(1)
if overrides:
patch_metadata(args.metadata, overrides)
if __name__ == '__main__':
main()

View File

@ -69,7 +69,6 @@ def combine_logical_lines(lines):
"""Combine logical lines together (backslash line-continuation)"""
pieces = []
for line in lines:
line = line.rstrip('\n')
# Whole-line comments *only* are removed before line-contionuation
if COMMENT_RE.match(line):
continue

View File

@ -2,18 +2,26 @@ import argparse
import fnmatch
import json
import os
import re
from collections import defaultdict
from keyword import iskeyword
from pathlib import PosixPath, PurePosixPath
from importlib.metadata import Distribution
from packaging.utils import canonicalize_name
# From RPM's build/files.c strtokWithQuotes delim argument
RPM_FILES_DELIMETERS = ' \n\t'
RPM_GLOB_SYMBOLS = '[]{}*?!'
# Combined for escape_rpm_path_4_19()
RPM_SPECIAL_SYMBOLS = RPM_FILES_DELIMETERS + RPM_GLOB_SYMBOLS + '"' + "\\"
RPM_ESCAPE_REGEX = re.compile(f"([{re.escape(RPM_SPECIAL_SYMBOLS)}])")
# See the comment in the macro that wraps this script
RPM_PERCENTAGES_COUNT = int(os.getenv('RPM_PERCENTAGES_COUNT', '2'))
RPM_FILES_ESCAPE = os.getenv('RPM_FILES_ESCAPE', '4.19')
PYCACHED_SUFFIX = '{,.opt-?}.pyc'
# RPM hardcodes the lists of manpage extensions and directories,
# so we have to maintain separate ones :(
@ -38,6 +46,24 @@ MANDIRS = [
]
def canonical_name_from_distinfo(distinfo_dir_name):
"""Extract the canonicalized distribution name from a .dist-info directory name.
Examples:
>>> canonical_name_from_distinfo('MarkupSafe-2.0.1.dist-info')
'markupsafe'
>>> canonical_name_from_distinfo('tldr-0.5.dist-info')
'tldr'
>>> canonical_name_from_distinfo('My.Package_name-1.0.dist-info')
'my-package-name'
"""
raw = distinfo_dir_name.removesuffix('.dist-info').rsplit('-', 1)[0]
return canonicalize_name(raw)
class BuildrootPath(PurePosixPath):
"""
This path represents a path in a buildroot.
@ -118,8 +144,9 @@ def pycached(script, python_version):
"""
assert script.suffix == ".py"
pyver = "".join(python_version.split(".")[:2])
pycname = f"{script.stem}.cpython-{pyver}{{,.opt-?}}.pyc"
pycname = f"{script.stem}.cpython-{pyver}{PYCACHED_SUFFIX}"
pyc = pycache_dir(script) / pycname
pyc.glob_suffix_len = len(PYCACHED_SUFFIX)
return [script, pyc]
@ -129,12 +156,11 @@ def add_file_to_module(paths, module_name, module_type, files_dirs, *files):
"""
for module in paths["modules"][module_name]:
if module["type"] == module_type:
if files[0] not in module[files_dirs]:
module[files_dirs].extend(files)
module[files_dirs].update(files)
break
else:
paths["modules"][module_name].append(
{"type": module_type, "files": [], "dirs": [], files_dirs: list(files)}
{"type": module_type, "files": set(), "dirs": set(), files_dirs: set(files)}
)
@ -212,10 +238,12 @@ def normalize_manpage_filename(prefix, path):
if fnmatch.fnmatch(str(path.parent), mandir) and path.name != "dir":
# "abc.1.gz2" -> "abc.1*"
if path.suffix[1:] in MANPAGE_EXTENSIONS:
return BuildrootPath(path.parent / (path.stem + "*"))
path = BuildrootPath(path.parent / (path.stem + "*"))
# "abc.1 -> abc.1*"
else:
return BuildrootPath(path.parent / (path.name + "*"))
path = BuildrootPath(path.parent / (path.name + "*"))
path.glob_suffix_len = 1
return path
else:
return path
@ -338,7 +366,7 @@ def classify_paths(
"docs": [], # to be used once there is upstream way to recognize READMEs
"licenses": [], # %license entries parsed from dist-info METADATA file
},
"lang": {}, # %lang entries: [module_name or None][language_code] lists of .mo files
"lang": {}, # %lang entries: [module_name or None][language_code] lists of .mo and .qm files
"modules": defaultdict(list), # each importable module (directory, .py, .so)
"module_names": set(), # qualified names of each importable module ("foo.bar.baz")
"other": {"files": []}, # regular %file entries we could not parse :(
@ -347,7 +375,7 @@ def classify_paths(
license_files = metadata.get_all('License-File')
license_directory = distinfo / 'licenses' # See PEP 639 "Root License Directory"
# setuptools was the first known build backend to implement License-File.
# Unfortunately they don't put licenses to the license directory (yet):
# Unfortunately they didn't put licenses to the license directory in setuptools<78:
# https://github.com/pypa/setuptools/issues/3596
# Hence, we check licenses in both licenses and dist-info
license_directories = (license_directory, distinfo)
@ -386,6 +414,9 @@ def classify_paths(
# extension modules can have 2 suffixes
name = BuildrootPath(path.stem).stem
add_file_to_module(paths, name, "extension", "files", path)
elif path.suffix == ".pyi":
name = path.stem
add_file_to_module(paths, name, "stub", "files", path)
elif path.suffix == ".py":
name = path.stem
# we add the .pyc files, but not top-level __pycache__
@ -402,7 +433,7 @@ def classify_paths(
for parent in list(path.parents)[:index]: # no direct slice until Python 3.10
add_file_to_module(paths, module_dir.name, "package", "dirs", parent)
is_lang = False
if path.suffix == ".mo":
if path.suffix == ".mo" or path.suffix == ".qm":
is_lang = add_lang_to_module(paths, module_dir.name, path)
if not is_lang:
if path.suffix == ".py":
@ -415,7 +446,7 @@ def classify_paths(
add_file_to_module(paths, module_dir.name, "package", "files", path)
break
else:
if path.suffix == ".mo":
if path.suffix == ".mo" or path.suffix == ".qm":
add_lang_to_module(paths, None, path) or paths["other"]["files"].append(path)
else:
path = normalize_manpage_filename(prefix, path)
@ -424,60 +455,139 @@ def classify_paths(
return paths
def escape_rpm_path(path):
def escape_rpm_path_4_19(path):
r"""
Escape special characters in string-paths or BuildrootPaths, RPM >= 4.19
E.g. a space in path otherwise makes RPM think it's multiple paths,
unless we escape it.
Or a literal % symbol in path might be expanded as a macro if not escaped by %%.
See https://github.com/rpm-software-management/rpm/pull/2103
and https://github.com/rpm-software-management/rpm/pull/2206
If the path ends with a glob produced by our other functions,
we cannot escape that part.
The BuildrootPath.glob_suffix_len attribute is used to indicate such globs.
When such suffix exists, it is not escaped.
Examples:
>>> escape_rpm_path_4_19(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
'/usr/lib/python3.9/site-packages/setuptools'
>>> escape_rpm_path_4_19('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
'/usr/lib/python3.9/site-packages/setuptools/script\\ (dev).tmpl'
>>> escape_rpm_path_4_19('/usr/share/data/100%valid.path')
'/usr/share/data/100%%valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/100 % valid.path')
'/usr/share/data/100\\ %%\\ valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/1000 %% valid.path')
'/usr/share/data/1000\\ %%%%\\ valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/spaces and "quotes" and ?')
'/usr/share/data/spaces\\ and\\ \\"quotes\\"\\ and\\ \\?'
>>> escape_rpm_path_4_19('/usr/share/data/spaces and [square brackets]')
'/usr/share/data/spaces\\ and\\ \\[square\\ brackets\\]'
>>> path = BuildrootPath('/whatever/__pycache__/bar.cpython-38{,.opt-?}.pyc')
>>> path.glob_suffix_len = len('{,.opt-?}.pyc')
>>> escape_rpm_path_4_19(path)
'/whatever/__pycache__/bar.cpython-38{,.opt-?}.pyc'
>>> path = BuildrootPath('/spa ces/__pycache__/bar.cpython-38{,.opt-?}.pyc')
>>> path.glob_suffix_len = len('{,.opt-?}.pyc')
>>> escape_rpm_path_4_19(path)
'/spa\\ ces/__pycache__/bar.cpython-38{,.opt-?}.pyc'
>>> path = BuildrootPath('/usr/man/man5/ipykernel.5*')
>>> path.glob_suffix_len = 1
>>> escape_rpm_path_4_19(path)
'/usr/man/man5/ipykernel.5*'
"""
Escape special characters in string-paths or BuildrootPaths
glob_suffix_len = getattr(path, "glob_suffix_len", 0)
suffix = ""
path = str(path)
if glob_suffix_len:
suffix = path[-glob_suffix_len:]
path = path[:-glob_suffix_len]
if "%" in path:
path = path.replace("%", "%%")
# Prepend all matched/special characters (\1) with a backslash (escaped, hence \\):
return RPM_ESCAPE_REGEX.sub(r'\\\1', path) + suffix
def escape_rpm_path_4_18(path):
"""
Escape special characters in string-paths or BuildrootPaths, RPM < 4.19
E.g. a space in path otherwise makes RPM think it's multiple paths,
unless we put it in "quotes".
Or a literal % symbol in path might be expanded as a macro if not escaped.
Due to limitations in RPM,
Due to limitations in RPM < 4.19,
some paths with spaces and other special characters are not supported.
See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
Examples:
>>> escape_rpm_path(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
>>> escape_rpm_path_4_18(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
'/usr/lib/python3.9/site-packages/setuptools'
>>> escape_rpm_path('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
>>> escape_rpm_path_4_18('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
'"/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl"'
>>> escape_rpm_path('/usr/share/data/100%valid.path')
'/usr/share/data/100%%valid.path'
>>> escape_rpm_path_4_18('/usr/share/data/100%valid.path')
'/usr/share/data/100%%%%%%%%valid.path'
>>> escape_rpm_path('/usr/share/data/100 % valid.path')
'"/usr/share/data/100 %% valid.path"'
>>> escape_rpm_path_4_18('/usr/share/data/100 % valid.path')
'"/usr/share/data/100 %%%%%%%% valid.path"'
>>> escape_rpm_path('/usr/share/data/1000 %% valid.path')
'"/usr/share/data/1000 %%%% valid.path"'
>>> escape_rpm_path_4_18('/usr/share/data/1000 %% valid.path')
'"/usr/share/data/1000 %%%%%%%%%%%%%%%% valid.path"'
>>> escape_rpm_path('/usr/share/data/spaces and "quotes"')
>>> escape_rpm_path_4_18('/usr/share/data/spaces and "quotes"')
Traceback (most recent call last):
...
NotImplementedError: ...
>>> escape_rpm_path('/usr/share/data/spaces and [square brackets]')
>>> escape_rpm_path_4_18('/usr/share/data/spaces and [square brackets]')
Traceback (most recent call last):
...
NotImplementedError: ...
"""
orig_path = path = str(path)
if "%" in path:
path = path.replace("%", "%" * RPM_PERCENTAGES_COUNT)
# Escaping an actual percentage sign in path by 8 signs
# has been verified in RPM 4.16 and 4.17:
path = path.replace("%", "%" * 8)
if any(symbol in path for symbol in RPM_FILES_DELIMETERS):
if '"' in path:
# As far as we know, RPM cannot list such file individually
# As far as we know, RPM < 4.19 cannot list such file individually
# See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
raise NotImplementedError(f'" symbol in path with spaces is not supported by %pyproject_save_files: {orig_path!r}')
raise NotImplementedError(f'" symbol in path with spaces is not supported by %pyproject_save_files on RPM < 4.19: {orig_path!r}')
if "[" in path or "]" in path:
# See https://bugzilla.redhat.com/show_bug.cgi?id=1990879
# and https://github.com/rpm-software-management/rpm/issues/1749
raise NotImplementedError(f'[ or ] symbol in path with spaces is not supported by %pyproject_save_files: {orig_path!r}')
raise NotImplementedError(f'[ or ] symbol in path with spaces is not supported by %pyproject_save_files on RPM < 4.19: {orig_path!r}')
return f'"{path}"'
return path
if RPM_FILES_ESCAPE == "4.19":
escape_rpm_path = escape_rpm_path_4_19
elif RPM_FILES_ESCAPE == "4.18":
escape_rpm_path = escape_rpm_path_4_18
else:
raise RuntimeError("RPM_FILES_ESCAPE must be 4.18 or 4.19")
def generate_file_list(paths_dict, module_globs, include_others=False):
"""
This function takes the classified paths_dict and turns it into lines
@ -525,8 +635,10 @@ def generate_file_list(paths_dict, module_globs, include_others=False):
# Users using '*' don't care about the files in the package, so it's ok
# not to fail the build when no modules are detected
# There can be legitimate reasons to create a package without Python modules
if not modules and fnmatch.fnmatchcase("", glob):
done_globs.add(glob)
if not modules:
for glob in module_globs:
if fnmatch.fnmatchcase("", glob):
done_globs.add(glob)
missed = module_globs - done_globs
if missed:
@ -667,15 +779,31 @@ def parse_varargs(varargs):
return globs, include_auto
def load_parsed_record(pyproject_record):
def load_parsed_record(pyproject_record, dist_name=None):
parsed_record = {}
with open(pyproject_record) as pyproject_record_file:
content = json.load(pyproject_record_file)
if len(content) > 1:
raise FileExistsError("%pyproject_install has found more than one *.dist-info/RECORD file. "
"Currently, %pyproject_save_files supports only one wheel → one file list mapping. "
"Feel free to open a bugzilla for pyproject-rpm-macros and describe your usecase.")
# Map each record path to its canonical dist name
dist_names = {rp: canonical_name_from_distinfo(BuildrootPath(rp).parent.name)
for rp in content}
available = sorted(dist_names.values()) # used in error messages
if dist_name:
normalized_target = canonicalize_name(dist_name)
content = {rp: files for rp, files in content.items()
if dist_names[rp] == normalized_target}
if not content:
raise ValueError(
f"No dist-info found matching dist name '{dist_name}'. "
f"Available: {', '.join(available)}"
)
elif len(content) > 1:
raise ValueError(
"%pyproject_install has found more than one *.dist-info/RECORD file. "
"Use %pyproject_save_files -D <name> to select a package. "
f"Available: {', '.join(available)}"
)
# Redefine strings stored in JSON to BuildRootPaths
for record_path, files in content.items():
@ -693,12 +821,12 @@ def dist_metadata(buildroot, record_path):
return dist.metadata
def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, assert_license, varargs):
def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, assert_license, allow_no_modules, auto, varargs, dist_name=None):
"""
Takes arguments from the %{pyproject_save_files} macro
Returns tuple: list of paths for the %files section and list of module names
for the %check section
Returns tuple: list of paths for the %files section, list of module names
for the %check section, and list of record paths (BuildrootPaths)
Raises ValueError when assert_license is true and no License-File (PEP 639)
is found.
@ -708,7 +836,17 @@ def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_versio
sitedirs = sorted({sitelib, sitearch})
globs, include_auto = parse_varargs(varargs)
parsed_records = load_parsed_record(pyproject_record)
include_auto = include_auto or auto
if not globs and not allow_no_modules:
raise ValueError(
"At least one module glob needs to be provided to %pyproject_save_files. "
"Alternatively, use -M to indicate no Python modules should be saved."
)
if globs and allow_no_modules:
raise ValueError(
"%pyproject_save_files -M cannot be used together with module globs."
)
parsed_records = load_parsed_record(pyproject_record, dist_name)
final_file_list = []
final_module_list = []
@ -739,11 +877,11 @@ def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_versio
"and include the %license file in %files manually."
)
return final_file_list, final_module_list
return final_file_list, final_module_list, list(parsed_records)
def main(cli_args):
file_section, module_names = pyproject_save_files_and_modules(
file_section, module_names, record_paths = pyproject_save_files_and_modules(
cli_args.buildroot,
cli_args.sitelib,
cli_args.sitearch,
@ -751,11 +889,27 @@ def main(cli_args):
cli_args.pyproject_record,
cli_args.prefix,
cli_args.assert_license,
cli_args.allow_no_modules,
cli_args.auto,
cli_args.varargs,
cli_args.dist_name,
)
cli_args.output_files.write_text("\n".join(file_section) + "\n", encoding="utf-8")
cli_args.output_modules.write_text("\n".join(module_names) + "\n", encoding="utf-8")
files_text = "\n".join(file_section) + "\n"
modules_text = "\n".join(module_names) + "\n"
cli_args.output_files.write_text(files_text, encoding="utf-8")
cli_args.output_modules.write_text(modules_text, encoding="utf-8")
# When no -D was given, also write to the named path (derived from dist-info)
# so that %{pyproject_files -D name} works even without -D in %pyproject_save_files
if not cli_args.dist_name:
record_path = record_paths[0]
normalized = canonical_name_from_distinfo(record_path.parent.name)
named_files = PosixPath(f"{cli_args.output_files}-{normalized}")
named_modules = PosixPath(f"{cli_args.output_modules}-{normalized}")
named_files.write_text(files_text, encoding="utf-8")
named_modules.write_text(modules_text, encoding="utf-8")
def argparser():
@ -764,7 +918,7 @@ def argparser():
prog="%pyproject_save_files",
add_help=False,
# custom usage to add +auto
usage="%(prog)s [-l|-L] MODULE_GLOB [MODULE_GLOB ...] [+auto]",
usage="%(prog)s [-l|-L] [-a|+auto] MODULE_GLOB|-M [MODULE_GLOB ...]",
)
parser.add_argument(
'--help', action='help',
@ -789,7 +943,19 @@ def argparser():
help="Don't fail when no License-File (PEP 639) is found (the default).",
)
parser.add_argument(
"varargs", nargs="+", metavar="MODULE_GLOB",
"-M", "--allow-no-modules", action="store_true", default=False,
help="Don't fail when no globs are provided, only include non-modules data in the generated filelist.",
)
parser.add_argument(
"-D", "--dist-name", type=str, default=None,
help="Save files for a specific distribution package (for multi-wheel installs).",
)
parser.add_argument(
"-a", "--auto", action="store_true", default=False,
help="Include all non-module, non-sitelib files (same as +auto).",
)
parser.add_argument(
"varargs", nargs="*", metavar="MODULE_GLOB",
help="Shell-like glob matching top-level module names to save into %%{pyproject_files}",
)
return parser

View File

@ -213,6 +213,10 @@ classified:
- /usr/lib/python3.7/site-packages/__pycache__/tldr.cpython-37{,.opt-?}.pyc
dirs: []
type: script
- files:
- /usr/lib/python3.7/site-packages/tldr.pyi
dirs: []
type: stub
other:
files:
- /usr/bin/tldr
@ -457,7 +461,7 @@ classified:
- /usr/lib/python3.7/site-packages/comic2pdf-3.1.0.dist-info/top_level.txt
- /usr/lib/python3.7/site-packages/comic2pdf-3.1.0.dist-info/zip-safe
licenses: []
modules: []
modules: {}
other:
files:
- /usr/bin/comic2pdf.py
@ -7572,6 +7576,7 @@ dumped:
- /usr/lib/python3.7/site-packages/tldr-0.5.dist-info/WHEEL
- /usr/lib/python3.7/site-packages/tldr-0.5.dist-info/top_level.txt
- /usr/lib/python3.7/site-packages/tldr.py
- /usr/lib/python3.7/site-packages/tldr.pyi
- /usr/share/man/man1/tldr*
- - tldr
- - mistune
@ -15552,6 +15557,46 @@ metadata:
License-File: LICENSE
License-File: LICENSE.python
Whatever: False data
kerberos:
path: /usr/lib64/python3.7/site-packages/kerberos-1.3.0.dist-info/METADATA
content: |
Name: kerberos
Version: 1.3.0
tensorflow:
path: /usr/lib64/python3.7/site-packages/tensorflow-2.1.0.dist-info/METADATA
content: |
Name: tensorflow
Version: 2.1.0
tldr:
path: /usr/lib/python3.7/site-packages/tldr-0.5.dist-info/METADATA
content: |
Name: tldr
Version: 0.5
mistune:
path: /usr/lib64/python3.7/site-packages/mistune-0.8.3.dist-info/METADATA
content: |
Name: mistune
Version: 0.8.3
ipykernel:
path: /usr/lib/python3.7/site-packages/ipykernel-5.2.1.dist-info/METADATA
content: |
Name: ipykernel
Version: 5.2.1
zope:
path: /usr/lib/python3.7/site-packages/zope.event-4.4.dist-info/METADATA
content: |
Name: zope.event
Version: 4.4
comic2pdf:
path: /usr/lib/python3.7/site-packages/comic2pdf-3.1.0.dist-info/METADATA
content: |
Name: comic2pdf
Version: 3.1.0
printrun:
path: /usr/lib/python3.7/site-packages/Printrun-2.0.0rc6.dist-info/METADATA
content: |
Name: Printrun
Version: 2.0.0rc6
records:
kerberos:
@ -15665,6 +15710,7 @@ records:
tldr-0.5.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92
tldr-0.5.dist-info/top_level.txt,sha256=xHSI9WD6Y-_hONbi2b_9RIn9oiO7RBGHU3A8geJq3mI,5
tldr.py,sha256=aJlA3tIz4QYYy8e7DZUhPyLCqTwnfFjA7Nubwm9bPe0,12779
tldr.pyi,sha256=GxQ4ZGLPQObN92QW_Hb8IJPEuYINNn186FjrRovM09g,13
mistune:
path: /usr/lib64/python3.7/site-packages/mistune-0.8.3.dist-info/RECORD

View File

@ -1,4 +1,5 @@
import argparse
import os
import sys
import subprocess
@ -56,21 +57,29 @@ def build_wheel(*, wheeldir, stdout=None, config_settings=None):
*get_config_settings_args(config_settings),
'.',
)
cp = subprocess.run(command, stdout=stdout)
env = {'PIP_CONFIG_FILE': '/dev/null', **os.environ}
cp = subprocess.run(command, env=env, stdout=stdout)
return cp.returncode
def parse_args(argv=None):
def argparser():
parser = argparse.ArgumentParser(prog='%pyproject_wheel')
parser.add_argument('wheeldir', help=argparse.SUPPRESS)
parser.add_argument(
'-C',
'-C', '--config-settings',
dest='config_settings',
action='append',
help='Configuration settings to pass to the PEP 517 backend',
)
parser.add_argument('-d', '--directory', help=argparse.SUPPRESS) # processed by RPM macro
return parser
def parse_args(argv=None):
parser = argparser()
args = parser.parse_args(argv)
args.config_settings = parse_config_settings_args(args.config_settings)
del args.directory
return args

View File

@ -9,3 +9,4 @@ tldr-0.5.dist-info/RECORD,,
tldr-0.5.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92
tldr-0.5.dist-info/top_level.txt,sha256=xHSI9WD6Y-_hONbi2b_9RIn9oiO7RBGHU3A8geJq3mI,5
tldr.py,sha256=aJlA3tIz4QYYy8e7DZUhPyLCqTwnfFjA7Nubwm9bPe0,12779
tldr.pyi,sha256=GxQ4ZGLPQObN92QW_Hb8IJPEuYINNn186FjrRovM09g,13

View File

@ -0,0 +1,591 @@
"""Unit tests for dependency override functionality."""
import subprocess
import sys
import pytest
from packaging.requirements import Requirement
from packaging.specifiers import Specifier, SpecifierSet
from pyproject_buildrequires import Requirements
from pyproject_dependency_overrides import (
parse_override_string, apply_overrides_to_specifiers,
)
import pyproject_patch_metadata
# ---- parse_override_string (shared module) ----
class TestParseOverrideString:
def test_basic(self):
assert parse_override_string('numpy:drop_upper') == ('numpy', 'drop_upper', None)
def test_with_value(self):
assert parse_override_string('numpy:set_upper:2.0') == ('numpy', 'set_upper', '2.0')
def test_name_normalization(self):
pkg, action, value = parse_override_string('My_Package:drop_upper')
assert pkg == 'my-package'
def test_invalid_format(self):
with pytest.raises(ValueError, match='Invalid dependency override format'):
parse_override_string('justpackage')
def test_invalid_action(self):
with pytest.raises(ValueError, match='Invalid dependency override action'):
parse_override_string('pkg:nonexistent')
def test_set_missing_value(self):
with pytest.raises(ValueError, match='requires a value'):
parse_override_string('pkg:set_upper')
def test_valueless_rejects_value(self):
with pytest.raises(ValueError, match='does not accept a value'):
parse_override_string('pkg:drop_upper:2.0')
def test_invalid_version(self):
with pytest.raises(ValueError, match='Invalid version'):
parse_override_string('pkg:set_upper:not_a_version!!!')
# ---- CLI validation (pyproject_dependency_overrides.py __main__) ----
SCRIPT = 'pyproject_dependency_overrides.py'
class TestCLIValidation:
def _run(self, arg):
return subprocess.run(
[sys.executable, '-Bs', SCRIPT, arg],
capture_output=True, text=True,
)
def test_valid_drop_upper(self):
assert self._run('numpy:drop_upper').returncode == 0
def test_valid_set_upper(self):
assert self._run('numpy:set_upper:2.0').returncode == 0
def test_valid_ignore_br_only(self):
assert self._run('pkg:ignore:br_only').returncode == 0
def test_valid_set_lower_br_only(self):
assert self._run('pkg:set_lower:1.0:br_only').returncode == 0
def test_invalid_no_colon(self):
r = self._run('justpackage')
assert r.returncode == 1
assert 'Invalid dependency override format' in r.stderr
def test_invalid_action(self):
r = self._run('pkg:bogus')
assert r.returncode == 1
assert 'Invalid dependency override action' in r.stderr
def test_set_upper_missing_value(self):
r = self._run('pkg:set_upper')
assert r.returncode == 1
assert 'requires a value' in r.stderr
def test_drop_upper_rejects_value(self):
r = self._run('pkg:drop_upper:2.0')
assert r.returncode == 1
assert 'does not accept a value' in r.stderr
def test_invalid_version(self):
r = self._run('pkg:set_upper:not_a_version!!!')
assert r.returncode == 1
assert 'Invalid version' in r.stderr
# ---- apply_overrides_to_specifiers (shared module) ----
class TestApplyOverridesToSpecifiers:
def _specs(self, spec_str):
return list(SpecifierSet(spec_str))
def test_drop_upper(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
assert len(result) == 1
assert result[0].operator == '>='
def test_drop_lower(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_lower', 'value': None}])
assert len(result) == 1
assert result[0].operator == '<'
def test_drop_constraints(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_constraints', 'value': None}])
assert result == []
def test_set_upper(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'set_upper', 'value': '3.0'}])
assert len(result) == 2
upper = [s for s in result if s.operator == '<'][0]
assert upper.version == '3.0'
def test_set_lower(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'set_lower', 'value': '0.5'}])
assert len(result) == 2
lower = [s for s in result if s.operator == '>='][0]
assert lower.version == '0.5'
def test_ignore_skipped(self):
specs = self._specs('>=1.0,<2.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'ignore', 'value': None}])
assert len(result) == 2
def test_drop_upper_decomposes_pin(self):
"""drop_upper on == decomposes to >= (keeps lower half)."""
specs = self._specs('==25.3.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
assert len(result) == 1
assert result[0].operator == '>='
assert result[0].version == '25.3.0'
def test_exclusions_preserved(self):
"""!= exclusions are always preserved."""
specs = self._specs('>=1.0,!=1.5,<3.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
ops = [s.operator for s in result]
assert '>=' in ops
assert '!=' in ops
assert '<' not in ops
def test_drop_upper_decomposes_tilde(self):
"""drop_upper on ~= decomposes to >= (PEP 440: ~=V.N is >=V.N, ==V.*)."""
specs = self._specs('~=1.4')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
assert len(result) == 1
assert result[0].operator == '>='
assert result[0].version == '1.4'
def test_drop_lower_decomposes_pin(self):
"""drop_lower on == decomposes to <= (keeps upper half)."""
specs = self._specs('==25.3.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_lower', 'value': None}])
assert len(result) == 1
assert result[0].operator == '<='
assert result[0].version == '25.3.0'
def test_drop_lower_removes_tilde(self):
"""drop_lower on ~= removes entirely (effective upper bound is complex)."""
specs = self._specs('~=1.4')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_lower', 'value': None}])
assert result == []
def test_drop_upper_removes_wildcard_pin(self):
"""==1.* (prefix match) can't be decomposed, removed entirely."""
specs = self._specs('==1.*')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
assert result == []
def test_drop_lower_removes_wildcard_pin(self):
"""==1.* (prefix match) can't be decomposed, removed entirely."""
specs = self._specs('==1.*')
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_lower', 'value': None}])
assert result == []
def test_drop_upper_removes_arbitrary_equality(self):
"""=== has no meaningful decomposition, removed atomically."""
specs = [Specifier('===1.0')]
result = apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}])
assert result == []
def test_set_upper_decomposes_pin(self):
"""set_upper on == decomposes to >= (keeps lower half) then adds <VALUE."""
specs = self._specs('==25.3.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'set_upper', 'value': '30.0'}])
assert len(result) == 2
ops = {s.operator: s.version for s in result}
assert ops['>='] == '25.3.0'
assert ops['<'] == '30.0'
def test_set_lower_decomposes_pin(self):
"""set_lower on == decomposes to <= (keeps upper half) then adds >=VALUE."""
specs = self._specs('==25.3.0')
result = apply_overrides_to_specifiers(
specs, [{'action': 'set_lower', 'value': '20.0'}])
assert len(result) == 2
ops = {s.operator: s.version for s in result}
assert ops['<='] == '25.3.0'
assert ops['>='] == '20.0'
def test_log_fn_called(self):
messages = []
specs = self._specs('>=1.0,<2.0')
apply_overrides_to_specifiers(
specs, [{'action': 'drop_upper', 'value': None}],
package_name='numpy', log_fn=messages.append)
assert len(messages) == 1
assert 'numpy' in messages[0]
# ---- Requirements._parse_dependency_overrides ----
class TestParseDependencyOverrides:
def _parse(self, overrides):
def mock_version(name):
raise Exception('not installed')
r = Requirements(mock_version, dependency_overrides=overrides)
return r.dependency_overrides
def test_empty(self):
assert self._parse([]) == {}
def test_basic_drop_upper(self):
result = self._parse(['cython:drop_upper'])
assert 'cython' in result
assert result['cython'] == [{'action': 'drop_upper', 'value': None}]
def test_set_upper_with_value(self):
result = self._parse(['numpy:set_upper:2.0'])
assert result['numpy'] == [{'action': 'set_upper', 'value': '2.0'}]
def test_set_lower_with_value(self):
result = self._parse(['requests:set_lower:2.28'])
assert result['requests'] == [{'action': 'set_lower', 'value': '2.28'}]
def test_drop_lower(self):
result = self._parse(['attrs:drop_lower'])
assert result['attrs'] == [{'action': 'drop_lower', 'value': None}]
def test_drop_constraints(self):
result = self._parse(['flask:drop_constraints'])
assert result['flask'] == [{'action': 'drop_constraints', 'value': None}]
def test_ignore(self):
result = self._parse(['sibling-pkg:ignore'])
assert result['sibling-pkg'] == [{'action': 'ignore', 'value': None}]
def test_br_only_suffix_stripped(self):
result = self._parse(['sibling-pkg:ignore:br_only'])
assert result['sibling-pkg'] == [{'action': 'ignore', 'value': None}]
def test_br_only_with_value(self):
result = self._parse(['numpy:set_upper:2.0:br_only'])
assert result['numpy'] == [{'action': 'set_upper', 'value': '2.0'}]
def test_name_normalization(self):
result = self._parse(['My_Package:drop_upper'])
assert 'my-package' in result
def test_multiple_overrides_same_package(self):
result = self._parse(['pkg:drop_upper', 'pkg:drop_lower'])
assert len(result['pkg']) == 2
def test_multiple_packages(self):
result = self._parse(['pkg-a:drop_upper', 'pkg-b:ignore'])
assert 'pkg-a' in result
assert 'pkg-b' in result
def test_invalid_format(self):
with pytest.raises(ValueError, match='Invalid dependency override format'):
self._parse(['justpackage'])
def test_invalid_action(self):
with pytest.raises(ValueError, match='Invalid dependency override action'):
self._parse(['pkg:nonexistent'])
def test_set_upper_missing_value(self):
with pytest.raises(ValueError, match='requires a value'):
self._parse(['pkg:set_upper'])
def test_set_lower_missing_value(self):
with pytest.raises(ValueError, match='requires a value'):
self._parse(['pkg:set_lower'])
def test_invalid_version(self):
with pytest.raises(ValueError, match='Invalid version'):
self._parse(['pkg:set_upper:not_a_version!!!'])
def test_drop_upper_rejects_value(self):
with pytest.raises(ValueError, match='does not accept a value'):
self._parse(['pkg:drop_upper:2.0'])
def test_ignore_rejects_value(self):
with pytest.raises(ValueError, match='does not accept a value'):
self._parse(['pkg:ignore:somevalue'])
# ---- Requirements._should_ignore_dependency ----
class TestShouldIgnoreDependency:
def _make_req(self, overrides):
def mock_version(name):
raise Exception('not installed')
return Requirements(mock_version, dependency_overrides=overrides)
def test_not_ignored(self):
r = self._make_req(['numpy:drop_upper'])
assert not r._should_ignore_dependency('numpy')
def test_ignored(self):
r = self._make_req(['sibling:ignore'])
assert r._should_ignore_dependency('sibling')
def test_not_in_overrides(self):
r = self._make_req(['other:ignore'])
assert not r._should_ignore_dependency('unrelated')
# ---- Requirements._apply_dependency_overrides ----
class TestApplyDependencyOverrides:
def _make_req(self, overrides):
def mock_version(name):
raise Exception('not installed')
return Requirements(mock_version, dependency_overrides=overrides)
def test_no_overrides(self):
r = self._make_req([])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
assert len(list(result.specifier)) == 2
def test_drop_upper(self):
r = self._make_req(['numpy:drop_upper'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '>='
def test_drop_lower(self):
r = self._make_req(['numpy:drop_lower'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '<'
def test_drop_constraints(self):
r = self._make_req(['numpy:drop_constraints'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
assert list(result.specifier) == []
def test_set_upper(self):
r = self._make_req(['numpy:set_upper:3.0'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
specs = list(result.specifier)
assert len(specs) == 2
ops = {s.operator for s in specs}
assert '>=' in ops
assert '<' in ops
upper = [s for s in specs if s.operator == '<'][0]
assert upper.version == '3.0'
def test_set_lower(self):
r = self._make_req(['numpy:set_lower:0.5'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
specs = list(result.specifier)
assert len(specs) == 2
ops = {s.operator for s in specs}
assert '>=' in ops
assert '<' in ops
lower = [s for s in specs if s.operator == '>='][0]
assert lower.version == '0.5'
def test_drop_upper_decomposes_pin(self):
"""drop_upper on == decomposes to >= (keeps lower half)."""
r = self._make_req(['attrs:drop_upper'])
result = r._apply_dependency_overrides(Requirement('attrs==25.3.0'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '>='
assert specs[0].version == '25.3.0'
def test_drop_lower_decomposes_pin(self):
"""drop_lower on == decomposes to <= (keeps upper half)."""
r = self._make_req(['attrs:drop_lower'])
result = r._apply_dependency_overrides(Requirement('attrs==25.3.0'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '<='
assert specs[0].version == '25.3.0'
def test_drop_upper_preserves_exclusions(self):
"""!= exclusions are always preserved."""
r = self._make_req(['pkg:drop_upper'])
result = r._apply_dependency_overrides(Requirement('pkg>=1.0,!=1.5,<3.0'))
ops = [s.operator for s in result.specifier]
assert '>=' in ops
assert '!=' in ops
assert '<' not in ops
def test_ignore_action_passthrough(self):
"""ignore action is handled separately; _apply_dependency_overrides skips it."""
r = self._make_req(['pkg:ignore'])
result = r._apply_dependency_overrides(Requirement('pkg>=1.0,<2.0'))
assert len(list(result.specifier)) == 2
def test_unrelated_package_not_affected(self):
r = self._make_req(['other:drop_upper'])
result = r._apply_dependency_overrides(Requirement('numpy>=1.0,<2.0'))
assert len(list(result.specifier)) == 2
def test_extras_stripped_for_lookup(self):
r = self._make_req(['numpy:drop_upper'])
result = r._apply_dependency_overrides(Requirement('numpy[extra1]>=1.0,<2.0'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '>='
def test_drop_upper_decomposes_tilde(self):
"""drop_upper on ~= decomposes to >= (PEP 440: ~=V.N is >=V.N, ==V.*)."""
r = self._make_req(['pkg:drop_upper'])
result = r._apply_dependency_overrides(Requirement('pkg~=1.4'))
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '>='
assert specs[0].version == '1.4'
# ---- pyproject_patch_metadata ----
class TestPatchMetadataParseOverrides:
def test_br_only_skipped(self):
result = pyproject_patch_metadata.parse_overrides(['pkg:ignore:br_only'])
assert result == {}
def test_non_br_only_included(self):
result = pyproject_patch_metadata.parse_overrides(['pkg:ignore'])
assert 'pkg' in result
def test_mixed(self):
result = pyproject_patch_metadata.parse_overrides([
'pkg-a:ignore:br_only',
'pkg-b:drop_upper',
])
assert 'pkg-a' not in result
assert 'pkg-b' in result
def test_rejects_value_for_valueless_action(self):
with pytest.raises(ValueError, match='does not accept a value'):
pyproject_patch_metadata.parse_overrides(['pkg:drop_upper:2.0'])
def test_rejects_invalid_version(self):
with pytest.raises(ValueError, match='Invalid version'):
pyproject_patch_metadata.parse_overrides(['pkg:set_upper:!!!'])
class TestApplyOverridesToRequirement:
def test_ignore_returns_none(self):
req = Requirement('numpy>=1.0')
result = pyproject_patch_metadata.apply_overrides_to_requirement(
req, [{'action': 'ignore', 'value': None}])
assert result is None
def test_drop_upper(self):
req = Requirement('numpy>=1.0,<2.0')
result = pyproject_patch_metadata.apply_overrides_to_requirement(
req, [{'action': 'drop_upper', 'value': None}])
assert result is not None
specs = list(result.specifier)
assert len(specs) == 1
assert specs[0].operator == '>='
def test_set_upper(self):
req = Requirement('numpy>=1.0,<2.0')
result = pyproject_patch_metadata.apply_overrides_to_requirement(
req, [{'action': 'set_upper', 'value': '3.0'}])
specs = list(result.specifier)
ops = {s.operator for s in specs}
assert '<' in ops
upper = [s for s in specs if s.operator == '<'][0]
assert upper.version == '3.0'
def test_drop_constraints(self):
req = Requirement('numpy>=1.0,<2.0')
result = pyproject_patch_metadata.apply_overrides_to_requirement(
req, [{'action': 'drop_constraints', 'value': None}])
assert result is not None
assert list(result.specifier) == []
def test_preserves_extras_and_markers(self):
req = Requirement('numpy[extra1]>=1.0,<2.0; python_version>="3"')
result = pyproject_patch_metadata.apply_overrides_to_requirement(
req, [{'action': 'drop_upper', 'value': None}])
assert result is not None
assert result.extras == {'extra1'}
assert result.marker is not None
class TestPatchMetadataFile:
def _write_metadata(self, tmp_path, content):
p = tmp_path / 'METADATA'
p.write_text(content)
return str(p)
def test_drop_upper(self, tmp_path):
metadata = self._write_metadata(tmp_path,
'Metadata-Version: 2.1\n'
'Name: mypackage\n'
'Requires-Dist: numpy (>=1.0,<2.0)\n'
'Requires-Dist: requests\n'
)
overrides = pyproject_patch_metadata.parse_overrides(['numpy:drop_upper'])
pyproject_patch_metadata.patch_metadata(metadata, overrides)
with open(metadata) as f:
content = f.read()
assert 'numpy' in content
assert '<2.0' not in content
assert 'requests' in content
def test_ignore(self, tmp_path):
metadata = self._write_metadata(tmp_path,
'Metadata-Version: 2.1\n'
'Name: mypackage\n'
'Requires-Dist: sibling-pkg\n'
'Requires-Dist: requests\n'
)
overrides = pyproject_patch_metadata.parse_overrides(['sibling-pkg:ignore'])
pyproject_patch_metadata.patch_metadata(metadata, overrides)
with open(metadata) as f:
content = f.read()
assert 'sibling-pkg' not in content
assert 'requests' in content
def test_no_overrides(self, tmp_path):
original = (
'Metadata-Version: 2.1\n'
'Name: mypackage\n'
'Requires-Dist: numpy (>=1.0,<2.0)\n'
)
metadata = self._write_metadata(tmp_path, original)
pyproject_patch_metadata.patch_metadata(metadata, {})
with open(metadata) as f:
content = f.read()
assert content == original
def test_br_only_not_applied_to_metadata(self, tmp_path):
original = (
'Metadata-Version: 2.1\n'
'Name: mypackage\n'
'Requires-Dist: sibling-pkg\n'
)
metadata = self._write_metadata(tmp_path, original)
overrides = pyproject_patch_metadata.parse_overrides(['sibling-pkg:ignore:br_only'])
pyproject_patch_metadata.patch_metadata(metadata, overrides)
with open(metadata) as f:
content = f.read()
assert 'sibling-pkg' in content

View File

@ -6,16 +6,36 @@ import pytest
import setuptools
import yaml
from pyproject_buildrequires import generate_requires
from pyproject_buildrequires import generate_requires, load_pyproject
SETUPTOOLS_VERSION = packaging.version.parse(setuptools.__version__)
SETUPTOOLS_60 = SETUPTOOLS_VERSION >= packaging.version.parse('60')
try:
import tox
except ImportError:
TOX_4_22 = False
else:
TOX_VERSION = packaging.version.parse(tox.__version__)
TOX_4_22 = TOX_VERSION >= packaging.version.parse('4.22')
testcases = {}
with Path(__file__).parent.joinpath('pyproject_buildrequires_testcases.yaml').open() as f:
testcases = yaml.safe_load(f)
@pytest.fixture(autouse=True)
def clear_pyproject_data():
"""
Clear pyproject data before each test.
In reality we build one RPM package at a time, so we can keep the once-loaded
pyproject.toml contents.
When testing, the cached data would leak the once-loaded data to all the
following test cases.
"""
load_pyproject.cache_clear()
@pytest.mark.parametrize('case_name', testcases)
def test_data(case_name, capfd, tmp_path, monkeypatch):
case = testcases[case_name]
@ -41,6 +61,11 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
for name, value in case.get('environ', {}).items():
monkeypatch.setenv(name, value)
fedora = int(case.get('environ', {}).get('FEDORA', 0))
rhel = int(case.get('environ', {}).get('RHEL', 0))
monkeypatch.setattr('pyproject_buildrequires.REJECT_INVALID_EXTRAS',
fedora >= 45 or rhel >= 11)
def get_installed_version(dist_name):
try:
return str(case['installed'][dist_name])
@ -49,8 +74,9 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
f'info not found for {dist_name}'
)
requirement_files = case.get('requirement_files', [])
requirement_files = [open(f) for f in requirement_files]
requirement_files = [Path(f) for f in requirement_files]
use_build_system = case.get('use_build_system', True)
pyproject_dependencies = case.get('pyproject_dependencies', False)
try:
generate_requires(
get_installed_version=get_installed_version,
@ -58,12 +84,15 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
build_wheel=case.get('build_wheel', False),
wheeldir=str(wheeldir),
extras=case.get('extras', []),
dependency_groups=case.get('dependency_groups', []),
toxenv=case.get('toxenv', None),
generate_extras=case.get('generate_extras', False),
requirement_files=requirement_files,
use_build_system=use_build_system,
pyproject_dependencies=pyproject_dependencies,
output=output,
config_settings=case.get('config_settings'),
dependency_overrides=case.get('dependency_overrides', []),
)
except SystemExit as e:
assert e.code == case['result']
@ -79,15 +108,15 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
assert 'expected' in case or 'stderr_contains' in case
out, err = capfd.readouterr()
dependencies = output.read_text()
dependencies = sorted(output.read_text().splitlines())
if 'expected' in case:
expected = case['expected']
if isinstance(expected, list):
# at least one of them needs to match
assert dependencies in expected
assert dependencies in [sorted(e.splitlines()) for e in expected]
else:
assert dependencies == expected
assert dependencies == sorted(expected.splitlines())
# stderr_contains may be a string or list of strings
stderr_contains = case.get('stderr_contains')
@ -96,6 +125,3 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
stderr_contains = [stderr_contains]
for expected_substring in stderr_contains:
assert expected_substring.format(**locals()) in err
finally:
for req in requirement_files:
req.close()

View File

@ -0,0 +1,632 @@
local pg = require("pyproject_getopt")
local M = {}
-- Helpers
local function assert_defined(name)
assert(
rpm.expand("%{defined __pyproject_opt_" .. name .. "}") == "1",
"expected __pyproject_opt_" .. name .. " to be defined"
)
end
local function assert_undefined(name)
assert(
rpm.expand("%{undefined __pyproject_opt_" .. name .. "}") == "1",
"expected __pyproject_opt_" .. name .. " to be undefined"
)
end
local function assert_value(name, expected)
local actual = rpm.expand("%{?__pyproject_opt_" .. name .. "}")
assert(
actual == expected,
"expected __pyproject_opt_" .. name .. "=" .. expected .. ", got " .. actual
)
end
local function assert_optflag(name, expected)
local actual = rpm.expand("%{?__pyproject_optflag_" .. name .. "}")
assert(
actual == expected,
"expected __pyproject_optflag_" .. name .. "=" .. expected .. ", got " .. actual
)
end
local function assert_optflag_undefined(name)
assert(
rpm.expand("%{undefined __pyproject_optflag_" .. name .. "}") == "1",
"expected __pyproject_optflag_" .. name .. " to be undefined"
)
end
local function assert_positional(expected)
local actual = rpm.expand("%{?__pyproject_positional_args}")
assert(
actual == expected,
"expected positional=" .. expected .. ", got " .. actual
)
end
local function assert_errors(fn)
local ok = pcall(fn)
assert(not ok, "expected an error but call succeeded")
end
local function skip(reason)
print("SKIP: " .. reason)
end
-- Specs
-- The specs here were copied from the implementation of the macros
-- when this test was added. But no need to keep them in sync,
-- as long as we test what needs to be tested.
local SAVE_FILES_SPEC = {
{short="l", long="assert-license"},
{short="L", long="no-assert-license"},
{short="M", long="allow-no-modules"},
}
local WHEEL_SPEC = {
{short="C", long="config-settings", value=true, separator=","},
{short="d", long="directory", value=true},
}
local BUILDREQUIRES_SPEC = {
{short="r", long="runtime"},
{short="R", long="no-runtime"},
{short="x", long="extras", value=true, separator=","},
{short="t", long="tox"},
{short="N", long="no-use-build-system"},
{short="w", long="wheel"},
{short="p", long="pyproject-dependencies"},
{short="e", long="toxenv", value=true, separator=","},
{short="g", long="dependency-groups", value=true, separator=","},
{short="C", long="config-settings", value=true, separator=","},
{short="d", long="directory", value=true},
}
local BUILDREQUIRES_EXCLUSIONS = {
{"R", {"r", "x", "e", "t", "w", "p"}},
{"N", {"r", "x", "e", "t", "w", "p", "C"}},
{"w", {"p"}},
}
local CHECK_IMPORT_SPEC = {
{short="e", long="exclude", value=true, separator=" -e "},
{short="t", long="top-level-only"},
}
local TOX_SPEC = {
{short="e", long="toxenv", value=true, separator=","},
}
-- Short flags
function M.test_single_flag()
pg.getopt(SAVE_FILES_SPEC, nil, {"-l"}, "test")
assert_defined("l")
assert_undefined("L")
assert_undefined("M")
end
function M.test_multiple_flags()
pg.getopt(SAVE_FILES_SPEC, nil, {"-l", "-M"}, "test")
assert_defined("l")
assert_defined("M")
assert_undefined("L")
end
function M.test_repeated_flag()
pg.getopt(SAVE_FILES_SPEC, nil, {"-l", "-l"}, "test")
assert_defined("l")
end
function M.test_bundled_flags()
pg.getopt(SAVE_FILES_SPEC, nil, {"-lM"}, "test")
assert_defined("l")
assert_defined("M")
end
-- Long flags
function M.test_single_long_flag()
pg.getopt(SAVE_FILES_SPEC, nil, {"--assert-license"}, "test")
assert_defined("l")
assert_undefined("L")
end
function M.test_multiple_long_flags()
pg.getopt(SAVE_FILES_SPEC, nil, {"--assert-license", "--allow-no-modules"}, "test")
assert_defined("l")
assert_defined("M")
end
-- Value options
function M.test_short_with_value()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo"}, "test")
assert_value("d", "foo")
end
function M.test_short_value_attached()
pg.getopt(WHEEL_SPEC, nil, {"-dfoo"}, "test")
assert_value("d", "foo")
end
function M.test_long_with_value()
pg.getopt(WHEEL_SPEC, nil, {"--directory", "foo"}, "test")
assert_value("d", "foo")
end
function M.test_long_with_equals()
pg.getopt(WHEEL_SPEC, nil, {"--directory=foo"}, "test")
assert_value("d", "foo")
end
function M.test_value_with_spaces()
pg.getopt(WHEEL_SPEC, nil, {"-d", "Path With Spaces"}, "test")
assert_value("d", "Path With Spaces")
end
function M.test_long_value_with_spaces()
pg.getopt(WHEEL_SPEC, nil, {"--directory", "Path With Spaces"}, "test")
assert_value("d", "Path With Spaces")
end
function M.test_empty_value()
pg.getopt(WHEEL_SPEC, nil, {"-d", ""}, "test")
assert_defined("d")
assert_value("d", "")
end
function M.test_empty_value_stays_single_arg()
if not pg._use_quote then return skip("%{quote:} not transparent on this RPM") end
pg.getopt(WHEEL_SPEC, nil, {"-d", ""}, "test")
rpm.define([[_test_countargs(-) %#]])
local nargs = rpm.expand("%_test_countargs %{__pyproject_opt_d}")
assert(nargs == "1",
"expected empty value to be 1 arg, got " .. nargs)
end
function M.test_value_with_spaces_stays_single_arg()
if not pg._use_quote then return skip("%{quote:} not transparent on this RPM") end
pg.getopt(WHEEL_SPEC, nil, {"-d", "Path With Spaces"}, "test")
-- Define a macro that expands to its argument count
rpm.define([[_test_countargs(-) %#]])
-- Pass the stored value to another macro — it must stay as 1 argument
local nargs = rpm.expand("%_test_countargs %{__pyproject_opt_d}")
assert(nargs == "1",
"expected value with spaces to be 1 arg, got " .. nargs)
end
function M.test_value_looking_like_option()
pg.getopt(WHEEL_SPEC, nil, {"--config-settings", "--with-dashes=1"}, "test")
assert_defined("C")
end
function M.test_short_value_attached_looking_like_option()
pg.getopt(WHEEL_SPEC, nil, {"-C--with-dashes=1"}, "test")
assert_defined("C")
end
-- Repeated options
function M.test_repeated_parse_option_comma_joins()
pg.getopt(TOX_SPEC, nil, {"-e", "env1", "-e", "env2"}, "test")
assert_value("e", "env1,env2")
end
function M.test_repeated_long_parse_option()
pg.getopt(TOX_SPEC, nil, {"--toxenv", "env1", "--toxenv", "env2"}, "test")
assert_value("e", "env1,env2")
end
function M.test_repeated_short_and_long_parse_option()
pg.getopt(TOX_SPEC, nil, {"-e", "env1", "--toxenv", "env2"}, "test")
assert_value("e", "env1,env2")
end
function M.test_repeated_long_and_short_parse_option()
pg.getopt(TOX_SPEC, nil, {"--toxenv", "env1", "-e", "env2"}, "test")
assert_value("e", "env1,env2")
end
function M.test_repeated_value_option_comma_joins()
pg.getopt(WHEEL_SPEC, nil, {"-C", "a", "-C", "b"}, "test")
assert_defined("C")
assert_value("C", "a,b")
end
-- Positional args
function M.test_positional_only()
pg.getopt(SAVE_FILES_SPEC, nil, {"foo", "bar"}, "test")
assert_positional("foo bar")
end
function M.test_mixed_options_and_positional()
pg.getopt(SAVE_FILES_SPEC, nil, {"-l", "foo", "bar"}, "test")
assert_defined("l")
assert_positional("foo bar")
end
function M.test_double_dash_stops_parsing()
pg.getopt(TOX_SPEC, nil, {"-e", "env1", "--", "--not-an-opt"}, "test")
assert_value("e", "env1")
assert_positional("--not-an-opt")
end
function M.test_double_dash_repeated()
pg.getopt(TOX_SPEC, nil, {"-e", "env1", "--", "a", "--", "-b", "--", "--c"}, "test")
assert_value("e", "env1")
assert_positional("a -- -b -- --c")
end
function M.test_no_arguments()
pg.getopt(SAVE_FILES_SPEC, nil, {}, "test")
assert_positional("")
assert_undefined("l")
end
function M.test_positional_with_spaces()
pg.getopt(SAVE_FILES_SPEC, nil, {"module with spaces", "other"}, "test")
assert_positional("module with spaces other")
end
function M.test_empty_positional_args()
pg.getopt(SAVE_FILES_SPEC, nil, {"", "foo", ""}, "test")
assert_positional(" foo ")
end
function M.test_whitespace_positional_args()
pg.getopt(SAVE_FILES_SPEC, nil, {" \t ", "foo", "\\\n"}, "test")
assert_positional("foo")
end
function M.test_newlines_embedded_in_tokens()
pg.getopt(CHECK_IMPORT_SPEC, nil,
{"\n-e", "mod.a", "-e", "mod.b\n-e", "mod.c", "-e", "mod.d\n"}, "test")
assert_value("e", "mod.a -e mod.b -e mod.c -e mod.d")
end
function M.test_trailing_backslash_stripped_after_newline_split()
pg.getopt(WHEEL_SPEC, nil,
{"-C\"key1=val1\" \\\n-C\"key2=val2\" \\\n-C\"key3=val3\"\n"}, "test")
assert_value("C", "\"key1=val1\",\"key2=val2\",\"key3=val3\"")
end
-- Errors
function M.test_unknown_short_option()
assert_errors(function() pg.getopt(SAVE_FILES_SPEC, nil, {"-z"}, "test") end)
end
function M.test_unknown_long_option()
assert_errors(function() pg.getopt(SAVE_FILES_SPEC, nil, {"--bogus"}, "test") end)
end
function M.test_missing_value_short()
assert_errors(function() pg.getopt(WHEEL_SPEC, nil, {"-d"}, "test") end)
end
function M.test_missing_value_long()
assert_errors(function() pg.getopt(WHEEL_SPEC, nil, {"--directory"}, "test") end)
end
function M.test_flag_given_value()
assert_errors(function() pg.getopt(SAVE_FILES_SPEC, nil, {"--assert-license=foo"}, "test") end)
end
-- Mutual exclusions
function M.test_R_and_x()
assert_errors(function()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"-R", "-x", "testing"}, "test")
end)
end
function M.test_long_form_exclusion()
assert_errors(function()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"--no-runtime", "--extras", "testing"}, "test")
end)
end
function M.test_N_and_C()
assert_errors(function()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"-N", "-C", "foo"}, "test")
end)
end
function M.test_w_and_p()
assert_errors(function()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"--wheel", "--pyproject-dependencies"}, "test")
end)
end
function M.test_non_conflicting_passes()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"-R", "-g", "tests"}, "test")
assert_defined("R")
assert_defined("g")
end
-- Cleanup
function M.test_options_do_not_leak_between_calls()
local spec = {
{short="d", long="directory", value=true},
{short="C", long="config-settings", value=true, separator=","},
}
pg.getopt(spec, nil, {"-d", "foo", "-C", "bar"}, "test")
assert_value("d", "foo")
assert_defined("C")
pg.getopt(spec, nil, {"-C", "baz"}, "test")
assert_undefined("d")
assert_defined("C")
end
-- Separator
function M.test_comma_separator()
pg.getopt(TOX_SPEC, nil, {"-e", "env1", "-e", "env2"}, "test")
assert_value("e", "env1,env2")
end
function M.test_custom_separator()
pg.getopt(CHECK_IMPORT_SPEC, nil, {"--exclude", "a", "--exclude", "b"}, "test")
assert_value("e", "a -e b")
end
function M.test_repeated_values_with_spaces_stay_separate_args()
if not pg._use_quote then return skip("%{quote:} not transparent on this RPM") end
pg.getopt(CHECK_IMPORT_SPEC, nil, {"--exclude", "a b", "--exclude", "c"}, "test")
-- Each value is quoted individually, separator splits them:
-- stored as: %{quote:a b} -e c
-- When passed to another macro as "-e %{__pyproject_opt_e}",
-- it should produce 4 args: -e, "a b", -e, c
rpm.define([[_test_countargs(-) %#]])
local nargs = rpm.expand("%_test_countargs -e %{__pyproject_opt_e}")
assert(nargs == "4",
"expected 4 args (-e, 'a b', -e, c), got " .. nargs)
end
function M.test_single_value_no_separator()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo"}, "test")
assert_value("d", "foo")
end
function M.test_repeated_without_separator_errors()
assert_errors(function()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo", "-d", "bar"}, "test")
end)
end
function M.test_repeated_long_without_separator_errors()
assert_errors(function()
pg.getopt(WHEEL_SPEC, nil, {"--directory", "foo", "--directory", "bar"}, "test")
end)
end
-- Optflag macros
function M.test_optflag_for_flag()
pg.getopt(SAVE_FILES_SPEC, nil, {"-l"}, "test")
assert_optflag("l", "-l")
assert_optflag_undefined("M")
end
function M.test_optflag_for_value()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo"}, "test")
assert_optflag("d", "-d foo")
end
function M.test_optflag_for_repeated_value()
pg.getopt(CHECK_IMPORT_SPEC, nil, {"--exclude", "a", "--exclude", "b"}, "test")
assert_optflag("e", "-e a -e b")
end
function M.test_optflag_cleanup_between_calls()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo"}, "test")
assert_optflag("d", "-d foo")
pg.getopt(WHEEL_SPEC, nil, {"-C", "bar"}, "test")
assert_optflag_undefined("d")
assert_optflag("C", "-C bar")
end
-- Nested macro scope: save/restore prevents inner getopt from clobbering outer values
function M.test_nested_getopt_with_restore_preserves_outer_opts()
rpm.define([[_test_inner(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}inner%{lua:require("pyproject_getopt").restore()}]]
)
rpm.define([[_test_outer(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_inner}/%{?__pyproject_opt_d}]]
)
local result = rpm.expand("%_test_outer -d hello")
assert(result == "inner/hello",
"expected 'inner/hello', got '" .. result .. "'")
end
function M.test_nested_getopt_with_restore_preserves_outer_optflags()
rpm.define([[_test_inner2(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}inner%{lua:require("pyproject_getopt").restore()}]]
)
rpm.define([[_test_outer2(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_inner2}/%{?__pyproject_optflag_d}]]
)
local result = rpm.expand("%_test_outer2 -d hello")
assert(result == "inner/-d hello",
"expected 'inner/-d hello', got '" .. result .. "'")
end
function M.test_nested_getopt_without_restore_clobbers()
rpm.define([[_test_inner3(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}inner]]
)
rpm.define([[_test_outer3(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_inner3}/%{?__pyproject_opt_d}]]
)
local result = rpm.expand("%_test_outer3 -d hello")
-- Without restore(), inner getopt clobbers outer's value
assert(result == "inner/",
"expected 'inner/' (clobbered without restore), got '" .. result .. "'")
end
function M.test_nested_getopt_with_restore_preserves_outer_positional_args()
rpm.define([[_test_inner4(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}inner%{lua:require("pyproject_getopt").restore()}]]
)
rpm.define([[_test_outer4(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_inner4}/%{?__pyproject_positional_args}]]
)
local result = rpm.expand("%_test_outer4 -d hello world")
assert(result == "inner/world",
"expected 'inner/world', got '" .. result .. "'")
end
function M.test_nested_getopt_with_different_values()
rpm.define([[_test_inner5(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{?__pyproject_opt_d}%{lua:require("pyproject_getopt").restore()}]]
)
rpm.define([[_test_outer5(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_inner5 -d other}/%{?__pyproject_opt_d}]]
)
local result = rpm.expand("%_test_outer5 -d hello")
assert(result == "other/hello",
"expected 'other/hello', got '" .. result .. "'")
end
function M.test_triple_nested_getopt_restores_all_levels()
rpm.define([[_test_innermost(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{?__pyproject_opt_d}%{lua:require("pyproject_getopt").restore()}]])
rpm.define([[_test_middle(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_innermost -d deep}+%{?__pyproject_opt_d}%{lua:require("pyproject_getopt").restore()}]])
rpm.define([[_test_outermost(-) %{lua:
require("pyproject_getopt").getopt({
{short="d", long="directory", value=true},
})}%{_test_middle -d mid}+%{?__pyproject_opt_d}]])
local result = rpm.expand("%_test_outermost -d top")
assert(result == "deep+mid+top",
"expected 'deep+mid+top', got '" .. result .. "'")
end
-- Raw error functions (without pcall) for Python-side stderr checking.
-- These are not discovered by list() since they don't start with test_.
function M.raw_unknown_short_option()
pg.getopt(SAVE_FILES_SPEC, nil, {"-z"}, "test")
end
function M.raw_unknown_long_option()
pg.getopt(SAVE_FILES_SPEC, nil, {"--bogus"}, "test")
end
function M.raw_missing_value_short()
pg.getopt(WHEEL_SPEC, nil, {"-d"}, "test")
end
function M.raw_flag_given_value()
pg.getopt(SAVE_FILES_SPEC, nil, {"--assert-license=foo"}, "test")
end
function M.raw_R_and_x()
pg.getopt(BUILDREQUIRES_SPEC, BUILDREQUIRES_EXCLUSIONS, {"-R", "-x", "testing"}, "test")
end
function M.raw_repeated_without_separator()
pg.getopt(WHEEL_SPEC, nil, {"-d", "foo", "-d", "bar"}, "test")
end
-- RPM macro integration (uses rpm.define to create a real parametric macro)
local _MACRO_DEF = [[_test_macro(-) %{lua:require("pyproject_getopt").getopt({{short="d", long="directory", value=true}})}]]
function M.test_macro_parses_short_option()
rpm.define(_MACRO_DEF)
rpm.expand("%_test_macro -d hello")
assert_value("d", "hello")
end
function M.test_macro_parses_long_option()
rpm.define(_MACRO_DEF)
rpm.expand("%_test_macro --directory hello")
assert_value("d", "hello")
end
function M.test_macro_preserves_quoted_spaces()
rpm.define(_MACRO_DEF)
rpm.expand("%_test_macro -d %{quote:Path With Spaces}")
assert_value("d", "Path With Spaces")
end
function M.test_macro_rejects_unknown_option()
rpm.define(_MACRO_DEF)
assert_errors(function() rpm.expand("%_test_macro --bogus") end)
end
function M.raw_macro_rejects_unknown_option()
rpm.define(_MACRO_DEF)
rpm.expand("%_test_macro --bogus")
end
-- List all tests for pytest discovery
function M.list()
local names = {}
for name in pairs(M) do
if type(M[name]) == "function" and name:sub(1, 5) == "test_" then
names[#names + 1] = name
end
end
table.sort(names)
return names
end
return M

View File

@ -0,0 +1,168 @@
"""Assert that short/long option mappings are consistent across
macros.pyproject, Python argparse scripts, and README.md."""
import argparse
import importlib
import re
from pathlib import Path
import pytest
BASEDIR = Path(__file__).parent
# Macros with corresponding Python scripts
MACROS_PYTHON = (
"pyproject_buildrequires",
"pyproject_save_files",
"pyproject_wheel",
)
# Macros that appear in the README reference table
MACROS_README = (
"pyproject_buildrequires",
"pyproject_check_import",
"pyproject_extras_subpkg",
"pyproject_save_files",
"pyproject_wheel",
"tox",
)
def parse_macros_file():
"""Extract {macro_name: set((short, long), ...)} from macros.pyproject."""
text = (BASEDIR / "macros.pyproject").read_text()
result = {}
# Match .getopt({ specs })
# The macro name comes from the %macro_name(-) definition line above the call
prev_end = 0
for m in re.finditer(r"\.getopt\(", text):
start = m.end()
# Find the macro name from the nearest preceding %name(-) line
preceding = text[prev_end:m.start()]
macro_name = re.findall(r"^%(\w+)\(-\)", preceding, re.MULTILINE)[-1]
prev_end = m.end()
# Find the opt_spec table (first argument)
brace_start = text.index("{", start)
depth = 0
pos = brace_start
while pos < len(text):
if text[pos] == "{":
depth += 1
elif text[pos] == "}":
depth -= 1
if depth == 0:
break
pos += 1
spec_text = text[brace_start : pos + 1]
opts = set()
for opt_m in re.finditer(
r'short\s*=\s*"([^"]+)"\s*,\s*long\s*=\s*"([^"]+)"', spec_text
):
opts.add((opt_m.group(1), opt_m.group(2)))
result[macro_name] = opts
return result
def parse_python_script(module_name):
"""Extract set((short, long), ...) from a module's argparser().
Only returns options that have both a short (-X) and long (--Y) form,
which are the user-facing options.
"""
module = importlib.import_module(module_name)
parser = module.argparser()
opts = set()
for action in parser._actions: # this is quite stable private API
if isinstance(action, argparse._HelpAction):
continue
strings = action.option_strings
if len(strings) == 2 and strings[0].startswith("-") and not strings[0].startswith("--"):
short = strings[0][1:] # strip the -
long = strings[1][2:] # strip the --
opts.add((short, long))
return opts
def parse_readme():
"""Extract {macro_name: set((short, long), ...)} from README tables."""
text = (BASEDIR / "README.md").read_text()
result = {}
current_macro = None
in_table = False
for line in text.splitlines():
# Detect ### `%macro_name` headings
if heading_m := re.match(r"^### `%(\w+)`", line):
current_macro = heading_m.group(1)
in_table = False
continue
if current_macro is None:
continue
# Detect table rows (skip header and separator)
if re.match(r"^\|[-\s|]+\|$", line):
in_table = True
continue
if not line.startswith("|"):
if in_table:
# Left the table
current_macro = None
in_table = False
continue
if not in_table:
# This is the header row
if "Short" in line and "Long" in line:
in_table = False # next line will be separator
continue
# Parse table data row: | `-x EXTRAS` | `--extras EXTRAS` | ... |
cols = [c.strip() for c in line.split("|")]
if len(cols) < 4:
continue
short_col = cols[1] # e.g. "`-x EXTRAS`"
long_col = cols[2] # e.g. "`--extras EXTRAS`"
short_m = re.match(r"`-(\w)", short_col)
long_m = re.match(r"`--([\w-]+)", long_col)
if short_m and long_m:
short = short_m.group(1)
long = long_m.group(1)
if current_macro not in result:
result[current_macro] = set()
result[current_macro].add((short, long))
return result
@pytest.fixture(scope="module")
def macro_opts():
return parse_macros_file()
@pytest.fixture(scope="module")
def readme_opts():
return parse_readme()
@pytest.mark.parametrize("macro_name", MACROS_PYTHON)
def test_macro_options_match_python(macro_opts, macro_name):
python_opts = parse_python_script(macro_name)
macro_set = macro_opts[macro_name]
assert macro_set
assert macro_set == python_opts
@pytest.mark.parametrize("macro_name", MACROS_README)
def test_macro_options_match_readme(macro_opts, readme_opts, macro_name):
macro_set = macro_opts[macro_name]
readme_set = readme_opts.get(macro_name, set())
assert macro_set
assert macro_set == readme_set

View File

@ -0,0 +1,98 @@
"""Unit tests for pyproject_getopt.lua option parsing.
Most tests are written in Lua (test_pyproject_getopt.lua) and run via
parametrized test_lua. Tests that need to inspect stderr for error
messages or define real RPM parametric macros stay in Python.
"""
import os
import subprocess
import textwrap
from pathlib import Path
import pytest
BASEDIR = Path(__file__).parent
def rpmlua(script):
"""Run a Lua script via rpm --eval, return (stdout, stderr, returncode)."""
full_script = textwrap.dedent(f"""\
package.path = "{BASEDIR}/?.lua"
""") + textwrap.dedent(script)
r = subprocess.run(
# ["rpmlua", "-e", full_script] when we no longer support c9s
["rpm", "--eval", "%{lua:" + full_script + "}"],
capture_output=True, text=True,
env={**os.environ, "LANG": "C.UTF-8"},
)
return r.stdout, r.stderr, r.returncode
def _get_lua_test_names():
stdout, stderr, rc = rpmlua("""\
local t = require("test_pyproject_getopt")
for _, name in ipairs(t.list()) do
io.write(name .. "\\n")
end
""")
assert rc == 0, f"Failed to list Lua tests:\n{stderr}"
names = stdout.strip().splitlines()
assert names, "No Lua tests found"
return names
@pytest.mark.parametrize("lua_test_name", _get_lua_test_names())
def test_lua(lua_test_name):
stdout, stderr, rc = rpmlua(f"""\
local t = require("test_pyproject_getopt")
t.{lua_test_name}()
""")
if rc != 0:
raise AssertionError(stderr)
if stdout.startswith("SKIP: "):
pytest.skip(stdout.removeprefix("SKIP: ").strip())
class TestErrorMessages:
"""Verify that error messages contain expected text (via stderr).
These call raw_* functions from the Lua test module which invoke
getopt() without pcall, so the RPM error propagates to stderr.
"""
def _run_raw(self, name):
_, stderr, rc = rpmlua(f"require('test_pyproject_getopt').{name}()")
assert rc != 0, "expected an error"
return stderr
def test_unknown_option_message(self):
stderr = self._run_raw("raw_unknown_short_option")
assert "unknown option: -z" in stderr
def test_unknown_long_option_message(self):
stderr = self._run_raw("raw_unknown_long_option")
assert "unknown option: --bogus" in stderr
def test_missing_value_message(self):
stderr = self._run_raw("raw_missing_value_short")
assert "requires a value" in stderr
def test_flag_given_value_message(self):
stderr = self._run_raw("raw_flag_given_value")
assert "does not take a value" in stderr
def test_mutual_exclusion_message(self):
stderr = self._run_raw("raw_R_and_x")
assert "mutually exclusive" in stderr
assert "--no-runtime" in stderr
assert "--extras" in stderr
def test_repeated_without_separator_message(self):
stderr = self._run_raw("raw_repeated_without_separator")
assert "cannot be repeated" in stderr
def test_macro_rejects_unknown_option_message(self):
stderr = self._run_raw("raw_macro_rejects_unknown_option")
assert "%_test_macro" in stderr
assert "unknown option: --bogus" in stderr

View File

@ -6,7 +6,7 @@ from pprint import pprint
from pyproject_preprocess_record import parse_record, read_record, save_parsed_record
from pyproject_save_files import argparser, generate_file_list, BuildrootPath
from pyproject_save_files import argparser, canonical_name_from_distinfo, generate_file_list, BuildrootPath
from pyproject_save_files import main as save_files_main
from pyproject_save_files import module_names_from_path
@ -25,6 +25,21 @@ TEST_RECORDS = yaml_data["records"]
TEST_METADATAS = yaml_data["metadata"]
# insert glob_suffix_len for .pyc files and man pages globs
for paths_dict in EXPECTED_DICT.values():
for modules in paths_dict["modules"].values():
for module in modules:
for idx, file in enumerate(module["files"]):
if file.endswith(".pyc"):
module["files"][idx] = BuildrootPath(file)
module["files"][idx].glob_suffix_len = len("{,.opt-?}.pyc")
if "other" in paths_dict and "files" in paths_dict["other"]:
for idx, file in enumerate(paths_dict["other"]["files"]):
if file.endswith("*"):
paths_dict["other"]["files"][idx] = BuildrootPath(file)
paths_dict["other"]["files"][idx].glob_suffix_len = len("*")
@pytest.fixture
def tldr_root(tmp_path):
prepare_pyproject_record(tmp_path, package="tldr")
@ -88,6 +103,7 @@ def test_parse_record_tldr():
str(SITELIB / "tldr-0.5.dist-info/WHEEL"),
str(SITELIB / "tldr-0.5.dist-info/top_level.txt"),
str(SITELIB / "tldr.py"),
str(SITELIB / "tldr.pyi"),
]
assert output == expected
@ -226,12 +242,12 @@ def test_cli_no_pyproject_record(tmp_path, pyproject_record):
def test_cli_too_many_RECORDS(tldr_root, output_files, output_modules, pyproject_record):
# Two calls to simulate how %pyproject_install process more than one RECORD file
prepare_pyproject_record(tldr_root,
content=("foo/bar/dist-info/RECORD", []))
content=("foo/bar-1.0.dist-info/RECORD", []))
prepare_pyproject_record(tldr_root,
content=("foo/baz/dist-info/RECORD", []))
content=("foo/baz-2.0.dist-info/RECORD", []))
cli_args = argparser().parse_args([*default_options(output_files, output_modules, tldr_root, pyproject_record), "tldr*"])
with pytest.raises(FileExistsError):
with pytest.raises(ValueError, match="Use %pyproject_save_files -D"):
save_files_main(cli_args)
@ -261,3 +277,66 @@ def test_cli_bad_namespace(tldr_root, output_files, output_modules, pyproject_re
with pytest.raises(ValueError):
save_files_main(cli_args)
def test_canonical_name_from_distinfo():
assert canonical_name_from_distinfo("MarkupSafe-2.0.1.dist-info") == "markupsafe"
assert canonical_name_from_distinfo("tldr-0.5.dist-info") == "tldr"
assert canonical_name_from_distinfo("my_package-1.0.0.dist-info") == "my-package"
assert canonical_name_from_distinfo("zope.event-1.0.dist-info") == "zope-event"
def test_cli_with_dist_name(tldr_root, pyproject_record):
output_files = tldr_root / "files"
output_modules = tldr_root / "modules"
cli_args = argparser().parse_args([
*default_options(output_files, output_modules, tldr_root, pyproject_record),
"-D", "tldr",
"tldr*",
])
save_files_main(cli_args)
assert output_files.exists()
assert "tldr" in output_files.read_text()
assert output_modules.exists()
def test_cli_with_dist_name_case_insensitive(tldr_root, pyproject_record):
output_files = tldr_root / "files"
output_modules = tldr_root / "modules"
cli_args = argparser().parse_args([
*default_options(output_files, output_modules, tldr_root, pyproject_record),
"-D", "TLDR",
"tldr*",
])
save_files_main(cli_args)
assert output_files.exists()
def test_cli_dist_name_not_found(tldr_root, pyproject_record):
output_files = tldr_root / "files"
output_modules = tldr_root / "modules"
cli_args = argparser().parse_args([
*default_options(output_files, output_modules, tldr_root, pyproject_record),
"-D", "nonexistent",
"tldr*",
])
with pytest.raises(ValueError, match="No dist-info found matching"):
save_files_main(cli_args)
def test_cli_single_record_named_output(tldr_root, pyproject_record):
output_files = tldr_root / "files"
output_modules = tldr_root / "modules"
cli_args = argparser().parse_args([
*default_options(output_files, output_modules, tldr_root, pyproject_record),
"tldr*",
])
save_files_main(cli_args)
# Without -D, base output should exist
assert output_files.exists()
# Named output (derived from dist-info) should also exist
named_files = tldr_root / "files-tldr"
named_modules = tldr_root / "modules-tldr"
assert named_files.exists()
assert named_modules.exists()
assert output_files.read_text() == named_files.read_text()

View File

@ -1,11 +1,12 @@
Name: pyproject-rpm-macros
Summary: RPM macros for PEP 517 Python packages
# SPDX
License: MIT
%bcond tests 1
# pytest-xdist and tox are not desired in RHEL
%bcond pytest_xdist %{undefined rhel}
%bcond tox_tests %{undefined rhel}
%bcond pytest_xdist %[%{undefined rhel} || %{defined epel}]
%bcond tox_tests %[%{undefined rhel} || %{defined epel}]
# The idea is to follow the spirit of semver
# Given version X.Y.Z:
@ -13,36 +14,44 @@ License: MIT
# Increment Y and reset Z when new macros or features are added
# Increment Z when this is a bugfix or a cosmetic change
# Dropping support for EOL Fedoras is *not* considered a breaking change
Version: 1.12.0
Version: 1.23.2
Release: 1%{?dist}
# Macro files
Source001: macros.pyproject
Source002: macros.aaa-pyproject-srpm
Source: macros.pyproject
Source: macros.aaa-pyproject-srpm
# Implementation files
Source101: pyproject_buildrequires.py
Source102: pyproject_save_files.py
Source103: pyproject_convert.py
Source104: pyproject_preprocess_record.py
Source105: pyproject_construct_toxenv.py
Source106: pyproject_requirements_txt.py
Source107: pyproject_wheel.py
# Implementation files, Python
Source: pyproject_buildrequires.py
Source: pyproject_convert.py
Source: pyproject_dependency_overrides.py
Source: pyproject_patch_metadata.py
Source: pyproject_preprocess_record.py
Source: pyproject_requirements_txt.py
Source: pyproject_save_files.py
Source: pyproject_wheel.py
# Implementation files, Lua
Source: pyproject_getopt.lua
# Tests
Source201: test_pyproject_buildrequires.py
Source202: test_pyproject_save_files.py
Source203: test_pyproject_requirements_txt.py
Source204: compare_mandata.py
Source: compare_mandata.py
Source: test_dependency_overrides.py
Source: test_pyproject_buildrequires.py
Source: test_pyproject_getopt.lua
Source: test_pyproject_getopt_consistency.py
Source: test_pyproject_getopt_parser.py
Source: test_pyproject_requirements_txt.py
Source: test_pyproject_save_files.py
# Test data
Source301: pyproject_buildrequires_testcases.yaml
Source302: pyproject_save_files_test_data.yaml
Source303: test_RECORD
Source: pyproject_buildrequires_testcases.yaml
Source: pyproject_save_files_test_data.yaml
Source: test_RECORD
# Metadata
Source901: README.md
Source902: LICENSE
Source: README.md
Source: LICENSE
URL: https://src.fedoraproject.org/rpms/pyproject-rpm-macros
@ -58,9 +67,9 @@ BuildRequires: python3dist(packaging)
BuildRequires: python3dist(pip)
BuildRequires: python3dist(setuptools)
%if %{with tox_tests}
BuildRequires: python3dist(tox-current-env) >= 0.0.6
BuildRequires: python3dist(tox-current-env) >= 0.0.16
%endif
BuildRequires: python3dist(wheel)
BuildRequires: (python3dist(wheel) if python3dist(setuptools) < 71)
BuildRequires: (python3dist(tomli) if python3 < 3.11)
# RHEL 9: We also run pytest with Python 3.11 and 3.12
@ -86,7 +95,7 @@ BuildRequires: python3-rpm-macros
Requires: python-rpm-macros
Requires: python-srpm-macros
Requires: python3-rpm-macros
Requires: (pyproject-srpm-macros = %{?epoch:%{epoch}:}%{version}-%{release} if pyproject-srpm-macros)
Requires: pyproject-srpm-macros = %{?epoch:%{epoch}:}%{version}-%{release}
# We use the following tools outside of coreutils
Requires: /usr/bin/find
@ -95,8 +104,10 @@ Requires: /usr/bin/sed
# This package requires the %%generate_buildrequires functionality.
# It has been introduced in RPM 4.15 (4.14.90 is the alpha of 4.15).
# What we need is rpmlib(DynamicBuildRequires), but that is impossible to (Build)Require.
Requires: (rpm-build >= 4.14.90 if rpm-build)
BuildRequires: rpm-build >= 4.14.90
# Also, we need to avoid 4.19.90..4.19.91-7 due to rhbz#2284187
# Also, we need 4.16.1.3-37 or newer to get RHEL-67161
Requires: ((rpm-build >= 4.16.1.3-37 with (rpm-build < 4.19.90 or rpm-build >= 4.19.91-8)) if rpm-build)
BuildRequires: rpm-build >= 4.16.1.3-37
%description
These macros allow projects that follow the Python packaging specifications
@ -116,7 +127,7 @@ which only work with setup.py.
%package -n pyproject-srpm-macros
Summary: Minimal implementation of %%pyproject_buildrequires
Requires: (pyproject-rpm-macros = %{?epoch:%{epoch}:}%{version}-%{release} if pyproject-rpm-macros)
Requires: (rpm-build >= 4.14.90 if rpm-build)
Requires: (rpm-build >= 4.16.1.3-37 if rpm-build)
%description -n pyproject-srpm-macros
This package contains a minimal implementation of %%pyproject_buildrequires.
@ -126,8 +137,7 @@ takes precedence.
%prep
# Not strictly necessary but allows working on file names instead
# of source numbers in install section
# Allows working on file names, as sources have no meaningful numbers
%setup -c -T
cp -p %{sources} .
@ -140,25 +150,22 @@ cp -p %{sources} .
%install
mkdir -p %{buildroot}%{_rpmmacrodir}
mkdir -p %{buildroot}%{_rpmconfigdir}/redhat
mkdir -p %{buildroot}%{_rpmluadir}/fedora/rpm
install -pm 644 macros.pyproject %{buildroot}%{_rpmmacrodir}/
install -pm 644 macros.aaa-pyproject-srpm %{buildroot}%{_rpmmacrodir}/
install -pm 644 pyproject_getopt.lua %{buildroot}%{_rpmluadir}/fedora/rpm/
install -pm 644 pyproject_buildrequires.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_convert.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_save_files.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_preprocess_record.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_construct_toxenv.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_requirements_txt.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_wheel.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_patch_metadata.py %{buildroot}%{_rpmconfigdir}/redhat/
install -pm 644 pyproject_dependency_overrides.py %{buildroot}%{_rpmconfigdir}/redhat/
%check
# assert the two signatures of %%pyproject_buildrequires match exactly
signature1="$(grep '^%%pyproject_buildrequires' macros.pyproject | cut -d' ' -f1)"
signature2="$(grep '^%%pyproject_buildrequires' macros.aaa-pyproject-srpm | cut -d' ' -f1)"
test "$signature1" == "$signature2"
# but also assert we are not comparing empty strings
test "$signature1" != ""
%if %{with tests}
%check
export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856356
%pytest -vv --doctest-modules %{?with_pytest_xdist:-n auto} %{!?with_tox_tests:-k "not tox"}
@ -181,9 +188,11 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
%{_rpmconfigdir}/redhat/pyproject_convert.py
%{_rpmconfigdir}/redhat/pyproject_save_files.py
%{_rpmconfigdir}/redhat/pyproject_preprocess_record.py
%{_rpmconfigdir}/redhat/pyproject_construct_toxenv.py
%{_rpmconfigdir}/redhat/pyproject_requirements_txt.py
%{_rpmconfigdir}/redhat/pyproject_wheel.py
%{_rpmconfigdir}/redhat/pyproject_patch_metadata.py
%{_rpmconfigdir}/redhat/pyproject_dependency_overrides.py
%{_rpmluadir}/fedora/rpm/pyproject_getopt.lua
%doc README.md
%license LICENSE
@ -194,6 +203,145 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
%changelog
* Sun Jul 26 2026 Miro Hrončok <mhroncok@redhat.com> - 1.23.2-1
- Fixup a regression in %%pyproject_extras_subpkg
* Thu Jul 23 2026 Miro Hrončok <mhroncok@redhat.com> - 1.23.1-1
- getopt: Fix global macro clobbering with save/restore stack
- %%pyproject_extras_subpkg: Add long options support
- %%pyproject_extras_subpkg: Make -D/--dist-name mutually exclusive with -i/-f/-F
- Set PIP_CONFIG_FILE=/dev/null by default when invoking pip to build the wheel
* Thu Jul 16 2026 Fedora Release Engineering <releng@fedoraproject.org> - 1.23.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_45_Mass_Rebuild
* Mon May 18 2026 Miro Hrončok <mhroncok@redhat.com> - 1.23.0-1
- %%pyproject_save_files: Implement support for multi-package specfiles
* Thu May 14 2026 Miro Hrončok <mhroncok@redhat.com> - 1.22.2-1
- %%pyproject_buildrequires: Fix "Requirement satisfied/not satisfied" messages to show overridden constraints from %%pyproject_patch_dependency
- %%pyproject_buildrequires: Stop using deprecated argparse.FileType
* Thu May 07 2026 Miro Hrončok <mhroncok@redhat.com> - 1.22.1-1
- Fix a regression wrt option parsing for macros with backslash-escaped newlines in argument list
* Thu May 07 2026 Miro Hrončok <mhroncok@redhat.com> - 1.22.0-1
- Add long option support for all public parametric macros
- E.g. %%pyproject_buildrequires --no-runtime is equivalent to %%pyproject_buildrequires -R
- %%pyproject_save_files: Allow to use --auto instead of +auto
- %%pyproject_patch_dependency: Validate arguments early, in %%prep
* Wed Apr 29 2026 Tomáš Hrnčiar <thrnciar@redhat.com> - 1.21.0-1
- Implement extras validation
- %%pyproject_buildrequires: validates if extras exist in upstream metadata, otherwise ValueError is raised on Fedora >=45 and RHEL >=11. It will emit a warning instead on older releases.
- Both user-specified extras and metadata extras are normalized following PEP 685 conventions.
* Tue Apr 28 2026 Benjamin A. Beasley <code@musicinmybrain.net> - 1.20.1-1
- Move %%pyproject_patch_dependency implementation to pyproject-srpm-macros
- Make pyproject-rpm-macros depend on pyproject-srpm-macros
- Resolves: rhbz#2463187
* Fri Apr 24 2026 Charalampos Stratakis <cstratak@redhat.com> - 1.20.0-1
- Add %%pyproject_patch_dependency macro for overriding dependency constraints
- Overrides apply to both BuildRequires and runtime Requires (via METADATA patching)
- Resolves: rhbz#2386906
* Tue Mar 31 2026 Miro Hrončok <mhroncok@redhat.com> - 1.19.0-1
- Add -d option for %%pyproject_buildrequires and %%pyproject_wheel to specify a working directory
- Fixes: rhbz#2371389
* Wed Mar 04 2026 Lumír Balhar <lbalhar@redhat.com> - 1.18.7-1
- pyproject_convert: Use deprecated LegacyVersion (_version) only if necessary
* Mon Dec 01 2025 Karolina Surma <ksurma@redhat.com> - 1.18.6-1
- Properly resolve self-referencing dependency groups in %%pyproject_buildrequires
* Thu Oct 16 2025 Miro Hrončok <mhroncok@redhat.com> - 1.18.5-1
- %%pyproject_extras_subpkg: Only %%ghost the dist-info directory, not the content
- That way, accidentally unpackaged files within are reported as errors
- %%pyproject_save_files: Also save top level typing stub files (.pyi)
* Mon Sep 01 2025 Miro Hrončok <mhroncok@redhat.com> - 1.18.4-1
- Don't exit from pyproject-srpm-macros implementation of %%pyproject_buildrequires
- Fixes: rhbz#2391290
- On RPM 4.20+ don't put pyproject-macros-specific files in %%buildsubdir
- Works around https://github.com/rpm-software-management/rpm/issues/3890
- Speed %%pyproject_save_files up significantly
* Fri Jul 25 2025 Fedora Release Engineering <releng@fedoraproject.org> - 1.18.3-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_43_Mass_Rebuild
* Fri Jul 11 2025 Miro Hrončok <mhroncok@redhat.com> - 1.18.3-1
- %%pyproject_buildrequires: Do not generate BuildRequires from Requires core metadata fields
- That field is deprecated and should include importable module names, not distribution packages
- Related: rhbz#2378463
* Mon May 19 2025 Maxwell G <maxwell@gtmx.me> - 1.18.2-1
- Fix handling of config_settings in %%pyproject_buildrequires
* Fri Mar 21 2025 Miro Hrončok <mhroncok@redhat.com> - 1.18.1-1
- Fix reverted conditional in %%pyproject_buildrequires -t/-e Fedora version comparison
* Tue Mar 11 2025 Miro Hrončok <mhroncok@redhat.com> - 1.18.0-1
- Make %%pyproject_buildrequires -t/-e and %%tox fail when no suitable tox configuration exists
- The %%pyproject_buildrequires -t/-e case is temporarily allowed on Fedora 40-42
- Requires tox-current-env >= 0.0.16
* Thu Jan 30 2025 Miro Hrončok <miro@hroncok.cz> - 1.17.0-1
- Add the -M flag to %%pyproject_save_files
- The flag can be used to indicate no Python modules should be saved
* Sat Jan 18 2025 Fedora Release Engineering <releng@fedoraproject.org> - 1.16.4-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_42_Mass_Rebuild
* Tue Dec 03 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.4-1
- Deprecate the provisional -w flag for %%pyproject_buildrequires
* Tue Dec 03 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.3-1
- Accept arbitrary options from %%pyproject_buildrequires in pyproject-srpm-macros
- This will make future additions smoother
* Wed Nov 13 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.2-1
- Fix one remaining test for setuptools 70+
* Thu Nov 07 2024 Miro Hrončok <miro@hroncok.cz> - 1.16.1-1
- Support for setuptools 70+
- wheel is no longer generated as a dependency of the default build system
* Mon Nov 04 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.0-1
- %%pyproject_buildrequires: Add support for dependency groups (PEP 735), via the -g flag
- This is implied when used tox testenvs depend on dependency groups (requires tox 4.22+)
- Fixes: rhbz#2318849
* Thu Oct 03 2024 Karolina Surma <ksurma@redhat.com> - 1.15.1-1
- Fix handling of self-referencing extras when reading pyproject.toml
* Tue Sep 17 2024 Python Maint <python-maint@redhat.com> - 1.15.0-1
- Add a possibility to read runtime requirements from pyproject.toml [project] table
- Fixes: rhbz#2261939
- Don't generate a dependency on pip when %%pyproject_buildrequires -N is used
- Fixes: rhbz#2294510
- Even when %%_auto_set_build_flags is disabled, set all compiler flags when building wheels
- Fixes: rhbz#2293616
* Tue Jul 23 2024 Miro Hrončok <mhroncok@redhat.com> - 1.14.0-1
- Add a provisional RPM Declarative Buildsystem (RPM 4.20+)
* Fri Jul 19 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.13.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_41_Mass_Rebuild
* Tue Jul 02 2024 Miro Hrončok <mhroncok@redhat.com> - 1.13.0-1
- Properly escape weird characters from paths in %%{pyproject_files} (RPM 4.19+ only)
- Revert the temporary workaround for RPM 4.20 alpha 2 leaking \x1f (unit separators)
- Fixes: rhbz#1990879
* Tue Jun 25 2024 Cristian Le <fedora@lecris.me> - 1.12.2-1
- %%pyproject_extras_subpkg: Allow passing -a or -A to %%python_extras_subpkg
* Tue Jun 04 2024 Miro Hrončok <mhroncok@redhat.com> - 1.12.1-1
- Add a temporary workaround for RPM 4.20 alpha 2 leaking \x1f (unit separators)
- Related: rhbz#2284187
* Fri Jan 26 2024 Miro Hrončok <miro@hroncok.cz> - 1.12.0-1
- Namespace pyproject-rpm-macros generated text files with %%{python3_pkgversion}
- That way, a single-spec can be used to build packages for multiple Python versions