diff --git a/SOURCES/README.md b/SOURCES/README.md index 192c04d..7b5b21c 100644 --- a/SOURCES/README.md +++ b/SOURCES/README.md @@ -80,7 +80,7 @@ using the `-R` flag: %pyproject_buildrequires -R Alternatively, if the project specifies its dependencies in the pyproject.toml -`[project]` table (as defined in [PEP 621](https://www.python.org/dev/peps/pep-0621/)), +`[project]` table (as defined in [PEP 621]), the runtime dependencies can be obtained by reading that metadata. This can be enabled by using the `-p` flag. @@ -92,28 +92,6 @@ Please note that not all build backends which use pyproject.toml support the For example, poetry-core (at least in 1.9.0) defines package metadata in the custom `[tool.poetry]` table which is not supported by the `%pyproject_buildrequires` macro. -Finally, the runtime dependencies can be obtained by building the wheel and reading the metadata from the built wheel. -This can be enabled with the `-w` flag and cannot be combined with `-p`. -Support for building wheels with `%pyproject_buildrequires -w` is **provisional** and the behavior might change. -Please subscribe to Fedora's [python-devel list] if you use the option. - - %generate_buildrequires - %pyproject_buildrequires -w - -When this is used, the wheel is going to be built at least twice, -becasue the `%generate_buildrequires` section runs repeatedly. -To avoid accidentally reusing a wheel leaking from a previous (different) build, -it cannot be reused between `%generate_buildrequires` rounds. -Contrarily to that, rebuilding the wheel again in the `%build` section is redundant -and the packager can omit the `%build` section entirely -to reuse the wheel built from the last round of `%generate_buildrequires`. -Be extra careful when attempting to modify the sources after `%pyproject_buildrequires`, -e.g. when running extra commands in the `%build` section: - - %build - cython src/wrong.pyx # this is too late with %%pyproject_buildrequires -w - %pyproject_wheel - For projects that specify test requirements using an [`extra` provide](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use), these can be added using the `-x` flag. @@ -167,7 +145,7 @@ in worst case, patch/sed the requirement out from the tox configuration. Note that neither `-x` or `-t` can be used with `-R` or `-N`, because runtime dependencies are always required for testing. You can only use those options if the build backend supports the [prepare-metadata-for-build-wheel hook], -or together with `-p` or `-w`. +or together with `-p`. However, using `-g` with `-R` or `-N` is supported because dependency groups don't need to be used for testing and can be obtained by reading `pyproject.toml` only. @@ -184,7 +162,7 @@ Dependencies will be loaded from them: For packages not using build system you can use `-N` to entirely skip automatical generation of requirements and install requirements only from manually specified files. `-N` option implies `-R` and cannot be used in combination with other options mentioned above -(`-w`, `-e`, `-t`, `-x`, `-p`). +(`-e`, `-t`, `-x`, `-p`). The `%pyproject_buildrequires` macro also accepts the `-r` flag for backward compatibility; it means "include runtime dependencies" which has been the default since version 0-53. @@ -243,7 +221,7 @@ The macro: - Always prepends `$PATH` with `%{buildroot}%{_bindir}` - If not defined, sets `$PYTHONPATH` to `%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}` - If not defined, sets `$TOX_TESTENV_PASSENV` to `*` - - Runs `tox` with `-q` (quiet), `--recreate` and `--current-env` (from [tox-current-env]) flags + - Runs `tox` with `-q` (quiet), `--recreate`, `--current-env` (from [tox-current-env]) and `--assert-config` (from [tox-current-env]) flags - Implicitly uses the tox environment name stored in `%{toxenv}` - as overridden by `%pyproject_buildrequires -e` By using the `-e` flag, you can use a different tox environment(s): @@ -311,10 +289,17 @@ However, in Fedora packages, always list executables explicitly to avoid uninten %doc README.rst %{_bindir}/downloader +If the package has no Python modules in it, you can explicitly use `-M` to denote that. + + %install + %pyproject_install + %pyproject_save_files -M + +Otherwise, at least one module-glob argument is required. + `%pyproject_save_files` can automatically mark license files with `%license` macro and language (`*.mo`) files with `%lang` macro and appropriate language code. Only license files declared via [PEP 639] `License-File` field are detected. -[PEP 639] is still provisional and can be changed in the future. It is possible to use the `-l` flag to declare that a missing license should terminate the build or `-L` (the default) to explicitly disable this check. Packagers are encouraged to use the `-l` flag when the `%license` file is not manually listed in `%files` @@ -417,7 +402,7 @@ Provisional: Declarative Buildsystem (RPM 4.20+) It is possible to reduce some of the spec boilerplate by using the provided pyproject [declarative buildsystem]. -This option is only available with RPM 4.20+ (e.g. in Fedora 41+). +This option is only available with RPM 4.20+ (e.g. in Fedora 41+, ELN/CentOS Stream 11+). The declarative buildsystem is **provisional** and the behavior might change. Please subscribe to Fedora's [python-devel list] if you use the feature. @@ -532,6 +517,7 @@ so be prepared for problems. [PEP 508]: https://www.python.org/dev/peps/pep-0508/ [PEP 517]: https://www.python.org/dev/peps/pep-0517/ [PEP 518]: https://www.python.org/dev/peps/pep-0518/ +[PEP 621]: https://www.python.org/dev/peps/pep-0621/ [PEP 639]: https://www.python.org/dev/peps/pep-0639/ [PEP 735]: https://www.python.org/dev/peps/pep-0735/ [pip's documentation]: https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support @@ -542,6 +528,11 @@ Deprecated The `%{pyproject_build_lib}` macro is deprecated, don't use it. +The `%pyproject_buildrequires` `-w` option is deprecated, don't use it. +If the build backend does not support the [prepare-metadata-for-build-wheel hook], +consider using the `-p` flag to read the metadata from the pyproject.toml +`[project]` table (as defined in [PEP 621]) instead. + Testing the macros ------------------ @@ -565,11 +556,11 @@ For each `$PKG.spec` in `tests/`: - download the sources: - spectool -g -R $PKG.spec + spectool -g $PKG.spec - build a SRPM: - rpmbuild -bs $PKG.spec + rpmbuild -bs --define '_sourcedir .' $PKG.spec - build in mock, using the path from the command above as `$SRPM`: diff --git a/SOURCES/macros.aaa-pyproject-srpm b/SOURCES/macros.aaa-pyproject-srpm index 1b06ac3..fd4dece 100644 --- a/SOURCES/macros.aaa-pyproject-srpm +++ b/SOURCES/macros.aaa-pyproject-srpm @@ -3,8 +3,8 @@ # When this file is installed but macros.pyproject is not # this macro will cause the package with the real macro to be installed. # When macros.pyproject is installed, it overrides this macro. -# Note: This needs to maintain the same set of options as the real macro. -%pyproject_buildrequires(rRxtNwpe:g:C:) echo 'pyproject-rpm-macros' && exit 0 +# Note: This takes arbitrary options, to ease addition of new options to the real macro. +%pyproject_buildrequires(-) echo 'pyproject-rpm-macros' # Declarative buildsystem, requires RPM 4.20+ to work diff --git a/SOURCES/macros.pyproject b/SOURCES/macros.pyproject index 449afe6..6bf5d01 100644 --- a/SOURCES/macros.pyproject +++ b/SOURCES/macros.pyproject @@ -2,8 +2,14 @@ # For the main Python it's empty, for all others it's "-3.X" %_pyproject_files_pkgversion %{expr:"%{python3_pkgversion}" != "3" ? "-%{python3_pkgversion}" : ""} +# In RPM < 4.20 (4.19.9x is 4.20 alpha), there is no guaranteed, RPM-controlled per-build directory (%%mkbuilddir step). +# Hence we use %%{buildsubdir} if available. +# On newer RPM 4.20+ this is no longer necessary and breaks the declarative buildsystem: +# https://github.com/rpm-software-management/rpm/issues/3890 +%_pyproject_buildsubdir_compat %[ v"0%{?rpmversion}" < v"4.19.90" ? "%{?buildsubdir:/%{buildsubdir}}" : ""] + # This is a directory where wheels are stored and installed from, absolute -%_pyproject_wheeldir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/pyproject-wheeldir%{_pyproject_files_pkgversion} +%_pyproject_wheeldir %{_builddir}%{_pyproject_buildsubdir_compat}/pyproject-wheeldir%{_pyproject_files_pkgversion} # This is a directory used as TMPDIR, where pip copies sources to and builds from, relative to PWD # For proper debugsource packages, we create TMPDIR within PWD @@ -12,7 +18,7 @@ # This will be used in debugsource package paths (applies to extension modules only) # NB: pytest collects tests from here if not hidden # https://docs.pytest.org/en/latest/reference.html#confval-norecursedirs -%_pyproject_builddir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/.pyproject-builddir%{_pyproject_files_pkgversion} +%_pyproject_builddir %{_builddir}%{_pyproject_buildsubdir_compat}/.pyproject-builddir%{_pyproject_files_pkgversion} # We prefix all created files with this value to make them unique # Ideally, we would put them into %%{buildsubdir}, but that value changes during the spec @@ -75,7 +81,7 @@ echo $(IFS=:; echo "${pyproject_build_lib[*]}") %pyproject_install() %{expand:\\\ specifier=$(ls %{_pyproject_wheeldir}/*.whl | xargs basename --multiple | sed -E 's/([^-]+)-([^-]+)-.+\\\.whl/\\\1==\\\2/') -if [ -z $specifier ]; then +if [ -z "$specifier" ]; then echo 'ERROR: %%%%pyproject_install found no wheel in %%%%{_pyproject_wheeldir} %{_pyproject_wheeldir}' >&2 exit 1 fi @@ -97,7 +103,7 @@ fi # Process all *.dist-info dirs in sitelib/sitearch for site_dir in ${site_dirs[@]}; do for distinfo in %{buildroot}$site_dir/*.dist-info; do - echo "%ghost ${distinfo#%{buildroot}}" >> %{_pyproject_ghost_distinfo} + echo "%ghost %dir ${distinfo#%{buildroot}}" >> %{_pyproject_ghost_distinfo} sed -i 's/pip/rpm/' ${distinfo}/INSTALLER PYTHONPATH=%{_rpmconfigdir}/redhat \\ %{__python3} -B %{_rpmconfigdir}/redhat/pyproject_preprocess_record.py \\ @@ -122,7 +128,7 @@ fi # https://github.com/rpm-software-management/rpm/issues/1749#issuecomment-1020420616 # Since we support both ways, we pass either 4.19 or 4.18 to the script, so it knows which one to use # Rather than passing the actual version, we let RPM compare the versions, as it is easier done here than in Python -%pyproject_save_files(lL) %{expand:\\\ +%pyproject_save_files(lLM) %{expand:\\\ %{expr:v"0%{?rpmversion}" >= v"4.18.90" ? "RPM_FILES_ESCAPE=4.19" : "RPM_FILES_ESCAPE=4.18" } \\ %{__python3} %{_rpmconfigdir}/redhat/pyproject_save_files.py \\ --output-files "%{pyproject_files}" \\ @@ -166,7 +172,6 @@ fi ]"\ ] -# Note: Keep the options in sync with this macro from macros.aaa-pyproject-srpm %pyproject_buildrequires(rRxtNwpe:g:C:) %{expand:\\\ %_set_pytest_addopts # The default flags expect the package note file to exist @@ -193,6 +198,10 @@ fi fi} } %{-w: +%{!?__pyproject_buildrequires_w_warned:%{warn:The %%pyproject_buildrequires -w option is deprecated. +It's not efficient to build the wheel several times during the build. +The option is not scheduled for removal, but packagers should use the -p option instead. +}%global __pyproject_buildrequires_w_warned 1} %{-p:%{error:The -w and -p options are mutually exclusive}} } %{-e:%{expand:%global toxenv %(%{__python3} -s %{_rpmconfigdir}/redhat/pyproject_construct_toxenv.py %{?**})}} @@ -216,7 +225,7 @@ if [ -f %{__python3} ]; then echo -n > %{_pyproject_buildrequires} %{_pyproject_build_flags} \\\ TMPDIR="%{_pyproject_builddir}" \\\ - RPM_TOXENV="%{toxenv}" HOSTNAME="rpmbuild" %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_buildrequires.py %{?!_python_no_extras_requires:--generate-extras} --python3_pkgversion %{python3_pkgversion} --wheeldir %{_pyproject_wheeldir} --output %{_pyproject_buildrequires} %{?**} >&2 + RPM_TOXENV="%{toxenv}" FEDORA=%{?fedora} HOSTNAME="rpmbuild" %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_buildrequires.py %{?!_python_no_extras_requires:--generate-extras} --python3_pkgversion %{python3_pkgversion} --wheeldir %{_pyproject_wheeldir} --output %{_pyproject_buildrequires} %{?**} >&2 cat %{_pyproject_buildrequires} fi # Incomplete .dist-info dir might confuse importlib.metadata @@ -231,7 +240,7 @@ PATH="%{buildroot}%{_bindir}:$PATH" \\ PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}}" \\ %{?__pytest_addopts:PYTEST_ADDOPTS="${PYTEST_ADDOPTS:-} %{__pytest_addopts}"}} \\ HOSTNAME="rpmbuild" \\ -%{__python3} -m tox --current-env -q --recreate -e "%{-e:%{-e*}}%{!-e:%{toxenv}}" %{?*} +%{__python3} -m tox --current-env --assert-config -q --recreate -e "%{-e:%{-e*}}%{!-e:%{toxenv}}" %{?*} } diff --git a/SOURCES/pyproject_buildrequires.py b/SOURCES/pyproject_buildrequires.py index e0ef7de..bba4f00 100644 --- a/SOURCES/pyproject_buildrequires.py +++ b/SOURCES/pyproject_buildrequires.py @@ -22,6 +22,15 @@ from pyproject_wheel import parse_config_settings_args # Allow only the forms we know we can handle. VERSION_RE = re.compile(r'[a-zA-Z0-9.-]+(\.\*)?') +# To avoid breakage on Fedora 40-42, +# we don't assert tox configuration there. +# This can be removed when Fedora 42 goes EOL. +# Note that %tox still uses --assert-config +# because %tox without config is dangerous (false sense of tests). +# Running %pyproject_buildrequires -t/-e without tox config is wrong, but not dangerous. +FEDORA = int(os.getenv('FEDORA') or 0) +TOX_ASSERT_CONFIG_OPTS = () if 40 <= FEDORA < 43 else ('--assert-config',) + class EndPass(Exception): """End current pass of generating requirements""" @@ -290,7 +299,7 @@ def get_backend(requirements): def generate_build_requirements(backend, requirements): get_requires = getattr(backend, 'get_requires_for_build_wheel', None) if get_requires: - new_reqs = get_requires(config_settings=requirements.config_settings) + new_reqs = get_requires(requirements.config_settings) requirements.extend(new_reqs, source='get_requires_for_build_wheel') requirements.check(source='get_requires_for_build_wheel') @@ -300,7 +309,7 @@ def parse_metadata_file(metadata_file): def requires_from_parsed_metadata_file(message): - return {k: message.get_all(k, ()) for k in ('Requires', 'Requires-Dist')} + return {k: message.get_all(k, ()) for k in ('Requires-Dist',)} def package_name_from_parsed_metadata_file(message): @@ -322,11 +331,10 @@ def generate_run_requirements_hook(backend, requirements): 'The build backend cannot provide build metadata ' '(incl. runtime requirements) before build. ' 'If the dependencies are specified in the pyproject.toml [project] ' - 'table, you can use the -p flag to read them.' - 'Alternatively, use the provisional -w flag to build the wheel and parse the metadata from it, ' - 'or use the -R flag not to generate runtime dependencies.' + 'table, you can use the -p flag to read them. ' + 'Alternatively, use the -R flag not to generate runtime dependencies.' ) - dir_basename = prepare_metadata('.', config_settings=requirements.config_settings) + dir_basename = prepare_metadata('.', requirements.config_settings) with open(dir_basename + '/METADATA') as metadata_file: name, requires = package_name_and_requires_from_metadata_file(metadata_file) for key, req in requires.items(): @@ -417,7 +425,7 @@ def generate_run_requirements(backend, requirements, *, build_wheel, read_pyproj def generate_tox_requirements(toxenv, requirements): toxenv = ','.join(toxenv) - requirements.add('tox-current-env >= 0.0.6', source='tox itself') + requirements.add('tox-current-env >= 0.0.16', source='tox itself') requirements.check(source='tox itself') with tempfile.NamedTemporaryFile('r') as deps, \ tempfile.NamedTemporaryFile('r') as extras, \ @@ -427,6 +435,7 @@ def generate_tox_requirements(toxenv, requirements): '--print-deps-to', deps.name, '--print-extras-to', extras.name, '--no-provision', provision.name, + *TOX_ASSERT_CONFIG_OPTS, '-q', '-r', '-e', toxenv], check=False, encoding='utf-8', @@ -669,7 +678,7 @@ def main(argv): parser.add_argument( '-w', '--wheel', action='store_true', default=False, help=('Generate run-time requirements by building the wheel ' - '(useful for build backends without the prepare_metadata_for_build_wheel hook)'), + '(useful for build backends without the prepare_metadata_for_build_wheel hook, deprecated)'), ) parser.add_argument( '-p', '--read-pyproject-dependencies', action='store_true', default=False, diff --git a/SOURCES/pyproject_buildrequires_testcases.yaml b/SOURCES/pyproject_buildrequires_testcases.yaml index a374fa0..ae26569 100644 --- a/SOURCES/pyproject_buildrequires_testcases.yaml +++ b/SOURCES/pyproject_buildrequires_testcases.yaml @@ -456,7 +456,7 @@ tox dependencies: setuptools: 50 wheel: 1 tox: 3.5.3 - tox-current-env: 0.0.6 + tox-current-env: 0.0.16 toxenv: - py3 setup.py: | @@ -479,13 +479,13 @@ tox dependencies: - | # tox 3 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(toxdep1) python3dist(toxdep2) python3dist(inst) - | # tox 4 with setuptools 70+ python3dist(setuptools) >= 40.8 - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) python3dist(toxdep1) python3dist(toxdep2) @@ -493,7 +493,7 @@ tox dependencies: - | # tox 4 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) python3dist(toxdep1) python3dist(toxdep2) @@ -505,7 +505,7 @@ tox extras: setuptools: 50 wheel: 1 tox: 3.5.3 - tox-current-env: 0.0.6 + tox-current-env: 0.0.16 generate_extras: true toxenv: - py3 @@ -536,7 +536,7 @@ tox extras: - | # tox 3 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(toxdep) python3dist(inst) python3dist(dep11) > 11.0 @@ -548,7 +548,7 @@ tox extras: python3dist(extra-dep[extra_dep]) - | # tox 4 with setuptools 70+ python3dist(setuptools) >= 40.8 - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) python3dist(toxdep) python3dist(inst) @@ -562,7 +562,7 @@ tox extras: - | # tox 4 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) python3dist(toxdep) python3dist(inst) @@ -580,7 +580,7 @@ tox provision unsatisfied: setuptools: 50 wheel: 1 tox: 3.5.3 - tox-current-env: 0.0.6 + tox-current-env: 0.0.16 toxenv: - py3 setup.py: | @@ -604,13 +604,13 @@ tox provision unsatisfied: - | # tox 3 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 3.999 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 - | # tox 4 with setuptools 70+ python3dist(setuptools) >= 40.8 - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 3.999 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 @@ -618,7 +618,7 @@ tox provision unsatisfied: - | # tox 4 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 3.999 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 @@ -630,7 +630,7 @@ tox provision satisfied: setuptools: 50 wheel: 1 tox: 3.5.3 - tox-current-env: 0.0.6 + tox-current-env: 0.0.16 toxenv: - py3 setup.py: | @@ -653,7 +653,7 @@ tox provision satisfied: - | # tox 3 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 3.5 python3dist(setuptools) > 40.0 python3dist(toxdep1) @@ -661,7 +661,7 @@ tox provision satisfied: python3dist(inst) - | # tox 4 with setuptools 70+ python3dist(setuptools) >= 40.8 - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(setuptools) > 40.0 python3dist(tox) >= 3.5 python3dist(toxdep1) @@ -670,7 +670,7 @@ tox provision satisfied: - | # tox 4 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(setuptools) > 40.0 python3dist(tox) >= 3.5 python3dist(toxdep1) @@ -683,7 +683,7 @@ tox provision no minversion: setuptools: 50 wheel: 1 tox: 3.5.3 - tox-current-env: 0.0.6 + tox-current-env: 0.0.16 toxenv: - py3 setup.py: | @@ -701,19 +701,19 @@ tox provision no minversion: - | # tox 3 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 - | # tox 4 with setuptools 70+ python3dist(setuptools) >= 40.8 - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 python3dist(tox) - | # tox 4 with setuptools < 70 python3dist(setuptools) >= 40.8 python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(setuptools) > 40.0 python3dist(wheel) > 2.0 python3dist(tox) @@ -1573,7 +1573,7 @@ tox with dependency_groups: setuptools: 50 wheel: 1 tox: 4.22 - tox-current-env: 0.0.14 + tox-current-env: 0.0.16 toxenv: - py3 pyproject.toml: | @@ -1594,7 +1594,7 @@ tox with dependency_groups: expected: - | # setuptools 70+ python3dist(setuptools) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 4.22 python3dist(tox) python3dist(pytest) >= 5 @@ -1602,9 +1602,29 @@ tox with dependency_groups: - | # setuptools < 70 python3dist(setuptools) python3dist(wheel) - python3dist(tox-current-env) >= 0.0.6 + python3dist(tox-current-env) >= 0.0.16 python3dist(tox) >= 4.22 python3dist(tox) python3dist(pytest) >= 5 python3dist(pytest-mock) result: 0 + +Plain Requires fields in core metadata is ignored: + installed: + setuptools: 50 + wheel: 1 + include_runtime: true + setup.py: | + from setuptools import setup + setup( + name='test', + version='0.1', + requires=['ignore_me'], + ) + expected: + - | # setuptools 70+ + python3dist(setuptools) >= 40.8 + - | # setuptools < 70 + python3dist(setuptools) >= 40.8 + python3dist(wheel) + result: 0 diff --git a/SOURCES/pyproject_save_files.py b/SOURCES/pyproject_save_files.py index 93de7a2..dcc53e4 100644 --- a/SOURCES/pyproject_save_files.py +++ b/SOURCES/pyproject_save_files.py @@ -137,12 +137,11 @@ def add_file_to_module(paths, module_name, module_type, files_dirs, *files): """ for module in paths["modules"][module_name]: if module["type"] == module_type: - if files[0] not in module[files_dirs]: - module[files_dirs].extend(files) + module[files_dirs].update(files) break else: paths["modules"][module_name].append( - {"type": module_type, "files": [], "dirs": [], files_dirs: list(files)} + {"type": module_type, "files": set(), "dirs": set(), files_dirs: set(files)} ) @@ -348,7 +347,7 @@ def classify_paths( "docs": [], # to be used once there is upstream way to recognize READMEs "licenses": [], # %license entries parsed from dist-info METADATA file }, - "lang": {}, # %lang entries: [module_name or None][language_code] lists of .mo files + "lang": {}, # %lang entries: [module_name or None][language_code] lists of .mo and .qm files "modules": defaultdict(list), # each importable module (directory, .py, .so) "module_names": set(), # qualified names of each importable module ("foo.bar.baz") "other": {"files": []}, # regular %file entries we could not parse :( @@ -357,7 +356,7 @@ def classify_paths( license_files = metadata.get_all('License-File') license_directory = distinfo / 'licenses' # See PEP 639 "Root License Directory" # setuptools was the first known build backend to implement License-File. - # Unfortunately they don't put licenses to the license directory (yet): + # Unfortunately they didn't put licenses to the license directory in setuptools<78: # https://github.com/pypa/setuptools/issues/3596 # Hence, we check licenses in both licenses and dist-info license_directories = (license_directory, distinfo) @@ -396,6 +395,9 @@ def classify_paths( # extension modules can have 2 suffixes name = BuildrootPath(path.stem).stem add_file_to_module(paths, name, "extension", "files", path) + elif path.suffix == ".pyi": + name = path.stem + add_file_to_module(paths, name, "stub", "files", path) elif path.suffix == ".py": name = path.stem # we add the .pyc files, but not top-level __pycache__ @@ -412,7 +414,7 @@ def classify_paths( for parent in list(path.parents)[:index]: # no direct slice until Python 3.10 add_file_to_module(paths, module_dir.name, "package", "dirs", parent) is_lang = False - if path.suffix == ".mo": + if path.suffix == ".mo" or path.suffix == ".qm": is_lang = add_lang_to_module(paths, module_dir.name, path) if not is_lang: if path.suffix == ".py": @@ -425,7 +427,7 @@ def classify_paths( add_file_to_module(paths, module_dir.name, "package", "files", path) break else: - if path.suffix == ".mo": + if path.suffix == ".mo" or path.suffix == ".qm": add_lang_to_module(paths, None, path) or paths["other"]["files"].append(path) else: path = normalize_manpage_filename(prefix, path) @@ -614,8 +616,10 @@ def generate_file_list(paths_dict, module_globs, include_others=False): # Users using '*' don't care about the files in the package, so it's ok # not to fail the build when no modules are detected # There can be legitimate reasons to create a package without Python modules - if not modules and fnmatch.fnmatchcase("", glob): - done_globs.add(glob) + if not modules: + for glob in module_globs: + if fnmatch.fnmatchcase("", glob): + done_globs.add(glob) missed = module_globs - done_globs if missed: @@ -782,7 +786,7 @@ def dist_metadata(buildroot, record_path): return dist.metadata -def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, assert_license, varargs): +def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, assert_license, allow_no_modules, varargs): """ Takes arguments from the %{pyproject_save_files} macro @@ -797,6 +801,15 @@ def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_versio sitedirs = sorted({sitelib, sitearch}) globs, include_auto = parse_varargs(varargs) + if not globs and not allow_no_modules: + raise ValueError( + "At least one module glob needs to be provided to %pyproject_save_files. " + "Alternatively, use -M to indicate no Python modules should be saved." + ) + if globs and allow_no_modules: + raise ValueError( + "%pyproject_save_files -M cannot be used together with module globs." + ) parsed_records = load_parsed_record(pyproject_record) final_file_list = [] @@ -840,6 +853,7 @@ def main(cli_args): cli_args.pyproject_record, cli_args.prefix, cli_args.assert_license, + cli_args.allow_no_modules, cli_args.varargs, ) @@ -853,7 +867,7 @@ def argparser(): prog="%pyproject_save_files", add_help=False, # custom usage to add +auto - usage="%(prog)s [-l|-L] MODULE_GLOB [MODULE_GLOB ...] [+auto]", + usage="%(prog)s [-l|-L] MODULE_GLOB|-M [MODULE_GLOB ...] [+auto]", ) parser.add_argument( '--help', action='help', @@ -878,7 +892,11 @@ def argparser(): help="Don't fail when no License-File (PEP 639) is found (the default).", ) parser.add_argument( - "varargs", nargs="+", metavar="MODULE_GLOB", + "-M", "--allow-no-modules", action="store_true", default=False, + help="Don't fail when no globs are provided, only include non-modules data in the generated filelist.", + ) + parser.add_argument( + "varargs", nargs="*", metavar="MODULE_GLOB", help="Shell-like glob matching top-level module names to save into %%{pyproject_files}", ) return parser diff --git a/SOURCES/pyproject_save_files_test_data.yaml b/SOURCES/pyproject_save_files_test_data.yaml index a3dd24f..82d47ac 100644 --- a/SOURCES/pyproject_save_files_test_data.yaml +++ b/SOURCES/pyproject_save_files_test_data.yaml @@ -213,6 +213,10 @@ classified: - /usr/lib/python3.7/site-packages/__pycache__/tldr.cpython-37{,.opt-?}.pyc dirs: [] type: script + - files: + - /usr/lib/python3.7/site-packages/tldr.pyi + dirs: [] + type: stub other: files: - /usr/bin/tldr @@ -7572,6 +7576,7 @@ dumped: - /usr/lib/python3.7/site-packages/tldr-0.5.dist-info/WHEEL - /usr/lib/python3.7/site-packages/tldr-0.5.dist-info/top_level.txt - /usr/lib/python3.7/site-packages/tldr.py + - /usr/lib/python3.7/site-packages/tldr.pyi - /usr/share/man/man1/tldr* - - tldr - - mistune @@ -15665,6 +15670,7 @@ records: tldr-0.5.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92 tldr-0.5.dist-info/top_level.txt,sha256=xHSI9WD6Y-_hONbi2b_9RIn9oiO7RBGHU3A8geJq3mI,5 tldr.py,sha256=aJlA3tIz4QYYy8e7DZUhPyLCqTwnfFjA7Nubwm9bPe0,12779 + tldr.pyi,sha256=GxQ4ZGLPQObN92QW_Hb8IJPEuYINNn186FjrRovM09g,13 mistune: path: /usr/lib64/python3.7/site-packages/mistune-0.8.3.dist-info/RECORD diff --git a/SOURCES/test_RECORD b/SOURCES/test_RECORD index e917ce9..cfc31d7 100644 --- a/SOURCES/test_RECORD +++ b/SOURCES/test_RECORD @@ -9,3 +9,4 @@ tldr-0.5.dist-info/RECORD,, tldr-0.5.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92 tldr-0.5.dist-info/top_level.txt,sha256=xHSI9WD6Y-_hONbi2b_9RIn9oiO7RBGHU3A8geJq3mI,5 tldr.py,sha256=aJlA3tIz4QYYy8e7DZUhPyLCqTwnfFjA7Nubwm9bPe0,12779 +tldr.pyi,sha256=GxQ4ZGLPQObN92QW_Hb8IJPEuYINNn186FjrRovM09g,13 diff --git a/SOURCES/test_pyproject_buildrequires.py b/SOURCES/test_pyproject_buildrequires.py index 815c916..d6f9852 100644 --- a/SOURCES/test_pyproject_buildrequires.py +++ b/SOURCES/test_pyproject_buildrequires.py @@ -102,15 +102,15 @@ def test_data(case_name, capfd, tmp_path, monkeypatch): assert 'expected' in case or 'stderr_contains' in case out, err = capfd.readouterr() - dependencies = output.read_text() + dependencies = sorted(output.read_text().splitlines()) if 'expected' in case: expected = case['expected'] if isinstance(expected, list): # at least one of them needs to match - assert dependencies in expected + assert dependencies in [sorted(e.splitlines()) for e in expected] else: - assert dependencies == expected + assert dependencies == sorted(expected.splitlines()) # stderr_contains may be a string or list of strings stderr_contains = case.get('stderr_contains') diff --git a/SOURCES/test_pyproject_save_files.py b/SOURCES/test_pyproject_save_files.py index 46aa230..5ee5893 100755 --- a/SOURCES/test_pyproject_save_files.py +++ b/SOURCES/test_pyproject_save_files.py @@ -103,6 +103,7 @@ def test_parse_record_tldr(): str(SITELIB / "tldr-0.5.dist-info/WHEEL"), str(SITELIB / "tldr-0.5.dist-info/top_level.txt"), str(SITELIB / "tldr.py"), + str(SITELIB / "tldr.pyi"), ] assert output == expected diff --git a/SPECS/pyproject-rpm-macros.spec b/SPECS/pyproject-rpm-macros.spec index da44d25..1b127da 100644 --- a/SPECS/pyproject-rpm-macros.spec +++ b/SPECS/pyproject-rpm-macros.spec @@ -14,7 +14,7 @@ License: MIT # Increment Y and reset Z when new macros or features are added # Increment Z when this is a bugfix or a cosmetic change # Dropping support for EOL Fedoras is *not* considered a breaking change -Version: 1.16.2 +Version: 1.18.5 Release: 1%{?dist} # Macro files @@ -59,9 +59,9 @@ BuildRequires: python3dist(packaging) BuildRequires: python3dist(pip) BuildRequires: python3dist(setuptools) %if %{with tox_tests} -BuildRequires: python3dist(tox-current-env) >= 0.0.6 +BuildRequires: python3dist(tox-current-env) >= 0.0.16 %endif -BuildRequires: python3dist(wheel) +BuildRequires: (python3dist(wheel) if python3dist(setuptools) < 71) BuildRequires: (python3dist(tomli) if python3 < 3.11) # RHEL 9: We also run pytest with Python 3.11 and 3.12 @@ -97,8 +97,9 @@ Requires: /usr/bin/sed # It has been introduced in RPM 4.15 (4.14.90 is the alpha of 4.15). # What we need is rpmlib(DynamicBuildRequires), but that is impossible to (Build)Require. # Also, we need to avoid 4.19.90..4.19.91-7 due to rhbz#2284187 -Requires: ((rpm-build >= 4.14.90 with (rpm-build < 4.19.90 or rpm-build >= 4.19.91-8)) if rpm-build) -BuildRequires: rpm-build >= 4.14.90 +# Also, we need 4.16.1.3-37 or newer to get RHEL-67161 +Requires: ((rpm-build >= 4.16.1.3-37 with (rpm-build < 4.19.90 or rpm-build >= 4.19.91-8)) if rpm-build) +BuildRequires: rpm-build >= 4.16.1.3-37 %description These macros allow projects that follow the Python packaging specifications @@ -118,7 +119,7 @@ which only work with setup.py. %package -n pyproject-srpm-macros Summary: Minimal implementation of %%pyproject_buildrequires Requires: (pyproject-rpm-macros = %{?epoch:%{epoch}:}%{version}-%{release} if pyproject-rpm-macros) -Requires: (rpm-build >= 4.14.90 if rpm-build) +Requires: (rpm-build >= 4.16.1.3-37 if rpm-build) %description -n pyproject-srpm-macros This package contains a minimal implementation of %%pyproject_buildrequires. @@ -152,15 +153,9 @@ install -pm 644 pyproject_construct_toxenv.py %{buildroot}%{_rpmconfigdir}/redha install -pm 644 pyproject_requirements_txt.py %{buildroot}%{_rpmconfigdir}/redhat/ install -pm 644 pyproject_wheel.py %{buildroot}%{_rpmconfigdir}/redhat/ -%check -# assert the two signatures of %%pyproject_buildrequires match exactly -signature1="$(grep '^%%pyproject_buildrequires' macros.pyproject | cut -d' ' -f1)" -signature2="$(grep '^%%pyproject_buildrequires' macros.aaa-pyproject-srpm | cut -d' ' -f1)" -test "$signature1" == "$signature2" -# but also assert we are not comparing empty strings -test "$signature1" != "" %if %{with tests} +%check export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856356 %pytest -vv --doctest-modules %{?with_pytest_xdist:-n auto} %{!?with_tox_tests:-k "not tox"} @@ -196,6 +191,51 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856 %changelog +* Thu Oct 16 2025 Miro Hrončok - 1.18.5-1 +- %%pyproject_extras_subpkg: Only %%ghost the dist-info directory, not the content +- That way, accidentally unpackaged files within are reported as errors +- %%pyproject_save_files: Also save top level typing stub files (.pyi) + +* Mon Sep 01 2025 Miro Hrončok - 1.18.4-1 +- Don't exit from pyproject-srpm-macros implementation of %%pyproject_buildrequires +- Fixes: rhbz#2391290 +- On RPM 4.20+ don't put pyproject-macros-specific files in %%buildsubdir +- Works around https://github.com/rpm-software-management/rpm/issues/3890 +- Speed %%pyproject_save_files up significantly + +* Fri Jul 25 2025 Fedora Release Engineering - 1.18.3-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_43_Mass_Rebuild + +* Fri Jul 11 2025 Miro Hrončok - 1.18.3-1 +- %%pyproject_buildrequires: Do not generate BuildRequires from Requires core metadata fields +- That field is deprecated and should include importable module names, not distribution packages +- Related: rhbz#2378463 + +* Mon May 19 2025 Maxwell G - 1.18.2-1 +- Fix handling of config_settings in %%pyproject_buildrequires + +* Fri Mar 21 2025 Miro Hrončok - 1.18.1-1 +- Fix reverted conditional in %%pyproject_buildrequires -t/-e Fedora version comparison + +* Tue Mar 11 2025 Miro Hrončok - 1.18.0-1 +- Make %%pyproject_buildrequires -t/-e and %%tox fail when no suitable tox configuration exists +- The %%pyproject_buildrequires -t/-e case is temporarily allowed on Fedora 40-42 +- Requires tox-current-env >= 0.0.16 + +* Thu Jan 30 2025 Miro Hrončok - 1.17.0-1 +- Add the -M flag to %%pyproject_save_files +- The flag can be used to indicate no Python modules should be saved + +* Sat Jan 18 2025 Fedora Release Engineering - 1.16.4-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_42_Mass_Rebuild + +* Tue Dec 03 2024 Miro Hrončok - 1.16.4-1 +- Deprecate the provisional -w flag for %%pyproject_buildrequires + +* Tue Dec 03 2024 Miro Hrončok - 1.16.3-1 +- Accept arbitrary options from %%pyproject_buildrequires in pyproject-srpm-macros +- This will make future additions smoother + * Wed Nov 13 2024 Miro Hrončok - 1.16.2-1 - Fix one remaining test for setuptools 70+