Dependencies are recorded to a text file that is catted at the end.
This should prevent subtle bugs like https://bugzilla.redhat.com/2183519 in the future.
An incomplete .dist-info directory in $PWD can confuse tests in %check.
For example, virtualenv uses importlib.metadata to load its
entry points and it does not work when it finds a virtualenv...dist-info without them.
pyproject-srpm-macros is intended to be installed in the default buildroot.
That way, no explicit BuildRequires for pyproject-rpm-macros are required,
as long as %pyproject_buildrequires is used in %generate_buildrequires.
When only pyproject-srpm-macros is installed, the minimal implementation of
%pyproject_buildrequires generates a dependency on pyproject-rpm-macros.
When pyproject-rpm-macros is installed, it overrides the implementation
of %pyproject_buildrequires with the full one.
Note that in Fedora, pyproject-rpm-macros is required by python3-devel,
but not in RHEL.
This allows us to keep pyproject-rpm-macros in the RHEL CRB repository.
The new %py3_test_envvars macro was added
to remove duplication of environment variables used in %check.
We reuse it in %tox to gain support e.g. for PYTEST_XDIST_AUTO_NUM_WORKERS.
However, we keep support for platforms where the macro is not yet available,
not to be forced to backport %py3_test_envvars everywhere right away.
Technically, this should change little, but it sets CFLAGS and LDFLAGS now,
hence a new Y version.
When extension modules are built in %pyproject_buildrequires,
we need to create the package note file.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2097535
This is tested via python-ldap -- %pyproject_buildrequires -w fails without the fix.
Neither python-markupsafe nor python-mistune can be used as a test
because they only warn when the extension module cannot be built
because they fallback to pure Python.
Apparently, when you repeatedly run `rpmbuild -ba`, files in %_builddir are not cleaned.
This way, we at least make sure the files are unique between different NVRAs,
so 2 unrelated builds don't share the files between each other.
Keeping files contained in the build subdirectory is the more common way of doing this,
but we cannot technically do that, because we don't know what's it gonna be (before %prep).
Should be backwards compatible, as we only modify underscore-prefixed macros and %{pyproject_files},
where the exact value should not matter to the packagers.
This package is already installed -- otherwise the macro would not even exist.
However, since python3-devel has started to Require pyproject-rpm-macros,
it is no longer possible to use `repoquery --whatrequires pyproject-rpm-macros`
to get a reliable list of packages that use the macros.
This was, all packages that use %pyproject_buildrequires will BuildRequire the macros explicitly.
(In the future, we could even include a stub version of %pyproject_buildrequires
in pyproject-srpm-macros (always installed in the buildroot),
that only echoes this package,
so packagers would not need to manually BuildRequire anything at all.)
Compressed manpages have different extension than those listed in the RECORD file,
so they were not recognized when %%pyproject_save_files '+auto' flag
was provided.
To enable the path recognition, if the manpage extension matches the one
listed in brp-compres, the extension is removed, and an asterisk is now added
to the manpages filenames.
Source: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages
Fixes: https://bugzilla.redhat.com/2033254
%%pyproject_save_files newly saves also a list of importable modules.
The list is used by %%pyproject_check_import to invoke the import test
on each module name.
%%pyproject_check_import accepts two options:
-t: filter only top-level modules
-e: exclude module names matching the given glob from the import check
%pyproject_buildrequires macro now accepts multiple file names to load
additional dependencies from them.
New option -N was added to disable automatical generation of requirements
in case package does not use build system. Option -N cannot be used in
combination with options -r, -e, -t, -x.
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
This allows users to do:
%build
cd somewhere
%pyproject_wheel
cd -
cd somewhere_else
%pyproject_wheel
cd -
%install
%pyproject_install
Without a need to copy paste the wheels to a common location.
This is in fact a breaking change, I'll make sure to adapt the affected packages in Fedora.
The macro already checks if pyproject.toml exists and echoes the dependency
on python3dist(toml) early. This adds an else branch to echo the default backend.
For projects without pyproject.toml, the number of installation rounds
is reduced. Previously:
1. (python3-devel +) pip + packaging
2. setuptools + wheel
3. ...
Now:
1. (python3-devel +) pip + packaging + setuptools + wheel
2. ...
This duplicates the information about the default build backend,
because the script still needs to handle projects with pyproject.toml without
an explicit build backend option.
Hence, the script was not adapted (except a comment).
The macro checks if pyproject.toml exists and echoes the dependency early.
For projects with pyproject.toml, this saves one installation round.
Previously, the installation steps by %generate_buildrequires were:
1. (python3-devel +) pip + packaging
2. toml
3. parsed dependencies from pyproject.toml
4. ...
Now they are:
1. (python3-devel +) pip + packaging + toml
2. parsed dependencies from pyproject.toml
3. ...
For projects without pyproject.toml, the number of rounds remains the same:
1. (python3-devel +) pip + packaging
2. setuptools + wheel
3. ...
This is also more consistent:
The Python script now only outputs dependencies of the probed project,
it no longer partially outputs dependencies for itself.
According to PEP 627, the RECORD file is optional and
doesn't make sense to keep it for system packages. Moreover,
its absence should indicate to other tools like pip that
they should not touch such packages.
Now, we process content of all RECORD files to one
pyproject-record (JSON) which is then used in
%pyproject_save_files. That way, we can remove the original
files in %pyproject_install and keep their content for
later.
PEP 627: https://www.python.org/dev/peps/pep-0627/#optional-record-file
Pros:
- projects without pyproject.toml will have 1 less dependency
- toml will be buildable with pyproject-rpm-macros out of the box
- easier bootstrap sequence (in theory)
Cons:
- projects with pyproject.toml will have 1 more %generate_buildrequires round
In %pyproject_buildrequires, don't run python with -I but -s.
This allows projects used by the script itself, such as packaging or toml,
to be packaged using the macros, using "self" -- i.e. the code from $PWD.
%pyproject_install currently has:
`pathfix%{python3_version}.py -pni "%{__python3}" -k%{?py3_shbang_opts: -a%{py3_shbang_opts_nodash}} %{buildroot}%{_bindir}/*`
We should replace it with `%py3_shebang_fix %{buildroot}%{_bindir}/*` which expands to:
~~~~
if [ -f /usr/bin/pathfix%{python3_version}.py ]; then
pathfix=/usr/bin/pathfix%{python3_version}.py
else
# older versions of Python don't have it and must BR /usr/bin/pathfix.py from python3-devel explicitly
pathfix=/usr/bin/pathfix.py
fi
$pathfix -pni %{__python3} -k%{?py3_shebang_flags:a %py3_shebang_flags}} %{buildroot}%{_bindir}/*
~~~~
Mainly so that we:
- switch to %py3_shebang_flags
- have only one place to fix the invocation
Fixes: rhbz#1868347
There is a slight problem when reporting that a dependency with extra is satisfied.
In fact, we only check the "base" dependency.
This can lead to a problem when a dependency is wrongly assumed as present
and the script proceeds to the "next stage" without restarting --
if the next stage tries to use (import) the missing dependency,
the script would crash.
However, that might be a very unlikely set of events and if such case ever happens,
we'll workaround it or fix it.
Tox calls socket.getfqdn() and that call does a DNS query.
In mock with disabled networking, it takes a minute until that times out.
When a spec file uses %pyproject_buildrequires -t and %tox, it is a 3 minute delay.
Since 3.17, tox does not call socket.getfqdn() when HOSTNAME variable is set to a value:
https://github.com/tox-dev/tox/pull/1616
The value is only used in result log, so setting it to "rpmbuild"
actually makes the logs more reproducible as well.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1856356
when tox is used in %pyproject_buildrequires -t or %tox.
With changes in PEP 610 there is new file direct_url.json created, since it is not useful
for us we prevent it's creation. This commit changes %pyproject_install macro to install wheel using
name instead of path.
This commit also includes new test to check if file direct_url.json wasn't created.
https://discuss.python.org/t/pep-610-usage-guidelines-for-linux-distributions/4012
This macro save generates file section to %pyproject_files. It should
simplify %files section and allow to build by some automatic machinery
Supposed use case in Fedora:
%install
%pyproject_install
%pyproject_save_files requests _requests
%files -n python3-requests -f %{pyproject_files}
%doc README.rst
%license LICENSE
Automatic build of arbitrary packages (e.g. in Copr):
%install
%pyproject_install
%pyproject_save_files * +bindir // save all modules with executables
%files -n python3-requests -f %{pyproject_files}
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
