Commit Graph

63 Commits

Author SHA1 Message Date
Lumir Balhar
e828b8291a Remove .dist-info directory at the end of %pyproject_buildrequires
An incomplete .dist-info directory in $PWD can confuse tests in %check.
For example, virtualenv uses importlib.metadata to load its
entry points and it does not work when it finds a virtualenv...dist-info without them.

Related: rhbz#2208971
2023-05-30 10:37:26 +02:00
Lumir Balhar
ea94129e35 Improve detection of lang files
Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
6923c14ba5 %check: Use %pytest -n auto for faster build
The speedup is tiny but noticeable (pytest took 8.49s, now 5.87s on my 8-core workstation).

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
ecf0a140a8 %pyproject_buildrequires: Avoid leaking stdout from subprocesses
When the build backend prints to stdout via non-Python means,
for example when a setup.py script calls a verbose program via os.system(),
the output leaked to stdout of %pyproject_buildrequires was treated as generated BuildRequires.

Fore example, if the setup.py script has:

    rv = os.system('/usr/bin/patch -N -p3 -d build/lib < lib/py-lmdb/env-copy-txn.patch')

(From https://github.com/jnwatson/py-lmdb/blob/py-lmdb_1.0.0/setup.py#L117)

The stdout of /usr/bin/patch leaked to stdout of %pyproject_buildrequires:

    [lmdb-1.0.0]$ /usr/bin/python3 -Bs /usr/lib/rpm/redhat/pyproject_buildrequires.py --python3_pkgversion 3 2>/dev/null
    python3dist(setuptools) >= 40.8
    python3dist(wheel)
    patching file lmdb.h
    patching file mdb.c
    python3dist(wheel)
    patching file lmdb.h
    patching file mdb.c

This resulted in DNF errors like this:

    No matching package to install: 'lmdb.h'
    No matching package to install: 'mdb.c'
    No matching package to install: 'patching'

Moreover, it resulted in bogus BuildRequires that may have existed (e.g. "file").

By replacing the usage of contextlib.redirect_stdout
(which only redirects Python's sys.stdout)
with a custom context manager that captures stdout on file descriptor level
(in addition to Python's sys.stdout),
we avoid this leak.

File descriptor magic heavily inspired by the capfd pytest fixture.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
159b22e742 Add pyproject-srpm-macros with a minimal %pyproject_buildrequires macro
pyproject-srpm-macros is intended to be installed in the default buildroot.

That way, no explicit BuildRequires for pyproject-rpm-macros are required,
as long as %pyproject_buildrequires is used in %generate_buildrequires.

When only pyproject-srpm-macros is installed, the minimal implementation of
%pyproject_buildrequires generates a dependency on pyproject-rpm-macros.
When pyproject-rpm-macros is installed, it overrides the implementation
of %pyproject_buildrequires with the full one.

Note that in Fedora, pyproject-rpm-macros is required by python3-devel,
but not in RHEL.
This allows us to keep pyproject-rpm-macros in the RHEL CRB repository.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
df1ddd0ccd Adjust %pyproject_buildrequires tests for tox 4
Fixes: https://bugzilla.redhat.com/2160687

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
9801a61106 Use %py3_test_envvars in %tox when available
The new %py3_test_envvars macro was added
to remove duplication of environment variables used in %check.

We reuse it in %tox to gain support e.g. for PYTEST_XDIST_AUTO_NUM_WORKERS.

However, we keep support for platforms where the macro is not yet available,
not to be forced to backport %py3_test_envvars everywhere right away.

Technically, this should change little, but it sets CFLAGS and LDFLAGS now,
hence a new Y version.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Karolina Surma
14b3f4b9c3 Filter discovered modules to match the given %pyproject_save_files globs
When certain modules are deliberately not included into the built RPM,
they shouldn't be listed in the list of qualified module names which are
used by %pyproject_check_import to test importability of the
distribution.

Related: rhbz#2168193
2023-02-13 16:42:57 +01:00
Miro Hrončok
064e6f283c %pyproject_save_files: Support License-Files installed into the *Root License Directory* from PEP 369
Files still need to be marked as License-File to be considered %license,
but if their path in METADATA is specified relative to dist-info/licenses,
they are correctly recognised.

This makes License-Files specified by hatchling 1.9.0+ marked as %license.

Related: rhbz#2168193
2023-02-13 16:42:39 +01:00
Otto Liljalaakso
e8135abc2f Fix typo in function name 'evaluate_all_environamnets'
The function is only used internally in these macros,
so name change should not break any users.

Related: rhbz#2168193
2023-02-13 16:27:15 +01:00
Karolina Surma
50012935df Don't fail %pyproject_save_files '*' if no modules are present
Users invoking %pyproject_save_files with glob: '*' don't care about the
files in the Python package, hence it shouldn't error when no modules
are detected.
There may be legitimate reasons to create a package without Python
modules in it, hence we shouldn't be blocking this possibility.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
5463b58b32 Preserve mtimes
Note that the mtimes are artificially created by git,
but at least this way they are consistent when a single SRPM is rebuilt.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
6000eccbc6 Use the %pytest macro
This macro is defined in python3-rpm-macros, so there's no problem using
it here. In fact, let's BuildRequire the macros we use.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
6bc995e61a Allign specfile to 16 characters
Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Miro Hrončok
589556411f %pyproject_buildrequires: Support Package information on ELF objects
When extension modules are built in %pyproject_buildrequires,
we need to create the package note file.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2097535

This is tested via python-ldap -- %pyproject_buildrequires -w fails without the fix.

Neither python-markupsafe nor python-mistune can be used as a test
because they only warn when the extension module cannot be built
because they fallback to pure Python.

Related: rhbz#2117571
2022-08-11 13:05:50 +02:00
Benjamin A. Beasley
e9ea68926b %pyproject_build_lib: support setuptools 62.1.0 and later
Related: rhbz#2117571
2022-08-11 13:05:34 +02:00
Miro Hrončok
0207225c97 Fix bogus date in changelog
Related: rhbz#2117571
2022-08-11 12:57:40 +02:00
Owen W. Taylor
681cc65e49 %pyproject_install: pass %{_prefix} explicitly to pip install
This makes things work for %{_prefix} other than user, when combined
with a change in python-rpm-macros to the definition of python3_sitelib/python3_sitearch.

https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/KEQMMNJ4HTTHSQLK6P4DJJTVPA36SS3W/

Related: rhbz#2117571

Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
2022-08-11 12:57:19 +02:00
Miro Hrončok
07598e61d1 Use tomllib from the standard library on Python 3.11+
Related: rhbz#2117571
2022-08-11 12:56:41 +02:00
Miro Hrončok
235e0c94a6 Allow building wheels in %pyproject_buildrequires to support other build backends
The hook is optional, see https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel

> If a build frontend needs this information and the method is not defined,
> it should call build_wheel and look at the resulting metadata directly.

This is not yet automatically detected because the feature is provisional.
Use `%pyproject_buildrequires -w` to opt-in.

Resolves: rhbz#2060109
2022-05-06 12:19:26 +02:00
Miro Hrončok
d7bc2a357f %pyproject_save_files: Support nested directories in dist-info
Related: rhbz#2060109
2022-05-06 12:19:08 +02:00
Miro Hrončok
be050ad171 Prefix paths of intermediate files (such as %{pyproject_files}) with NVRA
Apparently, when you repeatedly run `rpmbuild -ba`, files in %_builddir are not cleaned.
This way, we at least make sure the files are unique between different NVRAs,
so 2 unrelated builds don't share the files between each other.

Keeping files contained in the build subdirectory is the more common way of doing this,
but we cannot technically do that, because we don't know what's it gonna be (before %prep).

Should be backwards compatible, as we only modify underscore-prefixed macros and %{pyproject_files},
where the exact value should not matter to the packagers.

Resolves: rhbz#2060109
2022-03-23 15:13:05 +01:00
Miro Hrončok
71c4eaabf4 Release final version 1.0.0
There are no code changes.

Resolves: rhbz#2060109
2022-03-02 18:33:50 +01:00
Lumir Balhar
17cd08d6a0 Updated compatibility with tox4
Related: rhbz#2060109
2022-03-02 18:33:48 +01:00
Miro Hrončok
127a259768 Release version 1.0.0, first release candidate
Related: rhbz#1950291
2022-01-26 12:58:21 +01:00
Miro Hrončok
71dcd3ad74 %pyproject_buildrequires: Generate BuildRequires for this package
This package is already installed -- otherwise the macro would not even exist.

However, since python3-devel has started to Require pyproject-rpm-macros,
it is no longer possible to use `repoquery --whatrequires pyproject-rpm-macros`
to get a reliable list of packages that use the macros.

This was, all packages that use %pyproject_buildrequires will BuildRequire the macros explicitly.

(In the future, we could even include a stub version of %pyproject_buildrequires
in pyproject-srpm-macros (always installed in the buildroot),
that only echoes this package,
so packagers would not need to manually BuildRequire anything at all.)

Related: rhbz#1950291
2022-01-26 11:44:53 +01:00
Karolina Surma
2be56ab379 Include compressed manpages correctly in the RPM package
Compressed manpages have different extension than those listed in the RECORD file,
so they were not recognized when %%pyproject_save_files '+auto' flag
was provided.
To enable the path recognition, if the manpage extension matches the one
listed in brp-compres, the extension is removed, and an asterisk is now added
to the manpages filenames.
Source: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages

Related: rhbz#1950291
2022-01-26 11:44:41 +01:00
Miro Hrončok
93cf48f615 %pyproject_buildrequires: Make -r (include runtime) the default, use -R to opt-out
See the proposal:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/2R6NKELTHAWE6PI3CCZBVW5PMGO5VPDG/

 - -N now implies -R
 - the macro still guards against -Nr and now also against -Rr

Related: rhbz#1950291
2022-01-26 11:41:43 +01:00
Gordon Messmer
2abd0036ea Handle legacy version specifiers that would previously raise exceptions.
Related: rhbz#1950291
2022-01-26 11:36:40 +01:00
Miro Hrončok
5513c410bf Define provisional %pyproject_build_lib
Related: rhbz#1950291
2022-01-26 11:35:49 +01:00
Miro Hrončok
f768e230c1 Require sed, as we use it
Related: rhbz#1950291
2022-01-26 11:34:03 +01:00
Gordon Messmer
f8c2fede5b Sync dependency conversion with upstream pyreq2rpm.
Improve handling of > operator, preventing post-release from satisfying most rpm requirements.
Improve handling of < operator, preventing pre-release from satisfying rpm requirement.
Improve handling of != operator with prefix matching, preventing pre-release from satisfying rpm requirements.

Related: rhbz#1950291
2022-01-26 11:32:17 +01:00
Karolina Surma
d427052f40 Introduce %%pyproject_check_import
%%pyproject_save_files newly saves also a list of importable modules.
The list is used by %%pyproject_check_import to invoke the import test
on each module name.
%%pyproject_check_import accepts two options:
-t: filter only top-level modules
-e: exclude module names matching the given glob from the import check

Related: rhbz#1950291
2022-01-26 11:31:17 +01:00
Miro Hrončok
655f6dda0e %pyproject_buildrequires: Accept installed pre-releases for all requirements
Related: rhbz#1950291
2022-01-26 11:27:08 +01:00
Miro Hrončok
0d8d5ecd2e %pyproject_save_files: Add a workaround error for spaces and [brackets]
See the added links for details.
We cannot fix this for now,
so we error out early instead of producing a broken filelist.
Related: rhbz#1950291
2022-01-26 11:18:45 +01:00
Miro Hrončok
41fc715cc4 %pyproject_save_files: Expand the namespace error message, also display it with /
Related: rhbz#1950291
2022-01-26 11:11:52 +01:00
Tomas Hrnciar
bc3b3a778a Automatically detect LICENSE files and mark them with %license macro
Related: rhbz#1950291
2021-08-10 10:52:23 +02:00
Miro Hrončok
309172f9a8 Rename %_pyproject_ghost_distinfo and %_pyproject_record to indicate they are private
Related: rhbz#1950291
2021-08-10 10:52:23 +02:00
Petr Viktorin
a3a1caf32a Split requirements.txt parsing to its own module; test & improve it
Related: rhbz#1950291
2021-08-10 10:52:23 +02:00
Miro Hrončok
f190b5b225 %pyproject_buildrequires now fails when it encounters an invalid requirement
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1983053

Related: rhbz#1950291
2021-08-10 10:52:23 +02:00
Fedora Release Engineering
9fb8a6bd2b - Rebuilt for https://fedoraproject.org/wiki/Fedora_35_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>

Related: rhbz#1950291
2021-08-10 10:52:23 +02:00
Miro Hrončok
cbbb515b42 Explicitly require the "basic" Python RPM macros
Related: rhbz#1950291
2021-08-10 10:52:21 +02:00
Tomas Hrnciar
d74e61b485 %pyproject_buildrequires: Fallback to setuptools.build_meta:__legacy__ only if setup.py exists
Fixes: rhbz#1976459

Related: rhbz#1950291
2021-08-10 10:52:05 +02:00
Miro Hrončok
7602b9a77a %pyproject_buildrequires: Support x.* versions
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1981558

Related: rhbz#1950291
2021-08-10 10:51:46 +02:00
Miro Hrončok
be3e9d3b71 Escape weird paths generated by %pyproject_save_files
Related: rhbz#1950291
2021-08-10 10:51:23 +02:00
Mohan Boddu
8365fb6c2c Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
Related: rhbz#1991688
Signed-off-by: Mohan Boddu <mboddu@redhat.com>
2021-08-10 00:06:40 +00:00
Tomas Hrnciar
aac0af8401 Generate BuildRequires from file
%pyproject_buildrequires macro now accepts multiple file names to load
additional dependencies from them.

New option -N was added to disable automatical generation of requirements
in case package does not use build system. Option -N cannot be used in
combination with options -r, -e, -t, -x.

Co-authored-by: Miro Hrončok <miro@hroncok.cz>

Related: rhbz#1950291
2021-07-09 17:30:46 +02:00
Miro Hrončok
dd25ce59a6 Don't accidentally treat "~= X.0" requirement as "~= X"
Don't canonicalize the version twice.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1977060

Related: rhbz#1950291
2021-06-30 13:08:48 +02:00
Miro Hrončok
9c60342738 Avoid leaking %{_pyproject_builddir} to pytest collection
Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1935212

Requires a %pytest change: https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/102

Related: rhbz#1950291
2021-06-30 13:08:48 +02:00
Miro Hrončok
71f414ef2c Set %_pyproject_wheeldir and %_pyproject_builddir relative to the source tree, not $PWD
This allows users to do:

    %build
    cd somewhere
    %pyproject_wheel
    cd -
    cd somewhere_else
    %pyproject_wheel
    cd -

    %install
    %pyproject_install

Without a need to copy paste the wheels to a common location.

This is in fact a breaking change, I'll make sure to adapt the affected packages in Fedora.

Related: rhbz#1950291
2021-06-10 13:49:49 +02:00