Commit Graph

118 Commits

Author SHA1 Message Date
Miro Hrončok
e9b491535c Redirect stdout to stderr via Shell
Dependencies are recorded to a text file that is catted at the end.

This should prevent subtle bugs like https://bugzilla.redhat.com/2183519 in the future.

Related: rhbz#2208971
2023-05-30 10:43:35 +02:00
Miro Hrončok
000ca45f91 Remove duplicate import
Related: rhbz#2208971
2023-05-30 10:43:25 +02:00
Miro Hrončok
5ddb72e141 Acknowledge the license is MIT even in SPDX
$ askalono identify LICENSE
    License: MIT (original text)
    Score: 1.000

Related: rhbz#2208971
2023-05-30 10:39:13 +02:00
Lumir Balhar
e828b8291a Remove .dist-info directory at the end of %pyproject_buildrequires
An incomplete .dist-info directory in $PWD can confuse tests in %check.
For example, virtualenv uses importlib.metadata to load its
entry points and it does not work when it finds a virtualenv...dist-info without them.

Related: rhbz#2208971
2023-05-30 10:37:26 +02:00
Miro Hrončok
d46b0e5b01 CI: Unskip 1 skipped package and update comments for the remaining 2
Packages were introduced to EPEL9 in the meantime.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Lumir Balhar
ea94129e35 Improve detection of lang files
Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
1a404c23b3 CI: Adjust spec conditionals for F35 EOL
Also, use %elif instead of a nested %if.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
6923c14ba5 %check: Use %pytest -n auto for faster build
The speedup is tiny but noticeable (pytest took 8.49s, now 5.87s on my 8-core workstation).

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
ecf0a140a8 %pyproject_buildrequires: Avoid leaking stdout from subprocesses
When the build backend prints to stdout via non-Python means,
for example when a setup.py script calls a verbose program via os.system(),
the output leaked to stdout of %pyproject_buildrequires was treated as generated BuildRequires.

Fore example, if the setup.py script has:

    rv = os.system('/usr/bin/patch -N -p3 -d build/lib < lib/py-lmdb/env-copy-txn.patch')

(From https://github.com/jnwatson/py-lmdb/blob/py-lmdb_1.0.0/setup.py#L117)

The stdout of /usr/bin/patch leaked to stdout of %pyproject_buildrequires:

    [lmdb-1.0.0]$ /usr/bin/python3 -Bs /usr/lib/rpm/redhat/pyproject_buildrequires.py --python3_pkgversion 3 2>/dev/null
    python3dist(setuptools) >= 40.8
    python3dist(wheel)
    patching file lmdb.h
    patching file mdb.c
    python3dist(wheel)
    patching file lmdb.h
    patching file mdb.c

This resulted in DNF errors like this:

    No matching package to install: 'lmdb.h'
    No matching package to install: 'mdb.c'
    No matching package to install: 'patching'

Moreover, it resulted in bogus BuildRequires that may have existed (e.g. "file").

By replacing the usage of contextlib.redirect_stdout
(which only redirects Python's sys.stdout)
with a custom context manager that captures stdout on file descriptor level
(in addition to Python's sys.stdout),
we avoid this leak.

File descriptor magic heavily inspired by the capfd pytest fixture.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
159b22e742 Add pyproject-srpm-macros with a minimal %pyproject_buildrequires macro
pyproject-srpm-macros is intended to be installed in the default buildroot.

That way, no explicit BuildRequires for pyproject-rpm-macros are required,
as long as %pyproject_buildrequires is used in %generate_buildrequires.

When only pyproject-srpm-macros is installed, the minimal implementation of
%pyproject_buildrequires generates a dependency on pyproject-rpm-macros.
When pyproject-rpm-macros is installed, it overrides the implementation
of %pyproject_buildrequires with the full one.

Note that in Fedora, pyproject-rpm-macros is required by python3-devel,
but not in RHEL.
This allows us to keep pyproject-rpm-macros in the RHEL CRB repository.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
251f8e97c9 CI specs: Never use ! ... as a check (followup)
This is a followup of 6015300d34

This particular instance was not discovered before due to the macro.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
eb8a225689 CI: Adjust %toxenv tests for tox 4
Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
a6c5229070 CI: Manually BR pkg_resources from dns-lexicon to fix FTBFS
setuptools (pkg_resources) were transitively pulled in by tox

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
df1ddd0ccd Adjust %pyproject_buildrequires tests for tox 4
Fixes: https://bugzilla.redhat.com/2160687

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
81dd6d1601 CI: Keep patches near the specs to ease using the script from other dist-gits
That way, if a package (such as python-tox) uses it,
the patches in dist-git are available.
In the future, it also allows us to add patches without URLs to our tested spec files.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Ralf Ertzinger
05ff98103f Remove dead code in pyproject_requirements_txt.py
This removes some code from pyproject_requirements_txt.py that serves no
purpose (assigns to a variable that is never used).

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
9801a61106 Use %py3_test_envvars in %tox when available
The new %py3_test_envvars macro was added
to remove duplication of environment variables used in %check.

We reuse it in %tox to gain support e.g. for PYTEST_XDIST_AUTO_NUM_WORKERS.

However, we keep support for platforms where the macro is not yet available,
not to be forced to backport %py3_test_envvars everywhere right away.

Technically, this should change little, but it sets CFLAGS and LDFLAGS now,
hence a new Y version.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
4c1e725938 CI: Adjust setuptools versions to make the CI mockbuilds work on Fedora 36 and 38
Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
fe71860ca5 Reorder %pyproject_buildrequires arguments for a nicer usage string
Before:

    usage: %pyproject_buildrequires [-w] [-R] [-e TOXENVS] [-t] [-x EXTRAS] [-N] [REQUIREMENTS.TXT ...]

After:

    usage: %pyproject_buildrequires [-x EXTRAS] [-t] [-e TOXENVS] [-w] [-R] [-N] [REQUIREMENTS.TXT ...]

The order was determined as:

    0. suppressed options
    1. extras, the easiest way to specify test deps (x)
    2. tox related options (te)
    3. build wheel, as it is provisional (w)
    4. "disablers" (RN)
    5. varargs are always listed last

Previous order was pretty much random.

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Miro Hrončok
9eb042c336 Make %pyproject_buildrequires and %pyproject_save_files argparser errors nicer to macro users
Before:

    %pyproject_buildrequires bogus_arg
    usage: pyproject_buildrequires.py [-h] [-r] [-w] [--wheeldir PATH] [-R]
                                      [-e TOXENVS] [-t] [-x EXTRAS]
                                      [--generate-extras] [-p PYTHON3_PKGVERSION]
                                      [-N]
                                      [requirement_files ...]
    pyproject_buildrequires.py: error: argument requirement_files: can't open 'bogus_arg': ...

    %pyproject_save_files
    ...
    usage: pyproject_save_files.py [-h] --output-files OUTPUT_FILES
                                   --output-modules OUTPUT_MODULES --buildroot
                                   BUILDROOT --sitelib SITELIB --sitearch SITEARCH
                                   --python-version PYTHON_VERSION
                                   --pyproject-record PYPROJECT_RECORD --prefix
                                   PREFIX
                                   varargs [varargs ...]
    pyproject_save_files.py: error: the following arguments are required: varargs

After:

    %pyproject_buildrequires bogus_arg
    usage: %pyproject_buildrequires [-w] [-R] [-e TOXENVS] [-t] [-x EXTRAS] [-N]
                                    [REQUIREMENTS.TXT ...]
    %pyproject_buildrequires: error: argument REQUIREMENTS.TXT: can't open 'bogus_arg': ...

    %pyproject_save_files
    ...
    usage: %pyproject_save_files MODULE_GLOB [MODULE_GLOB ...] [+auto]
    %pyproject_save_files: error: the following arguments are required: MODULE_GLOB

Related: rhbz#2168193
2023-02-13 16:42:58 +01:00
Karolina Surma
14b3f4b9c3 Filter discovered modules to match the given %pyproject_save_files globs
When certain modules are deliberately not included into the built RPM,
they shouldn't be listed in the list of qualified module names which are
used by %pyproject_check_import to test importability of the
distribution.

Related: rhbz#2168193
2023-02-13 16:42:57 +01:00
Miro Hrončok
064e6f283c %pyproject_save_files: Support License-Files installed into the *Root License Directory* from PEP 369
Files still need to be marked as License-File to be considered %license,
but if their path in METADATA is specified relative to dist-info/licenses,
they are correctly recognised.

This makes License-Files specified by hatchling 1.9.0+ marked as %license.

Related: rhbz#2168193
2023-02-13 16:42:39 +01:00
Miro Hrončok
e339a9c835 Docs: Remove one more forgotten %pyproject_buildrequires -r example
Related: rhbz#2168193
2023-02-13 16:27:17 +01:00
Miro Hrončok
80e8038f9b Docs: Only mention -r for good measure, but don't mention it elsewhere
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2123167

Related: rhbz#2168193
2023-02-13 16:27:17 +01:00
Otto Liljalaakso
e8135abc2f Fix typo in function name 'evaluate_all_environamnets'
The function is only used internally in these macros,
so name change should not break any users.

Related: rhbz#2168193
2023-02-13 16:27:15 +01:00
Miro Hrončok
8c8a3ad80b CI: Invoke mock with --isolation=simple to workaround problems with systemd-nspawn
See https://pagure.io/fedora-ci/general/issue/217

Related: rhbz#2117571
2022-08-17 16:57:15 +02:00
Miro Hrončok
a438f38c28 CI: Apply the hack in python-ldap.spec also on RHEL 9
Related: rhbz#2117571
2022-08-16 16:31:33 +02:00
Miro Hrončok
804d83af30 CI: On CentOS Stream, use EPEL Next, not EPEL proper
Also, make sure to never pull from EPEL's Koji repo,
as is also exposes RHEL packages in the metadata, but they error 403.

    [MIRROR] libgcrypt-1.10.0-4.el9_0.x86_64.rpm: Status code: 403 for https://infrastructure.fedoraproject.org/repo/rhel/rhel9/x86_64/rhel-9-for-x86_64-baseos-rpms/Packages/l/libgcrypt-1.10.0-4.el9_0.x86_64.rpm

We just pull from the CentOS Stream Koji repo and EPEL Next Koji repo.
See the mock config for centos-stream+epel-next-9.

Related: rhbz#2117571
2022-08-16 16:31:33 +02:00
Karolina Surma
a81b91497f CI: Patch httpbin to build it in Rawhide with werkzeug
Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Karolina Surma
50012935df Don't fail %pyproject_save_files '*' if no modules are present
Users invoking %pyproject_save_files with glob: '*' don't care about the
files in the Python package, hence it shouldn't error when no modules
are detected.
There may be legitimate reasons to create a package without Python
modules in it, hence we shouldn't be blocking this possibility.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Karolina Surma
71dfcb503a Fix typo in License-File
PEP 639 defines License-File field.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
5463b58b32 Preserve mtimes
Note that the mtimes are artificially created by git,
but at least this way they are consistent when a single SRPM is rebuilt.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
6000eccbc6 Use the %pytest macro
This macro is defined in python3-rpm-macros, so there's no problem using
it here. In fact, let's BuildRequire the macros we use.

Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Maxwell G
6bc995e61a Allign specfile to 16 characters
Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Miro Hrončok
9899e0fa03 Fix CI tests for setuptools 61+
Related: rhbz#2117571
2022-08-11 13:05:53 +02:00
Miro Hrončok
589556411f %pyproject_buildrequires: Support Package information on ELF objects
When extension modules are built in %pyproject_buildrequires,
we need to create the package note file.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2097535

This is tested via python-ldap -- %pyproject_buildrequires -w fails without the fix.

Neither python-markupsafe nor python-mistune can be used as a test
because they only warn when the extension module cannot be built
because they fallback to pure Python.

Related: rhbz#2117571
2022-08-11 13:05:50 +02:00
Benjamin A. Beasley
e9ea68926b %pyproject_build_lib: support setuptools 62.1.0 and later
Related: rhbz#2117571
2022-08-11 13:05:34 +02:00
Miro Hrončok
0207225c97 Fix bogus date in changelog
Related: rhbz#2117571
2022-08-11 12:57:40 +02:00
Owen W. Taylor
681cc65e49 %pyproject_install: pass %{_prefix} explicitly to pip install
This makes things work for %{_prefix} other than user, when combined
with a change in python-rpm-macros to the definition of python3_sitelib/python3_sitearch.

https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/KEQMMNJ4HTTHSQLK6P4DJJTVPA36SS3W/

Related: rhbz#2117571

Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
2022-08-11 12:57:19 +02:00
Miro Hrončok
e082e4a71a CI and docs: Remove Fedora 34 conditionals and mentions
Fedora 34 is end of life.

Related: rhbz#2117571
2022-08-11 12:57:01 +02:00
Miro Hrončok
07598e61d1 Use tomllib from the standard library on Python 3.11+
Related: rhbz#2117571
2022-08-11 12:56:41 +02:00
Miro Hrončok
295b331296 CI: Enable hatchling test for Fedora 35 and EPEL 9
https://bodhi.fedoraproject.org/updates/FEDORA-2022-c0d0bd9fa8
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-b15c2993fe

Related: rhbz#2117571
2022-08-11 12:56:14 +02:00
Miro Hrončok
1dad31bc81 Markdown: Make the [prepare-metadata-for-build-wheel hook] link work in Pagure
(It works as it in on GitLab.)

Related: rhbz#2060109
2022-05-06 12:48:04 +02:00
Miro Hrončok
235e0c94a6 Allow building wheels in %pyproject_buildrequires to support other build backends
The hook is optional, see https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel

> If a build frontend needs this information and the method is not defined,
> it should call build_wheel and look at the resulting metadata directly.

This is not yet automatically detected because the feature is provisional.
Use `%pyproject_buildrequires -w` to opt-in.

Resolves: rhbz#2060109
2022-05-06 12:19:26 +02:00
Miro Hrončok
e92a87dee1 %_pyproject_wheeldir is absolute since 29157c19b0
Related: rhbz#2060109
2022-05-06 12:19:11 +02:00
Miro Hrončok
a02a1ae59f CI: More retrying for dnf, apparently 5 times 20 seconds is not enough
Related: rhbz#2060109
2022-05-06 12:19:08 +02:00
Miro Hrončok
d7bc2a357f %pyproject_save_files: Support nested directories in dist-info
Related: rhbz#2060109
2022-05-06 12:19:08 +02:00
Miro Hrončok
0b9b782c5c CI specs: Never use ! ... as a check
See https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org/thread/TFQGD7CSTD5WVKVT3WDIGF5D6DID5NK6/

Related: rhbz#2060109
2022-05-06 12:18:50 +02:00
Miro Hrončok
44772c3c74 CI: Install EPEL packages via full URL to support running tests on real RHEL
Resolves: rhbz#2065590
2022-03-23 16:46:15 +01:00
Miro Hrončok
a9aa69f463 CI: Move one %files section to test different order
Related: rhbz#2060109
2022-03-23 15:13:17 +01:00