262f6d3bc3
The usage without -r errored anyway, this way instead of forcing the user to add it, we do it ourselves. Machines stealing human's labor, yet again.
234 lines
8.0 KiB
Markdown
234 lines
8.0 KiB
Markdown
pyproject RPM macros
|
|
====================
|
|
|
|
This is a provisional implementation of pyproject RPM macros for Fedora.
|
|
|
|
These macros are useful for packaging Python projects that use the [PEP 517] `pyproject.toml` file, which specifies the package's build dependencies (including the build system, such as setuptools, flit or poetry).
|
|
|
|
|
|
Usage
|
|
-----
|
|
|
|
If your upstream sources include `pyproject.toml` and you want to use these macros, BuildRequire them:
|
|
|
|
BuildRequires: pyproject-rpm-macros
|
|
|
|
This will bring in python3-devel, so you don't need to require python3-devel explicitly.
|
|
|
|
In order to get automatic build dependencies on Fedora 31+, run `%pyproject_buildrequires` in the `%generate_buildrequires` section:
|
|
|
|
%generate_buildrequires
|
|
%pyproject_buildrequires
|
|
|
|
Only build dependencies according to [PEP 517] and [PEP 518] will be added.
|
|
All other build dependencies (such as non-Python libraries or test dependencies) still need to be specified manually.
|
|
|
|
Then, build a wheel in `%build` with `%pyproject_wheel`:
|
|
|
|
%build
|
|
%pyproject_wheel
|
|
|
|
And install the wheel in `%install` with `%pyproject_install`:
|
|
|
|
%install
|
|
%pyproject_install
|
|
|
|
`%pyproject_install` installs all wheels in `$PWD/pyproject-wheeldir/`. If you would like to save wheels somewhere else redefine `%{_pyproject_wheeldir}`.
|
|
|
|
|
|
Adding run-time and test-time dependencies
|
|
------------------------------------------
|
|
|
|
To run tests in the `%check` section, the package's runtime dependencies
|
|
often need to also be included as build requirements.
|
|
If the project's build system supports the [`prepare-metadata-for-build-wheel`
|
|
hook](https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel),
|
|
this can be done using the `-r` flag:
|
|
|
|
%generate_buildrequires
|
|
%pyproject_buildrequires -r
|
|
|
|
For projects that specify test requirements using an [`extra`
|
|
provide](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use),
|
|
these can be added using the `-x` flag.
|
|
For example, if upstream suggests installing test dependencies with
|
|
`pip install mypackage[testing]`, the test deps would be generated by:
|
|
|
|
%generate_buildrequires
|
|
%pyproject_buildrequires -x testing
|
|
|
|
For projects that specify test requirements in their [tox] configuration,
|
|
these can be added using the `-t` flag (default tox environment)
|
|
or the `-e` flag followed by the tox environment.
|
|
The default tox environment (such as `py37` assuming the Fedora's Python version is 3.7)
|
|
is available in the `%{toxenv}` macro.
|
|
For example, if upstream suggests running the tests on Python 3.7 with `tox -e py37`,
|
|
the test deps would be generated by:
|
|
|
|
%generate_buildrequires
|
|
%pyproject_buildrequires -t
|
|
|
|
If upstream uses a custom derived environment, such as `py37-unit`, use:
|
|
|
|
%pyproject_buildrequires -e %{toxenv}-unit
|
|
|
|
Or specify more environments if needed:
|
|
|
|
%pyproject_buildrequires -e %{toxenv}-unit,%{toxenv}-integration
|
|
|
|
The `-e` option redefines `%{toxenv}` for further reuse.
|
|
Use `%{default_toxenv}` to get the default value.
|
|
|
|
The `-t`/`-e` option uses [tox-current-env]'s `--print-deps-to-file` behind the scenes.
|
|
|
|
Note that both `-x` and `-t` imply `-r`,
|
|
because runtime dependencies are always required for testing.
|
|
|
|
[tox]: https://tox.readthedocs.io/
|
|
[tox-current-env]: https://github.com/fedora-python/tox-current-env/
|
|
|
|
|
|
Running tox based tests
|
|
-----------------------
|
|
|
|
In case you want to run the tests as specified in [tox] configuration,
|
|
you can use the `%tox` macro:
|
|
|
|
%check
|
|
%tox
|
|
|
|
The macro:
|
|
|
|
- Always prepends `$PATH` with `%{buildroot}%{_bindir}`
|
|
- If not defined, sets `$PYTHONPATH` to `%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}`
|
|
- If not defined, sets `$TOX_TESTENV_PASSENV` to `*`
|
|
- Runs `tox` with `-q` (quiet), `--recreate` and `--current-env` (from [tox-current-env]) flags
|
|
- Implicitly uses the tox environment name stored in `%{toxenv}` - as overridden by `%pyproject_buildrequires -t`
|
|
|
|
By using the `-e` flag, you can use a different tox environment(s):
|
|
|
|
%check
|
|
%tox
|
|
%if %{with integration_test}
|
|
%tox -e %{default_toxenv}-integration
|
|
%endif
|
|
|
|
If you wish to provide custom `tox` flags or arguments, add them after `--`:
|
|
|
|
%tox -- --flag-for-tox
|
|
|
|
If you wish to pass custom `posargs` to tox, use another `--`:
|
|
|
|
%tox -- --flag-for-tox -- --flag-for-posargs
|
|
|
|
Or (note the two sequential `--`s):
|
|
|
|
%tox -- -- --flag-for-posargs
|
|
|
|
**Warning:** This macro assumes you have used `%pyproject_buildrequires -t` or `-e`
|
|
in `%generate_buildrequires`. If not, you need to add:
|
|
|
|
BuildRequires: python3dist(tox-current-env)
|
|
|
|
|
|
Generating the %files section
|
|
-----------------------------
|
|
|
|
To generate the list of files in the `%files` section, you can use `%pyproject_save_files` after the `%pyproject_install` macro.
|
|
It takes toplevel module names (i.e. the names used with `import` in Python) and stores paths for those modules and metadata for the package (dist-info directory) to a file stored at `%{pyproject_files}`.
|
|
For example, if a package provides the modules `requests` and `_requests`, write:
|
|
|
|
%install
|
|
%pyproject_install
|
|
%pyproject_save_files requests _requests
|
|
|
|
To add listed files to the `%files` section, use `%files -f %{pyproject_files}`.
|
|
Note that you still need to add any documentation and license manually (for now).
|
|
|
|
%files -n python3-requests -f %{pyproject_files}
|
|
%doc README.rst
|
|
%license LICENSE
|
|
|
|
You can use globs in the module names if listing them explicitly would be too tedious:
|
|
|
|
%install
|
|
%pyproject_install
|
|
%pyproject_save_files *requests
|
|
|
|
In fully automated environmets, you can use the `*` glob to include all modules. In Fedora however, you should always use a more specific glob to avoid accidentally packaging unwanted files (for example, a top level module named `test`).
|
|
|
|
Speaking about automated environments, it is possible to also list all executables in `/usr/bin` by adding a special `+bindir` argument.
|
|
|
|
%install
|
|
%pyproject_install
|
|
%pyproject_save_files * +bindir
|
|
|
|
%files -n python3-requests -f %{pyproject_files}
|
|
|
|
However, in Fedora packages, always list executables explicitly to avoid unintended collisions with other packages or accidental missing executables:
|
|
|
|
%install
|
|
%pyproject_install
|
|
%pyproject_save_files requests _requests
|
|
|
|
%files -n python3-requests -f %{pyproject_files}
|
|
%doc README.rst
|
|
%license LICENSE
|
|
%{_bindir}/downloader
|
|
|
|
|
|
Limitations
|
|
-----------
|
|
|
|
`%pyproject_install` changes shebang lines of every Python script in `%{buildroot}%{_bindir}` to `#!%{__python3} %{py3_shbang_opt}` (`#!/usr/bin/python3 -s`).
|
|
Existing Python flags in shebangs are preserved.
|
|
For example `#!/usr/bin/python3 -Ru` will be updated to `#!/usr/bin/python3 -sRu`.
|
|
Sometimes, this can interfere with tests that run such scripts directly by name,
|
|
because in tests we usually rely on `PYTHONPATH` (and `-s` ignores that).
|
|
Would this behavior be undesired for any reason,
|
|
undefine `%{py3_shbang_opt}` to turn it off.
|
|
|
|
Extras are currently ignored.
|
|
|
|
Some valid Python version specifiers are not supported.
|
|
|
|
The `-x` flag does not yet support multiple (comma-separated) extras.
|
|
|
|
[PEP 517]: https://www.python.org/dev/peps/pep-0517/
|
|
[PEP 518]: https://www.python.org/dev/peps/pep-0518/
|
|
|
|
|
|
Testing the macros
|
|
------------------
|
|
|
|
This repository has two kinds of tests.
|
|
First, there is RPM `%check` section, run when building the `python-rpm-macros`
|
|
package.
|
|
|
|
Then there are CI tests.
|
|
There is currently [no way to run Fedora CI tests locally][ci-rfe],
|
|
but you can do what the tests do manually using mock.
|
|
For each `$PKG.spec` in `tests/`:
|
|
|
|
- clean your mock environment:
|
|
|
|
mock -r fedora-rawhide-x86_64 clean
|
|
|
|
- install the version of `python-rpm-macros` you're testing, e.g.:
|
|
|
|
mock -r fedora-rawhide-x86_64 install .../python-rpm-macros-*.noarch.rpm
|
|
|
|
- download the sources:
|
|
|
|
spectool -g -R $PKG.spec
|
|
|
|
- build a SRPM:
|
|
|
|
rpmbuild -bs $PKG.spec
|
|
|
|
- build in mock, using the path from the command above as `$SRPM`:
|
|
|
|
mock -r fedora-rawhide-x86_64 -n -N $SRPM
|
|
|
|
[ci-rfe]: https://pagure.io/fedora-ci/general/issue/4
|