pyproject-rpm-macros/README.md

pyproject RPM macros
====================

This is a provisional implementation of pyproject RPM macros for Fedora.

These macros are useful for packaging Python projects that use the [PEP 517] `pyproject.toml` file, which specifies the package's build dependencies (including the build system, such as setuptools, flit or poetry).


Usage
-----

If your upstream sources include `pyproject.toml` and you want to use these macros, BuildRequire them:

    BuildRequires: pyproject-rpm-macros

This will bring in python3-devel, so you don't need to require python3-devel explicitly.

In order to get automatic build dependencies on Fedora 31+, run `%pyproject_buildrequires` in the `%generate_buildrequires` section:

    %generate_buildrequires
    %pyproject_buildrequires

Only build dependencies according to [PEP 517] and [PEP 518] will be added.
All other build dependencies (such as non-Python libraries or test dependencies) still need to be specified manually.

Then, build a wheel in `%build` with `%pyproject_wheel`:

    %build
    %pyproject_wheel

And install the wheel in `%install` with `%pyproject_install`:

    %install
    %pyproject_install

`%pyproject_install` installs all wheels in `$PWD/pyproject-wheeldir/`. If you would like to save wheels somewhere else redefine `%{_pyproject_wheeldir}`.


Adding run-time and test-time dependencies
------------------------------------------

To run tests in the `%check` section, the package's runtime dependencies
often need to also be included as build requirements.
If the project's build system supports the [`prepare-metadata-for-build-wheel`
hook](https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel),
this can be done using the `-r` flag:

    %generate_buildrequires
    %pyproject_buildrequires -r

For projects that specify test requirements using an [`extra`
provide](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use),
these can be added using the `-x` flag.
Multiple extras can be supplied as a comma separated list.
For example, if upstream suggests installing test dependencies with
`pip install mypackage[testing]`, the test deps would be generated by:

    %generate_buildrequires
    %pyproject_buildrequires -x testing

For projects that specify test requirements in their [tox] configuration,
these can be added using the `-t` flag (default tox environment)
or the `-e` flag followed by the tox environment.
The default tox environment (such as `py37` assuming the Fedora's Python version is 3.7)
is available in the `%{toxenv}` macro.
For example, if upstream suggests running the tests on Python 3.7 with `tox -e py37`,
the test deps would be generated by:

    %generate_buildrequires
    %pyproject_buildrequires -t

If upstream uses a custom derived environment, such as `py37-unit`, use:

    %pyproject_buildrequires -e %{toxenv}-unit

Or specify more environments if needed:

    %pyproject_buildrequires -e %{toxenv}-unit,%{toxenv}-integration

The `-e` option redefines `%{toxenv}` for further reuse.
Use `%{default_toxenv}` to get the default value.

The `-t`/`-e` option uses [tox-current-env]'s `--print-deps-to-file` behind the scenes.

Note that both `-x` and `-t` imply `-r`,
because runtime dependencies are always required for testing.

[tox]: https://tox.readthedocs.io/
[tox-current-env]: https://github.com/fedora-python/tox-current-env/


Running tox based tests
-----------------------

In case you want to run the tests as specified in [tox] configuration,
you can use the `%tox` macro:

    %check
    %tox

The macro:

 - Always prepends `$PATH` with `%{buildroot}%{_bindir}`
 - If not defined, sets `$PYTHONPATH` to `%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}`
 - If not defined, sets `$TOX_TESTENV_PASSENV` to `*`
 - Runs `tox` with `-q` (quiet), `--recreate` and `--current-env` (from [tox-current-env]) flags
 - Implicitly uses the tox environment name stored in `%{toxenv}` - as overridden by `%pyproject_buildrequires -t`

By using the `-e` flag, you can use a different tox environment(s):

    %check
    %tox
    %if %{with integration_test}
    %tox -e %{default_toxenv}-integration
    %endif

If you wish to provide custom `tox` flags or arguments, add them after `--`:

    %tox -- --flag-for-tox

If you wish to pass custom `posargs` to tox, use another `--`:

    %tox -- --flag-for-tox -- --flag-for-posargs

Or (note the two sequential `--`s):

    %tox -- -- --flag-for-posargs

**Warning:** This macro assumes you have used `%pyproject_buildrequires -t` or `-e`
in `%generate_buildrequires`. If not, you need to add:

    BuildRequires: python3dist(tox-current-env)


Generating the %files section
-----------------------------

To generate the list of files in the `%files` section, you can use `%pyproject_save_files` after the `%pyproject_install` macro.
It takes toplevel module names (i.e. the names used with `import` in Python) and stores paths for those modules and metadata for the package (dist-info directory) to a file stored at `%{pyproject_files}`.
For example, if a package provides the modules `requests` and `_requests`, write:

    %install
    %pyproject_install
    %pyproject_save_files requests _requests

To add listed files to the `%files` section, use `%files -f %{pyproject_files}`.
Note that you still need to add any documentation and license manually (for now).

    %files -n python3-requests -f %{pyproject_files}
    %doc README.rst
    %license LICENSE

You can use globs in the module names if listing them explicitly would be too tedious:

    %install
    %pyproject_install
    %pyproject_save_files '*requests'

In fully automated environmets, you can use the `*` glob to include all modules (put it in single quotes to prevent Shell from expanding it). In Fedora however, you should always use a more specific glob to avoid accidentally packaging unwanted files (for example, a top level module named `test`).

Speaking about automated environments, some files cannot be classified with `%pyproject_save_files`, but it is possible to list all unclassified files by adding a special `+auto` argument.

    %install
    %pyproject_install
    %pyproject_save_files '*' +auto
    
    %files -n python3-requests -f %{pyproject_files}

However, in Fedora packages, always list executables explicitly to avoid unintended collisions with other packages or accidental missing executables:

    %install
    %pyproject_install
    %pyproject_save_files requests _requests
    
    %files -n python3-requests -f %{pyproject_files}
    %doc README.rst
    %license LICENSE
    %{_bindir}/downloader


Limitations
-----------

`%pyproject_install` changes shebang lines of every Python script in `%{buildroot}%{_bindir}` to `#!%{__python3} %{py3_shbang_opt}` (`#!/usr/bin/python3 -s`).
Existing Python flags in shebangs are preserved.
For example `#!/usr/bin/python3 -Ru` will be updated to `#!/usr/bin/python3 -sRu`.
Sometimes, this can interfere with tests that run such scripts directly by name,
because in tests we usually rely on `PYTHONPATH` (and `-s` ignores that).
Would this behavior be undesired for any reason,
undefine `%{py3_shbang_opt}` to turn it off.

Some valid Python version specifiers are not supported.

[PEP 517]: https://www.python.org/dev/peps/pep-0517/
[PEP 518]: https://www.python.org/dev/peps/pep-0518/


Testing the macros
------------------

This repository has two kinds of tests.
First, there is RPM `%check` section, run when building the `python-rpm-macros`
package.

Then there are CI tests.
There is currently [no way to run Fedora CI tests locally][ci-rfe],
but you can do what the tests do manually using mock.
For each `$PKG.spec` in `tests/`:

  - clean your mock environment:

        mock -r fedora-rawhide-x86_64 clean

  - install the version of `python-rpm-macros` you're testing, e.g.:

        mock -r fedora-rawhide-x86_64 install .../python-rpm-macros-*.noarch.rpm

  - download the sources:

        spectool -g -R $PKG.spec

  - build a SRPM:

        rpmbuild -bs $PKG.spec

  - build in mock, using the path from the command above as `$SRPM`:

        mock -r fedora-rawhide-x86_64 -n -N $SRPM

[ci-rfe]: https://pagure.io/fedora-ci/general/issue/4