pyproject RPM macros

This is a provisional implementation of pyproject RPM macros for Fedora.

These macros are useful for packaging Python projects that use the PEP 517 pyproject.toml file, which specifies the package's build dependencies (including the build system, such as setuptools, flit or poetry).

Usage

If your upstream sources include pyproject.toml and you want to use these macros, BuildRequire them:

BuildRequires: pyproject-rpm-macros

This will bring in python3-devel, so you don't need to require python3-devel explicitly.

In order to get automatic build dependencies on Fedora 31+, run %pyproject_buildrequires in the %generate_buildrequires section:

%generate_buildrequires
%pyproject_buildrequires

Only build dependencies according to PEP 517 and PEP 518 will be added. All other build dependencies (such as non-Python libraries or test dependencies) still need to be specified manually.

Then, build a wheel in %build with %pyproject_wheel:

%build
%pyproject_wheel

And install the wheel in %install with %pyproject_install:

%install
%pyproject_install

%pyproject_install installs all wheels in $PWD/pyproject-wheeldir/. If you would like to save wheels somewhere else redefine %{_pyproject_wheeldir}.

Adding run-time and test-time dependencies

To run tests in the %check section, the package's runtime dependencies often need to also be included as build requirements. If the project's build system supports the prepare-metadata-for-build-wheel hook, this can be done using the -r flag:

%generate_buildrequires
%pyproject_buildrequires -r

For projects that specify test requirements using an extra provide, these can be added using the -x flag. For example, if upstream suggests installing test dependencies with pip install mypackage[testing], the test deps would be generated by:

%generate_buildrequires
%pyproject_buildrequires -x testing

For projects that specify test requirements in their tox configuration, these can be added using the -t flag (default tox environment) or the -e flag followed by the tox environment. The default tox environment (such as py37 assuming the Fedora's Python version is 3.7) is available in the %{toxenv} macro. For example, if upstream suggests running the tests on Python 3.7 with tox -e py37, the test deps would be generated by:

%generate_buildrequires
%pyproject_buildrequires -t

If upstream uses a custom derived environment, such as py37-unit, use:

%pyproject_buildrequires -e %{toxenv}-unit

Or specify more environments if needed:

%pyproject_buildrequires -e %{toxenv}-unit,%{toxenv}-integration

The -e option redefines %{toxenv} for further reuse. Use %{default_toxenv} to get the default value.

The -t/-e option uses tox-current-env's --print-deps-to-file behind the scenes.

Note that both -x and -t imply -r, because runtime dependencies are always required for testing.

Running tox based tests

In case you want to run the tests as specified in tox configuration, you can use the %tox macro:

%check
%tox

The macro:

• Always prepends $PATH with %{buildroot}%{_bindir}
• If not defined, sets $PYTHONPATH to %{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}
• If not defined, sets $TOX_TESTENV_PASSENV to *
• Runs tox with -q (quiet), --recreate and --current-env (from tox-current-env) flags
• Implicitly uses the tox environment name stored in %{toxenv} - as overridden by %pyproject_buildrequires -t

By using the -e flag, you can use a different tox environment(s):

%check
%tox
%if %{with integration_test}
%tox -e %{default_toxenv}-integration
%endif

If you wish to provide custom tox flags or arguments, add them after --:

%tox -- --flag-for-tox

If you wish to pass custom posargs to tox, use another --:

%tox -- --flag-for-tox -- --flag-for-posargs

Or (note the two sequential --s):

%tox -- -- --flag-for-posargs

Warning: This macro assumes you have used %pyproject_buildrequires -t or -e in %generate_buildrequires. If not, you need to add:

BuildRequires: python3dist(tox-current-env)

Generating the %files section

To generate the list of files in the %files section, you can use %pyproject_save_files after the %pyproject_install macro. It takes toplevel module names (i.e. the names used with import in Python) and stores paths for those modules and metadata for the package (dist-info directory) to a file stored at %{pyproject_files}. For example, if a package provides the modules requests and _requests, write:

%install
%pyproject_install
%pyproject_save_files requests _requests

To add listed files to the %files section, use %files -f %{pyproject_files}. Note that you still need to add any documentation and license manually (for now).

%files -n python3-requests -f %{pyproject_files}
%doc README.rst
%license LICENSE

You can use globs in the module names if listing them explicitly would be too tedious:

%install
%pyproject_install
%pyproject_save_files *requests

In fully automated environmets, you can use the * glob to include all modules. In Fedora however, you should always use a more specific glob to avoid accidentally packaging unwanted files (for example, a top level module named test).

Speaking about automated environments, it is possible to also list all executables in /usr/bin by adding a special +bindir argument.

%install
%pyproject_install
%pyproject_save_files * +bindir

%files -n python3-requests -f %{pyproject_files}

However, in Fedora packages, always list executables explicitly to avoid unintended collisions with other packages or accidental missing executables:

%install
%pyproject_install
%pyproject_save_files requests _requests

%files -n python3-requests -f %{pyproject_files}
%doc README.rst
%license LICENSE
%{_bindir}/downloader

Limitations

%pyproject_install changes shebang lines of every Python script in %{buildroot}%{_bindir} to #!%{__python3} %{py3_shbang_opt} (#!/usr/bin/python3 -s). Existing Python flags in shebangs are preserved. For example #!/usr/bin/python3 -Ru will be updated to #!/usr/bin/python3 -sRu. Sometimes, this can interfere with tests that run such scripts directly by name, because in tests we usually rely on PYTHONPATH (and -s ignores that). Would this behavior be undesired for any reason, undefine %{py3_shbang_opt} to turn it off.

Extras are currently ignored.

Some valid Python version specifiers are not supported.

The -x flag does not yet support multiple (comma-separated) extras.

Testing the macros

This repository has two kinds of tests. First, there is RPM %check section, run when building the python-rpm-macros package.

Then there are CI tests. There is currently no way to run Fedora CI tests locally, but you can do what the tests do manually using mock. For each $PKG.spec in tests/:

• clean your mock environment:

  mock -r fedora-rawhide-x86_64 clean
  

• install the version of python-rpm-macros you're testing, e.g.:

  mock -r fedora-rawhide-x86_64 install .../python-rpm-macros-*.noarch.rpm
  

• download the sources:

  spectool -g -R $PKG.spec
  

• build a SRPM:

  rpmbuild -bs $PKG.spec
  

• build in mock, using the path from the command above as $SRPM:

  mock -r fedora-rawhide-x86_64 -n -N $SRPM