For packages with a lot of files, it took a really long time to run.
Profiling the code revealed most of the time is spent in PurePath.__eq__.
The code was using lists but checked
if to-be-added paths were not already in them.
Considering the order is insignificant
(the generated %files list is sorted at the end anyway),
the lists were essentially working as (very slow) sets.
Using sets instead of lists makes %pyproject_save_files over 20 times faster
(51.66 -> 2.39 seconds) for the ansible package (~62k files).
Checking if an item is in a list is O(N),
checking every added item is O(N**2).
Checking if an item is in a set is O(1),
checking every added item is O(N).
Additionally, with set, it is unnecessary to check for presence before addition,
so the code is easier.
(Commit message and removal of the check by Miro.)
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
- The flag can be used to indicate no Python modules should be saved
The change wrt users using '*' was necessary,
as `glob` was undefined when `module_globs` was empty.
- The -l flag can be used to assert at least 1 License-File was detected
- The -L flag explicitly disables this check (which remains the default)
Co-Authored-By: Maxwell G <maxwell@gtmx.me>
RPM 4.19 now requires 2 %s to escape a single literal % in the filelist.
The test has been adjusted to actually run our code
instead of only verifying the assumptions.
When certain modules are deliberately not included into the built RPM,
they shouldn't be listed in the list of qualified module names which are
used by %pyproject_check_import to test importability of the
distribution.
Resolves: https://bugzilla.redhat.com/2127958
Files still need to be marked as License-File to be considered %license,
but if their path in METADATA is specified relative to dist-info/licenses,
they are correctly recognised.
This makes License-Files specified by hatchling 1.9.0+ marked as %license.
Users invoking %pyproject_save_files with glob: '*' don't care about the
files in the Python package, hence it shouldn't error when no modules
are detected.
There may be legitimate reasons to create a package without Python
modules in it, hence we shouldn't be blocking this possibility.
Compressed manpages have different extension than those listed in the RECORD file,
so they were not recognized when %%pyproject_save_files '+auto' flag
was provided.
To enable the path recognition, if the manpage extension matches the one
listed in brp-compres, the extension is removed, and an asterisk is now added
to the manpages filenames.
Source: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages
Fixes: https://bugzilla.redhat.com/2033254
%%pyproject_save_files newly saves also a list of importable modules.
The list is used by %%pyproject_check_import to invoke the import test
on each module name.
%%pyproject_check_import accepts two options:
-t: filter only top-level modules
-e: exclude module names matching the given glob from the import check
This is done to avoid troubles with %lang files listed as duplicated.
1. It gets rid of a warning
2. It fixes a problem described in:
http://lists.rpm.org/pipermail/rpm-list/2020-November/002041.html
This is a backwards incompatible change,
packages that rename or remove the installed files after %pyproject_install
might no longer be compatible with %pyproject_save_files.
According to PEP 627, the RECORD file is optional and
doesn't make sense to keep it for system packages. Moreover,
its absence should indicate to other tools like pip that
they should not touch such packages.
Now, we process content of all RECORD files to one
pyproject-record (JSON) which is then used in
%pyproject_save_files. That way, we can remove the original
files in %pyproject_install and keep their content for
later.
PEP 627: https://www.python.org/dev/peps/pep-0627/#optional-record-file
This macro save generates file section to %pyproject_files. It should
simplify %files section and allow to build by some automatic machinery
Supposed use case in Fedora:
%install
%pyproject_install
%pyproject_save_files requests _requests
%files -n python3-requests -f %{pyproject_files}
%doc README.rst
%license LICENSE
Automatic build of arbitrary packages (e.g. in Copr):
%install
%pyproject_install
%pyproject_save_files * +bindir // save all modules with executables
%files -n python3-requests -f %{pyproject_files}
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>
