Commit Graph

95 Commits

Author SHA1 Message Date
Fedora Release Engineering
2ff265d8fd - Rebuilt for https://fedoraproject.org/wiki/Fedora_36_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2022-01-21 15:19:07 +00:00
Gordon Messmer
2c2f8bd984 Handle legacy version specifiers that would previously raise exceptions. 2021-12-19 14:08:05 -08:00
Gordon Messmer
a3ad67b505 Additional fix for dev releases. 2021-10-29 20:00:41 -07:00
Gordon Messmer
27f9733f0b Sync dependency conversion with upstream pyreq2rpm.
Improve handling of > operator, preventing post-release from satisfying most rpm requirements.
Improve handling of < operator, preventing pre-release from satisfying rpm requirement.
Improve handling of != operator with prefix matching, preventing pre-release from satisfying rpm requirements.
2021-10-28 21:50:58 -07:00
Fedora Release Engineering
98fa009fc8 - Rebuilt for https://fedoraproject.org/wiki/Fedora_35_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2021-07-23 09:18:27 +00:00
Miro Hrončok
9bd2a43a74 Support multiple vendor files in pythonbundles.py
Not bumping the release, will happily wait until it bubbles trough.
2021-06-22 18:28:43 +00:00
Miro Hrončok
04ae9b96f2 CI: Adapt pythondist.spec for Python 3.10 being the main Python version
Preserves comaptbility with Python 3.9.
2021-06-18 19:20:16 +02:00
Tomas Orsava
cc489bde7a pythondistdeps.py: Catch all exceptions and terminate build if one is raised 2021-05-25 18:51:44 +02:00
Tomas Orsava
27d363833e pythondistdeps.py: Detect and error when metadata is corrupted 2021-05-25 18:11:00 +02:00
Miro Hrončok
20f8b2c775 Fix python(abi) generator (the one written in Python)
There were three problems:

 - sys.version was not imported
 - sys.version[:3] is not reliable on Python 3.10+
 - distutils is deprecated on Python 3.10+

We were not hit by the missing import in Fedora because we only run the script
on .dist-info/.egg-info/.egg and not on .py files, so this if-branch never runs.

But when the script was fed with a .py path, it errored:

    Traceback (most recent call last):
      File "/usr/lib/rpm/pythondistdeps.py", line 344, in <module>
        purelib = get_python_lib(standard_lib=0, plat_specific=0).split(version[:3])[0]
    NameError: name 'version' is not defined

The sys.version[:3] thing kinda works for Python 3.10+ because *in this
particular case* splitting on '3.1' and taking the prefix yields the same
results as splitting on '3.10', but I consider that mere coincidence.

Finally, since the distutils import happened at module-level,
we got the Deprecation warning in all Fedora's Python packages:

    /usr/lib/rpm/pythondistdeps.py:16: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12

Backported from https://github.com/rpm-software-management/python-rpm-packaging/commit/d12e039037
2021-04-19 22:59:10 +02:00
Miro Hrončok
0a12aa5a2f Do not generate setuptools requirement for console_scripts on Python 3.10+
See https://fedoraproject.org/wiki/Changes/Reduce_dependencies_on_python3-setuptools
2021-03-31 11:56:16 +02:00
Tomas Orsava
a295a58559 Add __pycache__ into .gitignore 2021-03-11 13:41:54 +01:00
Tomas Orsava
3a4efade98 pythondistdeps.py: Always output extras names in lowercase 2021-03-11 13:41:54 +01:00
Tomas Orsava
b44c808358 pythondistdeps.py: Compare extras as lowercase
- New test sources tarball with added test data
2021-03-11 13:41:25 +01:00
Tomas Orsava
103464475f pythondistdeps.py: Changing order in test-data 2021-03-11 12:46:23 +01:00
Tomas Orsava
48510eebae scripts/pythondistdeps: Fix for Python 3.10
self.name in PathDistribution is a property in Python 3.10+ and thus we
can't redefine it as an instance variable. Instead we explicitly define
it as a property, which works on all supported Python versions.
2021-02-24 14:07:24 +01:00
Tomas Orsava
438d8d3b70 scripts/pythondistdeps: Backport switch to importlib.metadata from upstream
Upstream change to importlib.metadata: https://github.com/rpm-software-management/rpm/pull/1317

Due to extras packages being hadled slightly differently by importlib,
one test case for this was added.  And due to changes in handling
requires.txt files, comments were removed from the pyreq2rpm.tests
testing package.

Also because of the switch, we removed the dependency on setuptools and
added a dependency on packaging.

Note: Some packages with egg-info files might provide a different name
due to this change if there is a conflict between the filename and the
name in the metadata. Previously, the filename was sometimes used to
parse the name, now it is always the content of that file, which is what
packaging does, and thus also pip and other Python tooling. Currently,
this is known to affect only 1 package in Fedora (ntpsec).

The resulting script is different from upstream because of not yet upstreamed changes in Fedora:
- scripts/pythondistdeps: Rework error messages
- scripts/pythondistdeps: Add parameter --package-name
- scripts/pythondistdeps: Implement provides/requires for extras packages
- pythondistdeps.py: When parsing extras name, take the rightmost +

These changes are proposed in this upstream PR: https://github.com/rpm-software-management/rpm/pull/1546
2021-02-18 16:08:27 +01:00
Miro Hrončok
2d631762c5 Remove unused 2.7 from --majorver-provides-versions
Fixup for 8c2a1c0ac9.
This makes no real difference, just a cleanup, hence not bumping.
2021-02-08 10:53:40 +01:00
Miro Hrončok
8c2a1c0ac9 Disable the dist generators for Python 2
https://fedoraproject.org/wiki/Changes/Disable_Python_2_Dist_RPM_Generators_and_Freeze_Python_2_Macros

The regex previously matched any Python version in a form of <single digit>.<at least one digit>.

Now it matches anything from 3.0 above: <single digit (3 or higher)>.<at least one digit>

It still does not match <multiple digits>.<at least one digit>, e.g. 11.0.

This is a breaking change, hence the version bump.
2021-02-03 14:09:45 +01:00
Fedora Release Engineering
b65cf8549a - Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2021-01-27 13:13:21 +00:00
Tomas Orsava
f328c9dd18 Add executable bit to pythonbundles.py
pythondistdeps.py is executable already
2020-10-19 12:56:51 +02:00
Tomas Orsava
d77d134c10 Run scripts in an isolated environment (#1889080) 2020-10-19 12:56:43 +02:00
Miro Hrončok
bfb7f70b99 Add a test for a requires with multiple underscores 2020-09-25 14:22:03 +02:00
Tomas Orsava
cb3aaf6d26 Add a test for a requires with an underscore
We already have PyQt5_sip as a test of a provides with an underscore
2020-09-23 11:39:53 +02:00
Fedora Release Engineering
df7ed92279 - Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2020-07-29 03:42:44 +00:00
Tomas Orsava
64e5d7567a Sync tests for python dependency conversion with pyreq2rpm 2020-07-22 18:00:02 +02:00
Gordon Messmer
fbe1c77166 Sync python dependency conversion with pyreq2rpm. 2020-07-22 18:00:02 +02:00
Miro Hrončok
7398b71fbc pythondistdeps.py: When parsing extras name, take the rightmost + 2020-07-22 00:29:19 +02:00
Miro Hrončok
d1a02fdda7 pythondistdeps.py: Adapt Python version marker workaround for setuptools 42+
See https://bugzilla.redhat.com/show_bug.cgi?id=1853597#c11

pkg_resources from setuptools 42+ no longer only use platform.python_version(),
but also platform.python_version_tuple() -- this was updated in packaging 19.1+.

This fix makes it work again with both new and old setuptools,
hopefully for some while.

bf069fe9dd
86a443f318
2020-07-10 16:30:20 +02:00
Tomas Orsava
32a1b47f5b scripts/pythondistdeps: Tests: small tweaks 2020-07-10 15:46:19 +02:00
Tomas Orsava
c2e0f33565 scripts/pythondistdeps: Add tests for: Rework error messages 2020-07-10 15:37:11 +02:00
Tomas Orsava
098c48d46d scripts/pythondistdeps: Rework error messages 2020-07-10 13:45:59 +02:00
Tomas Orsava
b6e0638f7c Enable --require-extras-subpackages and bump release 2020-07-10 13:43:04 +02:00
Tomas Orsava
9df3e5bcb5 scripts/pythondistdeps: Add tests for: Implement provides/requires for extras packages 2020-07-10 13:42:12 +02:00
Tomas Orsava
0c9665427c scripts/pythondistdeps: Implement provides/requires for extras packages 2020-07-10 13:42:12 +02:00
Tomas Orsava
3b1100ba1f scripts/pythondistdeps: Add parameter --package-name 2020-07-10 13:42:12 +02:00
Miro Hrončok
48c0de39d9 Add a script to generate Python bundled provides
See https://src.fedoraproject.org/rpms/python-setuptools/pull-request/40

Strictly speaking, this is not an RPM generator, but:

 - it generates provides
 - it is tighly coupled with pythondistdeps.py

Usage:

 1. Run `$ /usr/lib/rpm/pythonbundles.py .../vendored.txt`

 2. Copy the output into the spec as a macro definition:

    %global bundled %{expand:
    Provides: bundled(python3dist(appdirs)) = 1.4.3
    Provides: bundled(python3dist(packaging)) = 16.8
    Provides: bundled(python3dist(pyparsing)) = 2.2.1
    Provides: bundled(python3dist(six)) = 1.15
    }

 3. Use the macro to expand the provides
 4. Verify the macro contents in %check:

    %check
    ...
    %{_rpmconfigdir}/pythonbundles.py src/_vendor/vendored.txt --compare-with '%{bundled}'
2020-07-07 16:01:37 +02:00
Miro Hrončok
e78c420523 Fix python(abi) requires generator, it picked files from almost good directories
The %__python_magic filter suddenly got actually working with file 5.39:

Before:

    file.cpython-38.opt-1.pyc: data

After:

    file.cpython-38.opt-1.pyc: python 3.8 byte-compiled

Hence, the filter started to pick all Python files regardless of their location.
Later, in the actual generator, paths like this were considered:

    /opt/usr/lib/python3.X/...

And generated requirements on python(abi).

We don't actually need to filter the files by file magic,
so we drop it to get the previously accidentally working behavior.

We could choose if the path and magic filters are applied as OR or AND.
However, we don't want either.

We actually want to mach any files in Python directories regardless of their magic.
We *could* filter by file type (and executable bit) for provides,
but that would require us to split the attr files into two.
2020-06-18 13:32:24 +02:00
Miro Hrončok
3a396fbf96 Use PEP 503 names for requires 2020-05-21 17:43:19 +02:00
Miro Hrončok
39315a6aa4 Adapt tests for the pythonXY -> pythonX.Y renaming
See https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/VIUS7WMQMDX6H2WEIH7TVTMBB6SUHY7E/
2020-05-07 17:59:18 +02:00
Miro Hrončok
33358b9a65 Deduplicate automatically provided names trough Python RPM Lua macros 2020-05-05 14:02:52 +02:00
Miro Hrončok
c3f90ed2e8 Fix reversed grep exit codes in integration tests
grep -v only fails if there are no unmatched lines, but that's not what we want to test.
2020-05-04 13:45:57 +02:00
Miro Hrončok
6beec97e9e Add integartion test for the dist generator 2020-05-04 13:45:57 +02:00
Tomas Orsava
54e4aa751b Bump version, enable new features, add test suite to Fedora CI 2020-05-04 13:45:57 +02:00
Tomas Orsava
79790d12af scripts/pythondistdeps: Modify handling of dev versions 2020-04-30 22:24:44 +02:00
Tomas Orsava
972beac29a scripts/pythondistdeps: Version handling exception with better information 2020-04-30 22:24:44 +02:00
Tomas Orsava
d48f3500d8 scripts/pythondistdeps: Do anything only when called as a main script
Note that the code is completely unchanged except for the indentation
under the new if __name__ == "__main__":

Note that this change is necessary, but not sufficient to use the
RpmVersion class.
The init of the RpmVersion class will fail when called from an outside
script, because the `parse_version()` function is lazily imported from
the code outside the class.  However, adding the import of
parse_version() to RpmVersion class is not done right now, because while
we would import it from `pkg_resources`, other scripts might want to
rely instead of the lightweight `packaging` module for the import. Thus
I'm leaving this conondrum to be addressed in the future.
2020-04-30 22:24:44 +02:00
Tomas Orsava
1523def34e scripts/pythondistdeps: Implement --normalized-name-* options
--normalized-names-format FORMAT
    FORMAT of normalized names can be `pep503` [default] or `legacy-dots` (dots allowed)

--normalized-names-provide-both
    Provede both `pep503` and `legacy-dots` format of normalized names (useful for a transition period)
2020-04-30 22:24:44 +02:00
Tomas Orsava
e33d4e94c8 scripts/pythondistdeps: Add option to generate major-version provides only for specified Python versions 2020-04-30 22:24:44 +02:00
Tomas Orsava
89e1676cee scripts/pythondistdeps: Add tests
The test data download themselves using pip if not present
2020-04-30 22:24:44 +02:00