Commit Graph

1587 Commits

Author SHA1 Message Date
Lubomír Sedlář 2930a1cc54
Retry 401 error from CTS
This could be a transient error caused by kerberos server instability.

JIRA: RHELCMP-11251
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 58036eab84)
2023-11-10 16:50:43 +02:00
Lubomír Sedlář 9c4d3d496d
gather: Better detection of debuginfo in lookaside
If the depsolver wants to include a package that is present in both the
source repo and a lookaside repo, it reliably detects binary packages
present in lookaside, but for debuginfo it's not so reliable.

There is a separate package object for each package in each repo.
Depending on which one is used, debuginfo could be included in the
result or not. This patch fixes that by actually looking if the same
package is present in any lookaside repo.

JIRA: RHELCMP-9373
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit a4476f2570)
2023-11-10 16:50:42 +02:00
Haibo Lin 4637fd6697
Log versions of all installed packages
JIRA: RHELCMP-9493
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit 8c06b7a3f1)
2023-11-10 16:50:41 +02:00
Lubomír Sedlář 2ff8132eaf
Use authentication for all CTS calls
The update of compose URL relied on environment being set from the
initial import. This got broken when a unique credentials cache started
to be used, and was cleaned up after the import.

JIRA: RHELCMP-11072
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 64ae81b416)
2023-11-10 16:50:40 +02:00
Lubomír Sedlář f9190d1fd1
Fix black complaints
These are newly detected by black 23.1.0.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 826169af7c)
2023-11-10 16:50:38 +02:00
Lubomír Sedlář 80ad0448ec
Add vhd.gz extension to compressed VHD images
JIRA: RHELCMP-11027
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit d97b8bdd33)
2023-11-10 16:50:37 +02:00
Lubomír Sedlář 027380f969
Add vhd-compressed image type
JIRA: RHELCMP-11027
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 8768b23cbe)
2023-11-10 16:50:36 +02:00
Stepan Oksanichenko 8b11bb81af AL-5220: Investigate why CL9 can't built on the new nebula
- Exclude the packages for using in a build
2023-07-24 18:26:51 +03:00
soksanichenko 114a73f100 - gather-module can find modules through symlinks
- Bump version
- Update changelog
2023-04-15 20:03:27 +03:00
soksanichenko 1c3e5dce5e - CLI option `--label` can be passed through a Pungi config file
- Bump version
- Update changelog
2023-04-13 00:57:39 +03:00
soksanichenko e81d78a1d1 - The log message contains a variant's name if Pungi didn't find one or more modules for that 2023-04-04 10:11:59 +03:00
soksanichenko 68915d04f8 - Excluded/included modules/packages will be processed correctly 2023-04-02 22:27:24 +03:00
soksanichenko 943fd8e77d ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Script `create extra repo` is fixed
- Unittests are fixed
2023-03-30 12:52:51 +03:00
soksanichenko 004fc4382f ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Review comments
2023-03-29 11:40:00 +03:00
soksanichenko 596c5c0b7f ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Refactoring
- Some absent packages are in packages.json now
2023-03-28 12:58:08 +03:00
soksanichenko 141d00e941 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- More info about unsigned packages
2023-03-24 16:39:10 +02:00
soksanichenko 4b64d20826 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Path.rglob/glob doesn't work with symlinks (it's the bug and reported)
- Refactoring
2023-03-24 12:45:28 +02:00
soksanichenko 0747e967b0 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Some refactoring
2023-03-23 09:36:52 +02:00
soksanichenko 6d58bc2ed8 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- [Generator of packages.json] Replace using CLI by config.yaml
- [Gather RPMs] os.path is replaced by Path
2023-03-22 15:56:58 +02:00
soksanichenko 53ed7386f3 ALBS-1030: Generate Devel section in packages.json
- Redundant empty lines are removed
2023-03-20 13:56:44 +02:00
soksanichenko ed43f0038e ALBS-1030: Generate Devel section in packages.json
- Style fix
2023-03-20 13:55:06 +02:00
soksanichenko fcc9b4f1ca ALBS-1030: Generate Devel section in packages.json
- Skip verifying an RPM signature if sigkeys are empty
2023-03-20 13:25:45 +02:00
soksanichenko d32c293bca ALBS-1030: Generate Devel section in packages.json
- Some upstream changes to KojiMock parts
2023-03-19 21:11:12 +02:00
soksanichenko f0bd1af999 ALBS-1030: Generate Devel section in packages.json
- Also the tool can combine (remove and add) packages in a variant from different
  sources according to an url's type of source
2023-03-19 18:21:33 +02:00
Tomáš Hozza 9e014fed6a osbuild: support specifying `package_sets` for repos
The `koji-osbuild` plugin supports additional formats for the `repo`
property since v4 [1]. Specifically, a repo can be specified as a
dictionary with `baseurl` key and `package_sets` list containing
specific package set names, that the repository should be used for.

Extend the configuration schema to reflect the plugin change.
Extend the documentation to cover the new repository format.
Extend an existing unit test to specify additional repository using the
added format.

[1] https://github.com/osbuild/koji-osbuild/pull/82

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit 8f0906be53)
2023-03-17 11:58:11 +02:00
Tomáš Hozza 7ccb1d4849 osbuild: don't use `util.get_repo_urls()`
Don't use `util.get_repo_urls()` to resolve provided repositories, but
implement osbuild-specific variant of the function named
`_get_repo_urls(). The reason is that the function from `utils`
transforms repositories defined as dicts to strings, which is
undesired for osbuild. The requirement for osbuild is to preserve the
dict as is, just to resolve the string in `baseurl` to the actual
repository URL.

Add a unit test covering the newly added function. It is inspired by a
similar test from `test_util.py`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit e3072c3d5f)
2023-03-17 11:58:11 +02:00
Tomáš Hozza abec28256d osbuild: update schema and config documentation
The `koji-osbuild` Hub schema has been relaxed a bit in the latest
release (v11). Adjust the schema in Pungi to reflect changes in
`koji-osbuild`.

For more information on the changes in `koji-osbuild`, see:
https://github.com/osbuild/koji-osbuild/pull/108

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit ef6d40dce4)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář 02b3adbaeb Stop sending compose paths to CTS
The tracking service will reject it as it's not an HTTP URL. Let's not
even try.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 147df93f75)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář d17e578645 Report errors from CTS
If the service returns a status code indicating a user error, report
that and do not retry.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit dd8c1002d4)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář 6c1c9d9efd createiso: Create Joliet tree with xorriso
This structure is important for isoinfo -J, which is in turn called by
virt-install.

This can be tested by using a bootable ISO by modifying it with a dummy
additional file and preserving boot records:

    $ xorriso -indev netinst.iso -outdev test.iso -boot_image any replay -map setup.py setup.py -end
    ...
    $ isoinfo -J -i test.iso
    isoinfo: Unable to find Joliet SVD
    $ rm test.iso
    $ xorriso -indev netinst.iso -outdev test.iso -joliet on -boot_image any replay -map setup.py setup.py -end
    ...
    $ isoinfo -J -i test.iso
    $

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2144105
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 12e3a46390)
2023-03-17 11:58:04 +02:00
soksanichenko fa4640f03e ALBS-1040: Investigate why Pungi doesn't put modules packages into the final repos
- Refactoring
- KojiMock extracts all modules which are suitable for the variant's arches
2023-03-14 18:25:21 +02:00
soksanichenko d56227ab4a ALBS-1032: Generate i686 section for all variants in packages.json
- Remove old non-necessary methods
- Some fixes to arch code
2023-03-09 12:32:11 +02:00
soksanichenko 4e0d2d14c9 - Unify branch for both RHEL versions 2022-11-11 16:31:43 +02:00
soksanichenko 54209f3643 ALBS-732 2022-11-09 21:42:13 +02:00
soksanichenko 80c4536eaa ALBS-732 2022-11-09 21:27:51 +02:00
soksanichenko 9bb5550d36 ALBS-732 2022-11-09 21:01:30 +02:00
soksanichenko 364ed6c3af - kojimock is added to pungi.phases.gather._make_lookaside_repo#prefixes
- unittests are fixed
2022-11-09 20:56:56 +02:00
soksanichenko 0b965096ee - PkgsetSourceKojiMock is added to ALL_SOURCES 2022-11-09 18:18:12 +02:00
soksanichenko d914626d92 - "kojimock" is valid value for option "pkgset_source" 2022-11-09 17:59:50 +02:00
soksanichenko 5a19ad2258 - Fix unittests 2022-11-08 12:47:14 +02:00
soksanichenko 9ae49dae5b - Fix unittests 2022-11-08 01:43:53 +02:00
soksanichenko c82cbfdc32 - Fix unittests 2022-11-08 00:59:10 +02:00
soksanichenko 323d31df2b Merge branch 'master' into a8_updated
# Conflicts:
#	pungi.spec
#	pungi/wrappers/kojiwrapper.py
#	setup.py
#	tests/test_extra_isos_phase.py
#	tests/test_pkgset_pkgsets.py
2022-11-07 23:38:38 +02:00
soksanichenko 0fa459eb9e - Right name of the class in constructor 2022-11-07 19:56:02 +02:00
soksanichenko b49ffee06d - Mock of Koji is moved to the separate modules, classes
- Unittests for mock of Koji are moved to the separate
2022-11-07 19:24:39 +02:00
Lubomír Sedlář 479849042f init: Filter comps for modular variants with tags
Modular variants can either be specified by a list of modules, or by a
list of Koji tags. In terms of comps preprocessing there should not be
any difference between the two.

Resolves: https://pagure.io/pungi/issue/1640
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-11-03 11:11:01 +01:00
Haibo Lin 8cd19605bd Retry failed cts requests
JIRA: RHELCMP-10033
Signed-off-by: Haibo Lin <hlin@redhat.com>
2022-10-31 12:29:44 +08:00
soksanichenko 750499eda1 - The unittests are fixed 2022-10-19 14:10:48 +03:00
soksanichenko b157a1825a Do not lose a module from koji if we have more than one arch (e.g. x86_64 + i686) 2022-10-19 04:33:34 +03:00
soksanichenko fd298d4f17 Replace list of cr.packages by cr.PackageIterator 2022-10-18 22:53:50 +03:00