Commit Graph

1580 Commits

Author SHA1 Message Date
Stepan Oksanichenko 8b11bb81af AL-5220: Investigate why CL9 can't built on the new nebula
- Exclude the packages for using in a build
2023-07-24 18:26:51 +03:00
soksanichenko 114a73f100 - gather-module can find modules through symlinks
- Bump version
- Update changelog
2023-04-15 20:03:27 +03:00
soksanichenko 1c3e5dce5e - CLI option `--label` can be passed through a Pungi config file
- Bump version
- Update changelog
2023-04-13 00:57:39 +03:00
soksanichenko e81d78a1d1 - The log message contains a variant's name if Pungi didn't find one or more modules for that 2023-04-04 10:11:59 +03:00
soksanichenko 68915d04f8 - Excluded/included modules/packages will be processed correctly 2023-04-02 22:27:24 +03:00
soksanichenko 943fd8e77d ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Script `create extra repo` is fixed
- Unittests are fixed
2023-03-30 12:52:51 +03:00
soksanichenko 004fc4382f ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Review comments
2023-03-29 11:40:00 +03:00
soksanichenko 596c5c0b7f ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Refactoring
- Some absent packages are in packages.json now
2023-03-28 12:58:08 +03:00
soksanichenko 141d00e941 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- More info about unsigned packages
2023-03-24 16:39:10 +02:00
soksanichenko 4b64d20826 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Path.rglob/glob doesn't work with symlinks (it's the bug and reported)
- Refactoring
2023-03-24 12:45:28 +02:00
soksanichenko 0747e967b0 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- Some refactoring
2023-03-23 09:36:52 +02:00
soksanichenko 6d58bc2ed8 ALBS-987: Generate i686 and dev repositories with pungi on building new distr. version automatically
- [Generator of packages.json] Replace using CLI by config.yaml
- [Gather RPMs] os.path is replaced by Path
2023-03-22 15:56:58 +02:00
soksanichenko 53ed7386f3 ALBS-1030: Generate Devel section in packages.json
- Redundant empty lines are removed
2023-03-20 13:56:44 +02:00
soksanichenko ed43f0038e ALBS-1030: Generate Devel section in packages.json
- Style fix
2023-03-20 13:55:06 +02:00
soksanichenko fcc9b4f1ca ALBS-1030: Generate Devel section in packages.json
- Skip verifying an RPM signature if sigkeys are empty
2023-03-20 13:25:45 +02:00
soksanichenko d32c293bca ALBS-1030: Generate Devel section in packages.json
- Some upstream changes to KojiMock parts
2023-03-19 21:11:12 +02:00
soksanichenko f0bd1af999 ALBS-1030: Generate Devel section in packages.json
- Also the tool can combine (remove and add) packages in a variant from different
  sources according to an url's type of source
2023-03-19 18:21:33 +02:00
Tomáš Hozza 9e014fed6a osbuild: support specifying `package_sets` for repos
The `koji-osbuild` plugin supports additional formats for the `repo`
property since v4 [1]. Specifically, a repo can be specified as a
dictionary with `baseurl` key and `package_sets` list containing
specific package set names, that the repository should be used for.

Extend the configuration schema to reflect the plugin change.
Extend the documentation to cover the new repository format.
Extend an existing unit test to specify additional repository using the
added format.

[1] https://github.com/osbuild/koji-osbuild/pull/82

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit 8f0906be53)
2023-03-17 11:58:11 +02:00
Tomáš Hozza 7ccb1d4849 osbuild: don't use `util.get_repo_urls()`
Don't use `util.get_repo_urls()` to resolve provided repositories, but
implement osbuild-specific variant of the function named
`_get_repo_urls(). The reason is that the function from `utils`
transforms repositories defined as dicts to strings, which is
undesired for osbuild. The requirement for osbuild is to preserve the
dict as is, just to resolve the string in `baseurl` to the actual
repository URL.

Add a unit test covering the newly added function. It is inspired by a
similar test from `test_util.py`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit e3072c3d5f)
2023-03-17 11:58:11 +02:00
Tomáš Hozza abec28256d osbuild: update schema and config documentation
The `koji-osbuild` Hub schema has been relaxed a bit in the latest
release (v11). Adjust the schema in Pungi to reflect changes in
`koji-osbuild`.

For more information on the changes in `koji-osbuild`, see:
https://github.com/osbuild/koji-osbuild/pull/108

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
(cherry picked from commit ef6d40dce4)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář 02b3adbaeb Stop sending compose paths to CTS
The tracking service will reject it as it's not an HTTP URL. Let's not
even try.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 147df93f75)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář d17e578645 Report errors from CTS
If the service returns a status code indicating a user error, report
that and do not retry.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit dd8c1002d4)
2023-03-17 11:58:11 +02:00
Lubomír Sedlář 6c1c9d9efd createiso: Create Joliet tree with xorriso
This structure is important for isoinfo -J, which is in turn called by
virt-install.

This can be tested by using a bootable ISO by modifying it with a dummy
additional file and preserving boot records:

    $ xorriso -indev netinst.iso -outdev test.iso -boot_image any replay -map setup.py setup.py -end
    ...
    $ isoinfo -J -i test.iso
    isoinfo: Unable to find Joliet SVD
    $ rm test.iso
    $ xorriso -indev netinst.iso -outdev test.iso -joliet on -boot_image any replay -map setup.py setup.py -end
    ...
    $ isoinfo -J -i test.iso
    $

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2144105
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 12e3a46390)
2023-03-17 11:58:04 +02:00
soksanichenko fa4640f03e ALBS-1040: Investigate why Pungi doesn't put modules packages into the final repos
- Refactoring
- KojiMock extracts all modules which are suitable for the variant's arches
2023-03-14 18:25:21 +02:00
soksanichenko d56227ab4a ALBS-1032: Generate i686 section for all variants in packages.json
- Remove old non-necessary methods
- Some fixes to arch code
2023-03-09 12:32:11 +02:00
soksanichenko 4e0d2d14c9 - Unify branch for both RHEL versions 2022-11-11 16:31:43 +02:00
soksanichenko 54209f3643 ALBS-732 2022-11-09 21:42:13 +02:00
soksanichenko 80c4536eaa ALBS-732 2022-11-09 21:27:51 +02:00
soksanichenko 9bb5550d36 ALBS-732 2022-11-09 21:01:30 +02:00
soksanichenko 364ed6c3af - kojimock is added to pungi.phases.gather._make_lookaside_repo#prefixes
- unittests are fixed
2022-11-09 20:56:56 +02:00
soksanichenko 0b965096ee - PkgsetSourceKojiMock is added to ALL_SOURCES 2022-11-09 18:18:12 +02:00
soksanichenko d914626d92 - "kojimock" is valid value for option "pkgset_source" 2022-11-09 17:59:50 +02:00
soksanichenko 5a19ad2258 - Fix unittests 2022-11-08 12:47:14 +02:00
soksanichenko 9ae49dae5b - Fix unittests 2022-11-08 01:43:53 +02:00
soksanichenko c82cbfdc32 - Fix unittests 2022-11-08 00:59:10 +02:00
soksanichenko 323d31df2b Merge branch 'master' into a8_updated
# Conflicts:
#	pungi.spec
#	pungi/wrappers/kojiwrapper.py
#	setup.py
#	tests/test_extra_isos_phase.py
#	tests/test_pkgset_pkgsets.py
2022-11-07 23:38:38 +02:00
soksanichenko 0fa459eb9e - Right name of the class in constructor 2022-11-07 19:56:02 +02:00
soksanichenko b49ffee06d - Mock of Koji is moved to the separate modules, classes
- Unittests for mock of Koji are moved to the separate
2022-11-07 19:24:39 +02:00
Lubomír Sedlář 479849042f init: Filter comps for modular variants with tags
Modular variants can either be specified by a list of modules, or by a
list of Koji tags. In terms of comps preprocessing there should not be
any difference between the two.

Resolves: https://pagure.io/pungi/issue/1640
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-11-03 11:11:01 +01:00
Haibo Lin 8cd19605bd Retry failed cts requests
JIRA: RHELCMP-10033
Signed-off-by: Haibo Lin <hlin@redhat.com>
2022-10-31 12:29:44 +08:00
soksanichenko 750499eda1 - The unittests are fixed 2022-10-19 14:10:48 +03:00
soksanichenko b157a1825a Do not lose a module from koji if we have more than one arch (e.g. x86_64 + i686) 2022-10-19 04:33:34 +03:00
soksanichenko fd298d4f17 Replace list of cr.packages by cr.PackageIterator 2022-10-18 22:53:50 +03:00
Lubomír Sedlář fa967f79b5 Ignore existing kerberos ticket for CTS auth
When there is an existing kerberos ticket, it gets precedence over the
environment variable with path to a keytab. That is not expected and the
user ticket can possibly lack permissions in CTS to be able to run the
compose successfully.

This patch fixes that by setting KRB5CCNAME to a fresh path. That way
there will not be any valid ticket, since the credentials cache does not
exist yet.

JIRA: RHELCMP-9742
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-09-13 12:38:20 +02:00
Tomas Hozza 57739c238f
osbuild: support specifying upload_options
Since version 9, the `koji-osbuild` plugin supports specifying upload
options as part of a Koji build. This enables one to upload the built
image directly to the cloud environment as part of the image build in
Koji.

Extend the configuration schema with `upload_options`.
Extend the documentation and describe valid `upload_options` values.
Add a unit test testing a scenario when `upload_options` are specified.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-09-06 11:20:01 +02:00
Tomas Hozza 805a1083a2
osbuild: accept only a single image type in the configuration
Modify the osbuild configuration schema to accept only an array with a
single value as the `image_types`, in addition to a single string. The
single string was supported by the schema also before, but this fact was
not mentioned in the documentation, nor it was supported by the
`koji-osbuild` plugin of version lower than `9`.

Update the documentation accordingly.

Add unit test for invalid configuration containing more than one image
type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-09-06 10:55:25 +02:00
Lubomír Sedlář c7121f9378 profiler: Flush stdout before printing
Apparently redirecting stderr to the same pipe as stdout does not
guarantee that the data will not be mangled together.

Flushing stdout before the profiler data is printed should ensure that
it does not end up in the middle of some RPM path.

Fixes: https://pagure.io/pungi/issue/1627
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-08-29 05:57:14 +00:00
Lubomír Sedlář 8aba2363e2 pkgset: Report better error when module is missing an arch
Pungi expects each module to be built for all architectures by default.
Unless the module is filtered out, missing metadata for a particular
arch would cause it to crash with a incomprehensible error message. This
should make it a little better.

Relates: https://pagure.io/releng/failed-composes/issue/3889
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-08-26 08:46:01 +00:00
Ondřej Budai 779793386c osbuild: add support for building ostree artifacts
In order to start building Fedora IoT images with osbuild, we need to be able
to pass ostree options from pungi to the koji's osbuildImage task.

This commit adds support for it via new configuration options: ostree_url,
ostree_url and ostree_parent.

A test was added to cover these new options and they are were also added
into the documentation.

JIRA: COMPOSER-1702
Merges: https://pagure.io/pungi/pull-request/1624
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-08-25 15:35:17 +02:00
Timothée Ravier 603c61a033 ostree: Add unified core mode for compose in rpm-ostree
rpm-ostree is moving to unified core composes and this is now working
for Silverblue & Kinoite.

This is untested for IoT but they should move to os-build with Fedora
37.

See: https://github.com/coreos/rpm-ostree/issues/729
Merges: https://pagure.io/pungi/pull-request/1626
Signed-off-by: Timothée Ravier <tim@siosm.fr>
2022-08-23 10:52:45 +02:00