Commit Graph

42 Commits

Author SHA1 Message Date
Haibo Lin cccfaea14e
Support OIDC Client Credentials authentication to CTS
JIRA: RHELCMP-11324
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit e4c525ecbf)
2023-11-10 16:54:58 +02:00
Lubomír Sedlář e891fe7b09
Pass parent id/respin id to CTS
When the --target-dir option is used, the compose can be created in CTS,
but the parent and respin information is not passed through. That leads
to data missing later on.

JIRA: RHELCMP-11411
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

(cherry picked from commit 0c3b6e22f9 (centos_master))
2023-11-10 16:51:33 +02:00
Haibo Lin 4637fd6697
Log versions of all installed packages
JIRA: RHELCMP-9493
Signed-off-by: Haibo Lin <hlin@redhat.com>
(cherry picked from commit 8c06b7a3f1)
2023-11-10 16:50:41 +02:00
soksanichenko 1c3e5dce5e - CLI option `--label` can be passed through a Pungi config file
- Bump version
- Update changelog
2023-04-13 00:57:39 +03:00
Haibo Lin 5e6248e3e0 Generate images.json for extra_isos phase
JIRA: RHELCMP-7241
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-11-15 14:38:53 +08:00
Lubomír Sedlář 80bd254347 Check dependencies after config validation
This way the checks can rely on default values from the config.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-04 13:37:51 +00:00
Lubomír Sedlář e8305f3978 extra_isos: Allow reusing old images
When nothing in configuration or the image itself changed, let's just
copy the older one.

JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 07:08:00 +00:00
Haibo Lin 8133676270 osbs: Reuse images from old compose
JIRA: RHELCMP-5972
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:22:19 +08:00
Haibo Lin e42e65783d image_build: Allow reusing old image_build results
JIRA: RHELCMP-5970
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:07:02 +08:00
Lubomír Sedlář ac061b2ea8 Work around ODCS creating COMPOSE_ID later
When ODCS starts a compose, it will provide base composeinfo file, but
it doesn't create COMPOSE_ID. This leads to a crash when updating CTS,
since the compose id can't be read from the file. We can instead use the
value we already have in memory.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-10-04 11:32:59 +00:00
Ozan Unsal 9612241396 Add COMPOSE_ID into the pungi log file
Jira: RHELCMP-6739

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-21 10:55:03 +02:00
Lubomír Sedlář a1ebd234a4 Only build CTS url when configured
JIRA: RHELCMP-6611
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-10 11:29:53 +02:00
Ozan Unsal b7666ba4a4 Enable pungi to send compose_url patches to CTS
If cts_keytab is also enabled then the HTTP requests are handled with
Kerberos Authentication otherwise no authentication is used.

If cts_url is defined in the configuration, translate_paths is required.
This is needed in order to get the host and the path of the composes.

Jira: RHELCMP-6318

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-07 16:41:35 +02:00
Ozan Unsal 980c7ba8fb Handle the pungi failures to ensure creation of log files
If the given directory is not a valid git directory, it raises RuntimeError.
This can be catched and raised as GitUrlResolveError, so compose can continue
to log the failure.

Jira: RHELCMP-6077

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-08-25 11:15:23 +00:00
Haibo Lin 035b37c566 Cancel koji tasks when pungi terminated
JIRA: RHELCMP-4148
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-03-23 14:47:48 +08:00
Lubomír Sedlář 477dcf37d9 Store extended traceback for gather errors
When a gathering thread raises an exception, it gets forwarded to the
main thread and re-raised there. However, during this transition it
loses details about exact location of the problem.

This patch creates an extended traceback in the worker, which should
make it easier to track the problem down later.

JIRA: RHELCMP-4259
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-02-17 11:11:26 +01:00
Lubomír Sedlář 40133074b3 Add image-container phase
This phase runs after image-build and osbuild and can embed an image
into a container.

JIRA: RHELCMP-3820
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-02-09 14:47:57 +01:00
Lubomír Sedlář 36373479db Move container metadata into compose object
Rather than tracking this directly in OSBS phase, move this into Compose
object, which will allow access to this from multiple phases.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-02-08 13:23:26 +01:00
Lubomír Sedlář 44f7eff1b7 Move UnsignedPackagesError to a separate file
This file can contain all Pungi specific exceptions.

It should also fix an issue encountered on Python 2.7:

    AttributeError: 'module' object has no attribute 'pkgsets'

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-02-08 10:16:32 +00:00
Lubomír Sedlář 49a5661521 pkgset: Remove reuse file when packages are not signed
In such case we never want to reuse the pkgset, as it risks leaking
unsigned packages. Safest option is to remove the file completely.

Fixes: https://pagure.io/pungi/issue/1480
JIRA: RHELCMP-3720
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-01-29 13:06:29 +01:00
Haibo Lin f470599f6c React to SIGINT signal
ODCS sends SIGINT signal.

JIRA: RHELCMP-3687
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-01-11 10:02:10 +08:00
Jan Kaluza c27e21ccf8 Add --respin-of argument.
It is used to defined the relation between original compose
and respun compose with a hotfix in CTS.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-11-19 12:26:38 +00:00
Lubomír Sedlář 99c1e2eb5e Kill all subprocess in signal handler
When Pungi receives a signal to terminate, it can sometimes get stuck
if there are threads running. It has to wait for all worker threads to
finish. They generally do finish, unless they get stuck waiting on a
subprocess.

This patch should reduce the likelihood of this happening by stopping
all subprocesses.

JIRA: RHELCMP-3056
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2020-11-09 08:45:57 +01:00
Lubomír Sedlář a45f4969f3 Add phase for building images with osbuild
This is similar to image-build in terms of what it does, and somewhat
similar to OSBS phase in how it's implemented.

The phase reads configuration, submits the build via XMLRPC call and
waits for the task to finish. Then it downloads the built image and
includes it in the compose metadata.

JIRA: RHELCMP-315
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2020-11-04 13:04:43 +01:00
Jan Kaluza 1595e188a9 Allow setting --development compose_type.
This is related to https://github.com/release-engineering/productmd/pull/149.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-09-25 07:31:20 +02:00
Jan Kaluza 2657a12c96 Allow setting CTS parent_compose_ids using --parent-compose-id option.
This is needed to track dependencies between composes in the Compose
Tracking Service.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-08-13 13:22:43 +02:00
Jan Kaluza b6573fab92 Check if composeinfo-base.json exists before creating it.
Previously, we checked only for `compose_dir`, but it makes
more sense to really check for `composeinfo-base.json` instead.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-06-23 15:27:53 +02:00
Jan Kaluza 4a15d1351a Allow building compose with scratch builds defined by `pkgset_koji_scratch_tasks`.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-06-11 12:05:47 +00:00
Jan Kaluza f1eea0b5a6 Allow getting the compose id from CTS (Compose Tracking Service).
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-05-26 13:18:38 +00:00
Haibo Lin 20ba1a7639 Execute image_checksum phase right after the dependent phases
JIRA: RHELCMP-468
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-05-08 16:12:30 +08:00
Lubomír Sedlář 9a1b9dd154 Fix warning about productimg in skip_phases option
If phase is skipped in a config option (and not on command line), we
should just print a warning or do nothing in quiet mode.

JIRA: RHELCMP-399
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2020-04-28 07:51:40 +00:00
Haibo Lin 477b43d4e9 Split repoclosure into separate phase
Move repoclosure out from test phase into its own phase and
run parallel with image building phases(osbs, imagebuild, ...)
to speed things up.

JIRA: RHELCMP-8
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-04-14 09:25:57 +08:00
Lon Hohberger 9391ce3065 pungi-koji: Allow user to specify version component count in latest symlink
This change adds an option which allows users to specify
the number of version components (i.e. values between dots)

The default behavior is preserved, and is equivalent to using
'-1' as the value.

- Negative values remove items from the end.
- Positive values specify the count of of components
- Zero will remove the version field entirely from the symlink
- When nonzero, at least one version component will appear

Merges: https://pagure.io/pungi/pull-request/1361
Signed-off-by: Lon Hohberger <lhh@redhat.com>
2020-03-13 10:50:08 +01:00
Jan Kaluza e70ad8aaa5 Run extra_files phase in parallel to other phases.
It seems the other phases executed in Weaver phase does not need
extra_files output, so it can be run in parallel to them.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-03-12 07:58:57 +01:00
Lubomír Sedlář 4734f9859a Silence all productimg related warnings
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2020-03-03 11:06:52 +01:00
Lubomír Sedlář 3cd94a4aa5 Silence productimg warning in quiet mode
When running with `--quiet`, the warning should not be printed. It would
confuse tools that expect only compose path to be printed.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2020-03-02 09:24:01 +01:00
Haibo Lin af5ee7030d Check skip_phases in config file
JIRA: COMPOSE-4143
Fixes: https://pagure.io/pungi/issue/1344
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-02-19 11:46:15 +08:00
Haibo Lin c0193c9fca Fix flake8 complaints - E501
E501 line too long (92 > 88 characters)
E501 line too long (103 > 88 characters)
...

JIRA: COMPOSE-4108
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-02-07 14:36:46 +08:00
Haibo Lin 41a629969c Format code base with black
https://black.readthedocs.io/en/stable/

JIRA: COMPOSE-4086
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-02-05 17:35:47 +08:00
Haibo Lin 3f111b559f Check dependency of --just-phase
JIRA: COMPOSE-4020
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-01-06 08:52:47 +00:00
Haibo Lin 794d151bef Remove productimg phase
JIRA: COMPOSE-4004
Signed-off-by: Haibo Lin <hlin@redhat.com>
2020-01-03 11:42:38 +00:00
Haibo Lin 3cf16eb42d cleanup: refactor scripts via entry points
Fixes: https://pagure.io/pungi/issue/1045
JIRA: COMPOSE-2946
Signed-off-by: Haibo Lin <hlin@redhat.com>
2019-12-10 10:53:31 +08:00