Commit Graph

2809 Commits

Author SHA1 Message Date
Haibo Lin 52c9816755 4.3.3 release
JIRA: RHELCMP-7691
Signed-off-by: Haibo Lin <hlin@redhat.com>
2022-01-13 15:45:30 +08:00
Lubomír Sedlář 32221e8f36 hybrid: Explicitly pull in debugsource packages
This should cover case where we there's a build like this:

foo-1-1.src.rpm
  foo-sub-1-1.noarch.rpm
  foo-debugsource-1-1.x86_64.rpm

The compose contains the noarch package, and should also have the
debugsource package. The original code only checked for
foo-sub-debugsource though.

JIRA: RHELCMP-7628
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2022-01-12 13:59:09 +00:00
Filip Valder fe986d68b9 Add module obsoletes feature
JIRA: MODULAR-113
Merges: https://pagure.io/pungi/pull-request/1578
Signed-off-by: Filip Valder <fvalder@redhat.com>
2022-01-12 12:51:37 +01:00
Ozan Unsal 42f668d969 buildinstall: Add ability to install extra packages in runroot
Resolves: https://pagure.io/pungi/issue/1461
Merges: https://pagure.io/pungi/pull-request/1580
JIRA: RHELCMP-2911
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2022-01-05 14:35:37 +01:00
Haibo Lin 894cce6a5a Ignore osbs/osbuild config when reusing iso images
JIRA: RHELCMP-7562
Signed-off-by: Haibo Lin <hlin@redhat.com>
2022-01-04 16:35:29 +08:00
soksanichenko 0484426e0c ALBS-97: Build AlmaLinux PPC64le repos and ISOs with pungi
- Changelog
- Version is bumped

@BS-NOBUILD
@BS-TARGET-CL8

Change-Id: I933925b7a27a5e1b642020e060f59212fdc6ebf4
2021-12-30 12:42:34 +02:00
soksanichenko b9d86b90e1 ALBS-97: Build AlmaLinux PPC64le repos and ISOs with pungi
- Scripts `create_packages_json` & `gather_modules` can process lzma compressed yaml files
- Script `create_package_json` can use repodata there are packages with different
  arch by compare with passed to the script

@BS-NOBUILD
@BS-TARGET-CL8

Change-Id: Ia9a3bacfa4344f0cf33b9f416649fd4a5f8d3c37
2021-12-28 16:08:04 +02:00
soksanichenko 58a16e5688 - The version is bumped
- The changelog is updated
- The test `create_packages_json` is fixed

@BS-NOBUILD
@BS-TARGET-CL8

Change-Id: I173013da990eb296e58ca8f3555a05913ca1c852
2021-12-20 14:11:17 +02:00
Lubomír Sedlář 260b3fce8d compose: Make sure temporary dirs are world readable
When the temporary directory is created with 0700, other programs
(potentially on another host) will have problems reading it.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: RHELCMP-7635
2021-12-17 08:10:48 +01:00
Haibo Lin 20c2e59218 Pass compose parameter for debugging git issue
With this param, get_dir_from_scm will try to copy the tmp git dir to
compose target dir when error occurs.

This does not fix the issue but it would be helpful for debugging when
it occurs again.

JIRA: RHELCMP-7244
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-11-26 15:40:35 +08:00
Haibo Lin 5e6248e3e0 Generate images.json for extra_isos phase
JIRA: RHELCMP-7241
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-11-15 14:38:53 +08:00
Haibo Lin f681956cf1 Fix tests for python 2.6
It failed to build RHEL 6 package as logging.NullHandler does not exist
in python 2.6

JIRA: RHELCMP-7188
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-11-12 17:04:17 +08:00
Haibo Lin cfb9882269 4.3.2 release
JIRA: RHELCMP-7182
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-11-11 16:08:54 +08:00
soksanichenko f2ed64d952 ALBS-66: Prepare Jenkins jobs for building distribution of AlmaLinux 8.5
- Script `create_packages_json` can duplicate the packages with
  same version in different variants

Change-Id: I3c79ad06c4c22442423c12d5fa06baf82d663a3f
2021-11-10 15:29:59 +02:00
Lubomír Sedlář b652119d54 gather: Load JSON mapping relative to config dir
JIRA: RHELCMP-7195
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-10 11:01:53 +01:00
Lubomír Sedlář 33d7290d78 gather: Stop requiring all variants/arches in JSON
The JSON source file should not require a mapping for all
variants/architectures. When something is specified, it should be
included.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-10 11:01:53 +01:00
Ken Dreyer 9bae86a51e doc: make dnf "backend" settings easier to discover
Mention the corresponding "gather" or "repoclosure" backend settings in
the documentation for each setting.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-11-04 10:55:54 -04:00
Lubomír Sedlář 1d654522be Remove with_jigdo argument
It was checked in a condition together with the configuration value, and
only ever explicitly used with the same value.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-04 13:37:51 +00:00
Lubomír Sedlář 80bd254347 Check dependencies after config validation
This way the checks can rely on default values from the config.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-04 13:37:51 +00:00
Ken Dreyer 94ffa1c5c6 default "with_jigdo" to False
Fedora has not composed with jigdo in a long time. Disable it by
default.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Merges: https://pagure.io/pungi/pull-request/1561
Fixes: https://pagure.io/pungi/issue/1560
2021-11-04 13:37:51 +00:00
Lubomír Sedlář 9d02f87c99 Stop trying to validate non-existent metadata
When a compose doesn't build any images, it won't produce any metadata
file for them, and thus it makes no sense to validate it.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Fixes: https://pagure.io/pungi/issue/1565
2021-11-04 09:57:20 +01:00
fdiprete 7b9e08ab28 test images for metadata deserialization error
Merges: https://pagure.io/pungi/pull-request/1559
Jira: https://issues.redhat.com/browse/RHELCMP-6685
Signed-off-by: fdiprete <fdipretre@redhat.com>
2021-11-03 08:50:07 +01:00
Lubomír Sedlář e2b3002726 repoclosure: Use --forcearch for dnf repoclosure
DNF repoclosure requires this option when checking a repository that is
not compatible with host architecture. It seems that when it is
compatible, it works as well.

Based on how the list of architectures is generated, we know that the
main one will always be first.

Fixes: https://pagure.io/pungi/issue/1562
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 08:51:40 +01:00
Lubomír Sedlář e8305f3978 extra_isos: Allow reusing old images
When nothing in configuration or the image itself changed, let's just
copy the older one.

JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 07:08:00 +00:00
Lubomír Sedlář ac66c3d7f3 createiso: Allow reusing old images
This patch allows Pungi to reuse ISO image created in previous compose
if a list of assumptions proves to hold:

 * If image is bootable, buildinstall phase must have been reused too.
 * Compose configuration must have not changed (except for a few
   whitelisted options).
 * Volume ID of the ISO much not have changed.
 * No RPM on the ISO must have changed.

The ISO also contains other files. Changes in extra files and product ID
certificates should be visible in configuration (the SHA will differ).
Similarly any repodata configuration would be reflected in
configuration.

JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-11-02 07:08:00 +00:00
Lubomír Sedlář eb61c97cdb Remove default runroot channel
When the value is not specified in the configuration file, let Koji pick
the default channel.

JIRA: RHELBLD-8088
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-10-25 15:56:24 +02:00
Ozan Unsal b03490bf18 4.3.1 release
JIRA: RHELCMP-7116
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-10-25 15:18:26 +02:00
Dan Čermák ab19043773 Correct irc network name & add matrix room
Signed-off-by: Dan Čermák <dan.cermak@cgc-instruments.com>
2021-10-25 07:26:18 +00:00
Lubomír Sedlář 204d88a351 Add missing mock to osbs tests
We don't want the test to try to a dummy URL.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-10-21 13:34:37 +02:00
Haibo Lin 8133676270 osbs: Reuse images from old compose
JIRA: RHELCMP-5972
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:22:19 +08:00
Haibo Lin e42e65783d image_build: Allow reusing old image_build results
JIRA: RHELCMP-5970
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-10-20 14:07:02 +08:00
Ozan Unsal 7475d2a3a9 Allow ISO-Level configuration within the config file
In order to enable this feature set "iso_level=<Value from 1 to 4>"
in config file

Jira: RHELCMP-6880

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-10-18 16:32:04 +02:00
Lubomír Sedlář ac061b2ea8 Work around ODCS creating COMPOSE_ID later
When ODCS starts a compose, it will provide base composeinfo file, but
it doesn't create COMPOSE_ID. This leads to a crash when updating CTS,
since the compose id can't be read from the file. We can instead use the
value we already have in memory.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-10-04 11:32:59 +00:00
Jan Kaluza 0530cf2712 When `cts_url` is configured, use CTS `/repo` API for buildContainer yum_repourls.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2021-09-24 10:29:28 +02:00
Ozan Unsal 9612241396 Add COMPOSE_ID into the pungi log file
Jira: RHELCMP-6739

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-21 10:55:03 +02:00
Lubomír Sedlář ba6f7429ee buildinstall: Add easy way to check if previous result was reused
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-14 10:09:44 +02:00
Lubomír Sedlář 72bcee01be 4.3.0 release
JIRA: RHELCMP-6614
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-10 11:38:11 +02:00
Lubomír Sedlář a1ebd234a4 Only build CTS url when configured
JIRA: RHELCMP-6611
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-10 11:29:53 +02:00
Lubomír Sedlář 5c26aa9127 Require requests_kerberos only when needed
If CTS integration is not used, let's not import a module that is not
needed.

JIRA: RHELCMP-6611
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-10 10:00:54 +02:00
Jan Kaluza 195bfbefa4 Allow specifying $COMPOSE_ID in the `repo` value for osbs phase.
There should be an option for `yum_repourls` to point to static
URL, for example when CTS is used. The idea is that instead of
setting `repo` to `AppStream`, we could use link similar to this one:

`https://cts.localhost/api/1/composes/$COMPOSE_ID/repo/?variant=AppStream`

This would be translated to real static link during the OSBS phase:

`https://cts.localhost/api/1/composes/CentOS-Stream-9-20210803.0/repo/?variant=AppStream`

That way this statis link would appear in the yum_repourls.

Merges: https://pagure.io/pungi/pull-request/1543
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2021-09-09 10:46:33 +02:00
Lubomír Sedlář 20dc4beb6b Make getting old compose config reusable
The file will only be loaded once, it gets cached afterwards.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-09 10:38:34 +02:00
Lubomír Sedlář d8d1cc520b paths: Allow customizing log file extension
If the file contents is JSON, it would be nice to have matching
extension.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-09 10:38:34 +02:00
Ozan Unsal 904a1c3271 Add authentication for updating the compose URL in CTS.
Put authentication steps in a function in order to prevent code duplication.

Jira: RHELCMP-6318

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-09 07:27:55 +00:00
Lubomír Sedlář e8ddacd10e Fix type detection for osbuild images
The image type value passed to the task doesn't match the type as it
will be recorded by Koji.

JIRA: RHELCMP-5727
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2021-09-08 09:07:00 +00:00
Ozan Unsal b7666ba4a4 Enable pungi to send compose_url patches to CTS
If cts_keytab is also enabled then the HTTP requests are handled with
Kerberos Authentication otherwise no authentication is used.

If cts_url is defined in the configuration, translate_paths is required.
This is needed in order to get the host and the path of the composes.

Jira: RHELCMP-6318

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-09-07 16:41:35 +02:00
Ozan Unsal 3d9335e90e Use xorriso instead of isoinfo when createiso_use_xorrisofs is enabled
Update get_mkisofs_cmd in createiso.py file in order to prevent using
default value. With this change it is possible to enable xorriso format

Jira: RHELCMP-6325

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-08-31 08:09:09 +00:00
Dominik Rumian 7c3e8d4276 Fix tests for createrepo
Tests for createrepo failed when pungi is installed in system.

JIRA: RHELCMP-6209

Signed-off-by: Dominik Rumian <drumian@redhat.com>
2021-08-30 13:48:29 +02:00
Dominik Rumian 9cd42a2b5e Formatted files according to flake8 and black feedback
Signed-off-by: Dominik Rumian <drumian@redhat.com>
2021-08-30 09:55:17 +02:00
Ozan Unsal 980c7ba8fb Handle the pungi failures to ensure creation of log files
If the given directory is not a valid git directory, it raises RuntimeError.
This can be catched and raised as GitUrlResolveError, so compose can continue
to log the failure.

Jira: RHELCMP-6077

Signed-off-by: Ozan Unsal <ounsal@redhat.com>
2021-08-25 11:15:23 +00:00
Haibo Lin 66dacb21e0 Add createrepo_enable_cache to configuration doc
JIRA: RHELCMP-5984
Signed-off-by: Haibo Lin <hlin@redhat.com>
2021-08-18 11:14:55 +08:00