Commit Graph

285 Commits

Author SHA1 Message Date
Lubomír Sedlář
20c3614fb3 scm: Add backend for downloading archives from Koji
Tests and documentation included.

JIRA: COMPOSE-3816
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-11-05 08:51:26 +01:00
Lubomír Sedlář
8829fc32ab 4.1.40 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-10-23 09:02:41 +02:00
Juliana Rodrigueiro
f8dcda9dcb doc: Make it clear that the field 'repo' has to be empty for 'file' backend
Leaving the field 'repo' out for the 'file' backend makes the compose fail.

    [ERROR   ] Compose run failed: 'repo'

Signed-off-by: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com>
2019-10-18 10:30:25 +02:00
Lubomír Sedlář
014560f8bd doc: Add full examples of compose configuration
JIRA: COMPOSE-3797
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-10-14 11:04:39 +02:00
Owen W. Taylor
72bf795bd4 ostree-install: allow configuring additional depenencies for runroot
A lorax template used for the ostree-installer might need an additional
package dependency (e.g., flatpak to embed a flatpak repository) - add
a config key 'extra_runroot_pkgs' to the ostree installer configuration
to allow supplementing the set of packages installed into the runroot.

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2019-10-10 15:13:38 -04:00
Lubomír Sedlář
c346492df4 buildinstall: Allow customizing dracut arguments
JIRA: COMPOSE-3853
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-10-09 15:42:10 +02:00
Lubomír Sedlář
21d45eb243 pkgset: Allow filtering modules from Koji tags
Add a configuration option to enable skipping some modules found in the
configured tag.

Fixes: https://pagure.io/pungi/issue/1260
JIRA: COMPOSE-3794
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-09-24 10:13:00 +02:00
Lubomír Sedlář
e814e1445a 4.1.39 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-09-09 09:30:45 +02:00
Haibo Lin
3811c0a176 config: Deprecate bootable option
JIRA: COMPOSE-2635
Fixes: https://pagure.io/pungi/issue/976
Signed-off-by: Haibo Lin <hlin@redhat.com>
2019-07-22 16:51:38 +08:00
Lubomír Sedlář
3097019338 test: Allow turning strictness off as well
When the value is explicitly set to `False`, the check should not be
strict.

JIRA: COMPOSE-3658
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-07-16 08:24:18 +02:00
Lubomír Sedlář
62c6d4ddcf test: Option to make size check strict
Sometimes it's practical not just warn when ISO is larger than expected,
but to also abort the compose.

JIRA: COMPOSE-3658
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-07-15 13:57:50 +02:00
Lubomír Sedlář
a340663f00 doc: Clarify relationship to Koji
Fixes: https://pagure.io/pungi/issue/1143
JIRA: COMPOSE-3372
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-07-03 09:06:49 +02:00
Lubomír Sedlář
46333b01ad 4.1.38 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-07-02 10:32:01 +02:00
Haibo Lin
c4ed2bf3b2 config: Deprecate runroot option
Fixes: https://pagure.io/pungi/issue/975
Signed-off-by: Haibo Lin <hlin@redhat.com>
2019-06-27 18:00:07 +08:00
Haibo Lin
ce066707c1 config: Deprecate release_is_layered option
Fixes: https://pagure.io/pungi/issue/977
Merges: https://pagure.io/pungi/pull-request/1204
Signed-off-by: Haibo Lin <hlin@redhat.com>
2019-06-06 13:45:49 +02:00
Lubomír Sedlář
163d69713d 4.1.37 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-05-27 10:40:38 +02:00
Jan Kaluza
cfb7b71fca Fix issues in OpenSSH Runroot method found by real tests.
- Pass the runroot_tag to init command in OpenSSH Runroot method.
  This is needed for the init command as a source for initial packages
  for the buildroot.
- Rename the "runroot_ssh_init_command" to "runroot_ssh_init_template"
  to make it consistent with the rest of "runroot_ssh_*" options.
- Add missing "runroot_ssh_*" options to checks.py.
- Use chmod/chown to `output_dir` in OpenSSH Runroot method the same way
  as it is used in Koji runroot method to make the runroot output readable
  for Pungi user.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2019-05-15 07:40:31 +02:00
Lubomír Sedlář
479d17c033 4.1.36 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-04-12 12:10:39 +02:00
Jan Kaluza
49d137b444 Extend "openssh" runroot_method to be able to execute "mock"
This adds few new config options which are well described in the
configuration documentation. Please refer to it for more information.

Merges: https://pagure.io/pungi/pull-request/1170
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2019-04-12 11:10:40 +02:00
Lubomír Sedlář
d521711957 osbs: Rework configuration for image pushes
Embedding the registry configuration into OSBS config itself is
simple, but makes it impossible to reuse the same configuration for
multiple different composes.

A nice example is a nightly pushing images to a testing registry, and
production compose building the same images but pushing to staging
location. The original design requires duplication of all the
configuration just because registries are different.

With this option, the push information is stored in a separate option as
a mapping from NVR patterns to arbitrary data. The patterns are used to
match finished builds to registry.

The old configuration is marked as deprecated in code and will
eventually be removed. The deprecation handling in config validation
does not allow emitting warnings for nested values.

JIRA: COMPOSE-3394
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-04-05 10:28:34 +02:00
Jan Kaluza
959d6979d4 Add "openssh" runroot method
Merges: https://pagure.io/pungi/pull-request/1166
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2019-04-04 14:44:05 +02:00
Lubomír Sedlář
5f8b519941 4.1.35 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-26 10:03:26 +01:00
Lubomír Sedlář
358fdd50ce buildinstall: Allow overwriting version for lorax
Sometimes the release version can be more specific than what should be
exposed to users of the boot iso.

JIRA: COMPOSE-3295
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-22 12:47:16 +01:00
Jan Kaluza
75bb48a882 Create new Runroot class and use it everywhere where runroot task is executed.
This adds new `Runroot` class and new `runroot_method` option which makes
it possible to choose between two currently available runroot methods:

- Local
- Koji

The main goal of this commit is to make it possible to add new runroot
methods in the future and this is the first step in that direction.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2019-03-21 14:15:58 +01:00
Lubomír Sedlář
45cdbb2faf orchestrator: Send messages about the main compose
Only start/finish messages will be sent if a handler is configured.

JIRA: COMPOSE-3288
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-21 09:29:54 +01:00
Lubomír Sedlář
088ea7fe37 orchestrator: Support generic pre- and post- scripts
Run arbitrary commands before and after the compose.

The example config is updated to generate latest symlink with a
post-compose script. The pre compose script runs always, post compose
runs only if the compose is not doomed.

JIRA: COMPOSE-3288
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-21 09:29:54 +01:00
Lubomír Sedlář
86fb93d603 orchestrator: Support getting kerberos ticket
If the configuration sets keytab path and principal, run kinit with
custom cache file, and delete the file at the end of the run.

JIRA: COMPOSE-3288
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-21 09:29:54 +01:00
Ken Dreyer
b5bef68c60 doc: explain koji_profile
Add more details to the documentation for the koji_profile setting.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2019-03-14 15:53:15 -06:00
Lubomír Sedlář
cc456d3e75 4.1.34 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-03-05 12:37:07 +01:00
Lubomír Sedlář
3efcfede6a image-build: Support repo/install_tree as path
If a repo or install tree is specified as an absolute path on the local
filesystem, we should either translate it using the configured mappings,
or if no mapping matches, it should be return unchanged. A variant name
can not start with a slash, so attempting that translation does not make
much sense.

JIRA: COMPOSE-3290
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-02-25 08:18:47 +01:00
Lubomír Sedlář
f33973ee65 Make the Apple/HFS compatibility configurable
The default is the original behaviour. On F30+ a new option should be
added to config to make it work.

Over time as users move to this option (which requires a new enough
version of lorax), the default should be switched and then the option
removed.

Resolves: https://pagure.io/pungi/issue/1126
Merges: https://pagure.io/pungi/pull-request/1128
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-02-21 16:11:01 +01:00
Lubomír Sedlář
e21c49d9e2 4.1.33 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-02-13 10:37:48 +01:00
Lubomír Sedlář
6c6d4759f5 isos: Check maximum expected size
This patch allows the configuration to express maximum expected size for
ISOs created in createiso and extra_isos phases. If the image is larger
than this limit, a warning is emitted in test phase. The compose itself
is not affected in any way.

JIRA: COMPOSE-2824
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-02-12 15:15:30 +01:00
Lubomír Sedlář
7693e562b1 osbs: Process data about pushing images to registries
This patch does not do any actual pushing. It will only extract data
about push targets from the main configuration and store it together
with exact Koji NVR in a well-defined location, and also send the data
to message bus for another service to handle.

JIRA: COMPOSE-3228
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-02-12 15:12:50 +01:00
Lubomír Sedlář
dbc0d5736c 4.1.32 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-01-08 12:17:29 +01:00
Lubomír Sedlář
90c60f8e64 Add script to orchestrate multiple composes
It may make sense to break a big compose into smaller chunks that can be
done independently. This script allows describing the smaller parts,
runs them with correct dependencies and arranges the result to look like
a single big compose.

All parts use the same koji event, that is either obtained from Koji, or
from command line argument.

JIRA: COMPOSE-2654
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-01-08 09:40:36 +01:00
Lubomír Sedlář
ba260c24e8 buildinstall: Expose lorax's --rootfs-size argument
We already make this possible for the ostree installer, but it was
missing from the traditional one. The default behaviour is to let lorax
decide, but if user knows better, they can overwrite in configuration.

JIRA: COMPOSE-3188
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-01-07 11:00:43 +01:00
Ondrej Nosek
70f46dfb62 Unify update-docs.sh script with rpkg
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2019-01-04 13:32:17 +01:00
Lubomír Sedlář
610bf4b125 Remove createrepo references from doc and spec
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2019-01-03 10:37:59 +01:00
Ken Dreyer
27d5073b98 doc: explain product_id_allow_missing results in detail
Add more details to the documentation for the product_id_allow_missing
setting:

- Pungi expects a product cert for each arch in each variant.

- Pungi will exit with an error by default if a variant+arch's cert is
  missing.

- Pungi will log a warning if product_id_allow_missing is True.

Merges: https://pagure.io/pungi/pull-request/1102
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2019-01-03 10:14:53 +01:00
Ken Dreyer
7c45157a92 doc: describe product_id's output and purpose
Document the "productid" files that Pungi creates when the user
configures the "product_id" setting. Describe where to find the
"productid" files, and how they relate to subscription-manager.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2019-01-02 14:22:21 -05:00
Lubomír Sedlář
7c48b808f1 4.1.31 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-11-26 13:14:09 +01:00
Lubomír Sedlář
dc692bc604 extra_iso: Stop including variant extra files
They are not always wanted, so let's not include them by default.
There's a new option to include the same files that extra files phases
uses, or alternatively they can be configured specifically and put into
the variant subdirectory.

JIRA: COMPOSE-3084
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-11-09 12:06:25 +01:00
Lubomír Sedlář
fa752eb2b5 Configure image name per variant
Up to now it was possible to change the pattern for all images, but
there are use-cases where different variants might want different names.
For example there could be one main variant that should only have
product name in the ISO filename, but addons should still be marked with
variant name.

JIRA: COMPOSE-3041
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-11-05 08:48:36 +01:00
Lubomír Sedlář
65a1779e3a 4.1.30 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-26 14:27:53 +02:00
Lubomír Sedlář
0c8702cd6d 4.1.29 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-10 12:51:04 +02:00
Jan Kaluza
37c89dfde6 Add 'pkgset_koji_builds' option to include extra builds in a compose
This PR adds new pkgset_koji_builds configuration option.

This option allows setting list of extra Koji build NVRs which will be
included in a compose. This is useful in two cases:

a) It allows generating standard composes with few packages update to
certain version to test how the compose behaves when the package is
updated for real.

b) It allows generating compose consisting only from particular builds
when pkgset_koji_tag = '' or None. This is useful when one want to
regenerate the compose with packages which are not tagged in single Koji
tag. This is very useful for ODCS when reproducing old composes.

Merges: https://pagure.io/pungi/pull-request/1049
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-09-21 14:06:49 +02:00
Lubomír Sedlář
21b8f3af3c 4.1.28 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-09-06 13:43:04 +02:00
Lubomír Sedlář
fbb739ef17 pkgset: Apply whitelist to modules in the tag
This patch changes the behaviour when both module tag and NSV?C? is
specified. The NSVC are used as a whitelist and only matching modules
will be included in the compose.

Additionally this patch adds filtering based on inheritance: when
finding the latest module for each N:S combination, only the top tag in
which the module is tagged is used. Even if a newer build is available
somewhere deeper in the inheritance, it's not going to be used.

Example inheritance and tagged modules

    f29-compose (foo:1:2018:cafe)
    └─ f29-candidate (foo:1:2019:cafe)

The compose will use 2018 version, because it's in the topmost tag.

JIRA: COMPOSE-2685
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-09-03 12:19:50 +02:00
Jan Kaluza
d2f392fac8 Use dogpile.cache to cache the listTaggedRPMS calls if possible
If the same tag is queried with the same event, Pungi can cache the
response and call the API again. Particularly for small composes this
can save up significant amount of time.

Merges: https://pagure.io/pungi/pull-request/1022
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-08-30 13:21:01 +02:00