Commit Graph

111 Commits

Author SHA1 Message Date
Lubomír Sedlář 1b6688d22c [buildinstall] Don't run lorax for empty variants
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-18 16:12:44 +01:00
Lubomír Sedlář 71369c7690 Merge #159 `use $basearch not $arch in livemedia tasks` 2016-02-18 13:56:15 +00:00
Tomas Mlcoch 5bffca5037 Support signing of rpm wrapped live images
With this patch, you can specify a command for
signing of koji builds. For example:

    signing_key_password_file = '~/file_with_password_for_key_fedora-24'
    signing_key_id = '81b46521'
    signing_command = '~/git/releng/scripts/sigulsign_unsigned.py -vv --password=%(signing_key_password)s fedora-24'

'signing_key_password_file' is a path to a file which contains
a password that will be formatted into 'signing_command' string
via '%(signing_key_password)s' string format syntax (if used).
Because pungi config is usualy stored in git and part of compose
logs we don't want password to be included directly in the config.
Note: If '-' is used instead of a filename, then you will be asked
for the password interactivelly right after pungi starts.

'signing_key_id' is ID of the key that will be used for the signing.
This ID will be used when crafting koji paths to signed files
(kojipkgs.fedoraproject.org/packages/NAME/VER/REL/data/signed/KEYID/..).

'signing_command' a command that will be run with a build as a single
argument. This command mustn't require any user interaction.
If you need to pass a password for a signing key to the command,
do this via command line option of the command with use of string
formatting syntax '%(signing_key_password)s' (see details
about 'signing_key_password_file').

Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:36:38 +01:00
Dennis Gilmore 1616de1e1e use $basearch not $arch in livemedia tasks
koji for livemedia acts different to image builds and does
not translate $arch to x86_64, i386, etc so we need to just pass
in $basearch so that yum/dnf will translate it for us

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-17 17:03:46 -06:00
Lubomír Sedlář bf5196af4a [live-images] Don't tweak kickstarts
Instead of downloading the kickstart file in Pungi and modifying it,
just pass it to Koji.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-17 14:35:28 +01:00
Lubomír Sedlář 454363fba8 Allow specifying empty variants
The variants.xml file can list a variant with is_empty="true" and no
groups. If such variant is found, not package gathering will be run for
it, and no repos will be created.

This only makes sense for a variant that will have some other
deliverables like live media or images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář adbc772fd0 [createrepo] Remove dead assignments
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář 284d3a3510 [image-build] Use dashes as arch separator in log
Since multiple images are created in a single Koji task, the path to log
file contains a list of arches. These used to be separated with comma.
For consistency with live-media they are now separated by dashes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-15 09:04:03 +01:00
Lubomír Sedlář abcc51fb85 [buildinstall] Stop parsing task_id
KojiWrapper returns the task id as an integer already (if at all). There
is no need to parse it again.

Pagure: #148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:39:50 +01:00
Dennis Gilmore 96489a1e78 Merge #146 `[live-media] Properly calculate iso dir` 2016-02-11 13:07:30 +00:00
Lubomír Sedlář 8ef2e7e308 [live-media] Properly calculate iso dir
The method expects to be called with a arch, not just variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 13:58:42 +01:00
Lubomír Sedlář e9a42ad54f [image-build] Fix tests
This patch updates tests so that they pass, and fixes a couple places in
image build that did not work correctly with the nested configuration.
The documentation is reformatted so that it horizontal scrolling is less
likely.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 13:22:26 +01:00
Lubos Kocman f067f38df0 add image-build sections
allows to use factory-parameters ...

Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2016-02-11 13:22:16 +01:00
Lubomír Sedlář 5138a9b6e6 [live-images] Add support for spin-appliance
The config can specify type=appliance so that koji uses spin-appliance
instead of spin-livecd.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 12:08:48 +01:00
Lubomír Sedlář ac29bf05ee [live-media] Koji option is ksfile, not kickstart
In config, it is still called kickstart to be consistent with how
image-build calls it.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:40:04 +01:00
Lubomír Sedlář 6ec151cde2 [live-media] Use install tree from another variant
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:24:12 +01:00
Lubomír Sedlář 0c4bd42f80 [live-media] Put images into iso dir
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:11:38 +01:00
Lubomír Sedlář 91faa0fafe [image-build] Koji expects arches as a comma separated string
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 07:50:02 +01:00
Dennis Gilmore 8bf1b09641 Merge #139 `Log more details when any deliverable fails` 2016-02-10 21:03:32 +00:00
Lubomír Sedlář b35264cfba [live-media] Version is required argument
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-10 18:29:29 +01:00
Lubomír Sedlář f57665f963 [buildinstall] Improve logging
This patch fixes how logs are stored if lorax is used as buildinstall
method. The logs for each variant are in a separate file now. If failure
is allowed, the global log will now show why it failed.

A couple tests are added as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 13:17:08 +01:00
Lubomír Sedlář d661de3e73 Log more details about failed deliverable
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 12:07:04 +01:00
Dennis Gilmore 2b897ec6ea Merge #135 `Add live media support` 2016-02-02 10:15:56 +00:00
Dennis Gilmore a2ea84934b Merge #133 `media_split: add logger support. Helps with debugging space issues on dvd media` 2016-02-02 10:11:46 +00:00
Lubomír Sedlář 439622d576 [live-media] Add live media phase
This phase builds live media in Koji using the Live Media Creator. It
runs in parallel with current live images, create ISO and image build
phases.

The documentation is updated to explain how to configure this.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-01 13:28:49 +01:00
Lubomír Sedlář ae30c07553 [koji-wrapper] Use more descriptive method names
The methods mentioning image build are generic and can work for other
task types.

get_image_build_paths -> get_image_paths
run_create_image_cmd -> run_blocking_cmd

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-29 09:02:02 +01:00
Lubomír Sedlář d85f00818d [image-build] Remove dead code
There are no mounts when running image-build tasks in Koji. This looks
like a copy-paste error.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-29 09:02:02 +01:00
Lubos Kocman f0a8899ea7 media_split: add logger support. Helps with debugging space issues on dvd media
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2016-01-26 13:02:14 +00:00
Lubomír Sedlář a698c7cc0a [image-build] Allow running image build scratch tasks
Issue #130

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-26 12:47:59 +01:00
Lubomír Sedlář dc4d502f2a [image-build] Allow dynamic release for images
When the release is explicitly set to None, generate a value from date
and respin. The documentation is updated to explain how it works.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-21 14:32:29 +01:00
Dennis Gilmore c77e78c4f1 Merge #123 `Live images: add repo from another variant` 2016-01-20 15:37:36 +00:00
Dennis Gilmore 09aebd3db9 Merge #125 `[image-build] Stop creating wrong arch dirs` 2016-01-20 15:37:27 +00:00
Lubomír Sedlář 8b4ad1ea02 Toggle multilib per variant
There is now a single option `multilib`, that maps variants and arches
to multilib methods. This replaces old `multilib_methods` option.

Multilib arches are implicitly deduced instead of using the
`multilib_arches` option.

The test compose is updated to only enable multilib on Server and its
addons.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-20 12:53:08 +01:00
Lubomír Sedlář 6d7fff5d1b [live-images] Code cleanup
This patch removes some duplicated variables that get passed on to the
build thread. It also moves creation of the command for generating image
manifest closer to where it is used. Finally it adds tests for the
thread.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:26:59 +01:00
Lubomír Sedlář 9059852ab5 [live-images] Add repos from other variants
This patch adds option for adding repositories from other variants to
the koji task for building images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:26:59 +01:00
Lubomír Sedlář e016015cf3 [image-build] Stop creating wrong arch dirs
There is no such arch as $arch or %(arch)s.

Pagure: #124
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:24:02 +01:00
Lubomír Sedlář f3f0ce0d35 Fix filtering of system release packages
The code was inconsistent with the documentation: the logic was
inverted.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-13 08:52:06 +01:00
Dennis Gilmore 5f7f23ed96 Merge #114 `Use install tree/repo from another variant for image build` 2016-01-12 17:56:38 +00:00
Lubomír Sedlář b78878ac69 Make system release package filtering optional
Fedora variants needs both general fedora-release and
fedora-release-$variant packages installed. This patch disables the code
that would throw fedora-release away (assuming correct config is set).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-12 17:04:37 +01:00
Lubomír Sedlář 5688f1cbae [image-build] Optionally do not break whole compose
The configuration can now specify image-build as a deliverable that can
fail but not abort the whole compose.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-12 08:39:30 +01:00
Lubomír Sedlář af11bebf1b [image-build] Refactoring
* Move getting arches to separate method.
 * Modify git url only if there is going to be a build. This is not very
   likely to happen, but could save a bit of time if it really does
   happen.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-11 16:42:05 +01:00
Lubomír Sedlář d2ea1dd288 [image-build] Use repo from another variant
The config can now additionally specify other variants whose repos will
be passed on to koji. The previous way of specifying extra repos is
still available, as is the automatic adding of repo for current variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-11 16:42:05 +01:00
Lubomír Sedlář 7f815d80f5 [image-build] Take install tree from another variant
Add a configuration option to use install tree from another variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-11 16:42:05 +01:00
Lubomír Sedlář 9c1418eb0a [image-build] Use single koji task per variant
Given a list of arches, koji can build multiple images in one go
(automatically starting children tasks for each one).

This causes a bunch of changes:
 * The configuration no longer allows changing config based on
   architecture, only variants are allowed. It is however possible to
   filter which arches are used for the building in the variant.
 * The configuration files for koji image-build are stored in
   work/image-build/$variant (not split based on arch).

This patch also changes the option name that is passed to koji
image-build: the repos should be specified under key `repo` (without the
trailing slash).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-08 14:33:54 +01:00
Lubomír Sedlář 88bb6ac3d7 Fix image-build modifying config
The code modifies the config in place, so it only works in the first
iteration. The fix is to create a copy which can be safely modified and
next iteration will still see the original data.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-06 16:53:00 +01:00
Lubomír Sedlář 770a81b76c Don't crash on generating volid without variant
First, this fixes the crash when volume id requires variant uid, but it
is not specified. This happened in buildinstall phase.

When lorax is used, it is run once for each variant, so we can actually
specify the variant and get the correct variant.

When using buildinstall, no volume id will be generated and pungi will
possibly crash later on.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-04 19:27:11 +01:00
Dennis Gilmore 1af17a5262 Merge #99 `Add option to specify non-failing stuff` 2015-12-17 18:18:12 +00:00
Lubomír Sedlář 0222d4dc6d Add repo from current compose
This gets added together with the any repos from configuration.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-17 12:15:17 -06:00
Lubomír Sedlář 7a965bcb83 Fix getting compose topdir in CreateImage build thread
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-17 12:15:13 -06:00
Lubomír Sedlář a6b673dbdd Add option to specify non-failing stuff
There is a new configuration option that allows listing what can fail
without aborting the whole compose. So far, only buildinstall, createiso
and liveimages phases react to this option.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-17 11:15:47 +01:00