Commit Graph

1880 Commits

Author SHA1 Message Date
Tomas Mlcoch
5bffca5037 Support signing of rpm wrapped live images
With this patch, you can specify a command for
signing of koji builds. For example:

    signing_key_password_file = '~/file_with_password_for_key_fedora-24'
    signing_key_id = '81b46521'
    signing_command = '~/git/releng/scripts/sigulsign_unsigned.py -vv --password=%(signing_key_password)s fedora-24'

'signing_key_password_file' is a path to a file which contains
a password that will be formatted into 'signing_command' string
via '%(signing_key_password)s' string format syntax (if used).
Because pungi config is usualy stored in git and part of compose
logs we don't want password to be included directly in the config.
Note: If '-' is used instead of a filename, then you will be asked
for the password interactivelly right after pungi starts.

'signing_key_id' is ID of the key that will be used for the signing.
This ID will be used when crafting koji paths to signed files
(kojipkgs.fedoraproject.org/packages/NAME/VER/REL/data/signed/KEYID/..).

'signing_command' a command that will be run with a build as a single
argument. This command mustn't require any user interaction.
If you need to pass a password for a signing key to the command,
do this via command line option of the command with use of string
formatting syntax '%(signing_key_password)s' (see details
about 'signing_key_password_file').

Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:36:38 +01:00
Tomas Mlcoch
be4d596c36 Fix terminology - Koji uses sigkey not level
Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:35:27 +01:00
Dennis Gilmore
1616de1e1e use $basearch not $arch in livemedia tasks
koji for livemedia acts different to image builds and does
not translate $arch to x86_64, i386, etc so we need to just pass
in $basearch so that yum/dnf will translate it for us

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-17 17:03:46 -06:00
Dennis Gilmore
2858b1aa54 do not uses pipes.quotes in livemedia tasks
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-17 14:39:51 -06:00
Lubomír Sedlář
bf5196af4a [live-images] Don't tweak kickstarts
Instead of downloading the kickstart file in Pungi and modifying it,
just pass it to Koji.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-17 14:35:28 +01:00
Lubomír Sedlář
454363fba8 Allow specifying empty variants
The variants.xml file can list a variant with is_empty="true" and no
groups. If such variant is found, not package gathering will be run for
it, and no repos will be created.

This only makes sense for a variant that will have some other
deliverables like live media or images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář
adbc772fd0 [createrepo] Remove dead assignments
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář
826afdc162 Keep empty query string in resolved git url
Pagure: #152
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 10:42:59 +01:00
Lubomír Sedlář
284d3a3510 [image-build] Use dashes as arch separator in log
Since multiple images are created in a single Koji task, the path to log
file contains a list of arches. These used to be separated with comma.
For consistency with live-media they are now separated by dashes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-15 09:04:03 +01:00
Lubomír Sedlář
abcc51fb85 [buildinstall] Stop parsing task_id
KojiWrapper returns the task id as an integer already (if at all). There
is no need to parse it again.

Pagure: #148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:39:50 +01:00
Lubomír Sedlář
e1895bff26 [koji-wrapper] Get task id from failed runroot
Even when runroot task fails, it can emit task_id.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:38:45 +01:00
Lubomír Sedlář
93015d679c [live-media] Pass ksurl to koji
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 17:09:30 +01:00
Dennis Gilmore
96489a1e78 Merge #146 [live-media] Properly calculate iso dir 2016-02-11 13:07:30 +00:00
Lubomír Sedlář
8ef2e7e308 [live-media] Properly calculate iso dir
The method expects to be called with a arch, not just variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 13:58:42 +01:00
Lubomír Sedlář
e9a42ad54f [image-build] Fix tests
This patch updates tests so that they pass, and fixes a couple places in
image build that did not work correctly with the nested configuration.
The documentation is reformatted so that it horizontal scrolling is less
likely.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 13:22:26 +01:00
Lubos Kocman
f067f38df0 add image-build sections
allows to use factory-parameters ...

Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2016-02-11 13:22:16 +01:00
Lubomír Sedlář
39c688f7ef [koji-wrapper] Add tests for get_create_image_cmd
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 12:23:54 +01:00
Lubomír Sedlář
5138a9b6e6 [live-images] Add support for spin-appliance
The config can specify type=appliance so that koji uses spin-appliance
instead of spin-livecd.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 12:08:48 +01:00
Lubomír Sedlář
ac29bf05ee [live-media] Koji option is ksfile, not kickstart
In config, it is still called kickstart to be consistent with how
image-build calls it.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:40:04 +01:00
Lubomír Sedlář
6ec151cde2 [live-media] Use install tree from another variant
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:24:12 +01:00
Lubomír Sedlář
0c4bd42f80 [live-media] Put images into iso dir
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 10:11:38 +01:00
Lubomír Sedlář
91faa0fafe [image-build] Koji expects arches as a comma separated string
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 07:50:02 +01:00
Dennis Gilmore
8bf1b09641 Merge #139 Log more details when any deliverable fails 2016-02-10 21:03:32 +00:00
Lubomír Sedlář
b35264cfba [live-media] Version is required argument
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-10 18:29:29 +01:00
Lubomír Sedlář
aec57dc72b [koji-wrapper] Only parse output on success
If Koji fails runroot task for some reason, the output will most likely
not have the required format and will crash Pungi.

Pagure: #140
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-10 09:08:21 +01:00
Lubomír Sedlář
8e5e893e5c [koji-wrapper] Add tests for runroot wrapper
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 13:27:18 +01:00
Lubomír Sedlář
f57665f963 [buildinstall] Improve logging
This patch fixes how logs are stored if lorax is used as buildinstall
method. The logs for each variant are in a separate file now. If failure
is allowed, the global log will now show why it failed.

A couple tests are added as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 13:17:08 +01:00
Lubomír Sedlář
d661de3e73 Log more details about failed deliverable
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 12:07:04 +01:00
Lubomír Sedlář
3baa386ed4 [image-build] Fix failable tests
A missing input value was causing tests to not check the expected
condition (even though they were still passing).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 10:54:24 +01:00
Dennis Gilmore
2b897ec6ea Merge #135 Add live media support 2016-02-02 10:15:56 +00:00
Dennis Gilmore
a2ea84934b Merge #133 media_split: add logger support. Helps with debugging space issues on dvd media 2016-02-02 10:11:46 +00:00
Lubomír Sedlář
439622d576 [live-media] Add live media phase
This phase builds live media in Koji using the Live Media Creator. It
runs in parallel with current live images, create ISO and image build
phases.

The documentation is updated to explain how to configure this.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-01 13:28:49 +01:00
Lubomír Sedlář
a5a0f3d69f [koji-wrapper] Add support for spin-livemedia
This patch adds support for live media creator in Koji. The intended
workflow is to create a command , run it and finally collect built
artifacts.

    get_live_media_cmd()
    run_blocking_cmd()
    get_image_paths()

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-29 09:02:02 +01:00
Lubomír Sedlář
ae30c07553 [koji-wrapper] Use more descriptive method names
The methods mentioning image build are generic and can work for other
task types.

get_image_build_paths -> get_image_paths
run_create_image_cmd -> run_blocking_cmd

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-29 09:02:02 +01:00
Lubomír Sedlář
d85f00818d [image-build] Remove dead code
There are no mounts when running image-build tasks in Koji. This looks
like a copy-paste error.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-29 09:02:02 +01:00
Lubos Kocman
f0a8899ea7 media_split: add logger support. Helps with debugging space issues on dvd media
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2016-01-26 13:02:14 +00:00
Lubomír Sedlář
a698c7cc0a [image-build] Allow running image build scratch tasks
Issue #130

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-26 12:47:59 +01:00
Lubomír Sedlář
dc4d502f2a [image-build] Allow dynamic release for images
When the release is explicitly set to None, generate a value from date
and respin. The documentation is updated to explain how it works.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-21 14:32:29 +01:00
Dennis Gilmore
39ce2556c3 4.0.4 release
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-01-20 09:41:18 -06:00
Dennis Gilmore
c77e78c4f1 Merge #123 Live images: add repo from another variant 2016-01-20 15:37:36 +00:00
Dennis Gilmore
09aebd3db9 Merge #125 [image-build] Stop creating wrong arch dirs 2016-01-20 15:37:27 +00:00
Lubomír Sedlář
8b4ad1ea02 Toggle multilib per variant
There is now a single option `multilib`, that maps variants and arches
to multilib methods. This replaces old `multilib_methods` option.

Multilib arches are implicitly deduced instead of using the
`multilib_arches` option.

The test compose is updated to only enable multilib on Server and its
addons.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-20 12:53:08 +01:00
Lubomír Sedlář
6d7fff5d1b [live-images] Code cleanup
This patch removes some duplicated variables that get passed on to the
build thread. It also moves creation of the command for generating image
manifest closer to where it is used. Finally it adds tests for the
thread.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:26:59 +01:00
Lubomír Sedlář
b16699ddfc [live-images] Add documentation
Add basic documentation for live-images phases. So far, there are no
examples, just a listing of accepted options.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:26:59 +01:00
Lubomír Sedlář
9059852ab5 [live-images] Add repos from other variants
This patch adds option for adding repositories from other variants to
the koji task for building images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:26:59 +01:00
Lubomír Sedlář
e016015cf3 [image-build] Stop creating wrong arch dirs
There is no such arch as $arch or %(arch)s.

Pagure: #124
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-19 08:24:02 +01:00
Lubomír Sedlář
6e0f6ee73e Enable identifying variants in exception traces
When pungi crashes, the logs will often contain useless description of
a variant. With this patch, the traceback will have details on the
variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-15 08:41:08 +01:00
Lubomír Sedlář
5f0675dd66 Store which deliverables failed
When compose is finished successfully, and there are some failed
deliverables, modify the final status to FINISHED_INCOMPLETE and log
what failed for which variants/arches.

This means the failures are logged twice, first time immediately after
it failed, second time in the summary at the end.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-15 08:41:08 +01:00
Lubos Kocman
7ec409a236 scm.py: use git clone instead git archive for http(s)://
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2016-01-14 06:42:57 -05:00
Lubomír Sedlář
f3f0ce0d35 Fix filtering of system release packages
The code was inconsistent with the documentation: the logic was
inverted.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-13 08:52:06 +01:00