Commit Graph

298 Commits

Author SHA1 Message Date
Lubomír Sedlář 19c3707aee [util] Add function to search for old composes
This was already implemented as part of pkgset phase. It is now moved to
the util module and covered with tests.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-07 14:49:17 +01:00
Dennis Gilmore f202d24961 Limit the variants with config option 'tree_variants'
'tree_variants' in configuration should be able to limit the tree
variants, just like the config option 'tree_arches' which is used to
limited the arches.

For example, if there is configuration (refer to doc/configuration.rst)
go with:

        tree_variants = ["Server"]

then only "Server" variants will be composed, and any other variants in
variants.xml will be ignored.

Signed-off-by: Qixiang Wan <qwan@redhat.com>

Rebased on master

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-03 13:00:52 -06:00
Lubomír Sedlář 9aed3364a6 [createrepo] Refactor code
This patch tries to simplify the code by:

 * breaking up too long lines
 * simplifying the loop to start createrepo jobs
 * removing duplication
 * general clean up

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-02 16:11:25 +01:00
Lubomír Sedlář 055b5e431b [paths] Use variant.uid explicitly
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-02 16:11:25 +01:00
Lubomír Sedlář 6ae48e2bf5 [createrepo-wrapper] Refactor code
Simplify the code a bit by removing conditions that don't help in any
way: executing a loop zero times is the same as not executing it at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-02 16:11:25 +01:00
Lubomír Sedlář d619f14dbf [createrepo-wrapper] Fix --deltas argument
There is no value, the flag is there or it is not there.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-02 16:11:25 +01:00
Lubomír Sedlář a813e926dc [koji-wrapper] Retry watching on connection errors
With this patch Pungi should be more tolerant of network failures when
running a blocking command (creating live media or live images).

If the connection drops and the output indicates network problems, Pungi
will try to watch the task with `koji watch-task`. This will be retried
until it finishes (successfully or with some other failure). There is an
increasing timeout after each retry. Currently the maximum number of
retries is not limited.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-02 14:01:17 +01:00
Lubomír Sedlář 44c4ef5c41 [testphase] Don't run repoclosure for empty variants
As a side effect, the non-existing repositories are no longer passed to
global repoclosure.

Fixes: #196
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-01 12:44:07 +01:00
Lubomír Sedlář 4a4ef23e3c [image-build] Fix resolving git urls
After config was modified to work with sections, the resolving broke.
This patch fixes it and adds a test to catch this problem in the future.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-29 08:23:26 +01:00
Lubomír Sedlář 73a560d63c [live-images] No manifest for appliances
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 19:48:52 +01:00
Dennis Gilmore e6596d818c Merge #190 `[live-images] Rename log file` 2016-02-26 13:54:01 +00:00
Dennis Gilmore 8670452b6b Merge #189 `[buildinstall] Use -dvd- in volume ids instead of -boot-` 2016-02-26 13:53:43 +00:00
Lubomír Sedlář abb27ac7d7 [live-images] Rename log file
The log file started with createiso-, which is utterly confusing. With
this patch, the logs are named with liveimage- prefix.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 12:05:58 +01:00
Lubomír Sedlář 452da86649 [buildinstall] Use -dvd- in volume ids instead of -boot-
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 11:42:41 +01:00
Lubomír Sedlář 636ac79186 [buildinstall] Hardlink boot isos
Instead of creating a symlink, try to hardlink the image, and copy it if
hardlinking fails.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 10:58:03 +01:00
Lubomír Sedlář 60d820d4fd [util] Resolve branches in git urls
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-25 08:39:01 +01:00
Lubomír Sedlář 6e8970e648 [live-images] Fix crash when repo_from is not a list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-24 19:33:28 +01:00
Lubomír Sedlář 98a9e02b1b [buildinstall] Don't copy files for empty variants
They are not there in the first place.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-24 07:31:13 +01:00
Lubomír Sedlář ca7c78d98c [live-images] Set type to raw-xz for appliances
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-23 09:41:03 +01:00
Lubomír Sedlář 456fbb8812 [live-images] Correctly create format
Extracting extensions from the file name is not reliable as there is no
way to determine where extensions start. There can very well be a
version.release separated by dot. To bypass this, just use hardcoded
list of possible formats.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-23 09:35:50 +01:00
Lubomír Sedlář 90954ddf0e [tests] Use real paths module in testing
This avoids problems of mock paths having different API that the real
ones.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 19:11:04 +01:00
Lubomír Sedlář 75b57ac25a [live-images] Create image dir if needed
We can't copy an image to a non-existing directory.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 18:59:31 +01:00
Lubomír Sedlář 7734ddf57e [live-images] Add images to manifest
The type is either "live" or "appliance", the format is set based on
filename extension.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 14:27:51 +01:00
Lubomír Sedlář f80c97e3ec [live-images] Fix path processing
* Add option to keep filenames generated by Koji.
 * Put results of spin-appliance into image dir
 * On failure images are no longer deleted.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 14:27:26 +01:00
Lubomír Sedlář 4a1e029c1d [live-images] Move repo calculation to separate method
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 12:27:26 +01:00
Lubomír Sedlář 9b748f752e [koji-wrapper] Fix getting results from spin-appliance
Also don't create new client proxy when one already exists.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 12:27:22 +01:00
Lubomír Sedlář ab44f3539e [live-images] Filter non-image results
Only keep results that have known extension.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 12:27:17 +01:00
Lubomír Sedlář 1044fb01f8 [live-images] Rename repos_from to repo_from
It should be consistent with live media and image build.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 08:48:12 +01:00
Lubomír Sedlář 7c81c5aa9c [live-images] Automatically populate release with date and respin
This is the same feature that is already available for image-build and
live-media.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 08:38:19 +01:00
Lubomír Sedlář ec03a8685a [live-media] Respect release set in configuration
Apparently just processing the config is not sufficient we don't pass
the value to Koji.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 08:32:27 +01:00
Lubomír Sedlář 32a966fc91 [live-images] Build all images specified in config
Not just the first one.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-22 08:21:51 +01:00
Lubomír Sedlář 8ed44dd153 [live-media] Don't create $basedir arch
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-19 09:01:45 +01:00
Dennis Gilmore b4a008f86a do not ad to image build and live tasks the variant if it is empty
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-19 08:51:03 +01:00
Dennis Gilmore 466a0eb24f when a variant is empty do not add it to the repolist for livemedia
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-19 08:47:05 +01:00
Lubomír Sedlář 1b6688d22c [buildinstall] Don't run lorax for empty variants
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-18 16:12:44 +01:00
Lubomír Sedlář 71369c7690 Merge #159 `use $basearch not $arch in livemedia tasks` 2016-02-18 13:56:15 +00:00
Lubomír Sedlář 921664c3b5 Merge #158 `do not uses pipes.quotes in livemedia tasks` 2016-02-18 13:55:14 +00:00
Tomas Mlcoch 5bffca5037 Support signing of rpm wrapped live images
With this patch, you can specify a command for
signing of koji builds. For example:

    signing_key_password_file = '~/file_with_password_for_key_fedora-24'
    signing_key_id = '81b46521'
    signing_command = '~/git/releng/scripts/sigulsign_unsigned.py -vv --password=%(signing_key_password)s fedora-24'

'signing_key_password_file' is a path to a file which contains
a password that will be formatted into 'signing_command' string
via '%(signing_key_password)s' string format syntax (if used).
Because pungi config is usualy stored in git and part of compose
logs we don't want password to be included directly in the config.
Note: If '-' is used instead of a filename, then you will be asked
for the password interactivelly right after pungi starts.

'signing_key_id' is ID of the key that will be used for the signing.
This ID will be used when crafting koji paths to signed files
(kojipkgs.fedoraproject.org/packages/NAME/VER/REL/data/signed/KEYID/..).

'signing_command' a command that will be run with a build as a single
argument. This command mustn't require any user interaction.
If you need to pass a password for a signing key to the command,
do this via command line option of the command with use of string
formatting syntax '%(signing_key_password)s' (see details
about 'signing_key_password_file').

Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:36:38 +01:00
Tomas Mlcoch be4d596c36 Fix terminology - Koji uses sigkey not level
Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:35:27 +01:00
Dennis Gilmore 1616de1e1e use $basearch not $arch in livemedia tasks
koji for livemedia acts different to image builds and does
not translate $arch to x86_64, i386, etc so we need to just pass
in $basearch so that yum/dnf will translate it for us

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-17 17:03:46 -06:00
Dennis Gilmore 2858b1aa54 do not uses pipes.quotes in livemedia tasks
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-17 14:39:51 -06:00
Lubomír Sedlář bf5196af4a [live-images] Don't tweak kickstarts
Instead of downloading the kickstart file in Pungi and modifying it,
just pass it to Koji.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-17 14:35:28 +01:00
Lubomír Sedlář 454363fba8 Allow specifying empty variants
The variants.xml file can list a variant with is_empty="true" and no
groups. If such variant is found, not package gathering will be run for
it, and no repos will be created.

This only makes sense for a variant that will have some other
deliverables like live media or images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář adbc772fd0 [createrepo] Remove dead assignments
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář 826afdc162 Keep empty query string in resolved git url
Pagure: #152
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 10:42:59 +01:00
Lubomír Sedlář 284d3a3510 [image-build] Use dashes as arch separator in log
Since multiple images are created in a single Koji task, the path to log
file contains a list of arches. These used to be separated with comma.
For consistency with live-media they are now separated by dashes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-15 09:04:03 +01:00
Lubomír Sedlář abcc51fb85 [buildinstall] Stop parsing task_id
KojiWrapper returns the task id as an integer already (if at all). There
is no need to parse it again.

Pagure: #148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:39:50 +01:00
Lubomír Sedlář e1895bff26 [koji-wrapper] Get task id from failed runroot
Even when runroot task fails, it can emit task_id.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:38:45 +01:00
Lubomír Sedlář 93015d679c [live-media] Pass ksurl to koji
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-11 17:09:30 +01:00
Dennis Gilmore 96489a1e78 Merge #146 `[live-media] Properly calculate iso dir` 2016-02-11 13:07:30 +00:00