Commit Graph

187 Commits

Author SHA1 Message Date
Lubomír Sedlář
18b6020ac5 Improve logging of failable deliverables
The pungi.global.log should show subvariant for each failed deliverable
(if available). When the compose finishes, there is a new log file in
logs/global/deliverables.json containing details about all deliverables
as triples of (variant, arch, subvariant).

 * `required` lists all deliverables that can not fail
 * `attempted` lists all failable deliverables that were started
 * `failed` is a subset of `attempted` and only contains deliverables
   that failed

If the compose fails, the lists may be incomplete.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-13 13:45:30 +02:00
Lubomír Sedlář
ad32b73918 [ostree-installer] Install ostree in runroot
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-12 12:47:02 +02:00
Lubomír Sedlář
31d922692f [pkgset] Print more detailed logs when rpm is not found
Fixes: #265
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-11 15:34:52 +02:00
Lubomír Sedlář
a696eb4f3a [ostree-installer] Clone repo with templates
We can't assume the templates will just be available. This patch adds a
configuration option to point to the git repo with them. It will be
cloned at compose box and relative paths to templates will be made
absolute respective to this clone.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-11 10:25:10 +02:00
Lubomír Sedlář
f17b5e13ad [ostree-installer] Copy all lorax outputs
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-08 12:48:42 +02:00
Lubomír Sedlář
8799d4cd85 [ostree-installer] Use separate directory for logs
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-08 11:36:06 +02:00
Dennis Gilmore
09222a87a0 Merge #260 Maybe fix ostree? 2016-04-08 09:26:11 +00:00
Lubomír Sedlář
81d84a030b [ostree-installer] Put lorax output into work dir
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-08 09:40:41 +02:00
Lubomír Sedlář
e705cdd165 [ostree] Move cloning repo back to compose box
and pass absolute path to the config into runroot. This could avoid the
problem of not being able to clone the repo in runroot.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-07 15:28:15 +02:00
Lubomír Sedlář
b7948f2d65 [ostree] Mount ostree directory in koji
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-07 14:04:50 +02:00
Lubomír Sedlář
ed12a64a84 [ostree] Enable marking ostree phase as failable
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-06 16:12:29 +02:00
Lubomír Sedlář
3e1a6edecc [createiso] Simplify code, test phase
A couple arguments passed from phase to worker threads were not
duplicated. Only one copy is passed now.

A test case was added both for the phase itself and for worker thread as
well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-06 13:25:49 +02:00
Lubomír Sedlář
df400002d8 [createiso] Move runroot work to separate script
Instead of running a long command line in the runroot (or locally), move
all that work into a separate script that will be installed. This means
chroot will need to install pungi.

Everything should work as it did before. The only exception to this is
that there is logic to find lorax templates instead of harcoding the
location. This is done using a separate script.

Related: #230
Fixes: #231
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-06 13:25:49 +02:00
Lubomír Sedlář
5b1b6c1c4f [ostree] Use explicit work directory
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 12:46:59 +02:00
Lubomír Sedlář
96e7ddd3ea [ostree] Rename atomic to ostree
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 09:14:00 +02:00
Lubomír Sedlář
ad87ea2d28 [ostree] Move cloning config repo to chroot
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 09:13:55 +02:00
Lubomír Sedlář
2ddaea5387 [atomic] Stop creating the os directory
The output directory for Lorax can not exist beforehand, or the process
will fail.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-04 15:39:38 +02:00
Lubomír Sedlář
1ebb9d1773 [checksum] Add arch to file name
This allows the checksum file to include the architecture.

Fixes: #243
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-04 09:38:48 +02:00
Lubomír Sedlář
e5f37016e0 Fix atomic/ostree config validations
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-01 07:42:49 +02:00
Lubomír Sedlář
98f7ef739e Refactor checking for failable deliverables
This uses a new common function that will ensure consistent logging
without duplicating the functionality all over the place.

A couple tests are updated to verify that correct data is logged.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-24 10:12:20 +01:00
Lubomír Sedlář
bb0267bc7c [buildinstall] Do not crash on failure
When the koji tasks fails but produces some output (e.g. .discinfo),
copying the artifacts would crash the compose. That should not happen if
the variant/arch is marked as failable.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-24 09:17:38 +01:00
Lubomír Sedlář
8d224b206b [atomic] Add atomic_installer phase
This phase runs lorax with extra templates in Koji runroot task, links
the boot.iso to proper location in compose directory and adds the
installer iso to image manifest. This phase runs concurrently with live
media etc.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-23 12:00:06 +01:00
Lubomír Sedlář
536c6c85b7 [ostree] Add ostree phase
This phase runs the script to make ostree repository in koji runroot
task. It runs right after regular yum repos are created.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-23 10:58:35 +01:00
Dennis Gilmore
6230b0ff3e Merge #232 Improve logging by adding subvariants 2016-03-22 14:15:26 +00:00
Lubomír Sedlář
ecbf08c6f8 Add compose type to release for images
Move getting of the precomputed value to a single place in Compose
class. The value now has format `date[.type_suffix].respin`.

Resolves: rhbz#1319924
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-22 08:57:40 +01:00
Lubomír Sedlář
1a1cfc65ce [image-build] Add traceback on failure
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-17 10:21:21 +01:00
Lubomír Sedlář
ea7f98ef4e [image-build] Use subvariants in logging output
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-17 10:19:28 +01:00
Lubomír Sedlář
f1974d3f03 [live-media] Use subvariants in logging
Print the subvariant together with variant UID and arches. This should
make the global log easier to understand.

The output of koji command should now be split into multiple files in
logs/{arch}/livemedia-{variant}-{subvariant}.{arch}.log instead of
having them all in one file.

The tests were updated to use more sensible value for the subvariant
field.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-17 09:04:50 +01:00
Lubomír Sedlář
4afd21952c Add tracebacks to all failable phases
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-16 18:47:12 +01:00
Lubomír Sedlář
0759c7fb48 [buildinstall] Add more debugging output
When it fails, a traceback will be printed.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-16 18:16:24 +01:00
Lubomír Sedlář
aab22b6f9f [checksums] Add label to file name
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-16 08:49:44 +01:00
Lubomír Sedlář
aa38eb1fd7 [buildinstall] Use customized dvd disc type
This should generate the same volume id in buildinstall and createiso
phases, with isolinux.cfg being consistent as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-14 08:20:43 +01:00
Adam Williamson
c321bca8ef image_build: fix subvariant handling
Dennis added a 'subvariant' property to the Docker image config,
but we weren't using it, we were just hard-coding the variant
as the subvariant...and doing it wrong, as well, using the
variant object rather than its uid property.

This fixes both problems: we'll now use the explicitly set
image config 'subvariant' property if there is one, otherwise
we use the variant.uid as the subvariant.

Note this is a high priority fix, as it broke F24 compose today.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2016-03-13 09:18:08 -07:00
Adam Williamson
254bfb8ca4 add 'subvariant' image property, create live/appliance names
This adds the new 'subvariant' property to all images produced by Pungi.
For most image types this is the same as the variant, for now. For
live_images and livemedia_phase, the image config may specify
'subvariant' and the value will be used if present, otherwise the
variant name will be used.

We also now synthesize the Koji image 'name' for live and appliance
images if the image config doesn't specify it, and use a name that
complies better with the naming policy for ARM disk images (partially
resolving https://pagure.io/pungi-fedora/issue/7 ).

This is completely untested as yet, just bashing stuff out on a Sunday,
will work more on it tomorrow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2016-03-10 16:59:34 -06:00
Lubomír Sedlář
2fd117367e Simplify koji pkgset
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 19:13:51 +01:00
Lubomír Sedlář
619e5323d6 [init] Iterate over arches just once
We can create the repos in one loop together with writing the comps
files.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 15:30:05 +01:00
Lubomír Sedlář
1efd4a9873 [init] Remove duplicated checks for comps
Check once before iterating through the variants. This greatly
simplifies the tests as each function now has one less code path.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 15:30:05 +01:00
Lubomír Sedlář
1c84dc6ea0 [init] Break long lines
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 15:30:03 +01:00
Lubomír Sedlář
41cdf54c05 [init] Don't overwrite the same log file
When creating the comps repo, use a separate log file for each variant
instead of overwriting the same global file again and again.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 15:29:40 +01:00
Lubomír Sedlář
2ca002d602 [init] Add config option for keeping original comps
The configuration can specify a list of variants for which the original
comps file will be copied without any modification.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 15:22:21 +01:00
Lubomír Sedlář
d909adf26c [pkgset] Respect inherit setting
It's logged, but not really used...

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-09 19:23:37 +01:00
Lubomír Sedlář
7e2cd28fb8 [live-media] Support release set to None globally
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-09 13:08:22 +01:00
Dennis Gilmore
0e7ea91d9d Merge #206 Add option to customize disc type 2016-03-08 13:40:43 +00:00
Dennis Gilmore
f898c7ca7d Merge #203 Simplify common live media options 2016-03-08 13:38:39 +00:00
Lubomír Sedlář
72f9819779 [createiso] Add customizing disc type
Relates: #105
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 13:04:34 +01:00
Lubomír Sedlář
44d7b31a80 [live-images] Add customizing disc type
Relates: #105
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 13:04:34 +01:00
Lubomír Sedlář
0c9ad96a31 [buildinstall] Add customizing disc type
This patch adds configuration option to change disc type used in file
name. So far this can only be changed for link to images/boot.iso.

Resolves: #109
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 13:04:28 +01:00
Lubomír Sedlář
ad104f994a [buildinstall] Rename method to not mention symlinks
It is either hardlinking or copying, so the name is quite misleading.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 11:26:14 +01:00
Lubomír Sedlář
27375788c2 [createrepo] Compute delta RPMS against old compose
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-07 14:49:17 +01:00
Lubomír Sedlář
19c3707aee [util] Add function to search for old composes
This was already implemented as part of pkgset phase. It is now moved to
the util module and covered with tests.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-07 14:49:17 +01:00