Commit Graph

40 Commits

Author SHA1 Message Date
Lubomír Sedlář d6dc269ef4 Remove FSF address from comments
The address is no longer correct. We can just as well simply point to
the web page describing the license.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-23 10:26:43 +02:00
Lubomír Sedlář f9a6c8418f Add JSON Schema for configuration
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.

There is a custom extension to the schema to report deprecated options.

The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.

Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.

The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.

The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-01 10:56:15 +02:00
Lubomír Sedlář b862fc5a50 [buildinstall] Fix cleaning output dir
Before the task is started, the output directory is checked and if it
exists and is not empty, the runroot task will be skipped. This is meant
for debugging when restarting the same compose. Under usual
circumstances, the directory will not be created in the first place.

The runroot task will start by removing the output directory. This way,
if koji restarts the task, lorax will not fail.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-08-18 09:11:30 +02:00
Lubomír Sedlář 463088d580 Refactor failables
This is a breaking change as big part of current failable_deliverables
options will be ignored.

There is no change for buildinstall and creatiso phase.

Failability for artifacts in other phases is now configured per
artifact. It already works correctly for ostree and ostree_installer
phases (even per-arch). For OSBS phase there is currently only a binary
switch as it does not handle multiple arches yet. When it gains that
support, the option should contain list of non-blocking architectures.

For live images, live media and image build phases each config block can
configure list of failable arches. If the list is not empty, it can
fail. Once we have a way to fail only some arches, the config will not
need to change.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-07-27 13:06:01 +02:00
Lubomír Sedlář 961648819f [buildinstall] Fix bad error message
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-27 12:18:16 +02:00
Lubomír Sedlář 4e3d87e658 [test] Add checks for created images
The performed checks:
 * If format is ISO, the file must have correct magic string
 * If it's bootable, there must be MBR or GPT

When a check fails on any failable deliverable, it will be logged and
the file removed from metadata (it will still remain on the disk). This
required a change to write the images.json file later (after test
phase).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-05 15:38:00 +02:00
Lubomír Sedlář 98f7ef739e Refactor checking for failable deliverables
This uses a new common function that will ensure consistent logging
without duplicating the functionality all over the place.

A couple tests are updated to verify that correct data is logged.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-24 10:12:20 +01:00
Lubomír Sedlář bb0267bc7c [buildinstall] Do not crash on failure
When the koji tasks fails but produces some output (e.g. .discinfo),
copying the artifacts would crash the compose. That should not happen if
the variant/arch is marked as failable.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-24 09:17:38 +01:00
Lubomír Sedlář 0759c7fb48 [buildinstall] Add more debugging output
When it fails, a traceback will be printed.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-16 18:16:24 +01:00
Lubomír Sedlář aa38eb1fd7 [buildinstall] Use customized dvd disc type
This should generate the same volume id in buildinstall and createiso
phases, with isolinux.cfg being consistent as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-14 08:20:43 +01:00
Adam Williamson 254bfb8ca4 add 'subvariant' image property, create live/appliance names
This adds the new 'subvariant' property to all images produced by Pungi.
For most image types this is the same as the variant, for now. For
live_images and livemedia_phase, the image config may specify
'subvariant' and the value will be used if present, otherwise the
variant name will be used.

We also now synthesize the Koji image 'name' for live and appliance
images if the image config doesn't specify it, and use a name that
complies better with the naming policy for ARM disk images (partially
resolving https://pagure.io/pungi-fedora/issue/7 ).

This is completely untested as yet, just bashing stuff out on a Sunday,
will work more on it tomorrow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2016-03-10 16:59:34 -06:00
Lubomír Sedlář 0c9ad96a31 [buildinstall] Add customizing disc type
This patch adds configuration option to change disc type used in file
name. So far this can only be changed for link to images/boot.iso.

Resolves: #109
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 13:04:28 +01:00
Lubomír Sedlář ad104f994a [buildinstall] Rename method to not mention symlinks
It is either hardlinking or copying, so the name is quite misleading.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-08 11:26:14 +01:00
Dennis Gilmore 8670452b6b Merge #189 `[buildinstall] Use -dvd- in volume ids instead of -boot-` 2016-02-26 13:53:43 +00:00
Lubomír Sedlář 452da86649 [buildinstall] Use -dvd- in volume ids instead of -boot-
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 11:42:41 +01:00
Lubomír Sedlář 636ac79186 [buildinstall] Hardlink boot isos
Instead of creating a symlink, try to hardlink the image, and copy it if
hardlinking fails.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-26 10:58:03 +01:00
Lubomír Sedlář 98a9e02b1b [buildinstall] Don't copy files for empty variants
They are not there in the first place.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-24 07:31:13 +01:00
Lubomír Sedlář 1b6688d22c [buildinstall] Don't run lorax for empty variants
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-18 16:12:44 +01:00
Lubomír Sedlář abcc51fb85 [buildinstall] Stop parsing task_id
KojiWrapper returns the task id as an integer already (if at all). There
is no need to parse it again.

Pagure: #148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-12 12:39:50 +01:00
Lubomír Sedlář f57665f963 [buildinstall] Improve logging
This patch fixes how logs are stored if lorax is used as buildinstall
method. The logs for each variant are in a separate file now. If failure
is allowed, the global log will now show why it failed.

A couple tests are added as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-08 13:17:08 +01:00
Lubomír Sedlář 770a81b76c Don't crash on generating volid without variant
First, this fixes the crash when volume id requires variant uid, but it
is not specified. This happened in buildinstall phase.

When lorax is used, it is run once for each variant, so we can actually
specify the variant and get the correct variant.

When using buildinstall, no volume id will be generated and pungi will
possibly crash later on.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-01-04 19:27:11 +01:00
Lubomír Sedlář a6b673dbdd Add option to specify non-failing stuff
There is a new configuration option that allows listing what can fail
without aborting the whole compose. So far, only buildinstall, createiso
and liveimages phases react to this option.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-17 11:15:47 +01:00
Lubomír Sedlář 0e237db5f6 Allow customizing image name and volume id
The computation of image name has been moved to a separate function.
This allowed simplification in how work dir for isos is computed, but
the result is not changed.

The live image phase has some special casing for names of RPM wrapped
ISOs. This is moved to the actual phase. Since this is (and has been)
undocumented, there should not be many users of this special case.

The documentation is updated to describe how image names and volume ids
are determined and how the process can be customized.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-14 14:39:21 +01:00
Dennis Gilmore f5862deea7 Merge #87 `Add support for customizing lorax options` 2015-12-08 17:36:18 +00:00
Dennis Gilmore 4b374278e4 we need to ensure that we send all the tasks to koji on the correct arch
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-08 09:04:32 -06:00
Lubomír Sedlář 716b2d521a Add support for customizing lorax options
The config file can now specify options for lorax per variant and arch.
This is needed for Fedora to set x86_64 images as bootable on mac.

The old config option `buildinstall_upgrade_image` was deprecated, as
the same can be specified via new `lorax_options`.

Documentation and tests are updated.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-08 12:47:21 +01:00
Lubomír Sedlář 4a96e8e313 Run lorax in separate dirs for each variant
Lorax fails when run on an existing non-empty directory. This patch runs
it in separate subdirectiories base on variant. Running with
buildinstall should not be changed at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-07 09:45:33 +01:00
Lubomír Sedlář cdec198b6d Start lorax for each variant separately
The buildinstall phase now starts lorax for each variant separately.
Running with buildinstall should not be affected in any way.

Only variants of type=variant are considered. The side-effect of this is
that if an architecture is only used by variants of other types, lorax
will not be called at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-12-03 15:19:12 +01:00
Dennis Gilmore e9699b7327 remove glibc32 from the runroot tasks
a workaround has been put in place on os's that need it. dnf errors
when a package is listed that is not available. breaking rawhide
composes.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-02 08:51:18 -06:00
Lubomír Sedlář 660c8bc2b3 Remove computing checksums from phases
These phases were computing checksums for images:

 * buildinstall
 * createiso
 * image_build
 * live_images
 * product_img

In each phase the checksummed thing would ultimately end-up in image
manifest.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-11-24 09:36:19 +01:00
Dennis Gilmore da0405a0f5 Revert "Added params needed for Atomic compose to LoraxWrapper"
This reverts commit 919a4fc619.
2015-09-04 12:12:36 -05:00
Adam Miller 05e9bbab9c Merge #36 `Add params needed for Atomic compose to LoraxWrapper` 2015-08-18 15:50:16 -05:00
Lubos Kocman 449b4e4c87 import duplicate import of errno from buildinstall
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2015-08-31 22:51:22 -05:00
Adam Miller 919a4fc619 Added params needed for Atomic compose to LoraxWrapper
Add add_template, add_template_var, add_arch_template, and
add_arch_template_var to the call for lorax.get_lorax_cmd for the Atomic
ISO composes.

Remove duplicate if statement for add_template_var in lorax wrapper
2015-08-18 15:47:41 -05:00
Daniel Mach b85307c683 Fix buildinstall for armhfp.
armhfp is not a valid RPM arch, it's only a tree arch or basearch.
Lorax requires valid RPM arch as --buildarch.
2015-07-22 13:00:36 -04:00
Daniel Mach 72302bd98e Rename product_* to release_*.
CHANGE: Rename product_* config options to release_* for consistency with productmd.
ACTION: Rename product_name, product_short, product_version, product_is_layered to release_* in config files.
        Rename //variant/product to //variant/release in variants XML.
2015-07-09 06:58:30 -04:00
Daniel Mach fd8cd62e79 Fix remaining productmd issues. 2015-06-06 13:46:20 -05:00
Adam Miller 5fa5fcdd06 Update compose, phases{buildinstall,createiso,gather/__ini__} to use correct productmd API calls 2015-06-06 13:45:48 -05:00
Dennis Gilmore 07e90f0f96 change pypungi imports to pungi 2015-03-12 16:12:38 -05:00
Daniel Mach f116d9384f Initial code merge for Pungi 4.0. 2015-02-10 08:19:34 -05:00