If the configuration does not specify version for images or live media,
Pungi will create a default value based on `release_version`. If label
is used for the compose, the milestone from it will be appended to the
version (unless it's RC).
This change is backwards compatible: nothing changes when version is set
in configuration. If the version was missing before, building the
artifacts would fail. With this patch, default values will be supplied.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.
There is a custom extension to the schema to report deprecated options.
The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.
Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.
The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.
The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a breaking change as big part of current failable_deliverables
options will be ignored.
There is no change for buildinstall and creatiso phase.
Failability for artifacts in other phases is now configured per
artifact. It already works correctly for ostree and ostree_installer
phases (even per-arch). For OSBS phase there is currently only a binary
switch as it does not handle multiple arches yet. When it gains that
support, the option should contain list of non-blocking architectures.
For live images, live media and image build phases each config block can
configure list of failable arches. If the list is not empty, it can
fail. Once we have a way to fail only some arches, the config will not
need to change.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The performed checks:
* If format is ISO, the file must have correct magic string
* If it's bootable, there must be MBR or GPT
When a check fails on any failable deliverable, it will be logged and
the file removed from metadata (it will still remain on the disk). This
required a change to write the images.json file later (after test
phase).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The live_images, live_media and image_build phases have same options
that need to be specified on each image. This leads to a lot of
duplication in the config file. This patch adds global settings and
phase-level settings that allow to significantly reduce duplication.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The pungi.global.log should show subvariant for each failed deliverable
(if available). When the compose finishes, there is a new log file in
logs/global/deliverables.json containing details about all deliverables
as triples of (variant, arch, subvariant).
* `required` lists all deliverables that can not fail
* `attempted` lists all failable deliverables that were started
* `failed` is a subset of `attempted` and only contains deliverables
that failed
If the compose fails, the lists may be incomplete.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This uses a new common function that will ensure consistent logging
without duplicating the functionality all over the place.
A couple tests are updated to verify that correct data is logged.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Move getting of the precomputed value to a single place in Compose
class. The value now has format `date[.type_suffix].respin`.
Resolves: rhbz#1319924
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Dennis added a 'subvariant' property to the Docker image config,
but we weren't using it, we were just hard-coding the variant
as the subvariant...and doing it wrong, as well, using the
variant object rather than its uid property.
This fixes both problems: we'll now use the explicitly set
image config 'subvariant' property if there is one, otherwise
we use the variant.uid as the subvariant.
Note this is a high priority fix, as it broke F24 compose today.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds the new 'subvariant' property to all images produced by Pungi.
For most image types this is the same as the variant, for now. For
live_images and livemedia_phase, the image config may specify
'subvariant' and the value will be used if present, otherwise the
variant name will be used.
We also now synthesize the Koji image 'name' for live and appliance
images if the image config doesn't specify it, and use a name that
complies better with the naming policy for ARM disk images (partially
resolving https://pagure.io/pungi-fedora/issue/7 ).
This is completely untested as yet, just bashing stuff out on a Sunday,
will work more on it tomorrow.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
After config was modified to work with sections, the resolving broke.
This patch fixes it and adds a test to catch this problem in the future.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Since multiple images are created in a single Koji task, the path to log
file contains a list of arches. These used to be separated with comma.
For consistency with live-media they are now separated by dashes.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch updates tests so that they pass, and fixes a couple places in
image build that did not work correctly with the nested configuration.
The documentation is reformatted so that it horizontal scrolling is less
likely.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The methods mentioning image build are generic and can work for other
task types.
get_image_build_paths -> get_image_paths
run_create_image_cmd -> run_blocking_cmd
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the release is explicitly set to None, generate a value from date
and respin. The documentation is updated to explain how it works.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The configuration can now specify image-build as a deliverable that can
fail but not abort the whole compose.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
* Move getting arches to separate method.
* Modify git url only if there is going to be a build. This is not very
likely to happen, but could save a bit of time if it really does
happen.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The config can now additionally specify other variants whose repos will
be passed on to koji. The previous way of specifying extra repos is
still available, as is the automatic adding of repo for current variant.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Given a list of arches, koji can build multiple images in one go
(automatically starting children tasks for each one).
This causes a bunch of changes:
* The configuration no longer allows changing config based on
architecture, only variants are allowed. It is however possible to
filter which arches are used for the building in the variant.
* The configuration files for koji image-build are stored in
work/image-build/$variant (not split based on arch).
This patch also changes the option name that is passed to koji
image-build: the repos should be specified under key `repo` (without the
trailing slash).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The code modifies the config in place, so it only works in the first
iteration. The fix is to create a copy which can be safely modified and
next iteration will still see the original data.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the image build configuration specifies kickstart URL as a HEAD of
a git repo, pungi now figures out what the actual hash of that commit is
and uses that hash instead. This might make logs clearer and should
prevent potential problems if someone pushes to that repo during
composing.
Documentation is updated to mention this.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
These phases were computing checksums for images:
* buildinstall
* createiso
* image_build
* live_images
* product_img
In each phase the checksummed thing would ultimately end-up in image
manifest.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Dennis Gilmore
Lubomír Sedlář
Dennis Gilmore
Adam Williamson
Lubos Kocman