Phases createiso, liveimages, image_build, ostree_installer and osbs are
done in parallel, logs from these phases are mixed and and it's not
obvious which log message belongs to which phase. This change adds phase
name in log message for these phases.
The new mixin 'PhaseLoggerMixin' is added to extend a Pungi phase with a
logging logger which copy handlers from compose's logger but with
formatter changed.
Fixes: #58
Signed-off-by: Qixiang Wan <qwan@redhat.com>
Sometimes addtional repos are required to get necessary packages for
composing OSTree repository. For example, RHEL doesn't have an 'Everyting'
variant, so composing OSTree repository from any of the RHEL variants
won't work, addtional source repos need to be enabled to achieve that.
The new option "extra_source_repos" enable the ability of allowing extra
source repos.
And a new option 'keep_original_sources' is introduced to keep the
original repos found in tree config file, if this is enabled, Pungi
will not remove the existing source repos from the tree config file,
just add new repos of "source_repo_from" + "extra_source_repos" to
the existing repos.
Signed-off-by: Qixiang Wan <qwan@redhat.com>
When a file is exported from an RPM in the compose, and there is no
matching RPM in the package set, we want a nice error message.
Fixes: #460
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The `get_variants()` method had a `recursive` argument with default
value of `False. However, this argument had no effect and the method
always returned all variants recursively.
We can just drop the argument. All callers are updated to not supply the
argument. Should any need for getting the top-level variants only arise,
they can be accessed as the `variants` attribute directly on the Compose
object.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The code to search for install tree and repo for image-build and
live-media was only looking at top-level variants. Addons, optional or
integrated layered products could not have been found. This would lead
to error messages such as "There is no variant Server-optional to get
repo from when building live image for Client" even though the variant
exists.
Various tests are updated to exercise this edge case.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Tell lorax to use a specific directory for log files so that we preserve
them despite koji not having any idea about them.
Fixes: #457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Added new option '--version' to pungi-make-ostree, and this can be
enabled in ostree settings with 'version'. The version string will be
added as versioning metadata if this is specified.
Signed-off-by: Qixiang Wan <qwan@redhat.com>
Added new option '--update-summary' to pungi-make-ostree, and this can
be enabled in ostree settings with 'update_summary'. A summary file will
be generated (or re-generated if it was presented in an existing ostree
repo) when it is enabled.
Signed-off-by: Qixiang Wan <qwan@redhat.com>
'mount -o loop' requires root privileges, guestmount from
libguestfs-tools-c package can work without root privileges.
Fixes: #19
Signed-off-by: Qixiang Wan <qwan@redhat.com>
If the media is bootable, we can not split it. Instead we will create an
ISO that is too big and issue a warning (aborting the whole compose
would be too much).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The older buildinstall method fails if it sees ppc64p7 instead of ppc64.
Nothing changes for composes using Lorax.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It really is just a group of independent functions, so we can simplify
it by removing the unused wrapper class. Instead of importing the
wrapper, instantiating it and calling its methods we can import the
module and call its functions directly.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of overloading the `dvd` value, use new value `ostree` that can
be overridden by a config change.
Fixes: https://pagure.io/pungi/issue/418
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
to complement https://pagure.io/koji/pull-request/162 we need to adjust pungi
rpm-ostree uses bublewrap that does not work in mock. --new-chroot to mock
enables the use of systemd-nspawn instead of chroot resulting in working
rpm-ostree again
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
The address is no longer correct. We can just as well simply point to
the web page describing the license.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The same information can be inferred from definitions in variants.xml:
if the variant has no groups defined, we include packages from all
groups. By the same logic we can also include all groups in the comps
file.
The config validation is updated to give a hint on how to remove the
option from the configuration.
Relates: #29
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It does not make sense for this phase to be skipped. If there are any
images, we need to generate the checksums so that writing metadata
works. If there are no images, the phase does not do anything and is
therefore very fast.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds a new config option createrepo_use_xz, which when set to
true will cause createrepo to compress sqlite databases with xz. The
default setting is False.
Fixes: #387
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Introduces a new metadata file to track arbitrary files added during the
extra-files phase. This file is placed in the root of each tree and is
called ``extra_files.json``. It is a JSON file containing a single
object, which contains a "header" key with an object describing the
metadata, and a "data" key, which is an array of objects, where each
object represents a file. Each object contains the "file", "checksums",
and "size" keys. "file" is the relative path from the tree root to the
extra file. "checksums" is an object containing one or more checksums,
where the key is the digest type and the value of that key is the hex
digest. Finally, the size is the size of the file in bytes.
For example:
{
"header": {"version": "1.0},
"data": [
{
"file": "GPL",
"checksums": {
"sha256": "8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643"
},
"size": 18092
},
{
"file": "release-notes/notes.html",
"checksums": {
"sha256": "82b1ba8db522aadf101dca6404235fba179e559b95ea24ff39ee1e5d9a53bdcb"
},
"size": 1120
}
]
}
Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Fixes: #295
If the configuration does not specify version for images or live media,
Pungi will create a default value based on `release_version`. If label
is used for the compose, the milestone from it will be appended to the
version (unless it's RC).
This change is backwards compatible: nothing changes when version is set
in configuration. If the version was missing before, building the
artifacts would fail. With this patch, default values will be supplied.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.
There is a custom extension to the schema to report deprecated options.
The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.
Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.
The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.
The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Currently the main log only says the phase started and finished. With
this patch each created image will be mentioned by its name, version,
release and arch.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Before the task is started, the output directory is checked and if it
exists and is not empty, the runroot task will be skipped. This is meant
for debugging when restarting the same compose. Under usual
circumstances, the directory will not be created in the first place.
The runroot task will start by removing the output directory. This way,
if koji restarts the task, lorax will not fail.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Originally, the idea was to remove non-blocking images that failed the
check. They were only removed from the manifest, which creates some
confusion as to what is going on. With this patch, a failed check on
non-blocking deliverable will only print an error message.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When ISO is marked as failable and check fails, the compose would still
be aborted. This patch fixes that
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of installing pungi itself in the runroot, we can prepare the
commands to be run on compose box, write the shell script into work/
directory, which is mounted in the chroot, and execute that. This way
there is no business logic in runroot (except for finding lorax
templates).
The main advantage of this approach is that we don't need to pull any
extra dependencies into buildroot.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When package has incompatible arch for a tree, the same error message
could be raised from two different places. This patch adds more
information to the message, so that it is easier to find the actual
problem.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a breaking change as big part of current failable_deliverables
options will be ignored.
There is no change for buildinstall and creatiso phase.
Failability for artifacts in other phases is now configured per
artifact. It already works correctly for ostree and ostree_installer
phases (even per-arch). For OSBS phase there is currently only a binary
switch as it does not handle multiple arches yet. When it gains that
support, the option should contain list of non-blocking architectures.
For live images, live media and image build phases each config block can
configure list of failable arches. If the list is not empty, it can
fail. Once we have a way to fail only some arches, the config will not
need to change.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Atomic Reactor does not honor this option. In the future we might need
to reintroduce this feature, but given that it does not work in the
current form it is better removed.
Fixes: #348
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Qixiang Wan
Colin Walters
Dennis Gilmore
Lubomír Sedlář
Patrick Uiterwijk
Jeremy Cline