There is a lot of mock objects needed: we bypass calls to Koji, use a
mock FileCache that does not need valid RPMs on disk and avoid any
multithreading.
The test data in tests/fixtures/tagged-rpms.json comes from Koji. It is
filtered down to only a few packages to make it manageable.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch also fixes the bug where using a preconfigured Koji event
would not actually work and instead the latest event would be used.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The live_images, live_media and image_build phases have same options
that need to be specified on each image. This leads to a lot of
duplication in the config file. This patch adds global settings and
phase-level settings that allow to significantly reduce duplication.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of spawning `cp x/* y` there is now Python code to the same
thing. This should help with debugging if something fails as the
traceback will be more informative (rather than saying a command
failed). As another benefit the tests get much simpler.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The pungi.global.log should show subvariant for each failed deliverable
(if available). When the compose finishes, there is a new log file in
logs/global/deliverables.json containing details about all deliverables
as triples of (variant, arch, subvariant).
* `required` lists all deliverables that can not fail
* `attempted` lists all failable deliverables that were started
* `failed` is a subset of `attempted` and only contains deliverables
that failed
If the compose fails, the lists may be incomplete.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We can't assume the templates will just be available. This patch adds a
configuration option to point to the git repo with them. It will be
cloned at compose box and relative paths to templates will be made
absolute respective to this clone.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
and pass absolute path to the config into runroot. This could avoid the
problem of not being able to clone the repo in runroot.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A couple arguments passed from phase to worker threads were not
duplicated. Only one copy is passed now.
A test case was added both for the phase itself and for worker thread as
well.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of running a long command line in the runroot (or locally), move
all that work into a separate script that will be installed. This means
chroot will need to install pungi.
Everything should work as it did before. The only exception to this is
that there is logic to find lorax templates instead of harcoding the
location. This is done using a separate script.
Related: #230Fixes: #231
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If umask is set to something too high (>0022), a warning will be
printed. It does not abort the compose though.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This causes a race condition: umask is process-wide setting, so any file
created by another thread while the umask is set to 0 will have wrong
permissions. This also removes the possible problem of not resetting the
umask if exception is raised.
Fixes: #239
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There already were config options tree_arches and tree_variants, but the
filtering was done at the compose level and there was not interaction
between the two options. This led to problems when a variant would have
all its arches filtered out.
This patch moves the filtering to the variant loader. It adds better
logging, so whenever a variant is filtered (for any reason), it will be
explicitly stated in the logs.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This uses a new common function that will ensure consistent logging
without duplicating the functionality all over the place.
A couple tests are updated to verify that correct data is logged.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the koji tasks fails but produces some output (e.g. .discinfo),
copying the artifacts would crash the compose. That should not happen if
the variant/arch is marked as failable.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs lorax with extra templates in Koji runroot task, links
the boot.iso to proper location in compose directory and adds the
installer iso to image manifest. This phase runs concurrently with live
media etc.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs the script to make ostree repository in koji runroot
task. It runs right after regular yum repos are created.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a wrapper over ostree and rpm-ostree. It is intended to be run
in either Koji or Mock chroot.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Move getting of the precomputed value to a single place in Compose
class. The value now has format `date[.type_suffix].respin`.
Resolves: rhbz#1319924
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Print the subvariant together with variant UID and arches. This should
make the global log easier to understand.
The output of koji command should now be split into multiple files in
logs/{arch}/livemedia-{variant}-{subvariant}.{arch}.log instead of
having them all in one file.
The tests were updated to use more sensible value for the subvariant
field.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should generate the same volume id in buildinstall and createiso
phases, with isolinux.cfg being consistent as well.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Check once before iterating through the variants. This greatly
simplifies the tests as each function now has one less code path.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When creating the comps repo, use a separate log file for each variant
instead of overwriting the same global file again and again.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The configuration can specify a list of variants for which the original
comps file will be copied without any modification.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It is used only in createiso and productimg phases.
For productimg, it needs to be present only when the compose is bootable
and productimg phase is explicitly enabled.
For createiso, it is only needed if runroot is not enabled.
Additionally, if we detect pungi running on arch for which syslinux is
not available, a warning is printed, but the compose is allowed to
continue (and possibly crash later).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the configuration specifically requests no jigdos, there is no point
in checking for the binary existence.
This is not 100% reliable. The jigdo option defaults to True, so if the
option is not specified the binary is required even if there are no
images configured.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds configuration option to change disc type used in file
name. So far this can only be changed for link to images/boot.iso.
Resolves: #109
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This was already implemented as part of pkgset phase. It is now moved to
the util module and covered with tests.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
With this patch Pungi should be more tolerant of network failures when
running a blocking command (creating live media or live images).
If the connection drops and the output indicates network problems, Pungi
will try to watch the task with `koji watch-task`. This will be retried
until it finishes (successfully or with some other failure). There is an
increasing timeout after each retry. Currently the maximum number of
retries is not limited.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
After config was modified to work with sections, the resolving broke.
This patch fixes it and adds a test to catch this problem in the future.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The log file started with createiso-, which is utterly confusing. With
this patch, the logs are named with liveimage- prefix.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There were asserts that actually did not test anything. They should have
checked that koji is called with correct arguments.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Extracting extensions from the file name is not reliable as there is no
way to determine where extensions start. There can very well be a
version.release separated by dot. To bypass this, just use hardcoded
list of possible formats.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
