The name brings a different expectation than how it actually worked.
This patch makes the code work similarly to the expectation.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a module is skipped from the compose, we should not add it to a
mapping of module tags. If it's there, we then spend time building a
repo for the module, and it get's passed to buildinstall, despite the
packages not being supposed to be included in the compose.
If the packages are not included in any variant, they shouldn't be
available to buildinstall either.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a gathering thread raises an exception, it gets forwarded to the
main thread and re-raised there. However, during this transition it
loses details about exact location of the problem.
This patch creates an extended traceback in the worker, which should
make it easier to track the problem down later.
JIRA: RHELCMP-4259
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If packages are appearing quickly in Koji, and signing them is triggered
by automation, there may be a delay between the package being signed and
compose running. In such case it may be preferable to wait for the
signed copy rather than fail the compose.
JIRA: RHELCMP-3932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs after image-build and osbuild and can embed an image
into a container.
JIRA: RHELCMP-3820
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Rather than tracking this directly in OSBS phase, move this into Compose
object, which will allow access to this from multiple phases.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This file can contain all Pungi specific exceptions.
It should also fix an issue encountered on Python 2.7:
AttributeError: 'module' object has no attribute 'pkgsets'
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When adding extra modules via option *pkgset_koji_module_builds*, all
other versions of the same stream potentially available in a Brew tag
should be skipped.
JIRA: RHELCMP-3689
Signed-off-by: Haibo Lin <hlin@redhat.com>
In such case we never want to reuse the pkgset, as it risks leaking
unsigned packages. Safest option is to remove the file completely.
Fixes: https://pagure.io/pungi/issue/1480
JIRA: RHELCMP-3720
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The task can only fail as a whole, thus it only makes sense to set all
architectures (`*`) as failable. The correct value needs to be checked
though.
JIRA: RHELCMP-3412
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of parsing the log file to get the NVR and then in turn
use that to get to the build info use the structured return value
from the koji task. The return value of the osbuild plugin is:
result = {
"composer": {
"server": <COMPOSER_URL>,
"id": <COMPOSE_ID>
},
"koji": {
"build": <BUILD_ID>
}
}
This means we have direct access to the koji build id, which was
returned by composer to the plugin via its status API. Using that
removes the need to parse the log file.
Adapt the test accordingly.
Merges: https://pagure.io/pungi/pull-request/1475
Signed-off-by: Christian Kellner <christian@kellner.me>
A new configuration *createiso_use_xorrisofs* is added to determine
which tool to use for creating ISOs.
By default, createiso_use_xorrisofs = False and genisoimage will be used.
When set to True, xorrisofs will be used.
JIRA: RHELCMP-2875
Fixes: https://pagure.io/pungi/issue/1130
Signed-off-by: Haibo Lin <hlin@redhat.com>
Previously, the pkg_object has been found by iterating over pkg_sets
for each package. This was quite slow given the number of RPMs in the
compose.
In this commit, the temporary dict is created which stores mapping
between path and pkg_obj and is used instead.
In this commit, the get_package_path is called just once instead of
twice in the loop which might also save some time.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
The pkgset_koji_builds influences the gather phase, but the
change itself is not a reason to not reuse old gather phase. if
new pkgset_koji_builds value leads to significant change in input
package set, we will find that later in this function when comparing
old and new package set.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
When testing if a variant has some modules,
it's no longer enough to look at `variant.modules`. That attribute
contains only names of modules configured in variants.xml.
With modules being specified in `pkgset_koji_module_builds` or
`pkgset_scratch_modules` options it is possible the code wouldn't
trigger even if there was a module.
JIRA: RHELCMP-3054
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is similar to image-build in terms of what it does, and somewhat
similar to OSBS phase in how it's implemented.
The phase reads configuration, submits the build via XMLRPC call and
waits for the task to finish. Then it downloads the built image and
includes it in the compose metadata.
JIRA: RHELCMP-315
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When list is used in gather_lookaside_repos, the Pungi currently
fails with an exception. This is caused by inverted condition
in the code which tries to filter-out the lookaside repos
generated during the Pungin execution pointing to different
compose variants.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
There are two thread pools for making package sets. If Pungi is being
terminated by external event and the exception is handled in the first
thread, the second one never gets to the `stop` method and the process
keeps hanging.
This patch should make sure that `stop()` is called on both pools.
JIRA: RHELCMP-2459
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The change allows for setting the parameters as described below to Lorax.
Lorax, a program called during the buildInstall phase, creates the SquashFS
during the buildInstall phase.
The Squash filesystem is present both on the DVD and the BOOT.ISO.
squashfs_only --- (str) passes --squashfs-only option.
configuration_file --- (str or scm_dict) passes -c option to Lorax.
The final goal of this change is to allow for optimization of
the installation medium size.
This pull request is related to the Fedora change proposal, which is available
at this location:
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
See the change proposal for more information about the benefits of higher
compression ratio.
Jira: RHELCMP-693
Signed-off-by: Bohdan Khomutskyi <bkhomuts@redhat.com>
This uses a previous change that added ability to retry and simply
extends the conditions when it fires.
JIRA: RHELCMP-1863
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Reusing old arch repo may fail for reasons such as arch not
available in old compose or unexpected error when copying data
from old compose.
JIRA: RHELCMP-994
Signed-off-by: Haibo Lin <hlin@redhat.com>
Logs of DepsolveError will be printed in the main log.
Fixes: https://pagure.io/pungi/issue/1399
JIRA: RHELCMP-955
Signed-off-by: Haibo Lin <hlin@redhat.com>
This patch updates the documentation to match the actual behavior, and
adds a configuration option to remove the leading prefix.
The extra wildcard is causing problems when there are two variants in
the compose and one UID is a suffix of the other (e.g. DevTools and
Tools), since multiple files will match the shorter name and an error
will be reported.
JIRA: RHELCMP-1086
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a workaround for modularity design issues and DNF bugs. If there
were gaps in contexts, DNF has trouble handling the upgrades. Thus we
may need to add module metadata for older versions of previously
released module streams and add the missing contexts.
JIRA: RHELCMP-982
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This commits changes `ostree` and `ostree_installer` phases
so they can run with Koji Pungi plugin instead of the plain runroot.
It is similar to `buildinstall` phase running with Koji plugin.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Haibo Lin
Lubomír Sedlář
Christian Kellner
Jan Kaluza
Jiri Konecny
Bohdan Khomutskyi