Prior to this change, if a container image used Cachito with OSBS, then
OSBS would store additional "remote-sources" files in the Koji archives
for the build. Pungi cannot parse the metadata for these archive
entries, so it would crash in add_metadata():
File "pungi/phases/osbs.py", line 81, in process
self.worker(compose, variant, config)
File "pungi/phases/osbs.py", line 141, in worker
nvr, archive_ids = add_metadata(variant, task_id, compose, scratch)
File "pungi/phases/osbs.py", line 447, in add_metadata
arch = archive["extra"]["image"]["arch"]
KeyError: 'image'
Tell Koji to only return container image archives, and ignore these
remote-source archives.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Clone the directory to the compose tmp directory
Update the test_scm in order to create real Compose object. Mock objects are not allowed
to create/delete files for preventing multiple clones
JIRA: RHELCMP-5250
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
This should cover case where we there's a build like this:
foo-1-1.src.rpm
foo-sub-1-1.noarch.rpm
foo-debugsource-1-1.x86_64.rpm
The compose contains the noarch package, and should also have the
debugsource package. The original code only checked for
foo-sub-debugsource though.
JIRA: RHELCMP-7628
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
With this param, get_dir_from_scm will try to copy the tmp git dir to
compose target dir when error occurs.
This does not fix the issue but it would be helpful for debugging when
it occurs again.
JIRA: RHELCMP-7244
Signed-off-by: Haibo Lin <hlin@redhat.com>
It was checked in a condition together with the configuration value, and
only ever explicitly used with the same value.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a compose doesn't build any images, it won't produce any metadata
file for them, and thus it makes no sense to validate it.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Fixes: https://pagure.io/pungi/issue/1565
DNF repoclosure requires this option when checking a repository that is
not compatible with host architecture. It seems that when it is
compatible, it works as well.
Based on how the list of architectures is generated, we know that the
main one will always be first.
Fixes: https://pagure.io/pungi/issue/1562
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When nothing in configuration or the image itself changed, let's just
copy the older one.
JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch allows Pungi to reuse ISO image created in previous compose
if a list of assumptions proves to hold:
* If image is bootable, buildinstall phase must have been reused too.
* Compose configuration must have not changed (except for a few
whitelisted options).
* Volume ID of the ISO much not have changed.
* No RPM on the ISO must have changed.
The ISO also contains other files. Changes in extra files and product ID
certificates should be visible in configuration (the SHA will differ).
Similarly any repodata configuration would be reflected in
configuration.
JIRA: RHELCMP-5969
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the value is not specified in the configuration file, let Koji pick
the default channel.
JIRA: RHELBLD-8088
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
- Use ModuleIndex's update_from_file/update_from_string instead of ModuleStream's
read_file/read_string which is deprecated.
- Extend tests to work with real module streams instead of mocks.
Signed-off-by: Filip Valder <fvalder@redhat.com>
Make sure that the function returns a path even for local files
specified by file:// urls.
JIRA: RHELCMP-5340
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
- Do not reuse when there is any external lookaside repo
- Do not reuse when lookaside variant is not reused
JIRA: RHELCMP-4596
Signed-off-by: Haibo Lin <hlin@redhat.com>
The original code ended up downloading all repodata from the lookaside
repo. This could cause a lot of memory to be used.
The new code only downloads the repomd.xml and then primary record,
which is sufficient to obtain all needed information. A lot less memory
is used and the code is also significantly faster.
Here are some alternative ways of getting a list of packages from the
lookaside repo and reasons why they did not work:
* dnf repoquery - this doesn't include modular packages unless the
stream is default
* dnf reposync - requires `--urls` option to only print the names,
which is not available on RHEL 7
JIRA: RHELCMP-4761
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There are two images because it's hard to install both Python 2
and Python 3 packages (e.g. libcomps) in latest fedora release.
JIRA: RHELCMP-4580
Signed-off-by: Haibo Lin <hlin@redhat.com>
OSBS will reject no scratch builds with arch_override.
When the option is not specified in Pungi, it would do `"".split(" ")`
to get list of arches, which returns a list with empty string instead of
an empty list.
With this fixed, it might be possible to have multiple images match the
spec (unless arch is used in the filter). To fix that, we can replace
arch with $basearch variable.
JIRA: RHELCMP-3824
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Previously commands to adjust permissions do not run when main
command failed and then files can't be cleaned up due to
Permission Denied problem.
JIRA: RHELCMP-4253
Signed-off-by: Haibo Lin <hlin@redhat.com>
The name brings a different expectation than how it actually worked.
This patch makes the code work similarly to the expectation.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a gathering thread raises an exception, it gets forwarded to the
main thread and re-raised there. However, during this transition it
loses details about exact location of the problem.
This patch creates an extended traceback in the worker, which should
make it easier to track the problem down later.
JIRA: RHELCMP-4259
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If packages are appearing quickly in Koji, and signing them is triggered
by automation, there may be a delay between the package being signed and
compose running. In such case it may be preferable to wait for the
signed copy rather than fail the compose.
JIRA: RHELCMP-3932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs after image-build and osbuild and can embed an image
into a container.
JIRA: RHELCMP-3820
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Rather than tracking this directly in OSBS phase, move this into Compose
object, which will allow access to this from multiple phases.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The task can only fail as a whole, thus it only makes sense to set all
architectures (`*`) as failable. The correct value needs to be checked
though.
JIRA: RHELCMP-3412
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of parsing the log file to get the NVR and then in turn
use that to get to the build info use the structured return value
from the koji task. The return value of the osbuild plugin is:
result = {
"composer": {
"server": <COMPOSER_URL>,
"id": <COMPOSE_ID>
},
"koji": {
"build": <BUILD_ID>
}
}
This means we have direct access to the koji build id, which was
returned by composer to the plugin via its status API. Using that
removes the need to parse the log file.
Adapt the test accordingly.
Merges: https://pagure.io/pungi/pull-request/1475
Signed-off-by: Christian Kellner <christian@kellner.me>
