Rather than trying to use local access when it's accessible, let user
make the decision:
* if koji_cache is configured use it and download stuff
* if not, fall back to local access
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a real build is downloaded, Koji can provide a checksum via API.
This commit adds verification of that checksum.
A mismatch will abort the compose. If Koji doesn't provide a checksum
for the particular sigkey, no checking will happen.
Nothing is still checked for scratch builds and images.
This patch requires Koji 1.32. When talking to an older version, there
is no checking done.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
With this patch, Pungi can be configured with a local directory to be
used as a cache for RPMs, and it will download packages from Koji over
HTTP instead of reading them from filesystem directly.
The files from the cache can then be hardlink as usual.
There is locking in place to avoid different composes running at the
same time to step on each other.
This is now supported for RPMs only, be it real builds or scratch
builds.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds an additional field `options` to scm_dict, which can be
used to provide additional information to the backends.
It implements a single new option for GitWrapper. This option allows
setting a custom git credentials wrapper. This can be useful if Pungi
needs to get files from a git repository that requires authentication.
The helper can be as simple as this (assuming the username is already
provided in the url):
#!/bin/sh
echo password=i-am-secret
The helper would need to be referenced by an absolute path from the
pungi configuration, or prefixed with ! to have git interpret it as a
shell script and look it up in PATH.
See https://git-scm.com/docs/gitcredentials for more details.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: RHELCMP-11808
When looking for task ID in output of koji runroot command, do not check
just the first line. Instead look for first line that contains just a
number.
Most of the time, this should really be the first line. But if koji
client decides to print any warnings, this patch should skip that.
JIRA: RHELCMP-8944
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The gssapi_login call is not retried automatically by Koji yet (see
koji#3170). Let's try to work around that by retrying in the calling
code.
JIRA: RHELCMP-8700
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Clone the directory to the compose tmp directory
Update the test_scm in order to create real Compose object. Mock objects are not allowed
to create/delete files for preventing multiple clones
JIRA: RHELCMP-5250
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
DNF repoclosure requires this option when checking a repository that is
not compatible with host architecture. It seems that when it is
compatible, it works as well.
Based on how the list of architectures is generated, we know that the
main one will always be first.
Fixes: https://pagure.io/pungi/issue/1562
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the value is not specified in the configuration file, let Koji pick
the default channel.
JIRA: RHELBLD-8088
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Update get_mkisofs_cmd in createiso.py file in order to prevent using
default value. With this change it is possible to enable xorriso format
Jira: RHELCMP-6325
Signed-off-by: Ozan Unsal <ounsal@redhat.com>
Previously commands to adjust permissions do not run when main
command failed and then files can't be cleaned up due to
Permission Denied problem.
JIRA: RHELCMP-4253
Signed-off-by: Haibo Lin <hlin@redhat.com>
When the wrapper processes comps file, it wasn't emitting "default"
argument for groupid element. The default is false and most entries are
actually using the default, so let's only emit it if set to true.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1882358
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration *createiso_use_xorrisofs* is added to determine
which tool to use for creating ISOs.
By default, createiso_use_xorrisofs = False and genisoimage will be used.
When set to True, xorrisofs will be used.
JIRA: RHELCMP-2875
Fixes: https://pagure.io/pungi/issue/1130
Signed-off-by: Haibo Lin <hlin@redhat.com>
These methods return command as list which is never serialized to str and
never executed using bash. It is instead passed directly to
`kobo.shortcuts.run`.
The `shlex_quote` usage here actually breaks the code, because it adds
quotes there which are needed only if this command would be serialized
to string and passed to bash. But this never happens. As a result,
the arguments passed to `kobo.shortcuts.run` contain those extra
quotes.
In this commit the shlex_quote is removed completely from this
part of code.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Without that, the resulting string is `--foo="'bar'"`
which results in `'bar'` being passed to Koji task.
With this commit, the resulting string is `"'--foo=bar'"`
which results in `bar` being passed to Koji which is expected.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Sometimes full clone failed with error fatal: Not a git repository
and we found that .git/refs/ dir is missing after shallow clone failed.
JIRA: RHELCMP-724
Signed-off-by: Haibo Lin <hlin@redhat.com>
Also, reformat the invocation of Lorax, since
Lorax does not accept -c=%s, only -c %s.
Jira: RHELCMP-713
Signed-off-by: Bohdan Khomutskyi <bkhomuts@redhat.com>
The change allows for setting the parameters as described below to Lorax.
Lorax, a program called during the buildInstall phase, creates the SquashFS
during the buildInstall phase.
The Squash filesystem is present both on the DVD and the BOOT.ISO.
squashfs_only --- (str) passes --squashfs-only option.
configuration_file --- (str or scm_dict) passes -c option to Lorax.
The final goal of this change is to allow for optimization of
the installation medium size.
This pull request is related to the Fedora change proposal, which is available
at this location:
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
See the change proposal for more information about the benefits of higher
compression ratio.
Jira: RHELCMP-693
Signed-off-by: Bohdan Khomutskyi <bkhomuts@redhat.com>
If the compose is aborted while koji tasks are running, we can be
left with empty log files. That complicates debugging.
JIRA: RHELCMP-1218
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
This commits changes `ostree` and `ostree_installer` phases
so they can run with Koji Pungi plugin instead of the plain runroot.
It is similar to `buildinstall` phase running with Koji plugin.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
We have seen the directory created with wrong permissions. Since we
haven't been able to find out why it happens this is a workaround.
JIRA: RHELCMP-142
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
GitWrapper._clone crashes occasionally but unfortunately
the root cause is not found yet. We need more info for debugging.
JIRA: COMPOSE-4219
Signed-off-by: Haibo Lin <hlin@redhat.com>
When partial cleanup messes up the guestfs cache, the call to guestmount
will fail. To fix that, let's check if there is a problem first and
clean up everything if needed.
Relates: https://bugzilla.redhat.com/show_bug.cgi?id=1771976
JIRA: COMPOSE-3932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We would like to start generating the buildinstall phase using the safer
Koji Pungi Buildinstall plugin and stop the direct use of Runroot plugin.
The plugin so far exists only as PR for Koji:
https://pagure.io/koji/pull-request/1939
This commit adds support for this plugin when `lorax_use_koji_plugin`
is set to `True`.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Lubomír Sedlář
Aditya Bisoi
Haibo Lin
Christopher O'Brien
Ozan Unsal
fdiprete
Jan Kaluza
Jiri Konecny
Bohdan Khomutskyi
Ondrej Nosek