The name brings a different expectation than how it actually worked.
This patch makes the code work similarly to the expectation.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If packages are appearing quickly in Koji, and signing them is triggered
by automation, there may be a delay between the package being signed and
compose running. In such case it may be preferable to wait for the
signed copy rather than fail the compose.
JIRA: RHELCMP-3932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs after image-build and osbuild and can embed an image
into a container.
JIRA: RHELCMP-3820
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration *createiso_use_xorrisofs* is added to determine
which tool to use for creating ISOs.
By default, createiso_use_xorrisofs = False and genisoimage will be used.
When set to True, xorrisofs will be used.
JIRA: RHELCMP-2875
Fixes: https://pagure.io/pungi/issue/1130
Signed-off-by: Haibo Lin <hlin@redhat.com>
This is similar to image-build in terms of what it does, and somewhat
similar to OSBS phase in how it's implemented.
The phase reads configuration, submits the build via XMLRPC call and
waits for the task to finish. Then it downloads the built image and
includes it in the compose metadata.
JIRA: RHELCMP-315
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The change allows for setting the parameters as described below to Lorax.
Lorax, a program called during the buildInstall phase, creates the SquashFS
during the buildInstall phase.
The Squash filesystem is present both on the DVD and the BOOT.ISO.
squashfs_only --- (str) passes --squashfs-only option.
configuration_file --- (str or scm_dict) passes -c option to Lorax.
The final goal of this change is to allow for optimization of
the installation medium size.
This pull request is related to the Fedora change proposal, which is available
at this location:
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
See the change proposal for more information about the benefits of higher
compression ratio.
Jira: RHELCMP-693
Signed-off-by: Bohdan Khomutskyi <bkhomuts@redhat.com>
This patch updates the documentation to match the actual behavior, and
adds a configuration option to remove the leading prefix.
The extra wildcard is causing problems when there are two variants in
the compose and one UID is a suffix of the other (e.g. DevTools and
Tools), since multiple files will match the shorter name and an error
will be reported.
JIRA: RHELCMP-1086
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a workaround for modularity design issues and DNF bugs. If there
were gaps in contexts, DNF has trouble handling the upgrades. Thus we
may need to add module metadata for older versions of previously
released module streams and add the missing contexts.
JIRA: RHELCMP-982
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This commits changes `ostree` and `ostree_installer` phases
so they can run with Koji Pungi plugin instead of the plain runroot.
It is similar to `buildinstall` phase running with Koji plugin.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If phase is skipped in a config option (and not on command line), we
should just print a warning or do nothing in quiet mode.
JIRA: RHELCMP-399
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
New `buildinstall.metadata` file is created once the buildinstall
phase is done. This file contains:
- list of lorax command line arguments.
- list of RPMs installed in the buildinstall buildroot.
- list of RPMs installed in the resulting boot.iso.
This file is checked in the next compose run to find out if
the result of buildinstall phase from the previous compose
can be reused. Following is checked:
- lorax commandline arguments are the same (except of expected
differences).
- The NVRAs of RPMs in the runroot_tag are the same as the ones
installed in the old buildinstall buildroot.
- The NVRAs of RPMs installed in the boot.iso are the same as
the ones in package sets in the current compose.
By its implementation, this reuse strategy is used only if
pungi_buildinstall Koji plugin is used.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Add tests for buildinstall reuse and buildinstall_allow_reuse option.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
- Get also requires/provides of RPMs in package set.
- Store the results of gather phase as pickle file.
- Reuse old gather phase results in case Pungi configuration
did not change, the "names" of RPMs in global package set
did not change and their requires/provides did not change.
- Add `gather_allow_reuse` option to enable this feature.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Add gather_allow_reuse, add more tests and better handling of gather_lookaside_repos.
Some composes might need extra validation to ensure they are following
certain strict rules - for example containing only signed packages or
packages only from particular Koji tag.
There is currently no way how to check that Pungi configuration fulfills
these extra requirements.
This commit adds new `--schema-override` option to
`pungi-config-validate` script which allows caller to specify path to
JSON schema overriding the default JSON schema and therefore limitting
it further.
For exmaple, to limit the `pkgset_source` to `koji`, one can use
following JSON schema override:
```
{
"properties": {
"pkgset_source": {
"enum": ["koji"]
}
}
}
```
It is possible to use `--schema-override` multiple times to apply
multiple schema overrides.
Merges: https://pagure.io/pungi/pull-request/1341
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
We would like to start generating the buildinstall phase using the safer
Koji Pungi Buildinstall plugin and stop the direct use of Runroot plugin.
The plugin so far exists only as PR for Koji:
https://pagure.io/koji/pull-request/1939
This commit adds support for this plugin when `lorax_use_koji_plugin`
is set to `True`.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
There was an attempt to achieve this by customizing --version and
--release arguments for Lorax, but Pungi does not take the [general] and
[release] sections from its .treeinfo. Instead it was always using
release version.
The value from this new option will be put into .treeinfo and used for
--version and --release arguments in Lorax (unless explicitly defined in
lorax_options to another value).
JIRA: COMPOSE-4029
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The `runroot_method` now accepts `dict` value with phase name as a key
and runroot method as a value. For backward compatibility, the `str`
value is still supported.
The new `global_runroot_method` option has been added which defines
the runroot method in case it is not set in `dict` in the `runroot_method`.
This commit allows running `createiso` phase locally while keeping the other
phases in Koji.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
It makes no sense to only skip phases needed by other phase.
Fixes: https://pagure.io/pungi/issue/1274
JIRA: COMPOSE-3871
Signed-off-by: Haibo Lin <hlin@redhat.com>
A lorax template used for the ostree-installer might need an additional
package dependency (e.g., flatpak to embed a flatpak repository) - add
a config key 'extra_runroot_pkgs' to the ostree installer configuration
to allow supplementing the set of packages installed into the runroot.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Add a configuration option to enable skipping some modules found in the
configured tag.
Fixes: https://pagure.io/pungi/issue/1260
JIRA: COMPOSE-3794
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds a new config option. This is expected to be a name of
subdirectory in the repo with module defaults. If supplied, overrides
from that location are loaded every time defaults are loaded.
This raises the minimal required version of libmodulemd to 2.8.0
JIRA: COMPOSE-3828
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Sometimes it's practical not just warn when ISO is larger than expected,
but to also abort the compose.
JIRA: COMPOSE-3658
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This also cleans up the runroot method detection code to not rely on the
now removed option.
JIRA: COMPOSE-2634
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch fixes some issues with inconsistent use of whitespace. It
only modifies lines that do not contain any code to not break git blame
too much.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
- Pass the runroot_tag to init command in OpenSSH Runroot method.
This is needed for the init command as a source for initial packages
for the buildroot.
- Rename the "runroot_ssh_init_command" to "runroot_ssh_init_template"
to make it consistent with the rest of "runroot_ssh_*" options.
- Add missing "runroot_ssh_*" options to checks.py.
- Use chmod/chown to `output_dir` in OpenSSH Runroot method the same way
as it is used in Koji runroot method to make the runroot output readable
for Pungi user.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If user configures branch as explicit None, we want to default to HEAD
(which is most likely refs/heads/master in git).
The original code was getting branch as None, which lead to wrong
resolver being used and the repo url being used as branch.
Fixes: https://pagure.io/pungi/issue/1181
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Embedding the registry configuration into OSBS config itself is
simple, but makes it impossible to reuse the same configuration for
multiple different composes.
A nice example is a nightly pushing images to a testing registry, and
production compose building the same images but pushing to staging
location. The original design requires duplication of all the
configuration just because registries are different.
With this option, the push information is stored in a separate option as
a mapping from NVR patterns to arbitrary data. The patterns are used to
match finished builds to registry.
The old configuration is marked as deprecated in code and will
eventually be removed. The deprecation handling in config validation
does not allow emitting warnings for nested values.
JIRA: COMPOSE-3394
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Sometimes the release version can be more specific than what should be
exposed to users of the boot iso.
JIRA: COMPOSE-3295
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Haibo Lin
Lubomír Sedlář
Bohdan Khomutskyi
Jan Kaluza
Owen W. Taylor