There is really no need to write out megabytes of logs that are not
really interesting. This should also help the parallelization. With the
verbose log createrepo fills the output buffer and needs to wait for the
busy python program to read it first.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The files created in koji runroot will be owned by root. If the compose
is done under different user, there could be a problem with copying the
files preserving the owner. Let's just copy them without that.
Fixes: https://pagure.io/pungi/issue/932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the ISO is meant to be bootable but lorax fails, there's no point in
creating the ISO as it will not behave as expected.
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1574585
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the parent task is successful, there can still be failed child tasks
for failable arches. We need to log those and potentially mark the
compose as incomplete.
Fixes: https://pagure.io/pungi/issue/874
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the config is syntactically correct, but it contains some issues
discovered in `validate` method of any phase, we need to raise an
exception, not exit immediately.
JIRA: COMPOSE-2431
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
With one compose combining traditional and modular content there might
be different requirements for tag inheritance. This patch adds a new
option that controls whether builds in modular tags should be inherited.
It defaults to False, which is the right option for current MBS
behavior.
JIRA: COMPOSE-2148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The code checked with `assert` that there is only one module matching
given NSV. In actual package that would not do anything and we would
silently pick the first value.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
On stdout it gets mixed with listing of gathered packages. This has no
effect in a real pungi-koji run, where both streams are merged into a
single file which is then parsed, but in manual debugging runs it's a
little obnoxious.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This speeds up the compose quite a bit and there is no need for the
database anyway.
Merges: https://pagure.io/pungi/pull-request/922
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If the phase is skipped, it could mean that we are doing a debug run and
we don't want to mess up the .treefile by missing arch specific images.
The other alternative is that the phase was really skipped, in which
case there will be no files generated and we already handle that fine.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Until now only list of top level variants was enough, but for variant
as lookaside we need a dict including even child variants.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If a package is in lookaside, we don't really need a debuginfo nor
source for it. Definitely we should not pull them from the package set
even if there is some suitable package. Therefore we should include them
in the output but marked as lookaside.
Fixes: https://pagure.io/pungi/issue/915
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The comps could potentially be different in different variants, so
instead we can create the comps repo for every variant separately and
use two repos instead of one (packages in one repository, comps in
another one).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The log file used to contain a dump of the configuration, which could
potentially be very large. This patch removes data that is directly
visible in the input kickstart file. Instead it just displays the number
of items.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the compose configuration includes the module_defaults_dir (an
scm_dict), clone the directory, read the module defaults contained
therein and include relevant defaults in the combined modulemd file.
Only defaults for modules present in the variant are included.
This requires libmodulemd 1.2.0+.
Merges: https://pagure.io/pungi/pull-request/891
Signed-off-by: Petr Šabata <contyk@redhat.com>
The recent comps related commit added a fixture file with new extension
which is not getting included in the source tarball and therefore causes
tests to fail.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Even for Everything we want to filter the comps file to make sure we
remove the stuff that is not compatible with current arch. All groups
are still preserved in that case.
This allows us to do the filtering once in init phase than just use the
prepared file in comps source.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If a variant contains both modules and comps groups, we need to include
builds from the compose tag in the package whitelist. However only
packages that are not already provided by any module should be added.
JIRA: COMPOSE-2435
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
All use cases that are actually used by pungi-koji are tested. There is
missing coverage for
* keeping only items with matching arch
* not reindenting the file
These aren't currently used and should be removed in the future, but
there may be other tools depending on the comps_filter executable.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The code should not live directly in the executable, that makes it very
hard to test.
Other than the move there is no functional change.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
- Add rpmdevtools and python-unittest2 to the list of packages
required to set up a testing environment
Signed-off-by: Robert Marshall <rmarshall@redhat.com>
We can't rely on the UID to be correctly joined with colons. There may
be historical data that still uses dashes.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There is no need to create the directory in work/, as it will get
deleted immediately. Let's move it to /tmp and use the context manager
to clean it up.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration option is added that allows users to point lorax to
extra repositories. This can be handy if some tools to create the
bootable image are not part of the product itself.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: COMPOSE-2253
Otherwise it tries to ensure it exists, and since the default is
/etc/dnf/modules.d, it's causing problems if the directory does not
exist and user does not have permissions to create it.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
