After cloning the repository with defaults, open each file and try to
check if there is more than one definition for the same module. It's not
a problem for the compose process, but consumers of the compose would
get confused and possibly explode. Better alert people early.
Conceptually this should be part of the test phase, but that would mean
waiting for the compose to finish before reporting the error. The
earlier the better.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the compose is not using comps, we can't pass the comps to lorax,
since it doesn't exist and it would cause a crash.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When multiple composes are chained, they should reuse the same event.
However it is tricky as the value would have to be passed by hand. This
patch makes it possible to read the value from another compose (the
first one in the chain).
JIRA: COMPOSE-2571
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The commands in runroot run as root every time. If they create files
that are not readable to other users, the reset of compose could have
problems with it if it does not run as root too. Particularly updates
composes in Bodhi run under apache user.
Relates: https://pagure.io/pungi/issue/932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
All the components are there already separately, but having the full NVR
should simplify searching the metadata with grep.
JIRA: COMPOSE-2519
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is an optimization for Yum. DNF does not care at all.
The behaviour is configurable, but the default depends on gather
backend, as that is what users should be using to consume the packages
from the repo.
Fixes: https://pagure.io/pungi/issue/951
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It should not be needed there, since the repo is empty anyway. A test is
added for the variant specific comps repo.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is used for mapping basearch to binary arch.
One use for this is when running depsolving to tell yum/dnf what arch to
work with. With this change it will get i686 instead of athlon when
working on i386 basearch.
The other use case is finding arches compatible with given basearch. The
change will remove athlon from the list for i386.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should give us better error reporting. The `copy_all` function
should preserve permissions on all files.
Relates: https://pagure.io/pungi/issue/932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It checks config file value 'skip_phases' for valid phases names.
Also checks command-line attribute 'skip-phase' of 'bin/pungi-koji'.
JIRA: COMPOSE-2493
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
This was added back when rpmutils module was used. Since we migrated to
Python 3 we have included the code directly in Pungi (because it was
coming from yum which is not going to Py3 any time soon). The mocks can
be dropped.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There is really no need to write out megabytes of logs that are not
really interesting. This should also help the parallelization. With the
verbose log createrepo fills the output buffer and needs to wait for the
busy python program to read it first.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The files created in koji runroot will be owned by root. If the compose
is done under different user, there could be a problem with copying the
files preserving the owner. Let's just copy them without that.
Fixes: https://pagure.io/pungi/issue/932
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the ISO is meant to be bootable but lorax fails, there's no point in
creating the ISO as it will not behave as expected.
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1574585
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the parent task is successful, there can still be failed child tasks
for failable arches. We need to log those and potentially mark the
compose as incomplete.
Fixes: https://pagure.io/pungi/issue/874
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the phase is skipped, it could mean that we are doing a debug run and
we don't want to mess up the .treefile by missing arch specific images.
The other alternative is that the phase was really skipped, in which
case there will be no files generated and we already handle that fine.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The comps could potentially be different in different variants, so
instead we can create the comps repo for every variant separately and
use two repos instead of one (packages in one repository, comps in
another one).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the compose configuration includes the module_defaults_dir (an
scm_dict), clone the directory, read the module defaults contained
therein and include relevant defaults in the combined modulemd file.
Only defaults for modules present in the variant are included.
This requires libmodulemd 1.2.0+.
Merges: https://pagure.io/pungi/pull-request/891
Signed-off-by: Petr Šabata <contyk@redhat.com>
Even for Everything we want to filter the comps file to make sure we
remove the stuff that is not compatible with current arch. All groups
are still preserved in that case.
This allows us to do the filtering once in init phase than just use the
prepared file in comps source.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
All use cases that are actually used by pungi-koji are tested. There is
missing coverage for
* keeping only items with matching arch
* not reindenting the file
These aren't currently used and should be removed in the future, but
there may be other tools depending on the comps_filter executable.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We can't rely on the UID to be correctly joined with colons. There may
be historical data that still uses dashes.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration option is added that allows users to point lorax to
extra repositories. This can be handy if some tools to create the
bootable image are not part of the product itself.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: COMPOSE-2253
Otherwise it tries to ensure it exists, and since the default is
/etc/dnf/modules.d, it's causing problems if the directory does not
exist and user does not have permissions to create it.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
