This should indicate that it's a modular variant, but there is no
modular content yet. We don't want to treat that as Everything.
The end result will be an empty repository.
Fixes: https://pagure.io/pungi/issue/871
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need to include all relevant arches, not just the base one (including
noarch and src). However the list can be shortened by only listing
NEVRs, because that should be unique.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
For the first pass we don't need to filter out exclusive architectures,
and we don't need to exclude source packages without any binary
packages. We just want to merge the two package sets as fast as
possible.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We now have a way to select even older version of package (since the
newer one can be left out of the whitelist), so we can include multiple
versions of the same package into global package set.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If we have a package set for the variant (which happens if there are
modules), include a list of all NEVRAs in the pungi kickstart.
This can be used to make sure only packages from correct tag get into
the compose. If two packages with same name but different version get
into the compose, this can help get even older version into a particular
variant.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need to check tags for the variant, not for the whole compose. This
results in merge always being done even if there is a single tag. For
f29 tag in Fedora this takes about 2 hours for each variant.
Relates: https://pagure.io/pungi/issue/860
Relates: https://bugzilla.redhat.com/show_bug.cgi?id=1551653
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The comps source should not return all groups when there are only
modules defined. This fixes part of the problem: non-modular packages
will not go in by default.
The second part is the comps file in the created repository. It will be
filtered to not contain any groups (because packages from there will not
be in the repo).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
OstreeInstaller phase will be moved to a different timeslot
and therefore needs different repo not to depend on Gather
phase which runs at the same time.
Related: https://pagure.io/pungi/issue/778
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
There is a valid use case for modules without any RPMs in them. This
patch makes it possible to include such modules in the repodata.
Merges: https://pagure.io/pungi/pull-request/856
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
In one place, there was an explicit check if pkgset_koji_tag was set,
in another it was blindly referenced and assumed to be a list (with
accidental semi-success for a scalar.)
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Use 'get_packages_to_gather' to fail early if these packages are not
signed with right key. This prevents us from having to wait for the
repo to be created and depsolving to finish. Unsigned dependencies will
still be reported later than previously.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
There can be packages in the tag that will not end up in the compose.
Instead of failing immediately with error, this patch delays the check
until after depsolving finishes and only checks packages that will
really be included.
This is not an issue for nodeps compose, as that already pulls in only
packages that will be composed and nothing else.
Merges: https://pagure.io/pungi/pull-request/843
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
With this patch the gather_source option is no longer used. Instead, all
sources are always used. If they return at least some input packages,
then a configured method is used and the returned lists of packages from
all sources are merged.
The method used for gathering can be configured for each variant and
gather source separately.
Additional packages are only added to the comps source.
Each gathering step is logged separately. All the logs are preserved for
later inspection.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Background story: if a compose is combining modular and traditional
compose, the configuration will contain multiple Koji tags to build
package set from (one tag for each module, plus at least one tag for the
traditional content). However some packages might be present in multiple
tags, and if the package set contains both, there's no way to control
which one will end up in the compose.
The solution for this is to give preference to the modular compose. If a
package with the same name exists in multiple tags, we only take the
first one we find. This relies on ordering of collected tags: modular
ones are always first, and traditional tags are at the end of the list.
If there are multiple modules that contain the same package, only one of
them will be used, which is not correct. Allegedly this should not
happen. In any case such use case does not work without this patch
either, so we're not losing anything.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When buildinstall fails, there will be no lorax generated files copied
into the compose directory. However they may still be mentioned in the
.treefile in work/ subdirectory where lorax runs.
To avoid possible issues, we should use the lorax created .treeinfo only
if the run was successful.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We already filtered a list of packages compatible with the architecture.
There's no need to do that again (with bugs). Instead of using the
global package set we should just restrict the code to an arch package
set.
This should not break anything. If a package is included such that it's
in global but not arch package set, the compose would crash as linking
packages takes paths from the arch package set.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Use global koji tag for populating package set even for modular
composes. To preserve backwards compatibility value "not-used" is
ignored. This should however be fixed in the configuration to simply not
specify the option at all.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When cherry picking packages from Koji tag, we need to make sure that
for every binary package we always have a corresponding source package.
Even if it does not go into the compose, we need it to get values for
Exclusive or Exclude Arch tags.
This means we need to process the binary packages first and only then
look at source ones. Instead of sorting a potentially very long list,
let's just iterate twice.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The list of RPMs in a module can contain multilib packages, but they are
never included because we only ask for compatible arches without
multilib. For x86_64 that only adds noarch, but not i686.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It allows specify what ref we want this compose to commit to.
New parameter 'ostree_ref' overrides the default value from the treefile json.
Relates: https://pagure.io/pungi/issue/777
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
A new `buildinstall_topdir` option allows using buildinstall even when
the compose is created on a different volume that Koji is using.
The files are created in this external directory and then copies into
the usual location.
Merges: https://pagure.io/pungi/pull-request/807
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If buildinstall is supposed to run, we can't pick only some packages
from the tag. The temporary repo is used as a source for lorax. There
could very well be packages not meant to be shipped in the compose that
are still essential for building the bootable images.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need to use `repository`, not `os_tree` path. For any non-addon
variant they are the same, but for addons the original path does not
work as it points to the parent really.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Without this there is almost no information on what's happening. This
patch should provide basic debugging capabilities.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This would make sure that e.g. "updates" composes don't try to use "updates-testing" as an
old_compose_path, which would create practically useless deltarpms and for no repodata
reuse at all.
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
The file extension in configuration is only used to tell Pungi which
files from the task results should be downloaded. The user has to get it
right or the phase will fail. Each format has a single valid suffix.
Pungi should not require users to specify the suffix, since it can just
as well just know the right value.
The old configuration will continue working, only the extension will be
ignored.
Fixes: https://pagure.io/pungi/issue/753
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Jan Kaluza
Lubomír Sedlář
Ondrej Nosek
Owen W. Taylor
Patrick Uiterwijk