This PR adds new pkgset_koji_builds configuration option.
This option allows setting list of extra Koji build NVRs which will be
included in a compose. This is useful in two cases:
a) It allows generating standard composes with few packages update to
certain version to test how the compose behaves when the package is
updated for real.
b) It allows generating compose consisting only from particular builds
when pkgset_koji_tag = '' or None. This is useful when one want to
regenerate the compose with packages which are not tagged in single Koji
tag. This is very useful for ODCS when reproducing old composes.
Merges: https://pagure.io/pungi/pull-request/1049
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If the same tag is queried with the same event, Pungi can cache the
response and call the API again. Particularly for small composes this
can save up significant amount of time.
Merges: https://pagure.io/pungi/pull-request/1022
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
If a module says to filter a package out, we can do it immediately when
getting the build information from Koji.
This avoids a possible problem of something pulling the module package
in as a dependency, but it should also make the package set slightly
smaller.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The `add_noarch` option of `get_valid_arches` is broken and doesn't
really do anything (noarch is always present in the result).
This causes packages that have ExclusiveArch including noarch to
actually not be excluded. They should be.
Changing this globally could have a very big impact. Therefore we can
hide it behind a configuration option so that it's opt-in.
JIRA: COMPOSE-2457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
For the first pass we don't need to filter out exclusive architectures,
and we don't need to exclude source packages without any binary
packages. We just want to merge the two package sets as fast as
possible.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We now have a way to select even older version of package (since the
newer one can be left out of the whitelist), so we can include multiple
versions of the same package into global package set.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Use 'get_packages_to_gather' to fail early if these packages are not
signed with right key. This prevents us from having to wait for the
repo to be created and depsolving to finish. Unsigned dependencies will
still be reported later than previously.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
There can be packages in the tag that will not end up in the compose.
Instead of failing immediately with error, this patch delays the check
until after depsolving finishes and only checks packages that will
really be included.
This is not an issue for nodeps compose, as that already pulls in only
packages that will be composed and nothing else.
Merges: https://pagure.io/pungi/pull-request/843
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Background story: if a compose is combining modular and traditional
compose, the configuration will contain multiple Koji tags to build
package set from (one tag for each module, plus at least one tag for the
traditional content). However some packages might be present in multiple
tags, and if the package set contains both, there's no way to control
which one will end up in the compose.
The solution for this is to give preference to the modular compose. If a
package with the same name exists in multiple tags, we only take the
first one we find. This relies on ordering of collected tags: modular
ones are always first, and traditional tags are at the end of the list.
If there are multiple modules that contain the same package, only one of
them will be used, which is not correct. Allegedly this should not
happen. In any case such use case does not work without this patch
either, so we're not losing anything.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When cherry picking packages from Koji tag, we need to make sure that
for every binary package we always have a corresponding source package.
Even if it does not go into the compose, we need it to get values for
Exclusive or Exclude Arch tags.
This means we need to process the binary packages first and only then
look at source ones. Instead of sorting a potentially very long list,
let's just iterate twice.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Currently `None` has to be included to allow using unsigned packages.
ODCS has trouble with including non-string value in the list though, so
we can treat empty string the same way (it's not a valid key ID anyway).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
For each tag we ask Koji about (there might be more than one in
modularity case), we create a log file with list of RPMs and details
about which tag they were pulled from. This makes it easier to find out
where the package is inherited from.
Fixes: #547
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The address is no longer correct. We can just as well simply point to
the web page describing the license.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There is a lot of mock objects needed: we bypass calls to Koji, use a
mock FileCache that does not need valid RPMs on disk and avoid any
multithreading.
The test data in tests/fixtures/tagged-rpms.json comes from Koji. It is
filtered down to only a few packages to make it manageable.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
CHANGE: pkgset_koji_url and pkgset_koji_path_prefix config options replaced with koji_profile.
ACTION: Add 'koji_profile = "<profile_name>"' (use "koji" for Fedora) to config files.
You can safely remove and pkgset_koji_url and pkgset_koji_path_prefix from config files.