If a module says to filter a package out, we can do it immediately when
getting the build information from Koji.
This avoids a possible problem of something pulling the module package
in as a dependency, but it should also make the package set slightly
smaller.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
PDC is deprecated in upstream. The usecase for getting list of modules
by NS, NSV or NSVC can however be satisfied by querying modules imported
into Koji.
This makes it possible to deprecate PDC configuration.
Merges: https://pagure.io/pungi/pull-request/985
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
With one compose combining traditional and modular content there might
be different requirements for tag inheritance. This patch adds a new
option that controls whether builds in modular tags should be inherited.
It defaults to False, which is the right option for current MBS
behavior.
JIRA: COMPOSE-2148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The code checked with `assert` that there is only one module matching
given NSV. In actual package that would not do anything and we would
silently pick the first value.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We can't rely on the UID to be correctly joined with colons. There may
be historical data that still uses dashes.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should indicate that it's a modular variant, but there is no
modular content yet. We don't want to treat that as Everything.
The end result will be an empty repository.
Fixes: https://pagure.io/pungi/issue/871
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
For the first pass we don't need to filter out exclusive architectures,
and we don't need to exclude source packages without any binary
packages. We just want to merge the two package sets as fast as
possible.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We now have a way to select even older version of package (since the
newer one can be left out of the whitelist), so we can include multiple
versions of the same package into global package set.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need to check tags for the variant, not for the whole compose. This
results in merge always being done even if there is a single tag. For
f29 tag in Fedora this takes about 2 hours for each variant.
Relates: https://pagure.io/pungi/issue/860
Relates: https://bugzilla.redhat.com/show_bug.cgi?id=1551653
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
In one place, there was an explicit check if pkgset_koji_tag was set,
in another it was blindly referenced and assumed to be a list (with
accidental semi-success for a scalar.)
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Use 'get_packages_to_gather' to fail early if these packages are not
signed with right key. This prevents us from having to wait for the
repo to be created and depsolving to finish. Unsigned dependencies will
still be reported later than previously.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
There can be packages in the tag that will not end up in the compose.
Instead of failing immediately with error, this patch delays the check
until after depsolving finishes and only checks packages that will
really be included.
This is not an issue for nodeps compose, as that already pulls in only
packages that will be composed and nothing else.
Merges: https://pagure.io/pungi/pull-request/843
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Background story: if a compose is combining modular and traditional
compose, the configuration will contain multiple Koji tags to build
package set from (one tag for each module, plus at least one tag for the
traditional content). However some packages might be present in multiple
tags, and if the package set contains both, there's no way to control
which one will end up in the compose.
The solution for this is to give preference to the modular compose. If a
package with the same name exists in multiple tags, we only take the
first one we find. This relies on ordering of collected tags: modular
ones are always first, and traditional tags are at the end of the list.
If there are multiple modules that contain the same package, only one of
them will be used, which is not correct. Allegedly this should not
happen. In any case such use case does not work without this patch
either, so we're not losing anything.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Use global koji tag for populating package set even for modular
composes. To preserve backwards compatibility value "not-used" is
ignored. This should however be fixed in the configuration to simply not
specify the option at all.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If buildinstall is supposed to run, we can't pick only some packages
from the tag. The temporary repo is used as a source for lorax. There
could very well be packages not meant to be shipped in the compose that
are still essential for building the bootable images.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should make all tests pass on both Python 2 and Python 3.
Unittest2 is required on Py 2.6 and Py 3.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
In many cases we need to open files as binary to avoid errors on Py3
about writing binary data to file opened in text mode.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Currently, this will fail two lines later when we try to access
`pdc_module['modulemd']` with an unhelpful `TypeError` since
`pdc_module` is `None`.
Signed-off-by: Ralph Bean <rbean@redhat.com>
For each tag we ask Koji about (there might be more than one in
modularity case), we create a log file with list of RPMs and details
about which tag they were pulled from. This makes it easier to find out
where the package is inherited from.
Fixes: #547
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
As long as the assertion is not violated, this would not be a problem as
the second argument is not evaluated. However if the condition was
broken, a NameError would be raised instead of AssertionError with a
nice message.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The address is no longer correct. We can just as well simply point to
the web page describing the license.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.
There is a custom extension to the schema to report deprecated options.
The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.
Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.
The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.
The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch also fixes the bug where using a preconfigured Koji event
would not actually work and instead the latest event would be used.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Jan Kaluza
Lubomír Sedlář
Martin Curlej
Patrick Uiterwijk
Ondrej Nosek
Owen W. Taylor
Dong Wang
Ralph Bean
Qixiang Wan