It's not a great name anymore though, because it will fail the compose
if any input package is missing, no matter whether it's from comps,
prepopulate or additional_packages.
JIRA: RHELCMP-12484
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 1c4275bbfa)
The initial version of the filtered the latest builds at the start. That
doesn't matter in many cases:
* When there are no lookaside repos, there is generally a single version
of each package.
* When lookaside repos do not overlap with compose repos, or contain
only older versions.
It is however a problem when the lookaside repos contain higher version
of a package than what is in a compose repo, and some package explicitly
requires the older version.
Consider this scenario:
* lookaside contains bar-1.1
* compose repo contains bar-1.0 and foo-1.0
* foo-1.0 `Requires: bar < 1.1`
The original code would filter out the bar-1.0 package, and then fail on
unresolved dependencies.
This patch moves the computation of latest packages much later, to part
of code where all options to satisfy a dependency are selected and the
best match is chosen. At that point if there are multiple versions
available, we do want the latest one.
JIRA: SPMM-13483
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit bcc440491e)
If lookaside contains an older version of a package, but with a
different arch, the depsolver doesn't notice that and prefers the
lookaside version.
This is not correct. The latest package should be used no matter if
there are different arches available.
The filtering in DNF doesn't ensure this, so we have to build it
ourselves. To limit the performance impact, only run this filtering when
there actually are some lookaside repos configured.
JIRA: RHELCMP-11728
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 2ad341a01c)
The documentation claims that dotarch syntax is supported for additional
packages. For yum backend this seems to be handled automatically, but
the dnf backend could not interpret this.
This patch checks if a package is specified in the syntax and contains a
valid architecture. If so, the query will honor the arch.
JIRA: RHELCMP-11728
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit 82ca4f4e65)
If the depsolver wants to include a package that is present in both the
source repo and a lookaside repo, it reliably detects binary packages
present in lookaside, but for debuginfo it's not so reliable.
There is a separate package object for each package in each repo.
Depending on which one is used, debuginfo could be included in the
result or not. This patch fixes that by actually looking if the same
package is present in any lookaside repo.
JIRA: RHELCMP-9373
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit a4476f2570)
Trying to compose from external classic repositories return an error trying the hardling from
a yum cache directory located in /tmp to the target directory in another filesystem.
This commit fixes this using the 'link' method form linker module which handle the link_type
configuration parameter instead of the hardcoded method 'hardlink'.
Change-Id: Ib79cfbd72f9def6462fddb2ae368730c55f257cd
Signed-off-by: Romain Forlot <romain.forlot@iot.bzh>
It is possible we only ever see a package from non-lookaside repo, but
it actually is in lookaside. This patch should check for that before
adding the package to result set.
JIRA: RCM-71946
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Until now, the behaviour was that all debuginfo from a build would be
included if at least one package with the same arch was included.
This resulted in many debuginfo packages being included even though
their corresponding package was not present.
With this patch, we will always pull in debugsource, but foo-debuginfo
will only be included if foo is included for the same arch. As a
consequence, it is necessary to resolve dependencies of debuginfo
packages. There are cases where foo-debuginfo needs foo-debuginfo-common
for example.
This change means that DNF and YUM backends are no longer identical in
the output. The tests where this is demonstrated are duplicated and
their results are modified.
JIRA: COMPOSE-3823
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This was a workaround to make some packages from the global repo
invisible for depsolving. This is now handled by packages being in
different repos. We can select which repos are enabled at which point.
This achieves the same result, but much faster.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The configuration for modules has changed.
Lookaside handling has changed, and there are now test failures. This
is a not a bug in DNF, so we need to fix it on our side.
Relates: https://bugzilla.redhat.com/show_bug.cgi?id=1603123
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If a package is in lookaside, we don't really need a debuginfo nor
source for it. Definitely we should not pull them from the package set
even if there is some suitable package. Therefore we should include them
in the output but marked as lookaside.
Fixes: https://pagure.io/pungi/issue/915
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The log file used to contain a dump of the configuration, which could
potentially be very large. This patch removes data that is directly
visible in the input kickstart file. Instead it just displays the number
of items.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need to include all relevant arches, not just the base one (including
noarch and src). However the list can be shortened by only listing
NEVRs, because that should be unique.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Basically everything not on the list is excluded. This has to be applied
before we filter only the latest versions (otherwise we could lose
packages that are on the whitelist).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This already works on YUM backend, and this patch makes it work for DNF
as well.
Both native and multilib debuginfo and debugsource packages will be
excluded. This matches the yum behavior.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
For sources and debuginfo package the flag should be set based on the
repo we pull that particular RPM from, not for the corresponding binary
package.
There could be a case where they come from different repos, so this
would cause problems.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When the same package is contained in the main repo for depsolving as
well as in a lookaside repo, there was a chance that we would pick a
package from main repo, but corresponding source from lookaside.
This would cause a crash when trying to link the packages into the
compose.
A solution is to try to get source rpm from the same repo as the binary
package. Only when it's not available there we get some other one.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should make all tests pass on both Python 2 and Python 3.
Unittest2 is required on Py 2.6 and Py 3.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When using repos as gather_source, we should use DNF backend even for
constructing initial package set and to download the packages from
source repos. Without this the repos source would not be usable on
Python 3.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is needed for correct trimming of addons and optional. A package
pulled in as a dependency but that matches something on fulltree exclude
list should have this flag. It will then be moved from addon to base
variant and therefore excluded from optional.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
These packages should behave like regular debuginfo packages (at least
for now).
Fixes: https://pagure.io/pungi/issue/684
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When looking at a package in a lookaside repo, it does not make much
sense to process its dependencies. We should just assume that the
lookaside can satisfy them.
In the worst case, this could result in packages being pulled into the
compose just so that they could satisfy a dep of something in lookaside.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This way we pay once for the expansion, and lookups are then done in
constant time instead of iterating over the list over and over again.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
These requires are accessible from a separate attribute, but we want to
handle them the same ways as regular Requires.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Lubomír Sedlář
Romain Forlot
Haibo Lin
Frédéric Pierret (fepitre)
Till Maas