Commit Graph

78 Commits

Author SHA1 Message Date
Jan Kaluza 6080b45178 Fix Koji search for modules with dash in stream
MBS replaces the dash by underscore. We have to match that.

Merges: https://pagure.io/pungi/pull-request/1005
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-07-18 10:31:19 +02:00
Lubomír Sedlář 98e7106f3e pkgset: Apply module filters on pkgset level
If a module says to filter a package out, we can do it immediately when
getting the build information from Koji.

This avoids a possible problem of something pulling the module package
in as a dependency, but it should also make the package set slightly
smaller.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-06-25 15:15:42 +02:00
Martin Curlej d8c03f6239 pkgset: Query Koji instead of PDC
PDC is deprecated in upstream. The usecase for getting list of modules
by NS, NSV or NSVC can however be satisfied by querying modules imported
into Koji.

This makes it possible to deprecate PDC configuration.

Merges: https://pagure.io/pungi/pull-request/985
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-06-18 15:17:04 +02:00
Ondrej Nosek c85d80f3c2 Handling multiple modules with the same NSV - PDC
Signed-off-by: Ondrej Nosek <onosek@redhat.com>

JIRA: COMPOSE-2510
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-05-21 11:24:46 +02:00
Lubomír Sedlář bd9a0ceda2 pkgset: Create global repo in parallel to merging pkgsets
The work in creating the repo is done in a separate process. This can
easily use two threads.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-05-07 08:38:23 +02:00
Lubomír Sedlář b8555b7869 modules: Allow context in variants XML
JIRA: COMPOSE-2508
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-26 09:26:37 +02:00
Jan Kaluza 9915c7d644 Allow reusing pkgset FileCache from old composes.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-04-23 09:57:03 +02:00
Lubomír Sedlář 267ff86f04 pkgset: Use modules PDC API
Instead of the deprecated and confusing unreleasedvariants.

JIRA: COMPOSE-2363
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-17 12:16:18 +02:00
Ondrej Nosek 7798174b30 Tests fail if unittest2 library is missing
Merges: https://pagure.io/pungi/pull-request/894
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-04-09 07:47:17 +02:00
Lubomír Sedlář f38770c67d pkgset: Construct UID for PDC modules
We can't rely on the UID to be correctly joined with colons. There may
be historical data that still uses dashes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-06 10:47:22 +02:00
Ondrej Nosek 1f0739831c Update tests for Python 2.6
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-03-29 16:34:52 +02:00
Ondrej Nosek e3aa2f769b Write module metadata
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-03-29 09:47:55 +02:00
Jan Kaluza 9be2d6a920 Allow setting <kojitag/> in <modules/> in variants.xml to get the modules from this Koji tag.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-21 14:33:45 +01:00
Jan Kaluza 1574f306c7 Move Modulemd import to pungi/__init__.py to remove duplicated code.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-20 12:59:17 +01:00
Jan Kaluza 340ae4d286 Use libmodulemd instead of modulemd Python module
Merges: https://pagure.io/pungi/pull-request/851
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-12 13:07:33 +01:00
Lubomír Sedlář c83316da31 pkgset: Merge initial package set without checks
For the first pass we don't need to filter out exclusive architectures,
and we don't need to exclude source packages without any binary
packages. We just want to merge the two package sets as fast as
possible.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-03-08 14:29:23 +01:00
Lubomír Sedlář 364d7f5229 Support multiple sources in one variant
With this patch the gather_source option is no longer used. Instead, all
sources are always used. If they return at least some input packages,
then a configured method is used and the returned lists of packages from
all sources are merged.

The method used for gathering can be configured for each variant and
gather source separately.

Additional packages are only added to the comps source.

Each gathering step is logged separately. All the logs are preserved for
later inspection.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-01 14:31:09 +01:00
Lubomír Sedlář 6f21576921 tests: Skip testing pdc logs if dependencies are not installed
On EPEL 6 modulemd is not available, so the test fails with an import
error. We should just skip this particular tests, as the rest of the
functionality is still perfectly fine.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-30 08:21:21 +01:00
Dong Wang 463fb961bc Log PDC communications and info for modular composes
Fixes: https://pagure.io/pungi/issue/664
Merges: https://pagure.io/pungi/pull-request/723
Signed-off-by: Dong Wang <dowang@redhat.com>
2017-10-27 16:20:21 +02:00
Lubomír Sedlář ed22e07ef9 Port to Python 3
This should make all tests pass on both Python 2 and Python 3.

Unittest2 is required on Py 2.6 and Py 3.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Jan Kaluza 894a5a11a4 Add support for new modules naming policy with colon delimiter
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-10-20 10:42:09 +02:00
Jan Kaluza 6208dae869 pkgset: Cherry-pick packages from Koji when we know already what packages will end up in compose
Merges: https://pagure.io/pungi/pull-request/763
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-10-19 13:43:59 +02:00
Qixiang Wan 18bd37ff2c pkgset: Allow populating packages from multiple koji tags
One of the use cases is https://pagure.io/odcs/issue/7.

Merges: #660
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-06-29 08:27:34 +02:00
Martin Curlej 079454c502 pkgset: Use release number of a module
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

updated regex, added comment, pep8 fix
2017-06-21 15:53:39 +02:00
Lubomír Sedlář b2554ce663 gather: Log tag from which we pulled a package
For each tag we ask Koji about (there might be more than one in
modularity case), we create a log file with list of RPMs and details
about which tag they were pulled from. This makes it easier to find out
where the package is inherited from.

Fixes: #547
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-19 10:51:37 +02:00
Jan Kaluza 9fcd71f831 Add support for modular composes
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-03-22 15:55:52 +01:00
Patrick Uiterwijk 59dbd9d35f Remove executable permissions on test scripts
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-25 10:42:16 +00:00
Lubomír Sedlář 01422983cb [pkgset] Add tests for Koji source
This patch also fixes the bug where using a preconfigured Koji event
would not actually work and instead the latest event would be used.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-22 09:44:20 +02:00