Commit Graph

38 Commits

Author SHA1 Message Date
Lubomír Sedlář
1a161982c0 gather: Make devel modules configurable
By default nothing should change. This patch adds a new config option
that enables all this new craziness:

 * it turns of applying module filters at pkgset level
 * it creates new modules and adds them to the compose

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-08 14:41:04 +02:00
Lubomír Sedlář
06b32b5d80 pkgset: Stop prefilling RPM artifacts
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-08 14:41:04 +02:00
Lubomír Sedlář
08d65bdde6 gather: Create devel module for each normal module
The module has same S:V:C, but the name is suffixed with `-devel`. The
module should contain all packages from the module koji tag that were
not included in the actual module.

The devel module has the same dependencies as the regular module, but
also additionally depends on the original module. The API and profiles
are cleared in the new module.

In the metadata it shows the same koji tag.

The test if package goes to the module is refactored to a function to
make work with the negated case a bit easier.

There may be unneeded multilib packages in the -devel module, because
there might be buildtime dependencies between things that we don't see.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>

Clear API and profiles

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-08 14:40:13 +02:00
Lubomír Sedlář
be81aeaa7d gather: Filter arches similarly to pkgset
There should not be multilib arches when checking if noarch package is
compatible with current arch. Otherwise we might be excluding a package
from x86_64 just because it does not work on i686.

JIRA: COMPOSE-2885
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-09-14 13:27:39 +02:00
Lubomír Sedlář
1350684c31 Index arch modulemd by full NSVC
There can be multiple modules with the same name and stream. They should
all have the same version, but will have different contexts. Fus takes
only N:S as input, but should pull in all matching modules. We just need
to give it correct data in the repo.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-09-05 13:10:53 +02:00
Lubomír Sedlář
03ee632cc8 gather: Honor module whitelist
If the modulemd contains a whitelist of packages (under buildopts), it
means the packages are possibly renamed, and we need to check that list
instead of components.rpms.

Multilib does not really work now, anything with non-native arch is
skipped.

JIRA: RCM-38019
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-08-27 09:08:16 +02:00
Stephen Gallagher
ec96757707 Copy modules instead of reparsing them
The Modulemd.copy() method has been available since libmodulemd 1.1
and is much faster than dumping to a string and parsing it again.

Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
Merges: https://pagure.io/pungi/pull-request/965
2018-05-31 09:04:28 +02:00
Lubomír Sedlář
8f1beeb54b init: Always filter comps file
Even for Everything we want to filter the comps file to make sure we
remove the stuff that is not compatible with current arch. All groups
are still preserved in that case.

This allows us to do the filtering once in init phase than just use the
prepared file in comps source.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-10 08:21:56 +02:00
Jan Kaluza
c6d507582a Support multilib in GatherSourceModule
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-28 13:38:46 +02:00
Jan Kaluza
1574f306c7 Move Modulemd import to pungi/__init__.py to remove duplicated code.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-20 12:59:17 +01:00
Jan Kaluza
3f71cdd384 Use Modulemd.Module for 'variant.arch_mmds' instead of yaml dump
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Merges: https://pagure.io/pungi/pull-request/872
2018-03-19 14:47:03 +01:00
Jan Kaluza
fedce5dff1 Remove the filtered RPMs from module metadata even in case all RPMs are filtered out.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-15 11:13:39 +01:00
Lubomír Sedlář
5c902592ae pkgset: Allow empty list of modules
This should indicate that it's a modular variant, but there is no
modular content yet. We don't want to treat that as Everything.

The end result will be an empty repository.

Fixes: https://pagure.io/pungi/issue/871
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-03-14 14:49:09 +01:00
Jan Kaluza
340ae4d286 Use libmodulemd instead of modulemd Python module
Merges: https://pagure.io/pungi/pull-request/851
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-03-12 13:07:33 +01:00
Lubomír Sedlář
bd852f4059 Remove comps groups from purely modular variants
The comps source should not return all groups when there are only
modules defined. This fixes part of the problem: non-modular packages
will not go in by default.

The second part is the comps file in the created repository. It will be
filtered to not contain any groups (because packages from there will not
be in the repo).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-03-02 08:46:23 +01:00
Jan Kaluza
18d005e593 Add modulemd metadata to repo even without components
There is a valid use case for modules without any RPMs in them. This
patch makes it possible to include such modules in the repodata.

Merges: https://pagure.io/pungi/pull-request/856
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-26 07:22:41 +01:00
Jan Kaluza
7ee920a085 Disable multilib for modules
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-22 11:54:02 +01:00
Lubomír Sedlář
5cc612f966 gather: Do not require variant for module source
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:06:56 +01:00
Lubomír Sedlář
df27164c1c gather: Comps source should not crash without comps file
There just isn't anything to return.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
670a68a5b8 gather: JSON source returns nothing without configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
cf77a6e413 modules: Allow multilib
The list of RPMs in a module can contain multilib packages, but they are
never included because we only ask for compatible arches without
multilib. For x86_64 that only adds noarch, but not i686.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:17:53 +01:00
Lubomír Sedlář
2bd3b85bb7 module-source: Log details about what packages are gathered
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-13 09:18:48 +01:00
Lubomír Sedlář
ed22e07ef9 Port to Python 3
This should make all tests pass on both Python 2 and Python 3.

Unittest2 is required on Py 2.6 and Py 3.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Jan Kaluza
5dd6b1b0e7 GatherSourceModule: return rpm_obj instead of the rpm_obj.name
Merges: https://pagure.io/pungi/pull-request/680
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-07-20 14:58:58 +02:00
Jan Kaluza
f7197ddbcc Allow gather source classes to return SimpleRpmWrapper objects from pkgset phase directly.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-06-14 08:51:37 +02:00
Lubomír Sedlář
700106facf arch: Move exclu(de|sive)arch check to a function
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-06 15:28:12 +02:00
Jan Kaluza
3601d6d1a8 gather-source: Check arch in module source
Skip the RPM if it is excluded on this arch or exclusive for different
arch.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-06-06 15:25:41 +02:00
Jan Kaluza
118444a311 Generate proper modular metadata when there are different versions of the same package in the variant
Merges: #629
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-05-30 15:18:09 +02:00
Jan Kaluza
d037d61521 Store RPM artifacts in resulting repository in modulemd metadata.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-05-04 21:12:30 +02:00
Lubomír Sedlář
20005adb98 comps-wrapper: Return IDs instead of yum.comps.Group
We never need anything more than the ID, so passing around a big
complicated object is not necessary.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-13 10:28:26 +02:00
Ralph Bean
01607602c6 Expand compatible arches when gathering from modules.
Fixes #596.

Signed-off-by: Ralph Bean <rbean@redhat.com>
2017-04-12 14:35:46 -04:00
Jan Kaluza
9fcd71f831 Add support for modular composes
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-03-22 15:55:52 +01:00
Lubomír Sedlář
a49bf3d74e gather: Filter comps group on depsolving input of optional
This is a follow up of 8bc65a8be5. When
resolving packages that go into optional variant, we only want to skip
filtering groups when working on a top-level variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-27 12:54:12 +01:00
Lubomír Sedlář
d6dc269ef4 Remove FSF address from comments
The address is no longer correct. We can just as well simply point to
the web page describing the license.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-23 10:26:43 +02:00
Lubomír Sedlář
f9a6c8418f Add JSON Schema for configuration
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.

There is a custom extension to the schema to report deprecated options.

The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.

Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.

The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.

The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-01 10:56:15 +02:00
Lubomír Sedlář
11bbbae2ed Treat variants without comps groups as having all of them
If the `<groups></groups>` section is not specified in the variants XML
file, all groups will be used in this variant. The section must be
omitted completely, not just empty. This is (and was) correct according
to the DTD, it just lead to crash before.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-28 09:01:30 +02:00
Dennis Gilmore
07e90f0f96 change pypungi imports to pungi 2015-03-12 16:12:38 -05:00
Daniel Mach
f116d9384f Initial code merge for Pungi 4.0. 2015-02-10 08:19:34 -05:00