Let's filter this list too, not just the grouplist tag.
JIRA: RHELCMP-7926
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
(cherry picked from commit d3f0701e01)
- The ability is added
- Also the generator includes only the latest by version packages to packages.json
- The generator has key `--is-reference` for an each repo. This key marks a repo as reference.
An reference repo is used as main source of packages. A not reference repo is used as source
of packages which don't exist in the reference repos.
- All cases are covered by the unittest
@BS-NOBUILD
@BS-TARGET-CL8
Change-Id: I2f80ba4fbfce27fb9a30500ae46c0b8a2f2aabcd
@BS-TARGET-CL8
@BS-NOBUILD
- It's added the script which can collect packages/modules
from the remote repos (including BS repos) and merge them
to an one local repo with the right repodata (including
modules.yaml.gz)
- The script `create_packages_json` can use regexps for list of excluded packages
Change-Id: I1365b712460959db6bb451d1199d640bff6ffe5e
- It's added the tool which can generate json like as `centos-packages.json` using repodata from completed repos.
@BS-LINKED-5ffda6156f44affc6c5ea239 # pungi & dependencies
@BS-TARGET-CL8
Change-Id: Ib0466a1d8e06feb855e81fb7160fe170e2e82e04
Some composes might need extra validation to ensure they are following
certain strict rules - for example containing only signed packages or
packages only from particular Koji tag.
There is currently no way how to check that Pungi configuration fulfills
these extra requirements.
This commit adds new `--schema-override` option to
`pungi-config-validate` script which allows caller to specify path to
JSON schema overriding the default JSON schema and therefore limitting
it further.
For exmaple, to limit the `pkgset_source` to `koji`, one can use
following JSON schema override:
```
{
"properties": {
"pkgset_source": {
"enum": ["koji"]
}
}
}
```
It is possible to use `--schema-override` multiple times to apply
multiple schema overrides.
Merges: https://pagure.io/pungi/pull-request/1341
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Make dummy-bash -> dummy-glibc dependency require archful. This avoids
potential race condition where order of dependency processing can result in
different packages being pulled in. The tests where this could happen are
updated.
Make dummy-glibc-debuginfo depend on dummy-glibc-debuginfo-common.
The filenames for the repo no longer include hash, and sqlite databases are not
generated.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The parts in multi compose are now named reasonably.
The package set repos are selected only once and not per variant. We
can't use repos with packages for specific arch only as that would
require more transformations during generating test data.
Lookasides for ResilientStorage part are added to point to Server part.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Run arbitrary commands before and after the compose.
The example config is updated to generate latest symlink with a
post-compose script. The pre compose script runs always, post compose
runs only if the compose is not doomed.
JIRA: COMPOSE-3288
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It may make sense to break a big compose into smaller chunks that can be
done independently. This script allows describing the smaller parts,
runs them with correct dependencies and arranges the result to look like
a single big compose.
All parts use the same koji event, that is either obtained from Koji, or
from command line argument.
JIRA: COMPOSE-2654
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The environment in comps for a variant can refer to groups in parent
variant (either for addons, or because of other configuration). We
should not remove the groups in this case.
This requires changes in two places:
* teaching `comps_filter` about groups that should not be removed
* fixing writing comps so that it does not actually change the data as
well
JIRA: COMPOSE-2612
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
With this patch the gather_source option is no longer used. Instead, all
sources are always used. If they return at least some input packages,
then a configured method is used and the returned lists of packages from
all sources are merged.
The method used for gathering can be configured for each variant and
gather source separately.
Additional packages are only added to the comps source.
Each gathering step is logged separately. All the logs are preserved for
later inspection.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This adds a new test for Requires(pre) and (post). The general structure
of the test now makes it easy to use the same test scenarios for
different backend.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The check for arch compatibility should not be performed if the
debuginfo package is noarch. Such packages should be included
unconditionally.
Fixes: #450
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
These are tests written by Daniel Mach originally for Distill-NG. They
are ported to current Pungi. The test repositories are committed in the
tests/fixtures/ directory. This is the same data that is used for test
compose, but the actual RPM files are not present. Some tests are
adapted from dmach's fork of Pungi.
Some of the packages are marked with a comment saying they are
important. These are the packages that the test is specifically trying
to get included in the package set. There are also explicit tests for
packages that should not be included.
Two tests are skipped for now as there is a bug preventing them from
passing. This is related to fulltree being done for packages that are
explicitly multilib.
The depsolver is called by invoking a separate executable, so the
coverage data is wrong.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When creating an ISO for a layered product, the name of the integrated
product should be included in the file name.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It uses bash specific features, so it should not claim /bin/sh in
shebang. We also want to use `set -e` to catch possible errors instead
of claiming success every time.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the `<groups></groups>` section is not specified in the variants XML
file, all groups will be used in this variant. The section must be
omitted completely, not just empty. This is (and was) correct according
to the DTD, it just lead to crash before.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is for layered products that have a variant mapped
to multiple variants in a base product, for example:
* Foo-Tools (id: FooTools, uid: Foo-Tools, name: Tools)
* Bar-Tools (id: BarTools, uid: Bar-Tools, name: Tools)
Requires productmd >= 1.2.
Signed-off-by: Daniel Mach <dmach@redhat.com>
If the configuration specifically requests no jigdos, there is no point
in checking for the binary existence.
This is not 100% reliable. The jigdo option defaults to True, so if the
option is not specified the binary is required even if there are no
images configured.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The variants.xml file can list a variant with is_empty="true" and no
groups. If such variant is found, not package gathering will be run for
it, and no repos will be created.
This only makes sense for a variant that will have some other
deliverables like live media or images.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There is now a single option `multilib`, that maps variants and arches
to multilib methods. This replaces old `multilib_methods` option.
Multilib arches are implicitly deduced instead of using the
`multilib_arches` option.
The test compose is updated to only enable multilib on Server and its
addons.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
On case-insensitive filesystems it is not such a good idea to have
directories that only differ in case. Packages should be always split
into lowercased directories.
The test data is modified to include some packages starting with
uppercase letters. The example in code can be verified by running
`nosetests --with-doctest`.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
CHANGE: Rename product_* config options to release_* for consistency with productmd.
ACTION: Rename product_name, product_short, product_version, product_is_layered to release_* in config files.
Rename //variant/product to //variant/release in variants XML.
CHANGE: createrepo_c is config option now defaults to True.
ACTION: You can safely remove 'createrepo_c = True' from config files.
Set 'createrepo_c = False' if you need legacy createrepo.
CHANGE: createrepo_checksum config option is now mandatory.
ACTION: Add 'createrepo_checksum = "sha256"' (or "sha") to config files.
Previous test data was insufficient for proper testing.
Test compose runs and depsolving tests require precisely
set NVRs, dependencies, sub-packages, etc.
Using rpmfluff for these would be an overkill, it's better
to create RPMs directly from specs.
Lubomír Sedlář
soksanichenko
Haibo Lin
Ken Dreyer
Jan Kaluza
Daniel Mach