If a file has multiple hard links, genisoimage will put the wrong number
on the ISO. This patch can work around it by copying hard-linked files
into a temporary staging directory.
JIRA: COMPOSE-2610
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
PDC is deprecated in upstream. The usecase for getting list of modules
by NS, NSV or NSVC can however be satisfied by querying modules imported
into Koji.
This makes it possible to deprecate PDC configuration.
Merges: https://pagure.io/pungi/pull-request/985
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
The `add_noarch` option of `get_valid_arches` is broken and doesn't
really do anything (noarch is always present in the result).
This causes packages that have ExclusiveArch including noarch to
actually not be excluded. They should be.
Changing this globally could have a very big impact. Therefore we can
hide it behind a configuration option so that it's opt-in.
JIRA: COMPOSE-2457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is an optimization for Yum. DNF does not care at all.
The behaviour is configurable, but the default depends on gather
backend, as that is what users should be using to consume the packages
from the repo.
Fixes: https://pagure.io/pungi/issue/951
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It checks config file value 'skip_phases' for valid phases names.
Also checks command-line attribute 'skip-phase' of 'bin/pungi-koji'.
JIRA: COMPOSE-2493
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
With one compose combining traditional and modular content there might
be different requirements for tag inheritance. This patch adds a new
option that controls whether builds in modular tags should be inherited.
It defaults to False, which is the right option for current MBS
behavior.
JIRA: COMPOSE-2148
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
A new configuration option is added that allows users to point lorax to
extra repositories. This can be handy if some tools to create the
bootable image are not part of the product itself.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
JIRA: COMPOSE-2253
OstreeInstaller phase will be moved to a different timeslot
and therefore needs different repo not to depend on Gather
phase which runs at the same time.
Related: https://pagure.io/pungi/issue/778
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
With this patch the gather_source option is no longer used. Instead, all
sources are always used. If they return at least some input packages,
then a configured method is used and the returned lists of packages from
all sources are merged.
The method used for gathering can be configured for each variant and
gather source separately.
Additional packages are only added to the comps source.
Each gathering step is logged separately. All the logs are preserved for
later inspection.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There are valid use cases for not specifying this option: specifically a
modular compose will get the tags to use from modules listed in the
variants file.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Followup from discussion in: https://pagure.io/pungi/issue/811
It's likely now that for Fedora Atomic Host we'll use this, to work
around other issues, after we fix the FAW change detection.
Signed-off-by: Colin Walters <walters@verbum.org>
This essentially forces everyone to install cvs even though they most
likely don't need it. There is no easy way to check what is actually
going to be needed, so let's just put the responsibility of decoding a
potential error message to users.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There is no place that would possibly call it, so we can drop the check
and function for generating cli.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When configured to use createrepo_c, we should also use modifyrepo_c.
That allows us to relax the check for createrepo package.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It allows specify what ref we want this compose to commit to.
New parameter 'ostree_ref' overrides the default value from the treefile json.
Relates: https://pagure.io/pungi/issue/777
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
A new `buildinstall_topdir` option allows using buildinstall even when
the compose is created on a different volume that Koji is using.
The files are created in this external directory and then copies into
the usual location.
Merges: https://pagure.io/pungi/pull-request/807
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Options that are currently marked as deprecated do not have any effect
anymore (other than printing warning). We should remove them and update
the message so that we can mark options as deprecated even when they
still work.
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
This would make sure that e.g. "updates" composes don't try to use "updates-testing" as an
old_compose_path, which would create practically useless deltarpms and for no repodata
reuse at all.
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
The file extension in configuration is only used to tell Pungi which
files from the task results should be downloaded. The user has to get it
right or the phase will fail. Each format has a single valid suffix.
Pungi should not require users to specify the suffix, since it can just
as well just know the right value.
The old configuration will continue working, only the extension will be
ignored.
Fixes: https://pagure.io/pungi/issue/753
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
To be more precise, new createrepo parameter "workers" was customized and
new default value was set. This fixes issue #752.
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
This should make all tests pass on both Python 2 and Python 3.
Unittest2 is required on Py 2.6 and Py 3.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Let's not duplicate the list. Productmd exports a list of valid values,
so Pungi should just pick and use that.
Closes: https://pagure.io/pungi/pull-request/773
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This would be useful for modularity. The templates can be added now and
variables set via the existing `lorax_option`.
It's not possible to use custom templates not shipped with lorax, as
passing the path to a random directory is a little bit more tricky.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It only affects gathering packages when gather_source is set to comps,
but it could still make sense to include the file in the repository even
if the not used for gathering.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of the old default value and custom handling introduce new
option `live_images_target` (for consistency with other phases) and use
the usual inheritance rules.
Fixes: https://pagure.io/pungi/issue/749
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
`pungi-gather` (the tool that underlies both the `pkgset` and `gather`
phases) contains profiling code that will log statistics about how long
different function calls take. However, pungi-koji did not contain a
way to pass the ``--profiler`` argument to enable this.
This change adds a new configuration option ``gather_profiler`` which,
when set to true, simply passes the argument to `pungi-koji`. Hopefully
this can help shed some light on what is happening in some of our
longer-running composes.
Merges: https://pagure.io/pungi/pull-request/727
Signed-off-by: Ralph Bean <rbean@redhat.com>
The configuration needs to be more granular than a single global option.
With this patch each tree can enable deltas separately.
Fixes: https://pagure.io/pungi/issue/715
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The config can change the default compose type. This can still be
overwritten by a CLI argument. A `--production` option is now added to
CLI (because that was the default before).
Fixes: https://pagure.io/pungi/issue/694
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is updated in all files for consistency, even the modules that will
never be ported to Py 3 completely due to dependency on Yum.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We can't reliably tell user what system packages are missing as the name
might be different on different systems. Addiotionally there's no reason
why not rely on the packaging to be correct.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It makes no sense to repeat the same configuration for multiple
architectures. Instead we should just list the architectures as another
key in the mapping. There is an option to specify multiple config dicts.
This preserves full backwards compatibility, the old config format is
still accepted.
Fixes: https://pagure.io/pungi/issue/678
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds a helper function for easy creation of a structure where
either X or list of Xs is accepted.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a file should be obtained from a git repository, allow running an
arbitrary command (like `make`) after clone but before copying the files
out. This only works for the Git backend.
The downside is that a clone is needed and we can no longer use `git
archive` to speed things up.
Fixes: https://pagure.io/pungi/issue/5
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a package mentioned in comps is not available in the package set,
print a warning about this. Additionally there is a config option that
allows to turn this warning into a fatal error.
Fixes: https://pagure.io/pungi/issue/50
Fixes: https://pagure.io/pungi/issue/683
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Alternatively the call to repoclosure can be turned off. This is
customizable per variant and architecture.
Fixes: https://pagure.io/pungi/issue/676
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Lubomír Sedlář
Martin Curlej
Patrick Uiterwijk
Ondrej Nosek
Petr Šabata
Patrick Uiterwijk
Jan Kaluza
Colin Walters
Ralph Bean