Commit Graph

1809 Commits

Author SHA1 Message Date
Ondrej Nosek
fbeb14344f Better INFO messages about modules
Relates: https://pagure.io/pungi/issue/779

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-22 17:23:31 +01:00
Lubomír Sedlář
778dbaef73 Updates composes should be marked as supported
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-19 13:13:42 +01:00
Lubomír Sedlář
121ffb403f pkgset: Only add missing packages from global tag
Background story: if a compose is combining modular and traditional
compose, the configuration will contain multiple Koji tags to build
package set from (one tag for each module, plus at least one tag for the
traditional content). However some packages might be present in multiple
tags, and if the package set contains both, there's no way to control
which one will end up in the compose.

The solution for this is to give preference to the modular compose. If a
package with the same name exists in multiple tags, we only take the
first one we find. This relies on ordering of collected tags: modular
ones are always first, and traditional tags are at the end of the list.

If there are multiple modules that contain the same package, only one of
them will be used, which is not correct. Allegedly this should not
happen. In any case such use case does not work without this patch
either, so we're not losing anything.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 16:57:22 +01:00
Ondrej Nosek
f301158974 ostree/utils: Drop timestamps from generated repo names - tests
Issue: https://pagure.io/pungi/issue/811

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-17 16:48:22 +01:00
Colin Walters
3427d6abe2 ostree/utils: Generate a single pungi.repo file, use repo-<num> IDs
Ideally, pungi would generate repository IDs like `fedora-updates`
or so, and we'd have versioning inside the rpm-md.  But for now
let's do this to avoid invalidating rpm-ostree's change detection.

Closes: https://pagure.io/pungi/issue/811
Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-17 16:48:22 +01:00
Colin Walters
b2bdc8a608 ostree/utils: Drop timestamps from generated repo names
Since we drop these files in a separate workdir each time,
there's no need to datestamp them.  Doing so is part of the
cause for invalidating's rpm-ostree input change hashing.

Issue: https://pagure.io/pungi/issue/811
Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-17 16:48:22 +01:00
Lubomír Sedlář
5cc612f966 gather: Do not require variant for module source
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:06:56 +01:00
Lubomír Sedlář
df27164c1c gather: Comps source should not crash without comps file
There just isn't anything to return.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
670a68a5b8 gather: JSON source returns nothing without configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
61a3be2307 buildinstall: Fix treeinfo generating on failure
When buildinstall fails, there will be no lorax generated files copied
into the compose directory. However they may still be mentioned in the
.treefile in work/ subdirectory where lorax runs.

To avoid possible issues, we should use the lorax created .treeinfo only
if the run was successful.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 13:57:49 +01:00
Jan Kaluza
930c2f1a42 Add buildinstall_use_guestmount boolean option
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-01-16 09:18:35 +01:00
Lubomír Sedlář
68a1370036 gather: Use arch packages in nodeps method
We already filtered a list of packages compatible with the architecture.
There's no need to do that again (with bugs). Instead of using the
global package set we should just restrict the code to an arch package
set.

This should not break anything. If a package is included such that it's
in global but not arch package set, the compose would crash as linking
packages takes paths from the arch package set.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-15 09:55:51 +01:00
Lubomír Sedlář
f4c3d2423d pkgset: Always use global tag if specified
Use global koji tag for populating package set even for modular
composes. To preserve backwards compatibility value "not-used" is
ignored. This should however be fixed in the configuration to simply not
specify the option at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-12 15:50:21 +01:00
Lubomír Sedlář
03293c725b config: Make pkgset_koji_tag optional
There are valid use cases for not specifying this option: specifically a
modular compose will get the tags to use from modules listed in the
variants file.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-12 15:50:21 +01:00
Ondrej Nosek
51cd359057 ostree: Add force_new_commit option - test added
Relates: https://pagure.io/pungi/issue/811

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-11 15:08:07 +01:00
Colin Walters
161b7f974b ostree: Add force_new_commit option
Followup from discussion in: https://pagure.io/pungi/issue/811

It's likely now that for Fedora Atomic Host we'll use this, to work
around other issues, after we fix the FAW change detection.

Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-11 15:03:36 +01:00
Lubomír Sedlář
5cc54cd587 gather: Fix checking string type
There is no unicode on Python 3, we should use six to hide this detail.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-10 13:50:21 +01:00
Ondrej Nosek
d4d264eb33 Improve logging for unsigned packages
Relates: https://pagure.io/pungi/issue/820

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-03 11:08:00 +01:00
Ondrej Nosek
2152e7ea26 Fall back to mount if guestmount is not available
Relates: https://pagure.io/pungi/issue/803

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-02 17:13:47 +01:00
Ondrej Nosek
116e7ca3bd El-Torito boot information on s390x
Relates: https://github.com/rhinstaller/lorax/pull/236

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-14 10:08:30 +01:00
Ondrej Nosek
59c162d46f Remove strace from buildinstall runroot
The package is not really needed.

Relates: https://pagure.io/pungi/issue/799

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-11 17:36:28 +01:00
Ondrej Nosek
ba3adf9bc4 Merge #818 doc: move "Phases" up, "Contributing" down 2017-12-11 08:54:32 +00:00
Ken Dreyer
6c3245bdee doc: fix "Miscellaneous" spelling in Config section
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-12-08 13:46:14 -07:00
Ken Dreyer
d66abc3aea doc: move "Phases" up, "Contributing" down
The "Phases" section breaks down what Pungi does in detail. Place it
towards the top of the documentation table of contents.

The "Contributing" and "Testing" sections are relevant to developers,
not all users, so move them to the end.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-12-08 13:43:39 -07:00
Lubomír Sedlář
25907f61ce 4.1.21 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:43:10 +01:00
Lubomír Sedlář
e15a49defd tests: Use correct python version for config validation test
When tests are running using Py3, the script should be called with Py3
as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
d5a0316877 Use dnf backend for repoclosure on PY3
When yum is not available, it makes no sense to offer it as an option.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
60cc3e5d55 Drop checks for git and cvs
This essentially forces everyone to install cvs even though they most
likely don't need it. There is no easy way to check what is actually
going to be needed, so let's just put the responsibility of decoding a
potential error message to users.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
fa4d728230 Relax check for gettext
It's only used in productimg phase. We don't have to check for it on
every run.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
aa9c137412 Drop check for repoquery command
There is no place that would possibly call it, so we can drop the check
and function for generating cli.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
001ae1d7b4 Use modifyrepo_c if possible
When configured to use createrepo_c, we should also use modifyrepo_c.
That allows us to relax the check for createrepo package.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
970143e59f pkgset: Add SRPMs to whitelist
When cherry picking packages from Koji tag, we need to make sure that
for every binary package we always have a corresponding source package.
Even if it does not go into the compose, we need it to get values for
Exclusive or Exclude Arch tags.

This means we need to process the binary packages first and only then
look at source ones. Instead of sorting a potentially very long list,
let's just iterate twice.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:23:31 +01:00
Lubomír Sedlář
cf77a6e413 modules: Allow multilib
The list of RPMs in a module can contain multilib packages, but they are
never included because we only ask for compatible arches without
multilib. For x86_64 that only adds noarch, but not i686.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:17:53 +01:00
Ondrej Nosek
9642c1171c add ability to specify ostree ref in OSTREE phase - update
Additionally ostree_ref (if parameter is given) should be placed into treefile.

Relates: https://pagure.io/pungi/issue/777

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-04 18:07:36 +01:00
Ondrej Nosek
a6c65e026a add ability to specify ostree ref in OSTREE phase
It allows specify what ref we want this compose to commit to.
New parameter 'ostree_ref' overrides the default value from the treefile json.

Relates: https://pagure.io/pungi/issue/777

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-04 09:27:19 +01:00
Jan Kaluza
953fb4c54c buildinstall: Allow using external dire for runroot task
A new `buildinstall_topdir` option allows using buildinstall even when
the compose is created on a different volume that Koji is using.

The files are created in this external directory and then copies into
the usual location.

Merges: https://pagure.io/pungi/pull-request/807
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-11-30 10:55:52 +01:00
Lubomír Sedlář
cf114a7fab pkgset: Remove package skip optimization for bootable products
If buildinstall is supposed to run, we can't pick only some packages
from the tag. The temporary repo is used as a source for lorax. There
could very well be packages not meant to be shipped in the compose that
are still essential for building the bootable images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-27 08:18:12 +01:00
Lubomír Sedlář
b068514471 Add documentation for modular composes
List all available configuration options and go into some detail on how
modules should be specified. This would probably deserver a more
thorough explanation, but it's still bit in a flux.

Fixes: https://pagure.io/pungi/issue/767
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-21 08:47:19 +01:00
Lubomír Sedlář
0f104cea40 osbs: Get correct path to repo for addons
We need to use `repository`, not `os_tree` path. For any non-addon
variant they are the same, but for addons the original path does not
work as it points to the parent really.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-20 12:58:49 +01:00
Ondrej Nosek
91ee1fb854 Remove deprecated options
Options that are currently marked as deprecated do not have any effect
anymore (other than printing warning). We should remove them and update
the message so that we can mark options as deprecated even when they
still work.

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-11-16 11:12:34 +01:00
Lubomír Sedlář
2bd3b85bb7 module-source: Log details about what packages are gathered
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-13 09:18:48 +01:00
Lubomír Sedlář
48d155d304 gather: Log details about nodeps method
Without this there is almost no information on what's happening. This
patch should provide basic debugging capabilities.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-13 09:18:48 +01:00
Lubomír Sedlář
69adf35e84 gather: get_packages_to_gather returns a tuple
It always needs to return a tuple, otherwise there would be crashes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-13 09:04:34 +01:00
Lubomír Sedlář
d2804b5d89 iso-wrapper: Fix calling wrong logger method
The mounting function receives logger as an argument, and its method is
called just error.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-13 09:04:34 +01:00
Patrick Uiterwijk
8181c5be48 Turn COMPOSE_ID version generator into DATE_RESPIN
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2017-11-10 11:14:11 +01:00
Lubomír Sedlář
388be481ea iso-wrapper: Remove hacks for sorting
We can use a key function instead of relying to the deprecated cmp. This
makes the code work on Python 2.6 and on recent versions it makes it
faster.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-08 15:35:29 +01:00
Lubomír Sedlář
cb740f063e Report missing module dependencies earlier
Fixes: https://pagure.io/pungi/issue/768
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-07 08:51:00 +01:00
Patrick Uiterwijk
1dbd0248d4 Implement version.compose_id version generator
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2017-11-06 16:14:44 +01:00
Patrick Uiterwijk
1a10a1fe83 Optionally do old_compose per release type
This would make sure that e.g. "updates" composes don't try to use "updates-testing" as an
old_compose_path, which would create practically useless deltarpms and for no repodata
reuse at all.

Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2017-11-06 15:17:33 +01:00
Lubomír Sedlář
385002fe94 4.1.20 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-01 15:01:02 +01:00