Commit Graph

2128 Commits

Author SHA1 Message Date
Lubomír Sedlář
3201648c37 gather: Allow filtering debuginfo packages
This already works on YUM backend, and this patch makes it work for DNF
as well.

Both native and multilib debuginfo and debugsource packages will be
excluded. This matches the yum behavior.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-28 10:07:42 +01:00
Ondrej Nosek
660c04368b Move ostree phase and pipelines for running phases
Signed-off-by: Ondrej Nosek <onosek@redhat.com>

Related: https://pagure.io/pungi/issue/778
2018-02-26 13:43:44 +01:00
Ondrej Nosek
5c081cb545 Other repo for OstreeInstaller
OstreeInstaller phase will be moved to a different timeslot
and therefore needs different repo not to depend on Gather
phase which runs at the same time.

Related: https://pagure.io/pungi/issue/778

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-02-26 13:10:36 +01:00
Jan Kaluza
18d005e593 Add modulemd metadata to repo even without components
There is a valid use case for modules without any RPMs in them. This
patch makes it possible to include such modules in the repodata.

Merges: https://pagure.io/pungi/pull-request/856
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-26 07:22:41 +01:00
Adam Williamson
a1d559fb93 Correct fix for volume ID substition sorting by length
The previous attempt - caed78e - is not really correct. It sorts
the dict item tuples according to the alphabetical sort order of
the first item of each tuple (reversed). This will always work
when both substitutions *start* with the same characters, as in
the case of two strings that start with the same characters but
have a different length, the shorter one sorts alphabetically
first, and we reverse that. But it is not safe if the shorter
substitution doesn't start with the same characters, as in the
case I put in the tests: we should sort 'zzzaaaaaazzz' before
'aaaaaa' (and hence apply the 'zzzaaaaaazzz' substitution to a
volume ID that contains that string and not the 'aaaaaa' one),
but the previous commit did not.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-02-23 14:48:04 -08:00
Ondrej Nosek
caed78e11a Ordering processing for volume ID substitutions
Related: https://pagure.io/pungi/issue/840

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-02-22 16:46:58 +01:00
Jan Kaluza
7ee920a085 Disable multilib for modules
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-22 11:54:02 +01:00
Lubomír Sedlář
a49704b2b8 scm: Stop decoding output of post-clone command
There is no guarantee that it will print any text. We don't even need
the output, we just print it to error log if there is a problem.

Fixes: https://pagure.io/pungi/issue/847
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-20 12:18:48 +01:00
Lubomír Sedlář
e557dfd61a Remove useless shebang
The module is not executable. This will remove a warning from rpmlint at
RPM build time.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-20 10:13:09 +01:00
Owen W. Taylor
303fb29a6c source_koji.py: Properly handle unset pkgset_koji_tag
In one place, there was an explicit check if pkgset_koji_tag was set,
in another it was blindly referenced and assumed to be a list (with
accidental semi-success for a scalar.)

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2018-02-15 15:55:13 -05:00
Lubomír Sedlář
cc8c7a702c pkgset: Only use package whitelist if enabled
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-14 10:33:36 +01:00
Jan Kaluza
ef058d1f9b Fail early if input packages are unsigned
Use 'get_packages_to_gather' to fail early if these packages are not
signed with right key. This prevents us from having to wait for the
repo to be created and depsolving to finish. Unsigned dependencies will
still be reported later than previously.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-13 12:35:55 +01:00
Jan Kaluza
c75f4a1e96 Allow composing from tag with unsigned packages
There can be packages in the tag that will not end up in the compose.
Instead of failing immediately with error, this patch delays the check
until after depsolving finishes and only checks packages that will
really be included.

This is not an issue for nodeps compose, as that already pulls in only
packages that will be composed and nothing else.

Merges: https://pagure.io/pungi/pull-request/843
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-02-13 12:35:17 +01:00
Ondrej Nosek
c7cc200246 Ostree can use pkgset repos
Related: https://pagure.io/pungi/issue/778

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-02-08 15:23:15 +01:00
Lubomír Sedlář
364d7f5229 Support multiple sources in one variant
With this patch the gather_source option is no longer used. Instead, all
sources are always used. If they return at least some input packages,
then a configured method is used and the returned lists of packages from
all sources are merged.

The method used for gathering can be configured for each variant and
gather source separately.

Additional packages are only added to the comps source.

Each gathering step is logged separately. All the logs are preserved for
later inspection.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-02-01 14:31:09 +01:00
Lubomír Sedlář
0074fe3f2c gather: Set lookaside flag focorrectly
For sources and debuginfo package the flag should be set based on the
repo we pull that particular RPM from, not for the corresponding binary
package.

There could be a case where they come from different repos, so this
would cause problems.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-25 15:54:00 +01:00
Lubomír Sedlář
e8fa2a13b1 gather: Try getting srpm from the same repo as rpm
When the same package is contained in the main repo for depsolving as
well as in a lookaside repo, there was a chance that we would pick a
package from main repo, but corresponding source from lookaside.
This would cause a crash when trying to link the packages into the
compose.

A solution is to try to get source rpm from the same repo as the binary
package. Only when it's not available there we get some other one.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-25 13:52:55 +01:00
Ondrej Nosek
2ae056e021 Minor correction for python backward compatibility
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-25 13:28:08 +01:00
Ondrej Nosek
f1f6ca74f1 4.1.22 release
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-24 10:11:30 +01:00
Ondrej Nosek
fbeb14344f Better INFO messages about modules
Relates: https://pagure.io/pungi/issue/779

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-22 17:23:31 +01:00
Lubomír Sedlář
778dbaef73 Updates composes should be marked as supported
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-19 13:13:42 +01:00
Lubomír Sedlář
121ffb403f pkgset: Only add missing packages from global tag
Background story: if a compose is combining modular and traditional
compose, the configuration will contain multiple Koji tags to build
package set from (one tag for each module, plus at least one tag for the
traditional content). However some packages might be present in multiple
tags, and if the package set contains both, there's no way to control
which one will end up in the compose.

The solution for this is to give preference to the modular compose. If a
package with the same name exists in multiple tags, we only take the
first one we find. This relies on ordering of collected tags: modular
ones are always first, and traditional tags are at the end of the list.

If there are multiple modules that contain the same package, only one of
them will be used, which is not correct. Allegedly this should not
happen. In any case such use case does not work without this patch
either, so we're not losing anything.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 16:57:22 +01:00
Ondrej Nosek
f301158974 ostree/utils: Drop timestamps from generated repo names - tests
Issue: https://pagure.io/pungi/issue/811

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-17 16:48:22 +01:00
Colin Walters
3427d6abe2 ostree/utils: Generate a single pungi.repo file, use repo-<num> IDs
Ideally, pungi would generate repository IDs like `fedora-updates`
or so, and we'd have versioning inside the rpm-md.  But for now
let's do this to avoid invalidating rpm-ostree's change detection.

Closes: https://pagure.io/pungi/issue/811
Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-17 16:48:22 +01:00
Colin Walters
b2bdc8a608 ostree/utils: Drop timestamps from generated repo names
Since we drop these files in a separate workdir each time,
there's no need to datestamp them.  Doing so is part of the
cause for invalidating's rpm-ostree input change hashing.

Issue: https://pagure.io/pungi/issue/811
Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-17 16:48:22 +01:00
Lubomír Sedlář
5cc612f966 gather: Do not require variant for module source
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:06:56 +01:00
Lubomír Sedlář
df27164c1c gather: Comps source should not crash without comps file
There just isn't anything to return.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
670a68a5b8 gather: JSON source returns nothing without configuration
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 14:02:46 +01:00
Lubomír Sedlář
61a3be2307 buildinstall: Fix treeinfo generating on failure
When buildinstall fails, there will be no lorax generated files copied
into the compose directory. However they may still be mentioned in the
.treefile in work/ subdirectory where lorax runs.

To avoid possible issues, we should use the lorax created .treeinfo only
if the run was successful.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-17 13:57:49 +01:00
Jan Kaluza
930c2f1a42 Add buildinstall_use_guestmount boolean option
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2018-01-16 09:18:35 +01:00
Lubomír Sedlář
68a1370036 gather: Use arch packages in nodeps method
We already filtered a list of packages compatible with the architecture.
There's no need to do that again (with bugs). Instead of using the
global package set we should just restrict the code to an arch package
set.

This should not break anything. If a package is included such that it's
in global but not arch package set, the compose would crash as linking
packages takes paths from the arch package set.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-15 09:55:51 +01:00
Lubomír Sedlář
f4c3d2423d pkgset: Always use global tag if specified
Use global koji tag for populating package set even for modular
composes. To preserve backwards compatibility value "not-used" is
ignored. This should however be fixed in the configuration to simply not
specify the option at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-12 15:50:21 +01:00
Lubomír Sedlář
03293c725b config: Make pkgset_koji_tag optional
There are valid use cases for not specifying this option: specifically a
modular compose will get the tags to use from modules listed in the
variants file.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-12 15:50:21 +01:00
Ondrej Nosek
51cd359057 ostree: Add force_new_commit option - test added
Relates: https://pagure.io/pungi/issue/811

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-11 15:08:07 +01:00
Colin Walters
161b7f974b ostree: Add force_new_commit option
Followup from discussion in: https://pagure.io/pungi/issue/811

It's likely now that for Fedora Atomic Host we'll use this, to work
around other issues, after we fix the FAW change detection.

Signed-off-by: Colin Walters <walters@verbum.org>
2018-01-11 15:03:36 +01:00
Lubomír Sedlář
5cc54cd587 gather: Fix checking string type
There is no unicode on Python 3, we should use six to hide this detail.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-01-10 13:50:21 +01:00
Ondrej Nosek
d4d264eb33 Improve logging for unsigned packages
Relates: https://pagure.io/pungi/issue/820

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-03 11:08:00 +01:00
Ondrej Nosek
2152e7ea26 Fall back to mount if guestmount is not available
Relates: https://pagure.io/pungi/issue/803

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-02 17:13:47 +01:00
Ondrej Nosek
116e7ca3bd El-Torito boot information on s390x
Relates: https://github.com/rhinstaller/lorax/pull/236

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-14 10:08:30 +01:00
Ondrej Nosek
59c162d46f Remove strace from buildinstall runroot
The package is not really needed.

Relates: https://pagure.io/pungi/issue/799

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-11 17:36:28 +01:00
Ondrej Nosek
ba3adf9bc4 Merge #818 doc: move "Phases" up, "Contributing" down 2017-12-11 08:54:32 +00:00
Ken Dreyer
6c3245bdee doc: fix "Miscellaneous" spelling in Config section
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-12-08 13:46:14 -07:00
Ken Dreyer
d66abc3aea doc: move "Phases" up, "Contributing" down
The "Phases" section breaks down what Pungi does in detail. Place it
towards the top of the documentation table of contents.

The "Contributing" and "Testing" sections are relevant to developers,
not all users, so move them to the end.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-12-08 13:43:39 -07:00
Lubomír Sedlář
25907f61ce 4.1.21 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:43:10 +01:00
Lubomír Sedlář
e15a49defd tests: Use correct python version for config validation test
When tests are running using Py3, the script should be called with Py3
as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
d5a0316877 Use dnf backend for repoclosure on PY3
When yum is not available, it makes no sense to offer it as an option.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
60cc3e5d55 Drop checks for git and cvs
This essentially forces everyone to install cvs even though they most
likely don't need it. There is no easy way to check what is actually
going to be needed, so let's just put the responsibility of decoding a
potential error message to users.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
fa4d728230 Relax check for gettext
It's only used in productimg phase. We don't have to check for it on
every run.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
aa9c137412 Drop check for repoquery command
There is no place that would possibly call it, so we can drop the check
and function for generating cli.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00
Lubomír Sedlář
001ae1d7b4 Use modifyrepo_c if possible
When configured to use createrepo_c, we should also use modifyrepo_c.
That allows us to relax the check for createrepo package.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-12-06 13:26:43 +01:00