Commit Graph

1759 Commits

Author SHA1 Message Date
Lubomír Sedlář cdfa3cb45f image-build: Drop suffixes from configuration
The file extension in configuration is only used to tell Pungi which
files from the task results should be downloaded. The user has to get it
right or the phase will fail. Each format has a single valid suffix.

Pungi should not require users to specify the suffix, since it can just
as well just know the right value.

The old configuration will continue working, only the extension will be
ignored.

Fixes: https://pagure.io/pungi/issue/753
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-01 12:47:36 +01:00
Lubomír Sedlář 026ba10987 kojiwrapper: Deal with multiple values for image-build
When the config for image-build command contains multiple values, they
should be joined with commas into a single value.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-01 12:47:36 +01:00
Patrick Uiterwijk 333045fb87 Add modulemd to the missing module error
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2017-11-01 12:41:23 +01:00
Lubomír Sedlář 390dff52ae notification: Add more info into the messages
Fixes: https://pagure.io/pungi/issue/771
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-31 11:11:04 +01:00
Lubomír Sedlář 28c3bc6268 notification: Fix running on Python 3
The input is given as unicode value, not a bytestring. We need universal
newlines to handle the conversion as needed.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-31 11:10:27 +01:00
Ondrej Nosek e0308a74ff remove remaining hard coded createrepo threads
To be more precise, new createrepo parameter "workers" was customized and
new default value was set. This fixes issue #752.

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-10-30 10:37:10 +01:00
Lubomír Sedlář 6d6cf6e233 tests: Fix remaining missing assertions
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-30 09:29:28 +01:00
Lubomír Sedlář f21e3a2d6d tests: Work with older unittest2
RHEL has an older version of the library which does not backport all the
assertions that we used. In order for the tests to pass there we need to
use names that exist everywhere.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-30 09:16:18 +01:00
Lubomír Sedlář 6f21576921 tests: Skip testing pdc logs if dependencies are not installed
On EPEL 6 modulemd is not available, so the test fails with an import
error. We should just skip this particular tests, as the rest of the
functionality is still perfectly fine.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-30 08:21:21 +01:00
Dong Wang 463fb961bc Log PDC communications and info for modular composes
Fixes: https://pagure.io/pungi/issue/664
Merges: https://pagure.io/pungi/pull-request/723
Signed-off-by: Dong Wang <dowang@redhat.com>
2017-10-27 16:20:21 +02:00
Ondrej Nosek 4ff3190935 Update documentation section "Contributing to Pungi".
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-10-25 10:28:08 +02:00
Lubomír Sedlář daf162503c Reject yum gather backend on Python 3
It will not run, and having a nice error message is better than a
cryptic crash.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Lubomír Sedlář f3806f7c77 Stop using deprecated pipes.quote
Instead use the definition from python-six. Once we drop Py 2 support
completely, we'll just swap underscores with dots.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Lubomír Sedlář 797b13b34a Convert configparser values to string
On Python 3, configparser will reject non string values, and
theoretically we could have some in the configuration.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Lubomír Sedlář 2efc4d8561 Explicitly decode test files as UTF-8
This way the test does not fail if run in a non-UTF-8 locale.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Lubomír Sedlář ed9d7f69a6 Use universal_newlines when running other commands
This will automatically convert the output to unicode/str and we will
not have to worry about decoding ourselves.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Lubomír Sedlář ed22e07ef9 Port to Python 3
This should make all tests pass on both Python 2 and Python 3.

Unittest2 is required on Py 2.6 and Py 3.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Patrick Uiterwijk 3088df8e60 checks: Use list of release types from productmd
Let's not duplicate the list. Productmd exports a list of valid values,
so Pungi should just pick and use that.

Closes: https://pagure.io/pungi/pull-request/773
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:05:08 +02:00
Patrick Uiterwijk 274f8b4e56 Add an option to make pungi-koji print its compose_dir to stdout
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2017-10-24 03:58:36 +02:00
Lubomír Sedlář be39dc3caf buildinstall: Expose template arguments for lorax
This would be useful for modularity. The templates can be added now and
variables set via the existing `lorax_option`.

It's not possible to use custom templates not shipped with lorax, as
passing the path to a random directory is a little bit more tricky.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-23 16:08:09 +02:00
Jan Kaluza 894a5a11a4 Add support for new modules naming policy with colon delimiter
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-10-20 10:42:09 +02:00
Jan Kaluza 8951e90882 Catch the issue when PDC does not contain RPMs, but the module definition says there should be some.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-10-19 13:47:03 +02:00
Jan Kaluza 6208dae869 pkgset: Cherry-pick packages from Koji when we know already what packages will end up in compose
Merges: https://pagure.io/pungi/pull-request/763
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-10-19 13:43:59 +02:00
Lubomír Sedlář 2819311d6e config: Allow comps_file for any gather_source
It only affects gathering packages when gather_source is set to comps,
but it could still make sense to include the file in the repository even
if the not used for gathering.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-18 15:24:24 +02:00
Lubomír Sedlář 68fdef451c pkgset: Allow unsigned packages by empty key
Currently `None` has to be included to allow using unsigned packages.
ODCS has trouble with including non-string value in the list though, so
we can treat empty string the same way (it's not a valid key ID anyway).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-16 13:48:27 +02:00
Lubomír Sedlář b79ff7d8dd gather: Nodeps should allow noarch packages
If the package name matches, we should take the package either if
architecture matches or the package is noarch.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-16 13:45:35 +02:00
Lubomír Sedlář bb6c5da7af pkgset: Clean up path generation
The path prefix for packages is identical for all architectures. There's
no reason to compute it multiple times and check the directory existence
repeatedly.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-16 09:17:24 +02:00
Lubomír Sedlář f2a2a5a0b6 createiso: Fix logging for media split
When splitting is not possible, we should not print a huge number as
free space on single media. This patch also adds more information so
that the numbers can actually be connected to variant and architecture.
The list of ignored files will only be printed now if not empty.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-13 09:10:39 +02:00
Randy Barlow e117c904c7
Raise the Exception when a symlink cannot be created.
Signed-off-by: Randy Barlow <randy@electronsweatshop.com>
2017-10-12 16:39:41 -04:00
Lubomír Sedlář da590d559e Use variant UID for subvariant fallback
The name can contains spaces, which would potentially cause problems.
Also this is not consistent with how it works in other phases.

Relates: https://pagure.io/pungi-fedora/pull-request/354
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-11 08:59:13 +02:00
Lubomír Sedlář a3b90f7474 Fixup for opening config dumps
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-11 08:55:46 +02:00
Ralph Bean 97d52d03c8 Open and close file descriptors.
We noticed this when working on the Bodhi integration.

Signed-off-by: Ralph Bean <rbean@redhat.com>
2017-10-10 13:22:19 -04:00
Lubomír Sedlář c89f033457 live-images: Honor global settings for target
Instead of the old default value and custom handling introduce new
option `live_images_target` (for consistency with other phases) and use
the usual inheritance rules.

Fixes: https://pagure.io/pungi/issue/749
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-09 13:46:54 +02:00
Lubomír Sedlář 842e2e810e unified-isos: Stop erasing metadata on failure
When saving new metadata fails, the images.json file would be left
empty. Instead we should not touch it at all.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-06 08:37:56 +02:00
Dennis Gilmore b51a639277 Merge #743 `Make ostree installer before cloud images` 2017-10-06 00:18:46 +00:00
Lubomír Sedlář 9ab3840085 Add directory name for checksum file
Fixes: https://pagure.io/pungi/issue/745
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-05 16:08:27 +02:00
Lubomír Sedlář dec00fe2f4 createrepo: Allow customizing number of threads
The default is now to use one thread per CPU.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-03 15:21:37 +02:00
Lubomír Sedlář 19436c66b2 Make ostree installer before cloud images
Fixes: https://pagure.io/pungi/issue/742
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-20 08:33:40 +02:00
Lubomír Sedlář 44c523339c 4.1.19 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-18 13:17:18 +02:00
Lubomír Sedlář 04836cfa9f docs: Mention how input package list are interpreted
It's really just an RPM name, not any random provide or source package
name.

Fixes: https://pagure.io/pungi/issue/725
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-18 12:06:03 +02:00
Dong Wang adaab46bf7 Fix pungi-koji --version
Signed-off-by: dowang <dowang@redhat.com>
2017-09-18 17:04:05 +08:00
Lubomír Sedlář 3ef2a65275 profiler: Fix sorting on Python 3
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-13 09:55:51 +02:00
Lubomír Sedlář 9dbf231080 util: Fix timezone offset
We need to negate the value: the values are in seconds west of UTC, but
ISO 8601 wants the offset to be negative for times behind UTC (i.e. to
the west).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 09:41:53 +02:00
Lubomír Sedlář d34c0a2777 gather(dnf): Remove dead code
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář 239e6b4301 gather(dnf): Don't exclude packages from lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář eead6ccc44 gather(yum): Don't exclude packages from lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář ed0a8249b1 gather: Add tests for excluding packages from lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář 4b40a1258a gather: Capture broken deps in test
This will help test more aspects of depsolving.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář 816ab917da gather-dnf: Warn about unresolvable dependencies
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář 6a425ee891 Fix formatting timezone offset
Displaying the offset in seconds makes very little sense. We should
adhere to ISO 8601 format of `+HH:MM` which is much easier to
understand.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-11 10:24:33 +02:00