Commit Graph

1359 Commits

Author SHA1 Message Date
Lubomír Sedlář
deb8623bd1 pungi: Include noarch debuginfo
The check for arch compatibility should not be performed if the
debuginfo package is noarch. Such packages should be included
unconditionally.

Fixes: #450
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-16 14:50:03 +01:00
Lubomír Sedlář
b7813d34ac pungi: Fix incorrectly skipped tests
The test was not correctly being skipped in Koji build, causing build
failure.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-16 09:43:19 +01:00
Lubomír Sedlář
3c1e2fba66 pungi: Fix tests on non-x86_64 arches
If the test did not specify an arch explicitly, it fell back on whatever
arch the current machine has. This was causing failures when building
RPM.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-16 09:20:10 +01:00
Lubomír Sedlář
b85d7ae787 Include test data in tarball
The repodata for pungi tests should be shipped in the tarball.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-16 08:53:10 +01:00
Dennis Gilmore
13c1c4da69 4.1.11 release
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-11-15 15:12:02 -06:00
Dennis Gilmore
69d949aaf9 Merge #463 [ostree] Allow extra repos to get packages for composing OSTree repository 2016-11-15 21:06:05 +00:00
Dennis Gilmore
3953914bd1 Merge #461 extra-files: Nice error message on missing RPM 2016-11-15 21:01:42 +00:00
Dennis Gilmore
d4a78a1553 Merge #459 Fix building images and live media for addons 2016-11-15 21:00:50 +00:00
Dennis Gilmore
5dc8132fa4 Merge #456 Add tests for dependency solving 2016-11-15 20:58:40 +00:00
Qixiang Wan
68e121e421 [ostree] Allow extra repos to get packages for composing OSTree repository
Sometimes addtional repos are required to get necessary packages for
composing OSTree repository. For example, RHEL doesn't have an 'Everyting'
variant, so composing OSTree repository from any of the RHEL variants
won't work, addtional source repos need to be enabled to achieve that.
The new option "extra_source_repos" enable the ability of allowing extra
source repos.

And a new option 'keep_original_sources' is introduced to keep the
original repos found in tree config file, if this is enabled, Pungi
will not remove the existing source repos from the tree config file,
just add new repos of "source_repo_from" + "extra_source_repos" to
the existing repos.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-15 18:56:06 +08:00
Lubomír Sedlář
0cdef2d8e3 pungi: Run in-process for testing
Instead of spawning a separate process for each test, move the code to
run in the main process. This gives us correct coverage information and
makes the tests a lot faster.

The input is still provided in a kickstart file. The output also goes
into a log file that is later parsed.

There is some extra work needed with logging setup to make the logs go
to standard output so that nose can correctly capture them.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 13:28:02 +01:00
Lubomír Sedlář
ea1bcf625b pungi: Only add logger once
The Pungi object reuses the same logger for all its instances. Every
time a new instance is created, a stream handler would be added. This
means that a second instance will print everything twice. When running
tests, tens of instances would be created.

This patch only adds the handler if there are no handlers configured
yet.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 13:28:02 +01:00
Lubomír Sedlář
4a7257f550 pungi: Connect yum callback to logger
This patch changes the format a little and makes sure the output goes to
the same logger as the rest of the output. The output no longer contains
control characters.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 13:27:58 +01:00
Lubomír Sedlář
88dbf8f849 extra-files: Nice error message on missing RPM
When a file is exported from an RPM in the compose, and there is no
matching RPM in the package set, we want a nice error message.

Fixes: #460
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 09:38:50 +01:00
Lubomír Sedlář
13871b64fb compose: Drop unused argument
The `get_variants()` method had a `recursive` argument with default
value of `False. However, this argument had no effect and the method
always returned all variants recursively.

We can just drop the argument. All callers are updated to not supply the
argument. Should any need for getting the top-level variants only arise,
they can be accessed as the `variants` attribute directly on the Compose
object.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 13:39:01 +01:00
Lubomír Sedlář
bd00920c62 compose: Search all nested variants
The code to search for install tree and repo for image-build and
live-media was only looking at top-level variants. Addons, optional or
integrated layered products could not have been found. This would lead
to error messages such as "There is no variant Server-optional to get
repo from when building live image for Client" even though the variant
exists.

Various tests are updated to exercise this edge case.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 13:18:34 +01:00
Lubomír Sedlář
99ba369901 ostree-installer: Capture all lorax logs
Fixes: #457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:37:00 +01:00
Lubomír Sedlář
9a72ea8f6d lorax-wrapper: Put all log files into compose logs
Tell lorax to use a specific directory for log files so that we preserve
them despite koji not having any idea about them.

Fixes: #457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:37:00 +01:00
Lubomír Sedlář
079cc46458 pungi: Fix reading multilib config files
When running from git, the files should be found relative to the python
module, not executable itself. This change makes it possible to load the
files when running tests.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:36:23 +01:00
Lubomír Sedlář
adaed6345c pungi: Fulltree should not apply for input multilib package
When the input explicitly lists a package as multilib, we should not
automatically add native version just because of fulltree.

The tests for this use case are now enabled and passing.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:36:23 +01:00
Lubomír Sedlář
8e5b197e3f pungi: Add tests for depsolving
These are tests written by Daniel Mach originally for Distill-NG. They
are ported to current Pungi. The test repositories are committed in the
tests/fixtures/ directory. This is the same data that is used for test
compose, but the actual RPM files are not present. Some tests are
adapted from dmach's fork of Pungi.

Some of the packages are marked with a comment saying they are
important. These are the packages that the test is specifically trying
to get included in the package set. There are also explicit tests for
packages that should not be included.

Two tests are skipped for now as there is a bug preventing them from
passing. This is related to fulltree being done for packages that are
explicitly multilib.

The depsolver is called by invoking a separate executable, so the
coverage data is wrong.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:36:23 +01:00
Lubomír Sedlář
4427769f6a Update ostree phase documentation
The message and the new option were not documented.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-04 09:35:08 +01:00
Qixiang Wan
fa3c5aff63 [ostree] Allow adding versioning metadata
Added new option '--version' to pungi-make-ostree, and this can be
enabled in ostree settings with 'version'. The version string will be
added as versioning metadata if this is specified.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-04 16:07:26 +08:00
Lubomír Sedlář
133b6a5cf5 Merge #453 [ostree] New option to enable generating ostree summary file 2016-11-04 07:08:09 +00:00
Qixiang Wan
06ba3b8551 [ostree] New option to enable generating ostree summary file
Added new option '--update-summary' to pungi-make-ostree, and this can
be enabled in ostree settings with 'update_summary'. A summary file will
be generated (or re-generated if it was presented in an existing ostree
repo) when it is enabled.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-03 21:02:05 +08:00
Lubomír Sedlář
a3a415e398 pungi: Avoid removing from list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
48bd3e6d2d pungi: Allow globs in %multilib-whitelist
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
2cd2e0e15f pungi: Exclude RPMs that are in lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
3e7f2dfa9d pungi: Fix excluding SRPMs
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
e6342ede18 pungi: Speed up blacklist processing
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Patrick Uiterwijk
a49510a9fb Update tests to use ostree write-commit-id
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-26 09:52:12 +00:00
Patrick Uiterwijk
754458823c ostree: Use the write-commitid-to feature rather than parsing ostree logs
This depends on an rpm-ostree release with
98332a3be4

Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-26 09:52:12 +00:00
Lubomír Sedlář
c5bd99da9e Merge #448 Remove executable permissions on test scripts 2016-10-26 07:31:37 +00:00
Lubomír Sedlář
bfd5a39ce6 checks: Check for createrepo_c
The createrepo package is needed always, but depending on configuration
we should also look for createrepo_c.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-25 16:00:37 +02:00
Lubomír Sedlář
f4cd25450b checks: Update tests to not require python modules
This way the tests are more independent of the system they are running
on.

Merges: #447
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-25 15:57:02 +02:00
Patrick Uiterwijk
59dbd9d35f Remove executable permissions on test scripts
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-25 10:42:16 +00:00
Patrick Uiterwijk
933c2608cd Add more require checks
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-25 10:37:55 +00:00
Patrick Uiterwijk
51a8f7a7fe Fix package name for createrepo and mergerepo
These binaries are in the createrepo package, not yum-utils.

Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-25 10:29:06 +00:00
Qixiang Wan
d067928a4e not using 'git -C path' which is not supported by git 1.x
'-C path' is new in git 2.x and RHEL 6 has git 1.8 by default

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-10-25 15:53:56 +08:00
Qixiang Wan
1345eb87a4 pungi-koji: add option for not creating latest symbol link
Fixes: #53
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-10-25 12:45:19 +08:00
Lubomír Sedlář
e02ed6c04b Merge #434 Replace mount/umount cmds with libguestfs tools 2016-10-24 07:52:29 +00:00
Dennis Gilmore
18128623b4 Merge #443 config: Don't abort on deprecated options 2016-10-20 20:26:17 +00:00
Dennis Gilmore
8f974ea109 Merge #442 metadata: Treeinfo should point to packages and repo 2016-10-20 20:25:17 +00:00
Qixiang Wan
a5273dc798 Replace mount/umount with guestfsmount and 'fusermount -u'
'mount -o loop' requires root privileges, guestmount from
libguestfs-tools-c package can work without root privileges.

Fixes: #19
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-10-20 19:38:35 +08:00
Lubomír Sedlář
c38bb4809b config: Don't abort on deprecated options
These options are in fact removed and have no effect anymore. This patch
changes the validation to print a warning that the option was removed
and what should be done instead. It no longer stops the whole compose.

The validation script still rejects configuration files with these
removed keys.

This change means we no longer check these removals with the JSON schema
(as that makes it hard to determine where exactly the problem is).

Fixes: #438
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 12:51:12 +02:00
Lubomír Sedlář
1bb1c2ba28 metadata: Treeinfo should point to packages and repo
Each variant section should contain a path to a directory with packages
and to repodata. There was code to do this, but due to a typo it did not
actually work.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 12:05:48 +02:00
Lubomír Sedlář
74aa41f8bd Send notification when compose fails to start
This is tricky as this early in the process we don't know the compose
ID. The new message gives the full command line that was called.

Relates: #439
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 09:39:18 +02:00
Lubomír Sedlář
6e55cc6419 metadata: Stop crashing for non-bootable products
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 09:25:39 +02:00
Dennis Gilmore
a7dc4e80a7 Merge #433 createiso: Do not split bootable media 2016-10-19 16:37:25 +00:00
Dennis Gilmore
2f184f9b23 Merge #432 metadata: Correctly clone buildinstall .treeinfo 2016-10-19 16:36:45 +00:00