Commit Graph

2219 Commits

Author SHA1 Message Date
Lubomír Sedlář
55035487de image-build: Correctly write can_fail option
Koji expects the value as comma separated list, otherwise it will ignore
it. This makes it possible for the image building to fail on per arch
basis.

Relates: #128
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-20 09:05:03 +01:00
Qixiang Wan
79e97dc845 pungi-koji: new cmd option '--latest-link-status'
Add a new option 'latest-link-status' to pungi-koji, if this is
specified, pungi will only create the latest symbol link to the compose
when compose's status matches the specified statuses. The status name is
case insensitive. If the option is not specified it will act as before.

Example:

pungi-koji --target-dir=_composes --config=data/dummy-pungi.conf \
--test --latest-link-status=finished --latest-link-status=finished_incomplete

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-20 11:15:30 +08:00
Lubomír Sedlář
23e53219eb live-cd: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
69af0a6a65 ostree-installer: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
263a990489 ostree: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
5ee285cc24 live-media: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
b10d275745 image-build: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Dennis Gilmore
b09641a708 Merge #568 buildinstall: Print debug info if unmount fails 2017-03-15 16:07:33 +00:00
Lubomír Sedlář
94b1159829 ostree-installer: Fix logging directory
Currently if there are multiple ostree installers for the same
architecture, the logfiles all end up in the same location and overwrite
each other.

This patch moves the logs to logs/<arch>/<variant>/ostree_installer-X/
for a unique value of X so that there can be multiple runs even for the
same tree.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 15:54:45 +01:00
Lubomír Sedlář
f0dc15bdc6 buildinstall: Print debug info if unmount fails
When even after retries the unmounting still fails, this patch runs `ls
-lA`, `fuser -vm` and `lsof +D` on the directory to give some idea of
what's blocking it.

Relates: #559
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-14 15:06:20 +01:00
Qixiang Wan
d081a4eda8 pkgset: report all unsigned packages
Fixes: #552
Merges: #567
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-14 09:50:28 +01:00
Qixiang Wan
ee21663c8b default createrepo_checksum to sha256
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-14 13:15:43 +08:00
Lubomír Sedlář
a1214525f5 unified-iso: Log better error when linking fails
When linking files fails due to target path already existing, we should
print details about the conflict.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-13 14:45:13 +01:00
Lubomír Sedlář
08b9d275a8 unified-iso: Blacklist extra files metadata
The file is in each variant that has extra files, and would create a
conflict when linking the files.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-13 14:45:13 +01:00
Lubomír Sedlář
5cd1c22e2d buildinstall: Retry unmounting image
If the image can not be unmounted because the device is busy, we should
retry. There will be increasing pauses between the attempts. At most 10
attempts will be done before giving up.

Fixes: #559
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-10 08:29:54 +01:00
Lubomír Sedlář
9041ccccc4 Remove indices from documentation
They are not generated as we don't actually document code. The search
page is removed as well (because it does not seem to work).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Fixes: #462
2017-03-09 12:47:12 +01:00
Lubomír Sedlář
cd805a1e6d iso-wrapper: Handle wrong implant md5
If the checkisomd5 command exits successfully but returns a wrong value,
we should catch and log that. In theory this should be impossible, but
we have seen it in production.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-07 09:58:14 +01:00
Lubomír Sedlář
1647f7612a image-build: Remove check for number of images
If task does not produce the expected number of images, it should fail
at Koji level. Also, this check does not really work when some
architectures are allowed to fail.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-06 19:31:08 +01:00
Lubomír Sedlář
63a3b6ccaa Merge #538 consolidate repo option names 2017-03-06 12:55:44 +00:00
Lubomír Sedlář
fa3d94bae3 Merge #554 osbs: write manifest for scratch osbs 2017-03-06 12:52:47 +00:00
Lubomír Sedlář
d3a2ceb8ce Extract only first version from specfile
When tagging a new version, rpm will give us the version twice, one for
main package and one for the -utils subpackage. We should only use the
first one (they should be the same anyway).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-06 09:08:13 +01:00
Qixiang Wan
2ae8710934 consolidate repo option names
live_images:            additional_repos        -> repo
ostree:                 source_repo_from        -> repo_from
                        extra_source_repos      -> repo
ostree_installer:       source_repo_from        -> repo_from

With the change, the phases have consolidate option names for variant
repos and external repos.

Old option names will continue to work, old names will be converted
to new names after validation automatically if new options are not
specified in config.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-06 09:26:42 +08:00
Qixiang Wan
c93207addb checks: extend validator with 'alias'
When a property has 'alias' defined, and it's not present in instance,
if the alias property is present, add the property with value from alias
property before remove the alias property from instance.

Examples:

with schema:
{
        "$schema": "http://json-schema.org/draft-04/schema#",
        "title": "Pungi Configuration",
        "type": "object",
        "properties": {
            "release_name": {"type": "string", "alias": "product_name"},
        },
        "required": ["release_name"],
        "additionalProperties": False,
}

1. config = {"release_name": "dummy product"}:

   validate pass, config not changed after validation.

2. config = {"product_name": "dummy product"}:

   validate pass, config updated to the following after validation:

   config: {"release_name": "dummy product"}

3. config = {"name": "dummy product"}:

   validate fail, errror message is "Failed validation in : 'release_name' is a required property",
   and warning message is "WARNING: Unrecognized config option: name."

4. config = {"product_name": "dummy product", "release_name": "dummy product"}

   validate fail, error message is "Failed validation in : product_name is an alias of release_name, only one can be used."

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-06 09:26:42 +08:00
Qixiang Wan
40e2874676 osbs: write manifest for scratch osbs
FIXES: #485

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-06 09:26:01 +08:00
Lubomír Sedlář
345308464f 4.1.13 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-03 14:34:45 +01:00
Lubomír Sedlář
069ee8d3df Merge #555 Make MANIFEST.in stricter 2017-03-03 11:56:41 +00:00
Qixiang Wan
a52561da6d Make MANIFEST.in stricter
Global exclude python bytecode files.
Global exclude vim & emace temporary files.
Only include specs dir under tests/data, there will be other dirs
generated for testing, we don't want them.

FIXES: #427

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-03 14:01:50 +08:00
Qixiang Wan
f43d3584c5 Remove one line of log print
The line of log printing can result in "No handlers could be found for
logger" warning in quiet mode when custom paths module is enabled,
because file handler is setup after that, since this line is only
printed to screen (when pungi-koji is not ran in quiet mode) and doesn't
contain important info, it doesn't hurt to remove it.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-02 11:46:33 +08:00
Lubomír Sedlář
a49bf3d74e gather: Filter comps group on depsolving input of optional
This is a follow up of 8bc65a8be5. When
resolving packages that go into optional variant, we only want to skip
filtering groups when working on a top-level variant.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-27 12:54:12 +01:00
Dennis Gilmore
8b1fb287d3 Merge #539 comps: Filter comps groups for optional variants 2017-02-23 20:13:39 +00:00
Lubomír Sedlář
4ea1916a87 Enable customizing runroot task weight
For different cases where runroot is used it's now possible to set
custom weight. The usecase for this is to avoid one builder taking too
many tasks. Especially buildinstall is quite resource intensive, so one
builder taking multiple tasks at the same time leads to very slow
compose time.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-23 12:44:27 +01:00
Lubomír Sedlář
8bc65a8be5 comps: Filter comps groups for optional variants
The optional variant can defined by just adding has_optional=True into
variant xml. In such case it has no comps groups and Pungi would copy
the original file unmodified. This leads to extra packages being pulled
into the optional variant.

In this case the correct solution is to filter the comps and remove all
groups.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-23 12:44:20 +01:00
Lubomír Sedlář
d9ab899920 Rename main logger
The name is not displayed anywhere. Changing it to lowercase will
however avoid possible problem if we create a sublogger somewhere else
in the codebase with name based on module name. Such logger would not
have any handlers defined.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-23 11:08:31 +01:00
Lubomír Sedlář
9e020c2782 ostree: Silence logger in tests
Instead of creating and configuring the logger at module import time, we
can only get the logger if it's actually needed and configure it from
the main script.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-23 11:08:31 +01:00
Lubomír Sedlář
3162fea60d ostree: Fix crash when extra repos are missing
The default needs to be an empty list, not `None`.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-22 15:48:06 +01:00
Lubomír Sedlář
857aee05c1 util: Add a utility for managing temporary files
In multiple situations we need to create temporary files or directories
that should not be preserved after compose is finished. Let's add
context managers that ensure these get cleaned up.

This fixes tests leaving garbage around in /tmp.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-20 09:14:25 +01:00
Qixiang Wan
a57bc13e30 Add --quiet option to pungi-koji
log will not be printed to screen when quiet mode is enabled.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-02-20 15:00:03 +08:00
Qixiang Wan
d496eeb090 handle opening empty images.json while re-running pungi-koji in debug mode
If there is an empty images.json file exists, re-running pungi-koji
in debug-mode will fail with exception. We can just ignore the exception
in such cases, pungi-koji will create a correct images.json later.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-02-16 17:00:14 +08:00
Qixiang Wan
8d41a004c3 minor change: remove an always true condition
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-02-14 16:36:47 +08:00
Daniel Mach
ad23efd323 Refactor depsolving tests
This adds a new test for Requires(pre) and (post). The general structure
of the test now makes it easy to use the same test scenarios for
different backend.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-13 12:32:07 +01:00
Lubomír Sedlář
e00776a413 multilib: Remove FileMultilibMethod class
In order to do something meaningful, the class needs to be instantiated
with arguments pointing the blacklist and whitelist.

The `file` multilib method used via `pungi-koji` or `pungi` directly has
no way to pass those in.

The only way this class can be useful would be if someone actually
imported the class directly in their own code. Pungi is not meant to be
used as a library though, so this is not really a supported use case.

Not to mention that the `select` method always returned `False`.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-10 12:32:45 +01:00
Lubomír Sedlář
c3cf09a2f7 pkgset: Use additional packages for initial pull
We call pungi to get the packages from external repos if
pkgset_source=repos. In this case we need to look at additional packages
as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Fixes: #528
2017-02-09 13:50:09 +01:00
Lubomír Sedlář
b5cfeaa6ca metadata: Fix .treeinfo paths for addons
Currently .treeinfo sections for addons are missing paths to packages
and repository.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-07 15:50:33 +01:00
Lubomír Sedlář
9533fca96c koji_wrapper: Always use --profile option with koji
This patch adds this option to all invoked koji commands.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-01-30 13:00:25 +01:00
Dennis Gilmore
42cdf047cc add missing koji_profile from test compose setting
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2017-01-30 05:06:46 -06:00
Dennis Gilmore
5a498f80b6 use koji --profile when calling koji for livemedia
take the profile from the koji config and apply it to the koji cli
when building livemedia

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2017-01-30 04:44:31 -06:00
Dennis Gilmore
c650f04d0b Revert "live-media: Call correct koji alias"
This was accidently merged and is not correct
This reverts commit 02beb35e67.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2017-01-30 04:44:17 -06:00
Lubomír Sedlář
dc7d3b36ab repoclosure: Don't run build deps check
Fixes: #521
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-01-30 10:16:03 +01:00
Dennis Gilmore
7feadb14ba Merge #518 Migrate test phase to dnf repoclosure 2017-01-28 16:49:10 +00:00
Lubomír Sedlář
02beb35e67 live-media: Call correct koji alias
We need call whatever command is configured in koji_profile. It's not
always `koji`.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-01-26 15:32:21 +01:00