Commit Graph

44 Commits

Author SHA1 Message Date
Patrick Uiterwijk
c9c5fcac94 Make sure ${basearch} is also replaced with config['ostree_ref']
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-01-09 21:39:33 +01:00
Lubomír Sedlář
5f6dcb37f6 ostree: Send correct ref in the message
If configuration requests a different ref than what treefile has, a
wrong message can be sent.

If the treefile is in JSON, the local copy will be updated in-place and
it works. However with YAML the updated version is still written as JSON
and thus we are still sending the original value.

Fixes: https://pagure.io/pungi/issue/1092
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-12-05 11:20:17 +01:00
Lubomír Sedlář
36c347eb79 ostree: Use --touch-if-changed
There are three different cases:

 * we expect commitid and it's there
 * we expect commitid and it's missing
 * we don't expect commitid

This patch helps differentiate between the second two. In former one we
should report an error and mark the phase as failed. The latter is
perfectly fine and no error should be reported

Fixes: https://pagure.io/pungi/issue/1046
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-09-19 13:07:03 +02:00
Lubomír Sedlář
58afece033 Only use comps repo if we really have comps
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-17 15:32:02 +02:00
Lubomír Sedlář
15ccd309fa Remove comps from arch repo
The comps could potentially be different in different variants, so
instead we can create the comps repo for every variant separately and
use two repos instead of one (packages in one repository, comps in
another one).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-04-12 13:52:36 +02:00
Ondrej Nosek
c7cc200246 Ostree can use pkgset repos
Related: https://pagure.io/pungi/issue/778

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-02-08 15:23:15 +01:00
Ondrej Nosek
51cd359057 ostree: Add force_new_commit option - test added
Relates: https://pagure.io/pungi/issue/811

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2018-01-11 15:08:07 +01:00
Ondrej Nosek
a6c65e026a add ability to specify ostree ref in OSTREE phase
It allows specify what ref we want this compose to commit to.
New parameter 'ostree_ref' overrides the default value from the treefile json.

Relates: https://pagure.io/pungi/issue/777

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-12-04 09:27:19 +01:00
Lubomír Sedlář
a63e4746c9 Report warning when config sections are not used
It's possible a variant is excluded via tree_variants option and the
section does not match anything. It can be confusing to users why
nothing is happening. This patch lets Pungi log all unmatched patterns.

Fixes: https://pagure.io/pungi/issue/692
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-21 09:07:10 +02:00
Lubomír Sedlář
804a0049f6 ostree: Simplify configuration
It makes no sense to repeat the same configuration for multiple
architectures. Instead we should just list the architectures as another
key in the mapping. There is an option to specify multiple config dicts.

This preserves full backwards compatibility, the old config format is
still accepted.

Fixes: https://pagure.io/pungi/issue/678
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-14 14:53:22 +02:00
Lubomír Sedlář
e246aef5f6 ostree: Add URL to repo to message
The message announcing new ostree commit contains hash of the commit,
the ref it's for, but there is no information about where the repo
actually is.

This patch adds `repo_path` key into the message with URL of the repo
and `local_repo_path` with path to the repo on local filesystem.

Relates: https://pagure.io/pungi/issue/650
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-11 10:24:49 +02:00
Lubomír Sedlář
98f40f6138 ostree: Autogenerate a version
If the value is a particular magic string, we will expand it to a proper
value.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-13 10:17:59 +02:00
Qixiang Wan
2f5d6d7dcd unify repo and repo_from options
Config option 'repo' and 'repo_from' are used in several phases, merge
them with one option 'repo'. 'append' in schema is used for appending
the values from deprecated options to 'repo', so it won't break on any
existing config files that have the old options of 'repo_from' and
'source_repo_from' (which is an alias of 'repo_from').

And 'repo' schema is updated to support repo dict as the value or an
item in the values, a repo dict is just a dict contains repo options,
'baseurl' is required in the dict, like:

{"baseurl": "http://example.com/url/to/repo"}

or:

{"baseurl": "Serer"}

currently this is used in ostree phase to support extra repo options
like:

{"baseurl": "Server", "exclude": "systemd-container"}

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-29 10:12:32 +08:00
Qixiang Wan
0f508e2228 move translate_path from paths.py to util.py
So translate_path can be used in util.py, or it will result in
circular import error.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-29 10:12:32 +08:00
Lubomír Sedlář
263a990489 ostree: Print task ID on success
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Qixiang Wan
2ae8710934 consolidate repo option names
live_images:            additional_repos        -> repo
ostree:                 source_repo_from        -> repo_from
                        extra_source_repos      -> repo
ostree_installer:       source_repo_from        -> repo_from

With the change, the phases have consolidate option names for variant
repos and external repos.

Old option names will continue to work, old names will be converted
to new names after validation automatically if new options are not
specified in config.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-06 09:26:42 +08:00
Lubomír Sedlář
4ea1916a87 Enable customizing runroot task weight
For different cases where runroot is used it's now possible to set
custom weight. The usecase for this is to avoid one builder taking too
many tasks. Especially buildinstall is quite resource intensive, so one
builder taking multiple tasks at the same time leads to very slow
compose time.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-02-23 12:44:27 +01:00
Qixiang Wan
a57bc13e30 Add --quiet option to pungi-koji
log will not be printed to screen when quiet mode is enabled.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-02-20 15:00:03 +08:00
Dennis Gilmore
59dd4dbcd8 replace ${basearch} when updating the ref
pungi-make-ostree has to run on the target arch so that rpm
scriptlets can be ran. as a reult we can ask rpm what the
basearch is for the running environment. For notifications
we have to pass in the arch we are running for.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2017-01-18 21:21:26 -06:00
Qixiang Wan
e043604822 [ostree] Add 'tree' sub-command to pungi-make-ostree script
Update pungi-make-ostree to supourt sub-command 'tree', which is just
as the original feature of pungi-make-ostree to compose OSTree tree.
With the change we can add other sub commands later to build other
OSTree artifacts, like the installer image.

Inaddtional to the change, now the the 'tree' command can accept an
optional '--extra-config' parameter to update the original tree
configuration with extra configurations specified in a json file
before composing the OSTree tree.

Example:

pungi-make-ostree tree --repo=/ostree --treefile=/path/to/treefile \
--log-dir=/path/to/log --extra-config=/path/to/extra-config.json

The extra-config file can contains the same configuration as OSTree
phase, the difference is it doesn't understand variant UID as source
repo since it's not ran in the chain of phases. A valid configuration
can be like:

{
    "source_repo_from": "http://example.com/repo/x86_64/Server",
    "extra_source_repos": [
        {
            "name": "optional",
            "baseurl": "http://example.com/repo/x86_64/optional",
            "exclude": "systemd-container",
            "gpgcheck": False
        },
        {
            "name": "extra",
            "baseurl": "http://example.com/repo/x86_64/extra",
        }
    ],
    "keep_original_sources": True
}

The OSTree phase is updated to move out the task of updating treefile,
instead of that, it writes the extra configurations to a json file,
then 'pungi-make-ostree tree' will take it by option '--extra-config'.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-12-12 19:30:35 +08:00
Dennis Gilmore
69d949aaf9 Merge #463 [ostree] Allow extra repos to get packages for composing OSTree repository 2016-11-15 21:06:05 +00:00
Qixiang Wan
68e121e421 [ostree] Allow extra repos to get packages for composing OSTree repository
Sometimes addtional repos are required to get necessary packages for
composing OSTree repository. For example, RHEL doesn't have an 'Everyting'
variant, so composing OSTree repository from any of the RHEL variants
won't work, addtional source repos need to be enabled to achieve that.
The new option "extra_source_repos" enable the ability of allowing extra
source repos.

And a new option 'keep_original_sources' is introduced to keep the
original repos found in tree config file, if this is enabled, Pungi
will not remove the existing source repos from the tree config file,
just add new repos of "source_repo_from" + "extra_source_repos" to
the existing repos.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-15 18:56:06 +08:00
Lubomír Sedlář
bd00920c62 compose: Search all nested variants
The code to search for install tree and repo for image-build and
live-media was only looking at top-level variants. Addons, optional or
integrated layered products could not have been found. This would lead
to error messages such as "There is no variant Server-optional to get
repo from when building live image for Client" even though the variant
exists.

Various tests are updated to exercise this edge case.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 13:18:34 +01:00
Qixiang Wan
fa3c5aff63 [ostree] Allow adding versioning metadata
Added new option '--version' to pungi-make-ostree, and this can be
enabled in ostree settings with 'version'. The version string will be
added as versioning metadata if this is specified.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-04 16:07:26 +08:00
Qixiang Wan
06ba3b8551 [ostree] New option to enable generating ostree summary file
Added new option '--update-summary' to pungi-make-ostree, and this can
be enabled in ostree settings with 'update_summary'. A summary file will
be generated (or re-generated if it was presented in an existing ostree
repo) when it is enabled.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-11-03 21:02:05 +08:00
Patrick Uiterwijk
754458823c ostree: Use the write-commitid-to feature rather than parsing ostree logs
This depends on an rpm-ostree release with
98332a3be4

Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-10-26 09:52:12 +00:00
Lubomír Sedlář
8880b1ea8a ostree: Use $basearch in repo file
Fixes: #413
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-05 18:41:55 +02:00
Dennis Gilmore
4e90f81fc1 use --new-chroot when making ostree's
to complement https://pagure.io/koji/pull-request/162 we need to adjust pungi
rpm-ostree uses bublewrap that does not work in mock. --new-chroot to mock
enables the use of systemd-nspawn instead of chroot resulting in working
rpm-ostree again

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-09-28 20:58:17 -05:00
Patrick Uiterwijk
2c401a4a5c Send fedmsg message on ostree compose finishg
This sends a message when we finish composing an ostree.

Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
2016-09-15 11:16:18 +00:00
Lubomír Sedlář
f9a6c8418f Add JSON Schema for configuration
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.

There is a custom extension to the schema to report deprecated options.

The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.

Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.

The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.

The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-01 10:56:15 +02:00
Lubomír Sedlář
463088d580 Refactor failables
This is a breaking change as big part of current failable_deliverables
options will be ignored.

There is no change for buildinstall and creatiso phase.

Failability for artifacts in other phases is now configured per
artifact. It already works correctly for ostree and ostree_installer
phases (even per-arch). For OSBS phase there is currently only a binary
switch as it does not handle multiple arches yet. When it gains that
support, the option should contain list of non-blocking architectures.

For live images, live media and image build phases each config block can
configure list of failable arches. If the list is not empty, it can
fail. Once we have a way to fail only some arches, the config will not
need to change.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-07-27 13:06:01 +02:00
Lubomír Sedlář
b634a18a7f Stop using str.format
On Python 2.6, it requires the format placeholder to have explicit index
of argument, so using % formatting is easier.

There are a couple places where the method is still used because the
same argument is used twice.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-25 13:41:49 +02:00
Lubomír Sedlář
9dce30a78d [ostree] Use unique work and log paths
Each variant.arch combination can have multiple ostree repos configured,
so we need to make sure the filesystem paths don't clash.

The paths used now are:

    logs/<arch>/<variant>/ostree-<x>/
    work/ostree-<x>/config_repo

The x stands for a number identifying the task. It has no relation to
actual contents of the repo.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-18 08:03:31 +02:00
Lubomír Sedlář
eeacb5b9f1 [ostree] Put variant name in ostree log dir
This will prevent log files from being mangled when multiple ostree
repos are created.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-16 18:29:09 +02:00
Lubomír Sedlář
12652ad1fa [ostree] Initialize empty repo
Make sure the directory exists before calling Koji (because otherwise
the mounting will fail). Update the runroot script to initialize the
repo when there are no files in the target destination.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-10 08:27:43 +02:00
Lubomír Sedlář
46ab1cb6f6 [ostree] Set each repo to point to current compose
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-15 08:17:40 +02:00
Lubomír Sedlář
e705cdd165 [ostree] Move cloning repo back to compose box
and pass absolute path to the config into runroot. This could avoid the
problem of not being able to clone the repo in runroot.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-07 15:28:15 +02:00
Lubomír Sedlář
b7948f2d65 [ostree] Mount ostree directory in koji
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-07 14:04:50 +02:00
Lubomír Sedlář
ed12a64a84 [ostree] Enable marking ostree phase as failable
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-06 16:12:29 +02:00
Lubomír Sedlář
5b1b6c1c4f [ostree] Use explicit work directory
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 12:46:59 +02:00
Lubomír Sedlář
96e7ddd3ea [ostree] Rename atomic to ostree
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 09:14:00 +02:00
Lubomír Sedlář
ad87ea2d28 [ostree] Move cloning config repo to chroot
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 09:13:55 +02:00
Lubomír Sedlář
e5f37016e0 Fix atomic/ostree config validations
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-01 07:42:49 +02:00
Lubomír Sedlář
536c6c85b7 [ostree] Add ostree phase
This phase runs the script to make ostree repository in koji runroot
task. It runs right after regular yum repos are created.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-23 10:58:35 +01:00