This patch does not do any actual pushing. It will only extract data
about push targets from the main configuration and store it together
with exact Koji NVR in a well-defined location, and also send the data
to message bus for another service to handle.
JIRA: COMPOSE-3228
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This makes it hard to programatically access the dump.
The timestamp was included originally to make sure that restarting a
compose in debug mode will preserve all configs. However we don't really
support that usecase anyway.
JIRA: COMPOSE-3063
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When multiple composes are chained, they should reuse the same event.
However it is tricky as the value would have to be passed by hand. This
patch makes it possible to read the value from another compose (the
first one in the chain).
JIRA: COMPOSE-2571
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It's a bit unfortunate we can't use the logger for printing these, but
there is cyclic dependency: to set up logger, we need Compose object (to
generate path to log file), but to create Compose object we need a valid
config.
JIRA: COMPOSE-2577
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It checks config file value 'skip_phases' for valid phases names.
Also checks command-line attribute 'skip-phase' of 'bin/pungi-koji'.
JIRA: COMPOSE-2493
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
If the ISO is meant to be bootable but lorax fails, there's no point in
creating the ISO as it will not behave as expected.
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1574585
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If the config is syntactically correct, but it contains some issues
discovered in `validate` method of any phase, we need to raise an
exception, not exit immediately.
JIRA: COMPOSE-2431
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When buildinstall fails, there will be no lorax generated files copied
into the compose directory. However they may still be mentioned in the
.treefile in work/ subdirectory where lorax runs.
To avoid possible issues, we should use the lorax created .treeinfo only
if the run was successful.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead use the definition from python-six. Once we drop Py 2 support
completely, we'll just swap underscores with dots.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should make all tests pass on both Python 2 and Python 3.
Unittest2 is required on Py 2.6 and Py 3.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Displaying the offset in seconds makes very little sense. We should
adhere to ISO 8601 format of `+HH:MM` which is much easier to
understand.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Timestamps in the main log are currently printed in local time. This
patch adds UTC offset to the top of the log so it's possible to find out
exactly when events happen.
Fixes: https://pagure.io/pungi/issue/710
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Save the config files which are imported by the following syntax:
from some_config import *
Note: This requires kobo >= 0.6
Signed-off-by: Qixiang Wan <qwan@redhat.com>
The config can change the default compose type. This can still be
overwritten by a CLI argument. A `--production` option is now added to
CLI (because that was the default before).
Fixes: https://pagure.io/pungi/issue/694
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
In once case we can completely drop try-except statements: it only
logs the exception details but that will be logged again anyway.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is updated in all files for consistency, even the modules that will
never be ported to Py 3 completely due to dependency on Yum.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The notification hooks can be useful for doing other things than just
announcing status on message bus. For this to be truly usable, we need
the ability to use multiple scripts.
This patch allows the command line option to be specified multiple
times. Each given script will be called. Even if the script fails, it
does not block the compose.
Additionally the output of the notification scripts is logged now to
make it possible to debug possible failure.
Relates: https://pagure.io/pungi/issue/650
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Optparse is deprecated since Python 2.7, long live argparse.
This also allows us to remove some of the manual error checking and make
the library check required or conflicting options.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
After the patch to ensure that all phases are stopped the livemedia
phase was apparently dropped by accident.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
If a phase is started successfully, it needs to be stopped as well. In
most cases when `stop` is called immediately after `start`, this is not
a problem.
Only when something else happens while a phase is runnning and this
something fails, Pungi will deadlock and never exit. This something
could be another phase or just main thread raising an exception.
Fixes: #625
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Add a new option 'latest-link-status' to pungi-koji, if this is
specified, pungi will only create the latest symbol link to the compose
when compose's status matches the specified statuses. The status name is
case insensitive. If the option is not specified it will act as before.
Example:
pungi-koji --target-dir=_composes --config=data/dummy-pungi.conf \
--test --latest-link-status=finished --latest-link-status=finished_incomplete
Signed-off-by: Qixiang Wan <qwan@redhat.com>
The name is not displayed anywhere. Changing it to lowercase will
however avoid possible problem if we create a sublogger somewhere else
in the codebase with name based on module name. Such logger would not
have any handlers defined.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Given the way config files can include other files, it is entirely
possible to use the same config file for both Pungi and Distill. Pungi
will however complain about unknown options.
This patch reverts part of c38bb480 and moves deprecation handling back
into schema validation. The validation method then returns a list of
errors and a list of warnings.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Phases createiso, liveimages, image_build, ostree_installer and osbs are
done in parallel, logs from these phases are mixed and and it's not
obvious which log message belongs to which phase. This change adds phase
name in log message for these phases.
The new mixin 'PhaseLoggerMixin' is added to extend a Pungi phase with a
logging logger which copy handlers from compose's logger but with
formatter changed.
Fixes: #58
Signed-off-by: Qixiang Wan <qwan@redhat.com>
The `get_variants()` method had a `recursive` argument with default
value of `False. However, this argument had no effect and the method
always returned all variants recursively.
We can just drop the argument. All callers are updated to not supply the
argument. Should any need for getting the top-level variants only arise,
they can be accessed as the `variants` attribute directly on the Compose
object.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
These options are in fact removed and have no effect anymore. This patch
changes the validation to print a warning that the option was removed
and what should be done instead. It no longer stops the whole compose.
The validation script still rejects configuration files with these
removed keys.
This change means we no longer check these removals with the JSON schema
(as that makes it hard to determine where exactly the problem is).
Fixes: #438
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is tricky as this early in the process we don't know the compose
ID. The new message gives the full command line that was called.
Relates: #439
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Once we have the checksums, nothing else will change in the metadata. We
should make sure it is on disk as soon as possible.
Fixes: #436
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
We need both full dump and the copy of original file. Having just the
copy is not sufficient because while it preserves comments and
formatting, there can be include statements for other files and those
are not copied.
With this patch we create the dump in the same way as before, and
additionally create a copy of of the config file specified on command
line.
Fixes: #398
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.
There is a custom extension to the schema to report deprecated options.
The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.
Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.
The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.
The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This should help with debugging by providing better information on which
Pungi version created the compose. In development, the version will show
output of git describe, in production it asks which version is installed
in site-packages/. The egg-info directory must be installed for this to
work.
It is no longer necessary to synchronize version in `setup.py` with
`pungi/__init__.py`.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It will take RPM repo from a variant in this compose and a Dockerfile
from configured git and use it to build an image.
The build images are uploaded to some a Docker registry by OSBS and are
not directly part of compose (because there is no export function).
There is a new metadata file `osbs.json` that has some information that
can be used to find the image.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
