Commit Graph

39 Commits

Author SHA1 Message Date
Lubomír Sedlář
13871b64fb compose: Drop unused argument
The `get_variants()` method had a `recursive` argument with default
value of `False. However, this argument had no effect and the method
always returned all variants recursively.

We can just drop the argument. All callers are updated to not supply the
argument. Should any need for getting the top-level variants only arise,
they can be accessed as the `variants` attribute directly on the Compose
object.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 13:39:01 +01:00
Qixiang Wan
1345eb87a4 pungi-koji: add option for not creating latest symbol link
Fixes: #53
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2016-10-25 12:45:19 +08:00
Dennis Gilmore
18128623b4 Merge #443 config: Don't abort on deprecated options 2016-10-20 20:26:17 +00:00
Lubomír Sedlář
c38bb4809b config: Don't abort on deprecated options
These options are in fact removed and have no effect anymore. This patch
changes the validation to print a warning that the option was removed
and what should be done instead. It no longer stops the whole compose.

The validation script still rejects configuration files with these
removed keys.

This change means we no longer check these removals with the JSON schema
(as that makes it hard to determine where exactly the problem is).

Fixes: #438
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 12:51:12 +02:00
Lubomír Sedlář
74aa41f8bd Send notification when compose fails to start
This is tricky as this early in the process we don't know the compose
ID. The new message gives the full command line that was called.

Relates: #439
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-20 09:39:18 +02:00
Lubomír Sedlář
b5e4ea848a Dump images.json after checksumming
Once we have the checksums, nothing else will change in the metadata. We
should make sure it is on disk as soon as possible.

Fixes: #436
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-19 15:02:33 +02:00
Lubomír Sedlář
bfe1068b8e Recreate JSON dump of configuration
We need both full dump and the copy of original file. Having just the
copy is not sufficient because while it preserves comments and
formatting, there can be include statements for other files and those
are not copied.

With this patch we create the dump in the same way as before, and
additionally create a copy of of the config file specified on command
line.

Fixes: #398
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-21 08:31:43 +02:00
Chenxiong Qi
912f81ca16 Dump original pungi conf
Fix #383

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2016-09-19 17:31:43 +08:00
Lubomír Sedlář
f9a6c8418f Add JSON Schema for configuration
The schema is written in Python to reduce duplication. When
configuration is loaded, the validation checks if it's correct and fills
in default values.

There is a custom extension to the schema to report deprecated options.

The config dependencies are implemented as a separate pass. While it's
technically possible to express the dependencies in the schema itself,
the error messages are not very helpful and it makes the schema much
harder to read.

Phases no longer define `config_options`. New options should be added to
the schema. Since the default values are populated automatically during
validation, there is no need to duplicate them into the code.

The `pungi-config-validate` script is updated to use the schema and
report errors even for deeply nested fields.

The dependencies are updated: pungi now depends on `python-jsonschema`
(which is already available in Fedora).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-01 10:56:15 +02:00
Lubomír Sedlář
a72a38b278 Add full Pungi version to log output
This should help with debugging by providing better information on which
Pungi version created the compose. In development, the version will show
output of git describe, in production it asks which version is installed
in site-packages/. The egg-info directory must be installed for this to
work.

It is no longer necessary to synchronize version in `setup.py` with
`pungi/__init__.py`.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-08-16 08:12:40 +02:00
Lubomír Sedlář
b4fc97be03 [osbs] Add a phase to build images in OSBS
It will take RPM repo from a variant in this compose and a Dockerfile
from configured git and use it to build an image.

The build images are uploaded to some a Docker registry by OSBS and are
not directly part of compose (because there is no export function).
There is a new metadata file `osbs.json` that has some information that
can be used to find the image.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-22 10:02:03 +02:00
Lubomír Sedlář
95cfbfb3fe Setup global log file before logging anything
This should fix the issue with only printing information about automatic
toggling of `supported` flag to standard output and not to a file.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-14 08:48:36 +02:00
Dennis Gilmore
d13f655300 pungi-koji: fix up latest symlink creation
If release_version does not have a . in it the current logic gives us
and empty result. in those cases, such as 23 or Rawhide use the value
of release_version as is.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-26 09:46:44 +02:00
Lubomír Sedlář
4e3d87e658 [test] Add checks for created images
The performed checks:
 * If format is ISO, the file must have correct magic string
 * If it's bootable, there must be MBR or GPT

When a check fails on any failable deliverable, it will be logged and
the file removed from metadata (it will still remain on the disk). This
required a change to write the images.json file later (after test
phase).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-05 15:38:00 +02:00
Lubomír Sedlář
96e7ddd3ea [ostree] Rename atomic to ostree
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-05 09:14:00 +02:00
Lubomír Sedlář
490514e263 [checks] Add a check for too restrictive umask
If umask is set to something too high (>0022), a warning will be
printed. It does not abort the compose though.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-31 10:10:18 +02:00
Lubomír Sedlář
8d224b206b [atomic] Add atomic_installer phase
This phase runs lorax with extra templates in Koji runroot task, links
the boot.iso to proper location in compose directory and adds the
installer iso to image manifest. This phase runs concurrently with live
media etc.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-23 12:00:06 +01:00
Lubomír Sedlář
536c6c85b7 [ostree] Add ostree phase
This phase runs the script to make ostree repository in koji runroot
task. It runs right after regular yum repos are created.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-23 10:58:35 +01:00
Lubomír Sedlář
337d2a999c [checks] Don't always require jigdo
If the configuration specifically requests no jigdos, there is no point
in checking for the binary existence.

This is not 100% reliable. The jigdo option defaults to True, so if the
option is not specified the binary is required even if there are no
images configured.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-03-10 09:08:19 +01:00
Tomas Mlcoch
5bffca5037 Support signing of rpm wrapped live images
With this patch, you can specify a command for
signing of koji builds. For example:

    signing_key_password_file = '~/file_with_password_for_key_fedora-24'
    signing_key_id = '81b46521'
    signing_command = '~/git/releng/scripts/sigulsign_unsigned.py -vv --password=%(signing_key_password)s fedora-24'

'signing_key_password_file' is a path to a file which contains
a password that will be formatted into 'signing_command' string
via '%(signing_key_password)s' string format syntax (if used).
Because pungi config is usualy stored in git and part of compose
logs we don't want password to be included directly in the config.
Note: If '-' is used instead of a filename, then you will be asked
for the password interactivelly right after pungi starts.

'signing_key_id' is ID of the key that will be used for the signing.
This ID will be used when crafting koji paths to signed files
(kojipkgs.fedoraproject.org/packages/NAME/VER/REL/data/signed/KEYID/..).

'signing_command' a command that will be run with a build as a single
argument. This command mustn't require any user interaction.
If you need to pass a password for a signing key to the command,
do this via command line option of the command with use of string
formatting syntax '%(signing_key_password)s' (see details
about 'signing_key_password_file').

Signed-off-by: Tomáš Mlčoch <tmlcoch@redhat.com>
2016-02-18 13:36:38 +01:00
Lubomír Sedlář
454363fba8 Allow specifying empty variants
The variants.xml file can list a variant with is_empty="true" and no
groups. If such variant is found, not package gathering will be run for
it, and no repos will be created.

This only makes sense for a variant that will have some other
deliverables like live media or images.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-16 15:08:15 +01:00
Lubomír Sedlář
439622d576 [live-media] Add live media phase
This phase builds live media in Koji using the Live Media Creator. It
runs in parallel with current live images, create ISO and image build
phases.

The documentation is updated to explain how to configure this.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-02-01 13:28:49 +01:00
Dennis Gilmore
10a90f97b4 Merge #75 Start of development guide 2015-12-08 17:41:00 +00:00
Lubomír Sedlář
bd8d814230 Move messaging into cli options
The messaging is not really part of compose settings. It is an
infrastructure part. As such, it should really be set up as part of
pungi invocation, not compose configuration.

The documentation is updated to reflect this. Some updates to the
documentation are done as well: listing messages about ISOs and minor
formatting updates.

The test_compose.sh script can now accept additional command line
options and pass them on to pungi-koji to simplify testing.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-11-27 10:35:08 +01:00
Lubomír Sedlář
ba8ae15eb1 Allow running scripts with any python in PATH
Instead of hardcoding /usr/bin/python in shebangs, use /usr/bin/env.
This allows Pungi to work with dependencies installed in virtualenv.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-11-27 08:38:47 +01:00
Dennis Gilmore
6f00f20b3d Merge #68 Add support for sending messages 2015-11-25 14:56:51 +00:00
Lubomír Sedlář
066855a039 Add ability to send messages about progress
With this patch, Pungi can invoke an arbitrary command on various
moments of the compose process. The invoked command can the decide on
what message to send (and using what messaging system).

The actual command is specified in the config file.

There is a script provided that sends the messages via fedmsg.

The documentation is updated to have details about the new config option
as well as the interface for the messaging script.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-11-25 15:46:50 +01:00
Lubomír Sedlář
359eb444e5 Move image checksum collecting to separate phase
The checksums are still generated in the ImageBuild and CreatISO phases,
but collecting of them into single file is moved from pungi-koji script
into a separate phase.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2015-11-24 09:36:13 +01:00
Tomas Mlcoch
5c8694b286 Merge #57 Initialize phases in their usage order 2015-09-09 08:00:21 -04:00
Tomas Mlcoch
ecc6b0f62a Initialize phases in their usage order 2015-09-09 08:00:21 -04:00
Tomas Mlcoch
4d93221005 Add missing config check for phases: createrepo, extrafiles, liveimages and imate_build 2015-09-10 07:12:08 -04:00
Lubos Kocman
8e90a2a32a Add image-build support
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2015-09-01 08:03:34 +00:00
Pavol Babincak
853c27d1be pungi-koji: Don't print traceback if error occurred.
If compose directory was created and logger setup properly traceback is
written in traceback.log. Otherwise traceback is printed on stdout.
2015-08-25 08:11:14 -04:00
Daniel Mach
72302bd98e Rename product_* to release_*.
CHANGE: Rename product_* config options to release_* for consistency with productmd.
ACTION: Rename product_name, product_short, product_version, product_is_layered to release_* in config files.
        Rename //variant/product to //variant/release in variants XML.
2015-07-09 06:58:30 -04:00
Lubos Kocman
7d714e49f3 pungi-koji: use logger instead of print when it's available 2015-06-07 18:28:15 +00:00
Lubos Kocman
008d80186e pungi-koji: fix bad module path to verify_label() 2015-06-07 18:27:58 +00:00
Dennis Gilmore
b6825f3471 fix up productmd import in the executable
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-04-24 17:58:59 -05:00
Dennis Gilmore
5197fed16a add missing ) causing a syntax error
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-03-14 14:20:31 -05:00
Dennis Gilmore
a3158ec144 rename binaries
rename the pungi binary to pungi-koji since it does is tasks in koji
rename pungi-gather to pungi as it is the standalone old pungi binary

there is scripts that expect pungi to be the old pungi, the new binary
is not yet in use, pungi-koji semes to make sense, open to better ideas
2015-03-12 13:15:29 -05:00