Commit Graph

188 Commits

Author SHA1 Message Date
Lubomír Sedlář
cdfa3cb45f image-build: Drop suffixes from configuration
The file extension in configuration is only used to tell Pungi which
files from the task results should be downloaded. The user has to get it
right or the phase will fail. Each format has a single valid suffix.

Pungi should not require users to specify the suffix, since it can just
as well just know the right value.

The old configuration will continue working, only the extension will be
ignored.

Fixes: https://pagure.io/pungi/issue/753
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-11-01 12:47:36 +01:00
Ondrej Nosek
e0308a74ff remove remaining hard coded createrepo threads
To be more precise, new createrepo parameter "workers" was customized and
new default value was set. This fixes issue #752.

Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-10-30 10:37:10 +01:00
Ondrej Nosek
4ff3190935 Update documentation section "Contributing to Pungi".
Signed-off-by: Ondrej Nosek <onosek@redhat.com>
2017-10-25 10:28:08 +02:00
Lubomír Sedlář
daf162503c Reject yum gather backend on Python 3
It will not run, and having a nice error message is better than a
cryptic crash.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:15:31 +02:00
Patrick Uiterwijk
3088df8e60 checks: Use list of release types from productmd
Let's not duplicate the list. Productmd exports a list of valid values,
so Pungi should just pick and use that.

Closes: https://pagure.io/pungi/pull-request/773
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-24 15:05:08 +02:00
Lubomír Sedlář
be39dc3caf buildinstall: Expose template arguments for lorax
This would be useful for modularity. The templates can be added now and
variables set via the existing `lorax_option`.

It's not possible to use custom templates not shipped with lorax, as
passing the path to a random directory is a little bit more tricky.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-23 16:08:09 +02:00
Lubomír Sedlář
68fdef451c pkgset: Allow unsigned packages by empty key
Currently `None` has to be included to allow using unsigned packages.
ODCS has trouble with including non-string value in the list though, so
we can treat empty string the same way (it's not a valid key ID anyway).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-16 13:48:27 +02:00
Lubomír Sedlář
c89f033457 live-images: Honor global settings for target
Instead of the old default value and custom handling introduce new
option `live_images_target` (for consistency with other phases) and use
the usual inheritance rules.

Fixes: https://pagure.io/pungi/issue/749
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-09 13:46:54 +02:00
Dennis Gilmore
b51a639277 Merge #743 Make ostree installer before cloud images 2017-10-06 00:18:46 +00:00
Lubomír Sedlář
9ab3840085 Add directory name for checksum file
Fixes: https://pagure.io/pungi/issue/745
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-05 16:08:27 +02:00
Lubomír Sedlář
dec00fe2f4 createrepo: Allow customizing number of threads
The default is now to use one thread per CPU.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-10-03 15:21:37 +02:00
Lubomír Sedlář
19436c66b2 Make ostree installer before cloud images
Fixes: https://pagure.io/pungi/issue/742
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-20 08:33:40 +02:00
Lubomír Sedlář
44c523339c 4.1.19 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-18 13:17:18 +02:00
Lubomír Sedlář
04836cfa9f docs: Mention how input package list are interpreted
It's really just an RPM name, not any random provide or source package
name.

Fixes: https://pagure.io/pungi/issue/725
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-18 12:06:03 +02:00
Ralph Bean
5b6e468952 Allow extracting profiling information from pungi-gather.
`pungi-gather` (the tool that underlies both the `pkgset` and `gather`
phases) contains profiling code that will log statistics about how long
different function calls take.  However, pungi-koji did not contain a
way to pass the ``--profiler`` argument to enable this.

This change adds a new configuration option ``gather_profiler`` which,
when set to true, simply passes the argument to `pungi-koji`.  Hopefully
this can help shed some light on what is happening in some of our
longer-running composes.

Merges: https://pagure.io/pungi/pull-request/727
Signed-off-by: Ralph Bean <rbean@redhat.com>
2017-09-04 10:25:11 +02:00
Lubomír Sedlář
e21a27bdc9 createrepo: Allow selecting variants for delta RPMs
The configuration needs to be more granular than a single global option.
With this patch each tree can enable deltas separately.

Fixes: https://pagure.io/pungi/issue/715
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-04 09:15:20 +02:00
Lubomír Sedlář
2e3a9385a3 config: Allow setting default compose type
The config can change the default compose type. This can still be
overwritten by a CLI argument. A `--production` option is now added to
CLI (because that was the default before).

Fixes: https://pagure.io/pungi/issue/694
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-25 10:07:33 +02:00
Lubomír Sedlář
c14c28a157 docs: Abort update script on error
If building documentation fails, there's no point in trying to update
the repo.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-22 10:28:38 +02:00
Lubomír Sedlář
c9d95f5f6f 4.1.18 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-22 10:22:27 +02:00
Lubomír Sedlář
804a0049f6 ostree: Simplify configuration
It makes no sense to repeat the same configuration for multiple
architectures. Instead we should just list the architectures as another
key in the mapping. There is an option to specify multiple config dicts.

This preserves full backwards compatibility, the old config format is
still accepted.

Fixes: https://pagure.io/pungi/issue/678
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-14 14:53:22 +02:00
Lubomír Sedlář
7e03133c8f docs: Add documentation for scm_dict
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-10 15:32:24 +02:00
Lubomír Sedlář
cef8650c3e gather: Report missing comps packages
When a package mentioned in comps is not available in the package set,
print a warning about this. Additionally there is a config option that
allows to turn this warning into a fatal error.

Fixes: https://pagure.io/pungi/issue/50
Fixes: https://pagure.io/pungi/issue/683
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-09 11:04:14 +02:00
Lubomír Sedlář
e2962dc547 repoclosure: Allow aborting compose when repoclosure fails
Alternatively the call to repoclosure can be turned off. This is
customizable per variant and architecture.

Fixes: https://pagure.io/pungi/issue/676
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-07 15:17:33 +02:00
Qixiang Wan
0a3e5b27bf remove the dependency of rpmUtils
The rpmUtils module is provided yum-utils package, which is only
available for Python 2. There is no replacement for the functionality in
DNF.

There is a proposal to add this functionality to rpm itself, but it's
not really moving forward very much:
https://bugzilla.redhat.com/show_bug.cgi?id=1072972

As a short term solution let's copy the needed parts of rpmUtils.arch
module directly to pungi code base.

Fixes: https://pagure.io/pungi/issue/533
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-07-26 11:23:16 +02:00
Lubomír Sedlář
c0bac63f4d 4.1.17 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-17 09:58:35 +02:00
Lubomír Sedlář
6cf912f555 docs: Convert phases diagram to PNG
Sphinx and Latex are unable to include SVG apparently:
https://github.com/sphinx-doc/sphinx/issues/1907

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-17 09:58:35 +02:00
Lubomír Sedlář
543c184e0d docs: Minor improvements to documentation
* add description for `gather_method`
 * fix typo in `createrepo_deltas`

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-13 10:03:54 +02:00
Qixiang Wan
18bd37ff2c pkgset: Allow populating packages from multiple koji tags
One of the use cases is https://pagure.io/odcs/issue/7.

Merges: #660
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-06-29 08:27:34 +02:00
Lubomír Sedlář
65bc6969e2 docs: Add a basic info about gathering packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-21 09:55:29 +02:00
Lubomír Sedlář
3e64238fca docs: Kobo can be installed via pip now
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-20 11:11:16 +02:00
Dennis Gilmore
72081a3a35 Merge #649 docs: Add overview of what each phase does 2017-06-19 17:22:43 +00:00
Lubomír Sedlář
9ab3dc89b2 docs: Add overview of what each phase does
Fixes: #520
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-19 10:58:01 +02:00
Lubomír Sedlář
2a8d7f8843 docs: Document config file format
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-19 10:51:31 +02:00
Lubomír Sedlář
bef573d222 docs: Move logo to _static subdir
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-19 10:51:23 +02:00
Lubomír Sedlář
d3dd7aa7ce 4.1.16 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-09 13:12:19 +02:00
Lubomír Sedlář
e549c732a4 4.1.15 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-05-05 08:16:11 +02:00
Lubomír Sedlář
68a1051f14 docs: Add a logo on the About page
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-26 10:46:57 +02:00
Lubomír Sedlář
a2c3274585 docs: Document origin of the name
Fixes: #600
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-26 10:46:57 +02:00
Lubomír Sedlář
bab2a125d4 osbs: Make git_branch required option
In order to avoid conflicting tags, OSBS allows only one build for a
repo/branch pair at the same time. To avoid race conditions, we should
make sure we always pass in the branch. This commit makes it a required
option.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-24 10:05:25 +02:00
Lubomír Sedlář
21952955ad docs: Update createrepo_checksum allowed values
Relates: #591
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-20 15:22:58 +02:00
Lubomír Sedlář
c293a1e147 extra-files: Allow configuring used checksums
Instead of adding a new config option, we can just reuse the existing
`media_checksums` value. If the value is good for image checksums, it
should work for extra files as well.

Relates: #591
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-20 09:28:09 +02:00
Lubomír Sedlář
532b6b1fbc doc: Document options for media checksums
Pungi does not really care about the values as long as hashlib supports
them.

Relates: #591
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-20 09:28:09 +02:00
Lubomír Sedlář
f6d07c1651 Be explicit about generating release for images
The config now uses similar logic what previous commit did for OSTree.
Also we should report error when an unknown generator is used.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-13 10:17:59 +02:00
Lubomír Sedlář
0f4b6b1947 docs: Add examples for generated versions
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-13 10:17:59 +02:00
Qixiang Wan
2f5d6d7dcd unify repo and repo_from options
Config option 'repo' and 'repo_from' are used in several phases, merge
them with one option 'repo'. 'append' in schema is used for appending
the values from deprecated options to 'repo', so it won't break on any
existing config files that have the old options of 'repo_from' and
'source_repo_from' (which is an alias of 'repo_from').

And 'repo' schema is updated to support repo dict as the value or an
item in the values, a repo dict is just a dict contains repo options,
'baseurl' is required in the dict, like:

{"baseurl": "http://example.com/url/to/repo"}

or:

{"baseurl": "Serer"}

currently this is used in ostree phase to support extra repo options
like:

{"baseurl": "Server", "exclude": "systemd-container"}

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-29 10:12:32 +08:00
Qixiang Wan
0f508e2228 move translate_path from paths.py to util.py
So translate_path can be used in util.py, or it will result in
circular import error.

Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-29 10:12:32 +08:00
Lubomír Sedlář
9784961568 4.1.14 release
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-27 18:04:26 +02:00
Lubomír Sedlář
5dd2b3947d Rename option to switch gather backend
Calling it gather_backend is similar to repoclosure_backend we already
have. It's also more obvious what it does.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář
76cf4a7540 Update to use python-multilib
We also rename the old multilib module used by dnf code to multilib_yum
to make it clear what is imported where.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář
77673e1eac Improve documentation
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00