Commit Graph

437 Commits

Author SHA1 Message Date
Lubomír Sedlář 11bbbae2ed Treat variants without comps groups as having all of them
If the `<groups></groups>` section is not specified in the variants XML
file, all groups will be used in this variant. The section must be
omitted completely, not just empty. This is (and was) correct according
to the DTD, it just lead to crash before.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-28 09:01:30 +02:00
Lubomír Sedlář 8323a735e7 Always generate rpms.json file
Even if gather phase is skipped, it will still generate a rpms.json
metadata file. It will have no payload, but the header structure is
there.

Createrepo phase had to be updated to ignore variants that have no RPMs
listed in metadata.

There is one more fix for writing treeinfo files: if repomd.xml files
are not generated, it will no longer crash.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-27 08:50:19 +02:00
Lubomír Sedlář e1ab519542 [scm] Add logging for exporting local files
This will avoid confusion when the file can not be found, but previous
export from remote location worked well. In such case the log said:

    Exporting from SCM...
    Boom, no files found...

This commit changes it into:

    Exporting from SCM...
    Exporting file from current working dir...
    Boom, no files found...

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-23 08:07:23 +02:00
Lubomír Sedlář 8e4b1e278c [extra-files] Only copy files when there is a config
Instead of running the copy function for all variant.arch pairs
unconditionally, only do it if there is something to do. This makes the
log more understandable.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-23 07:25:52 +02:00
Lubomír Sedlář 170cd8f72d [extra-files] Refactoring
* remove duplicated function arguments
 * move logic into smaller and well documented functions

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-23 07:25:52 +02:00
Lubomír Sedlář 74001d5aa5 [extra-files] Skip whole phase if not configured
This checks the configuration in one place rather than doing it for each
variant/arch combination.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-23 07:25:52 +02:00
Lubomír Sedlář b67f6369db [extra-files] Copy files using existing function
Instead of calling cp as a system command. This will help debug problems
if something fails.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-23 07:25:52 +02:00
Lubomír Sedlář b4fc97be03 [osbs] Add a phase to build images in OSBS
It will take RPM repo from a variant in this compose and a Dockerfile
from configured git and use it to build an image.

The build images are uploaded to some a Docker registry by OSBS and are
not directly part of compose (because there is no export function).
There is a new metadata file `osbs.json` that has some information that
can be used to find the image.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-22 10:02:03 +02:00
Lubomír Sedlář 95cfbfb3fe Setup global log file before logging anything
This should fix the issue with only printing information about automatic
toggling of `supported` flag to standard output and not to a file.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-14 08:48:36 +02:00
Lubomír Sedlář 658ef90458 [metadata] Correctly save final flag
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-13 19:01:38 +02:00
Dennis Gilmore 6800501217 Merge #325 `Fix checking optional ISO images in test phase` 2016-06-06 11:56:51 +00:00
Dennis Gilmore 469c275670 Merge #321 `Add support for top-level variant IDs with dashes.` 2016-06-06 11:56:27 +00:00
Dennis Gilmore 5b5f9b7460 Merge #320 `images.json: Move src images under binary arches.` 2016-06-06 11:56:11 +00:00
Lubomír Sedlář b2a266c3e4 Fix checking optional ISO images in test phase
Instead of iterating over image manifest, loop through all variants and
arches and see if there are any images. This avoids a crash for variants
nested under other variants (layered products, optionals or addons).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-06 10:41:06 +02:00
Daniel Mach f78709aaf9 images.json: Move src images under binary arches.
Signed-off-by: Daniel Mach <dmach@redhat.com>
2016-06-02 10:17:06 -04:00
Daniel Mach f0aecf6744 Add support for top-level variant IDs with dashes.
This is for layered products that have a variant mapped
to multiple variants in a base product, for example:
 * Foo-Tools (id: FooTools, uid: Foo-Tools, name: Tools)
 * Bar-Tools (id: BarTools, uid: Bar-Tools, name: Tools)

Requires productmd >= 1.2.

Signed-off-by: Daniel Mach <dmach@redhat.com>
2016-06-02 10:16:16 -04:00
Lubomír Sedlář b62b468ccf [createiso] Enable customizing media reserve
Add a documented and tested config options for setting ISO parameters
instead of hardcoding magic values.

Fixes: #256
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-02 08:34:49 +02:00
Lubomír Sedlář a8d7f8a63e [createiso] Add test for splitting media
This patch also fixes a bug where packages in hashed directories would
not be recognized as such and would not be placed after other files.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-02 08:34:49 +02:00
Lubomír Sedlář 7655756a4f [media-split] Remove commented-out code
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-01 14:11:56 +02:00
Lubomír Sedlář e5454a23eb [media-split] Simplify code
Some parts can be written in a more straight-forward way. This also
fixes the issue with not raising error on adding to big file.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-01 14:06:25 +02:00
Lubomír Sedlář 79035ea7e6 [media-split] Add code documentation
Add a docstring to MediaSplitter class explaining what it does and how
to use it.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-06-01 14:05:49 +02:00
Lubomír Sedlář 961648819f [buildinstall] Fix bad error message
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-27 12:18:16 +02:00
Dennis Gilmore 264c7b1ddf Merge #309 `Add compatibility for Python 2.6` 2016-05-27 01:23:34 +00:00
Dennis Gilmore 0e0e333982 Merge #293 `Add tests for generating discinfo and media.repo files` 2016-05-27 01:23:22 +00:00
Dennis Gilmore e2ee4b0cc1 Merge #287 `Use koji profiles to list RPMs in buildroot` 2016-05-27 01:23:03 +00:00
Lubomír Sedlář 022a4d36f5 [ostree-installer] Put images to os/ directory
Instead of putting everything to <variant>/<arch>/iso, move images/,
EFI/ and isolinux/ subdirs to <variant>/<arch>/os. The nicely named ISO
image is still in the original location.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-26 09:58:56 +02:00
Lubomír Sedlář 9eb85c481c [util] Use koji profile for getting RPMs from buildroot
Instead of having a separate config option, just use the koji profile.
According to release notes, this should have already been done.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-26 09:50:57 +02:00
Lubomír Sedlář b634a18a7f Stop using str.format
On Python 2.6, it requires the format placeholder to have explicit index
of argument, so using % formatting is easier.

There are a couple places where the method is still used because the
same argument is used twice.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-25 13:41:49 +02:00
Lubomír Sedlář c8341e1806 [ostree-installer] Allow using external repos as source
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-24 10:06:06 +02:00
Lubomír Sedlář 17bb3d2122 [metadata] Simplify writing media.repo
This commit also removes the ability to generate timestamp. It must now
always be passed explicitly. The feature was not used anywhere and it
actually did not work.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-23 18:34:04 +02:00
Lubomír Sedlář 8ef9478879 [discinfo] Use context manager for file access
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-23 18:34:04 +02:00
Lubomír Sedlář 2113b6475c [metadata] Add tests for discinfo files
This tests writing and reading them.

Also, it makes sure the description for layered products is correct:
there was a missing space.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-23 18:34:04 +02:00
Lubomír Sedlář bb4afea4f1 [image-build] Allow using external install trees
This is needed for the twoweek Fedora atomic release to point to regular
nightly composes.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-23 18:30:14 +02:00
Lubomír Sedlář 385f0e950c Add type to base product for layered releases
Default value is "ga", can be overwritten by `base_product_type` config
option.

Fixes: #305
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-23 14:12:23 +02:00
Lubomír Sedlář 9dce30a78d [ostree] Use unique work and log paths
Each variant.arch combination can have multiple ostree repos configured,
so we need to make sure the filesystem paths don't clash.

The paths used now are:

    logs/<arch>/<variant>/ostree-<x>/
    work/ostree-<x>/config_repo

The x stands for a number identifying the task. It has no relation to
actual contents of the repo.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-18 08:03:31 +02:00
Lubomír Sedlář eeacb5b9f1 [ostree] Put variant name in ostree log dir
This will prevent log files from being mangled when multiple ostree
repos are created.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-16 18:29:09 +02:00
Dennis Gilmore e998bbe436 Merge #294 `[ostree] Initialize empty repo` 2016-05-16 13:39:55 +00:00
Lubomír Sedlář d3900296af [util] Resolve git+https URLs
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-16 08:01:50 +02:00
Lubomír Sedlář 12652ad1fa [ostree] Initialize empty repo
Make sure the directory exists before calling Koji (because otherwise
the mounting will fail). Update the runroot script to initialize the
repo when there are no files in the target destination.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-10 08:27:43 +02:00
Lubomír Sedlář 4e3d87e658 [test] Add checks for created images
The performed checks:
 * If format is ISO, the file must have correct magic string
 * If it's bootable, there must be MBR or GPT

When a check fails on any failable deliverable, it will be logged and
the file removed from metadata (it will still remain on the disk). This
required a change to write the images.json file later (after test
phase).

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-05 15:38:00 +02:00
Lubomír Sedlář 5aadf4ac39 Fix caching global ksurl
The config object from Kobo does not allow keys starting with
underscore, so we prefix it with `private_` instead.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-05-05 10:44:27 +02:00
Dennis Gilmore c80b7c6894 Merge #273 `Deduplicate configuration a bit` 2016-04-29 12:26:51 +00:00
Dennis Gilmore 8615c96624 Merge #280 `[createrepo] Use more verbose output` 2016-04-29 12:26:15 +00:00
Dennis Gilmore f9498744af Merge #283 `Pungi should log when it tries to publish notifications.` 2016-04-29 12:25:57 +00:00
Dennis Gilmore 5c4d3a5196 [createiso] Add back running isohybrid on x86 disk images
rhbz#1331317 when we refactored how we make dvds in
df400002d8 we lost the ability to boot
the dvd as a disk image.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-04-29 09:00:44 +02:00
Lubomír Sedlář b949bfd615 [createiso] Remove chdir()
Explicitly pass workdir to all invoked command. This is easier to follow
and test.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-29 08:25:49 +02:00
Lubomír Sedlář d4effc1610 [pkgset] Fix caching RPMs
The test mock did not actually match what is provided by Kobo.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-28 09:05:03 +02:00
Lubomír Sedlář 4da77c5e5d [createrepo] Use more verbose output
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-27 18:35:43 +02:00
Ralph Bean dafeb613c6 Pungi should log when it tries to publish notifications.
Signed-off-by: Ralph Bean <rbean@redhat.com>
2016-04-27 12:09:57 -04:00
Lubomír Sedlář 9f3408fec0 [pkgset] Use context manager for opening file list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-04-26 14:58:18 +02:00