In multiple situations we need to create temporary files or directories
that should not be preserved after compose is finished. Let's add
context managers that ensure these get cleaned up.
This fixes tests leaving garbage around in /tmp.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This adds a new option repoclosure_backend that changes what tool is
used for repoclosure.
Checking build dependencies is currently not supported, as `dnf` does
not have the corresponding option.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There are a lot of options that are not used by any code path in Pungi.
There is not reason to provide access to them.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Using getattr so we also work with versions of koji that do not
read principal and keytab from the config file.
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
This uses the --can-fail option in koji. Failing an optional image will
not abort whole task. If the whole task fails (or there is a problem on
the compose side), we abort unless all arches are optional.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch uses the `--can-fail` option of koji command line. If only
optional arches fail, the task will report as success. Failures on
compose box side are ignored if and only if all architectures are
optional.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of spawning a separate process for each test, move the code to
run in the main process. This gives us correct coverage information and
makes the tests a lot faster.
The input is still provided in a kickstart file. The output also goes
into a log file that is later parsed.
There is some extra work needed with logging setup to make the logs go
to standard output so that nose can correctly capture them.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Tell lorax to use a specific directory for log files so that we preserve
them despite koji not having any idea about them.
Fixes: #457
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It really is just a group of independent functions, so we can simplify
it by removing the unused wrapper class. Instead of importing the
wrapper, instantiating it and calling its methods we can import the
module and call its functions directly.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
to complement https://pagure.io/koji/pull-request/162 we need to adjust pungi
rpm-ostree uses bublewrap that does not work in mock. --new-chroot to mock
enables the use of systemd-nspawn instead of chroot resulting in working
rpm-ostree again
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
Some modules can be executed as a sort-of test. However, the files do
not have executable bit set, so there is no need for them to have
shebangs. If someone wants to call them directly, they should do so via
python.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The address is no longer correct. We can just as well simply point to
the web page describing the license.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When a variant has optional, it is possible to explictly list it in the
XML file and add extra groups. However, the original DTD did not allow
multiple variants with `id="optional"` as `id` attribute has to be
globally unique. This patch changes its type to `CDATA`, which has less
restrictions. This also means we can no longer define `<ref>` as
`IDREF` and instead check the existence of referenced variant in parser.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This patch adds a new config option createrepo_use_xz, which when set to
true will cause createrepo to compress sqlite databases with xz. The
default setting is False.
Fixes: #387
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Introduces a new metadata file to track arbitrary files added during the
extra-files phase. This file is placed in the root of each tree and is
called ``extra_files.json``. It is a JSON file containing a single
object, which contains a "header" key with an object describing the
metadata, and a "data" key, which is an array of objects, where each
object represents a file. Each object contains the "file", "checksums",
and "size" keys. "file" is the relative path from the tree root to the
extra file. "checksums" is an object containing one or more checksums,
where the key is the digest type and the value of that key is the hex
digest. Finally, the size is the size of the file in bytes.
For example:
{
"header": {"version": "1.0},
"data": [
{
"file": "GPL",
"checksums": {
"sha256": "8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643"
},
"size": 18092
},
{
"file": "release-notes/notes.html",
"checksums": {
"sha256": "82b1ba8db522aadf101dca6404235fba179e559b95ea24ff39ee1e5d9a53bdcb"
},
"size": 1120
}
]
}
Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Fixes: #295
This will avoid confusion when the file can not be found, but previous
export from remote location worked well. In such case the log said:
Exporting from SCM...
Boom, no files found...
This commit changes it into:
Exporting from SCM...
Exporting file from current working dir...
Boom, no files found...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
It will take RPM repo from a variant in this compose and a Dockerfile
from configured git and use it to build an image.
The build images are uploaded to some a Docker registry by OSBS and are
not directly part of compose (because there is no export function).
There is a new metadata file `osbs.json` that has some information that
can be used to find the image.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is for layered products that have a variant mapped
to multiple variants in a base product, for example:
* Foo-Tools (id: FooTools, uid: Foo-Tools, name: Tools)
* Bar-Tools (id: BarTools, uid: Bar-Tools, name: Tools)
Requires productmd >= 1.2.
Signed-off-by: Daniel Mach <dmach@redhat.com>
On Python 2.6, it requires the format placeholder to have explicit index
of argument, so using % formatting is easier.
There are a couple places where the method is still used because the
same argument is used twice.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
rhbz#1331317 when we refactored how we make dvds in
df400002d8 we lost the ability to boot
the dvd as a disk image.
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
It should have raised NotImplementedError, but we better remove it
altogher since we are missing export_dir anyway.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This is a sure way to have it removed, without having to duplicate the
logic in all places where it is used.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
The logic for expanding the list of patterns into a list of actual RPM
paths was duplicated for both exporting a directory and file.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of spawning `cp x/* y` there is now Python code to the same
thing. This should help with debugging if something fails as the
traceback will be more informative (rather than saying a command
failed). As another benefit the tests get much simpler.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
When multiple threads call `get_profile_module` at the same time, some
of them may get an exception from koji. This should prevent the problem.
Fixes: #253
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Instead of looking for the DTD at module load time do it only when
required. Use a logger (if supplied) to print the warning.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
There already were config options tree_arches and tree_variants, but the
filtering was done at the compose level and there was not interaction
between the two options. This led to problems when a variant would have
all its arches filtered out.
This patch moves the filtering to the variant loader. It adds better
logging, so whenever a variant is filtered (for any reason), it will be
explicitly stated in the logs.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
This phase runs lorax with extra templates in Koji runroot task, links
the boot.iso to proper location in compose directory and adds the
installer iso to image manifest. This phase runs concurrently with live
media etc.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Simplify the code a bit by removing conditions that don't help in any
way: executing a loop zero times is the same as not executing it at all.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
Lubomír Sedlář
Dennis Gilmore
Patrick Uiterwijk
Colin Walters
Jeremy Cline
Daniel Mach
Dennis Gilmore
Peter Robinson