When there is no run or new symlink do one last check to make sure no
STATUS file was written. If it is missing, go ahead and remove the
results directory.
(cherry picked from commit a8f616c6da)
Related: rhbz#1659129
If another CANCEL request has already been made just exit from
uuid_cancel. If the build is FINISHED before it times out just exit,
don't remove the finished results.
(cherry picked from commit 962d10b3b7)
Related: rhbz#1659129
In addition to monitoring the logs for errors, call a function (or
functions) that tell it to cancel the anaconda process and cleanup.
Also check for a cancel after creating the squashfs image for live-iso
since that's a long running process.
This required adding a new argument to a number of existing functions,
passing it down to VirtualInstall and novirt_install where the function
is called.
(cherry picked from commit 4b84475612)
Resolves: rhbz#1659129
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.
So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
Resolves: rhbz#1664128
Support both
[customizations]
hostname = "whatever"
and
[[customizations]]
hostname = "whatever"
in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.
Add some tests for converting customizations to kickstarts.
(cherry picked from commit 35ab6a1336)
Resolves: rhbz#1666517
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.
composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
The projects_list function uses a different yum request, and doesn't
return a full project dict. Updated it to also return only the unique
project names.
Related: rhbz#1657055
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.
This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.
Resolves: rhbz#1657055
(cherry picked from commit 663a0dcd73)
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
* Removes broken symlinks from queue/new and queue/run
* Removes symlinks from run and sets the build to FAILED
* Sets builds w/o a STATUS to FAILED
* Sets builds with STATUS of RUNNING to FAILED
* Creates missing queue/new symlinks to results with STATUS of WAITING
So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.
Resolves: rhbz#1657054
(cherry picked from commit f0bac40d7f)
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning
(cherry picked from commit 1056bfc25b)
Resolves: rhbz#1656105
This is similar to the AMI compose type, with a handful of additional
changes specific to Azure:
* Add waagent (but leave NetworkManager enabled, despite some of the
docs)
* Disable cloud-init
* Add Hyper-V modules into initrams.
Fixes specific for RHEL:
* Create ifcfg-eth0 required by waagent.
* Install python3 and net-tools required by waagent.
Recommended changes:
* Use recommended kernel boot args.
* Disable kdump.
(cherry picked from commit e0c236ff36)
(cherry picked from commit da0435bc90)
(cherry picked from commit b594fa99bc)
Resolves: rhbz#1656105
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
(cherry picked from commit 18188bf6cf)
(cherry picked from commit 81d38b6445)
Resolves: rhbz#1656105
Some kickstart templates use globbing to match packages, searchNames
doesn't support this.
Add fallback code to find packages that are just a dep, matching the
behavior of yum.install()
This fixes things so that it can depsolve package names with globs in
them like grub2-efi-*-cdboot, and deps like shim and grub2-efi.
Results are still filtered by the version glob, with the highest NEVRA
selected for installation.
Resolves: rhbz#1641601
The problem this solves is that yum really isn't designed to be part of\
a long running daemon. So when repodata changes upstream, even when
you force it to download the new metadata, it doesn't change in memory
so you end up with lorax-composer depsolving against old versions, and
anaconda depsolving against new versions (because it sets up its own
YumBase and cache) and then the kickstart is no longer valid.
To solve this I have
- Added a 6h timeout to the metadata check (because yum's doesn't work
in this situation).
- Added a metadata check to the YumLock .lock property, but only when
the timeout expires.
- Added a new .lock_check property to YumLock that always checks the
metadata and resets the timeout.
If it has changed it does its best to tear down the existing YumBase,
deleting as much as it can in hopes it doesn't leak memory. And then it
sets up a totally new YumBase with the new repodata.
Resolves: rhbz#1632962
Use a common _depsolve function for projects_depsolve and
projects_depsolve_with_size so that it always uses the correct version
glob support when depsolving blueprints and templates.
Resolves: rhbz#1628114
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.
So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.
Related: rhbz#1626120
The blueprint version glob was being applied to the whole package NEVRA
by yum (it lacks a separate API for just globbing versions), so this
implements that in filterVersionGlob using fnmatchcase on the package
names, and the yum package verGT comparison on the versions for the
selected package.
Also includes tests.
Resolves: rhbz#1628114
This adds the following optional arguments to the /compose/status route:
- type, matches the compose_type field
- status, matches the queue_status field
- blueprint, matches the blueprint field
Currently we are making MBR disk images for qcow2 and partitioned disk,
so the UEFI packages aren't required at this point.
Move the clearpart command into compose.py so that in the futute it can
use clearpart --disklabel to create a GPT image, and add the required
packages to the package set.
The idea here is to make sure all return points have the same type for
the error cases. There's not really all that many, so they just go in
one patch. Some of these could potentially turn into more specialized
errors later.
Note the exception string checking around compose_type. I didn't really
want to introduce a new exception type just for this, but also didn't
want to duplicate strings. I'd be open to other suggestions for how to
do this.
This adds some fairly redundant code to the beginning of all the
blueprint routes to attempt reading a commit from git for the
blueprint's recipe. If it succeeds, the blueprint exists and the route
can continue. Otherwise, return an error. Hopefully this doesn't slow
things down too much.
Note that this also changes the return type of uuid_info to return None
when an unknown ID is given. The other uuid_* functions are fine
because they are checked ahead of time.
Each element in the errors value is now a dict, with a msg field and an
id field. The id field contains a value out of errors.py that can be
used by the front end to key on. The msg field is the same as what's
been there.
The idea is to keep the number of IDs somewhat limited so there's not a
huge number of things for the front end to know.
Currently the code is not UTF8 safe, so we need to return a clear error
when invalid characters are passed in.
This also adds tests for the routes to confirm that an error is
correctly returned.
This handles the case where a route is requested, but without a required
parameter. So, /blueprints/info is requested instead of
/blueprints/info/http-server. It accomplishes this via a decorator, so
a lot of these route-related functions now have quite a few decorators
attached to them.
Typo'd URLs (/blueprints/nfo for instance) will still return a 404. I
think this is a reasonable thing to do.
Unfortunately, this isn't very useful if /modules/info is provided with
multiple modules. yum doesn't traceback when doPackageLists is given
something that doesn't exist. It just returns an empty list. If
/modules/info is given just one module and yum gives us an empty list,
it's easy to say what happened. If /modules/info is given several
modules and just one does not exist, we will not be able to detect that.
Fixing this would require doing more yum operations, which is likely to
slow things down and isn't the direction I want to be going.
