This is responsible for writing out a new times.toml file, containing
important timestamps in the life of a compose. This seems a little more
reliable than attempting to infer things from the filesystem, especially
in light of the fact that we can't ever really know when a file was
created.
A bad system repo can cause lorax-composer to fail to start. Instead of
a traceback log the error and exit.
(note that the exit still results in an OSError traceback due to part of
it running as root, this needs to be addressed in another commit).
This adds a new argument to projects_depsolve and
projects_depsolve_with_size that contains the group list, unfortunately.
I would have prefered adding a function that just returns a list of all
the contents of a group and then add that to what was being passed into
projects_depsolve. However, there does not appear to be any good way to
do that in yum aside from a lot of grubbing around in the comps object,
which I am unwilling to do.
Depsolve the packages included in the templates and report any errors
using the /api/status 'msgs' field. This should help narrow down
problems with package sources not being setup correctly.
Previously it was impossible to know which package in a blueprint caused
a failure, if it was just one of them, or all of them, etc. This catches
the error when calling yb.install and lists all the failures in the
error message that is raised.
The current version of libgit2 available (0.26.3) has different behavior
with SortMode.TIME. It works correctly when left at the default (which
is also how the rawhide version works).
Yum needs to have some other attrs setup on the YumRepository object, so
use the function provided to ensure that everything is correct. Also
switch the related functions to use a dict instead of a YumRepository
object.
yum also has a cache it uses for listEnabled(), but the cache isn't
invalidated when a repo is deleted it any following metadata update
will fail because it is still using the deleted repo.
We are forced to use the heavy hammer on a yum private variable yet
again to force the cache to be cleared so that it won't crash.
yum TumRepository.dump() function cannot be used as a .repo file Add a
new function to write this in the correct format, and limited to the
fields we use.
Add a test for the new function.
Fix /projects/source/info to return an error 400 if a nonexistant TOML
source is requested. If JSON is used the error is part of the standard
response.
Update test_server.py to check for the correct error code.
When adding a source failed it wasn't being removed from the dnf object.
This fixes that, and returns an error when setting up the source fails.
Also adds a test for it.
Otherwise the user creation fails when anaconda sees there is already a
group with that name. Log a warning and continue on.
(cherry picked from commit a363aee971)
This adds support for the optional blueprint section [customizations].
Use it like this:
[customizations]
hostname = yourhostnamehere
[[customizations.sshkey]]
user = root
key = root user key
And change recipe_names API variable to blueprint_names. This *only*
changes the API variable, it does not change any subsequent usage of
'recipe'. The goal here is to change the public API, not all of the
code.
The default size is always going to be wrong, so try to estimate a more
reasonable amount of space. This is more complicated than you would
expect, yum's installedsize doesn't take into account the block size of
the filesystem, nor any extra artifacts generated by pre/post scripts.
So in the end we end up with a minimum image size of 1GiB, a partition
that is 40% larger than the estimated space needed, and a disk image
that increases size in 1GiB increments. This is still better than having
a fixed 4GiB / partition that was either too large or too small.
This will return the recipe in TOML format. Note that this does not
include any extra information about errors. Just the recipes, any
unrecognized recipe names will be skipped.
This ended up requiring more intrusive changes, but it should be the
most complex of the output types. After moving the core of
livemedia-creator into a function I added more settings to compose_args,
and more defaults to start_build. It now pulls the release information
from /etc/os-release, and produces a bootable .iso
Because of how Anaconda is run it needs to be passed a baseurl (using
--repo on the anaconda cmdline), not a mirrorlist url. This fixes it so
that the first mirror is used if the main repository is using a
mirrorlist.
This will allow testing without having a full system setup with
anaconda, if ?test=1 is passed to the POST /compose command it will wait
10 seconds instead of running Anaconda, and then raise an error to
generate a failed build.
Passing ?test=2 will also wait 10 seconds instead of running Anaconda,
but will finish successfully.
This allows the client to request the end of the anaconda.log during and
after a build. The amount of data returned can be set by adding
?size=<kbytes>
Output is raw bytes, starting on the next available line boundry.
If the build hasn't started yet (state is WAITING) try removing the
symlink to it. If this succeeds, delete the partial results directory.
If the build makes it to RUNNING then it writes a CANCEL file in the
results directory. The callback that is passed to execWithRedirect
catches this, causing a SIGTERM to be sent to anaconda. It then exits
and cleanup happens normally. The partial results directory is then
removed.
.get_default() returns string so make sure we're actually parsing
the value as boolean and not evaluating a non-empty string in a
boolean context (which will always return True)
Also fix a bug with the name of the queue status in the status results
(it is now 'queue_status' not 'status' which is used for error
responses).
This adds the following routes:
- /compose/metadata/<uuid> to retrieve a .tar of the build metadata
- /compose/results/<uuid> to retrieve .tar of all of the build results
- /compose/logs/<uuid> to retrieve a .tar of just the logs from the build
- /compose/image/<uuid> to retrieve the output image from the build
The results is a JSON string with the following information:
* id - The uuid of the comoposition
* config - containing the configuration settings used to run Anaconda
* recipe - The depsolved recipe used to generate the kickstart
* commit - The (local) git commit hash for the recipe used
* deps - The NEVRA of all of the dependencies used in the composition
* compose_type - The type of output generated (tar, iso, etc.)
* queue_status - The final status of the composition (FINISHED or FAILED)
This adds returning the commit id from read_commit, and a new function
read_recipe_and_id() that returns the commit id and the recipe in a
tuple.
If the commit is passed in, it is used as is. If no commit is passed in
it finds the most recent commit for the file on the selected branch and
returns that.
Missing recipes now raise a RecipeError with an informative message.
eg. "No commits for missing-recipe.toml on the master branch."
Write original as recipe.toml and the depsolved version as frozen.toml
Also write 'WAITING' to the STATUS file as its first state.
The STATUS states are now WAITING -> RUNNING -> FINISHED|FAILED
Also adds .package_names and .module_names properties. Call
recipe.freeze with a list of NEVRA dependencies and it will return a new
Recipe object with all of the packages and modules set to the depsolved
version.
This adds the ability to build a tar output image. The /compose and
/compose/types API routes are now available.
To start a build POST a JSON body to /compose, like this:
{"recipe_name":"glusterfs", "compose_type":"tar", "branch":"master"}
This will return a unique build id:
{
"build_id": "4d13abb6-aa4e-4c80-a671-0b867e6e77f6",
"status": true
}
which will be used to keep track of the build status (routes for this
do not exist yet).
The queue is in /var/lib/weldr/queue/new by default. It watches the
directory for new symlinks (to /var/lib/weldr/results/<dirname>) and
handles running anaconda on the kickstart found in final-kickstart.ks
inside the symlinked directory.
Passing ?branch=<branch-name> will use the specified branch instead of
master.
The new branch will not exist until a /recipes/new?branch=new-branch
POST is made. At that time the branch will be created based on the
current master branch and the new commit will be added to it.
- Fix `projects_depsolve()` to not consider a successful empty response
(rc == 0) as an error.
- Fix recipe_from_dict() to default modules and packages to empty lists
instead of `None`, to avoid a Python-ism in the API for consumers and
stay compatible to the bdcs API.
Fixes#290
