Commit Graph

769 Commits

Author SHA1 Message Date
Brian C. Lane bb8c7851d2 Fix composer-cli command help text
sources shouldn't be indented
details for a compose are 'details' not 'info'
2018-06-18 10:45:36 -07:00
Brian C. Lane 4bfcd8fcd2 Remove SortMode from list_commits
The current version of libgit2 available (0.26.3) has different behavior
with SortMode.TIME. It works correctly when left at the default (which
is also how the rawhide version works).
2018-06-11 16:54:59 -07:00
Brian C. Lane 3cfb895054 Use yum.add_enable_repo() for new repos
Yum needs to have some other attrs setup on the YumRepository object, so
use the function provided to ensure that everything is correct. Also
switch the related functions to use a dict instead of a YumRepository
object.
2018-06-11 16:54:59 -07:00
Brian C. Lane 957def1264 Add support for sources to composer-cli
This adds the sources command which can be used to list, add, change,
and delete sources using the TOML formatted source file.
2018-06-11 16:54:59 -07:00
Brian C. Lane a5eaebeefc yum repos has a delete() function. But it doesn't clear the cache.
yum also has a cache it uses for listEnabled(), but the cache isn't
invalidated when a repo is deleted it any following metadata update
will fail because it is still using the deleted repo.

We are forced to use the heavy hammer on a yum private variable yet
again to force the cache to be cleared so that it won't crash.
2018-06-11 16:54:59 -07:00
Brian C. Lane 2e95b56508 Fix DNF related issues with source selection
yum TumRepository.dump() function cannot be used as a .repo file Add a
new function to write this in the correct format, and limited to the
fields we use.

Add a test for the new function.

Fix /projects/source/info to return an error 400 if a nonexistant TOML
source is requested. If JSON is used the error is part of the standard
response.

Update test_server.py to check for the correct error code.
2018-06-11 16:54:59 -07:00
Brian C. Lane 3f7997d7ae Fix handling bad source repos and add a test
When adding a source failed it wasn't being removed from the dnf object.
This fixes that, and returns an error when setting up the source fails.
Also adds a test for it.
2018-06-11 16:54:59 -07:00
Brian C. Lane 25bae61520 Fix make_dnf_dirs
It was chopping off an extra directory level due to realpath removing
the trailing / from the paths when they are setup.
2018-06-11 16:54:59 -07:00
Brian C. Lane 33c84331fe Add support for user defined package sources API
This lives under /api/v0/projects/source/*

See the documentation for details
2018-06-11 16:54:59 -07:00
Brian C. Lane 82c8c3a491 gevent has deprecated .wsgi, should use .pywsgi instead
https://github.com/gevent/gevent/blob/master/doc/api/gevent.wsgi.rst
(cherry picked from commit c9ca451568)
2018-06-11 16:54:59 -07:00
Brian C. Lane b99d8d7f6b Add support for version globs to blueprints
You can use '*' wildcards and '?' for single character matching.
2018-06-11 16:54:59 -07:00
Brian C. Lane ce715ad15f Check to make sure blueprints directory exists 2018-06-11 16:54:59 -07:00
Brian C. Lane 3c60f07d98 Remove temporary files after run_compose
A crash can also leave temporary lmc-* files, remove them as well.
2018-06-11 16:54:59 -07:00
Brian C. Lane f26a6212d6 Add --proxy to lorax-composer cmdline
Overrides the [yum] proxy setting in the config file.
2018-06-11 16:54:59 -07:00
Brian C. Lane 9978503a69 Pass the --tmp value into run_creator and cleanup after a crash
Crashing can sometimes leave directories in /var/tmp/lmc-* so clean
those up after run_creator is finished.
2018-06-11 16:54:59 -07:00
Brian C. Lane 3a38a57ae1 Add --tmp to lorax-composer and set default tempdir
It was using /tmp/ which can fill up quickly when building images.
Default to /var/tmp unless the user passes --tmp /tmp/
2018-06-11 16:54:59 -07:00
Brian C. Lane 050f87168a Skip creating groups with the same name as a user
Otherwise the user creation fails when anaconda sees there is already a
group with that name. Log a warning and continue on.

(cherry picked from commit a363aee971)
2018-06-11 16:54:59 -07:00
Brian C. Lane 5b7b37860a Add user and group creation to blueprint
[[customizations.user]] and [[customizations.group]]
2018-06-11 16:54:59 -07:00
Brian C. Lane ccafa76019 Add blueprint customization support for hostname and ssh key
This adds support for the optional blueprint section [customizations].

Use it like this:

[customizations]
hostname = yourhostnamehere

[[customizations.sshkey]]
user = root
key = root user key
2018-06-11 16:54:59 -07:00
Brian C. Lane ffc3195d77 Add support for systemd socket activation
Instead of enabling lorax-composer.service enable lorax-composer.socket
and it will start lorax-composer on first access to
/run/weldr/api.socket
2018-06-11 16:54:59 -07:00
Brian C. Lane 85f365bd73 Sort the list of supported output types 2018-06-11 16:54:59 -07:00
Brian C. Lane f40f7cc0fd Update the error responses to just return lists of strings.
This makes error handling consistent and easier than a couple layers of
fields to fetch.
2018-06-11 16:54:59 -07:00
Brian C. Lane 520439a7c4 Move status to /api/status
Also drops /api/v0/test, and redirects / to /api/docs/

db and schema are fixed at 0 since lorax-composer uses yum, not the bdcs
metadata.db
2018-06-11 16:54:59 -07:00
Brian C. Lane 0887bd3d95 Update the queue to use blueprint.toml
Also update the output JSON to use 'blueprint' instead of 'recipe'
2018-06-11 16:54:59 -07:00
Brian C. Lane 5efeb05aa7 Update composer-cli to use blueprint instead of recipe 2018-06-11 16:54:59 -07:00
Brian C. Lane 651e111d1f Change the API code to use blueprint
This includes in the JSON output. The only exception is when using
functions from the recipes module. There are no plans to change that.
2018-06-11 16:54:59 -07:00
Brian C. Lane 540f36eee4 Change the /recipes/ routes to /blueprints/
And change recipe_names API variable to blueprint_names.  This *only*
changes the API variable, it does not change any subsequent usage of
'recipe'. The goal here is to change the public API, not all of the
code.
2018-06-11 16:54:59 -07:00
Brian C. Lane 2939feadec Change recipe in API documentation to blueprint 2018-06-11 16:54:59 -07:00
Brian C. Lane 2697bb2bc7 Add support for building ext4 filesystem images. 2018-06-11 16:54:59 -07:00
Brian C. Lane 020d9de4b3 Add the image size to the composer-cli status output 2018-06-11 16:54:59 -07:00
Brian C. Lane 211f1bf764 Add image_size to the compose/info JSON
The size will be 0 until the compose is finished. It will then be set
to the size of the image in bytes.
2018-06-11 16:54:59 -07:00
Brian C. Lane 78c02dd7e7 Add image size to the compose details
If the image is finished include the size of the image file as
"image_size" in the ouput JSON from /compose/finished
2018-06-11 16:54:59 -07:00
Brian C. Lane b2f5fe2f60 Removed the fixed partition size from composer ks templates
The default size is always going to be wrong, so try to estimate a more
reasonable amount of space. This is more complicated than you would
expect, yum's installedsize doesn't take into account the block size of
the filesystem, nor any extra artifacts generated by pre/post scripts.

So in the end we end up with a minimum image size of 1GiB, a partition
that is 40% larger than the estimated space needed, and a disk image
that increases size in 1GiB increments. This is still better than having
a fixed 4GiB / partition that was either too large or too small.
2018-06-11 16:54:59 -07:00
Brian C. Lane 47a3980b12 Fix some pylint warnings 2018-06-11 16:54:59 -07:00
Brian C. Lane c15f3c89fa Add the compose type to the output from compose status 2018-06-11 16:54:59 -07:00
Brian C. Lane 423e8cba88 Fix composer-cli handling of log and detail errors. 2018-06-11 16:54:59 -07:00
Brian C. Lane 2481c486a5 Fix a couple of error responses
Errors should always be "error:{"msg":"message"}
2018-06-11 16:54:59 -07:00
Brian C. Lane c58e1994e9 Add missing checks on return value from uuid_status 2018-06-11 16:54:59 -07:00
Brian C. Lane 97eb262d1e Fix handling of missing STATUS file
If lorax-composer is interrupted by ^C it can leave a result directory
without a STATUS file. Don't crash when that happens, just skip the
directory.
2018-06-11 16:54:59 -07:00
Brian C. Lane 258cecf75d Fix compose types command
Missed a function when adding the testmode arguments. Also add
documentation for testmode.
2018-06-11 16:54:59 -07:00
Brian C. Lane af68a98abe Add qcow2 image type
Very similar to partitioned-disk, image is named disk.qcow2 instead of
disk.img
2018-06-11 16:54:58 -07:00
Brian C. Lane 5446cad1b0 Fix prettyDiffEntry output
It was returning old -> old for Description and Version, not old -> new
2018-06-11 16:54:58 -07:00
Brian C. Lane 9e16e1c0c5 Default composer-cli log should be in ./composer-cli.log
Multiple users may use the tool, each one gets their own logfile.
2018-06-11 16:54:58 -07:00
Brian C. Lane 73011c6bb0 Update Sphinx documentation for composer.cli 2018-06-11 16:54:58 -07:00
Brian C. Lane 5c878a695f Add --test option to composer-cli
The API server will run a mock compose when a test mode is passed to it.
Passing a 1 will queue a build, pretend to run the build for 10 seconds,
and then fail. Passing a 2 will do the same thing, but it will finish as
if it was successful. All results are available but the output file is
only the string "TEST IMAGE"

This should allow running tests inside docker without calling anaconda
(becuase it will not run in docker, it needs a VM).
2018-06-11 16:54:58 -07:00
Brian C. Lane 1a4a4defcc Make sure lorax-composer tests only use temporary directories
It was using /var/tmp/composer/, now it places everything under a
temporary directory that is removed when it is finished.
2018-06-11 16:54:58 -07:00
Brian C. Lane d4b99b5ee9 Refactor get_filename so it can be tested
Just pass in the headers instead of the whole response object so that it
can be tested without needing an actual server.
2018-06-11 16:54:58 -07:00
Brian C. Lane d0676dbf7c Fix bug in prettyDiffEntry output
It was not handling adding or removing modules correctly.
2018-06-11 16:54:58 -07:00
Brian C. Lane aa076ee53b composer-cli: Handle download errors
The API will return an error 400 with a json error object if the
download fails, catch this and report the error to the user without a
traceback.
2018-06-11 16:54:58 -07:00
Brian C. Lane de604b37a6 Add a pid file for lorax-composer
This will prevent accidentally running more than 1 instance.
Uses /run/lorax-composer.pid and checks to make sure that the PID
written to it isn't stale.
2018-06-11 16:54:58 -07:00