Commit Graph

1860 Commits

Author SHA1 Message Date
David Shea
40f23f093d Add filter arguments to /compose/status
This adds the following optional arguments to the /compose/status route:

  - type, matches the compose_type field
  - status, matches the queue_status field
  - blueprint, matches the blueprint field
2018-08-21 16:42:07 -07:00
Brian C. Lane
844ff9998b composer-cli should not log to a file by default
The user can pass --log /path/to/logfile.log if they want logging
enabled.
2018-08-21 11:49:28 -07:00
Brian C. Lane
04fd94f44b Add documentation for using a DVD as the package source 2018-08-21 11:28:35 -07:00
Lars Karlitski
be5d50e6f3 Set TCP listen backlog for API socket to SOMAXCONN
A value of 1 is too low for heavy users of the API, such as the weldr-web
interface.

This is also systemd's default for sockets it opens. Using lorax-composer with
socket activation already results in a backlog of SOMAXCONN connections.
2018-08-20 14:54:07 -07:00
Peter Robinson
94453ff94e Update Arm architectures for the latest requirements
ARMv7 will now support UEFI and grub2-efi so add those deps. We also don't
require uboot-tools as part of the install process so don't add that
explicitly, and drop grubby from the aarch64 explicit list as it's in the
general list, add grub2-tools-efi.

Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2018-08-16 15:34:19 -07:00
Brian C. Lane
c638926880 New lorax documentation - 29.11 2018-08-13 16:43:20 -07:00
Brian C. Lane
83e5f3f885 Add a note about using lorax-composer.service
Update the docs to make it clear that .socket will be slow to respond to
the first request, and to use .service to speed it up.
2018-08-13 16:41:44 -07:00
Brian C. Lane
07de4c371a Ignore dnf.logging when building docs 2018-08-13 16:32:26 -07:00
Radek Vykydal
b1daf6fcd3 Bring back import-state.service (#1615332)
The service is a part of initscripts package which is no more pulled in as a
transitive dependcy so we have to require it explicitly.
2018-08-13 14:34:28 -07:00
Chris Lumens
47d3a72014 Fix a little bug in running "modules list".
(cherry picked from commit 5dddef6f49)
2018-08-10 12:00:23 -04:00
Brian C. Lane
b1e2bbfd17 Fix bash_completion.d typo 2018-08-09 16:25:03 -07:00
Brian C. Lane
c8283adc17 Move disklabel and UEFI support to compose.py
Currently we are making MBR disk images for qcow2 and partitioned disk,
so the UEFI packages aren't required at this point.

Move the clearpart command into compose.py so that in the futute it can
use clearpart --disklabel to create a GPT image, and add the required
packages to the package set.
2018-08-09 15:17:27 -07:00
Chris Lumens
fb9e5c4a2b Fix more tests.
These were broken due to me rebasing before sending a PR off.  They
should work now.

(cherry picked from commit 4cb15e0a0f)
2018-08-09 16:20:29 -04:00
Chris Lumens
faba254be5 Change INVALID_NAME to INVALID_CHARS.
It's getting used in a whole lot more places now, so make it a bit more
generic sounding.

(cherry picked from commit bf0318eab6)
2018-08-09 16:20:23 -04:00
Chris Lumens
473ee99eb9 Update composer-cli for the new error return types.
(cherry picked from commit bc96f75992)
2018-08-09 16:20:17 -04:00
Chris Lumens
d03a198a8c Add default error IDs everywhere else.
The idea here is to make sure all return points have the same type for
the error cases.  There's not really all that many, so they just go in
one patch.  Some of these could potentially turn into more specialized
errors later.

(cherry picked from commit fd901c5e3f)
2018-08-09 16:20:01 -04:00
Chris Lumens
6a3ccd6ade Add error IDs to things that can go wrong when running a compose.
Note the exception string checking around compose_type.  I didn't really
want to introduce a new exception type just for this, but also didn't
want to duplicate strings.  I'd be open to other suggestions for how to
do this.

(cherry picked from commit b3bb438254)
2018-08-09 16:19:19 -04:00
Chris Lumens
c1af7dd69d Add error IDs for common source-related errors.
(cherry picked from commit e43adfc7af)
2018-08-09 16:19:12 -04:00
Chris Lumens
f67e2c5d52 Add error IDs for unknown modules and unknown projects.
(cherry picked from commit 2adcfc9563)
2018-08-09 16:19:08 -04:00
Chris Lumens
785ee7bcc3 Add error IDs for when an unknown commit is requested.
(cherry picked from commit 07528a431e)
2018-08-09 16:19:03 -04:00
Chris Lumens
2f8f076b1f Add error IDs for when an unknown blueprint is requested.
This adds some fairly redundant code to the beginning of all the
blueprint routes to attempt reading a commit from git for the
blueprint's recipe.  If it succeeds, the blueprint exists and the route
can continue.  Otherwise, return an error.  Hopefully this doesn't slow
things down too much.

(cherry picked from commit a925cc7ddb)
2018-08-09 16:18:49 -04:00
Chris Lumens
a2ce0686ca Add error IDs for when an unknown build UUID is requested.
Note that this also changes the return type of uuid_info to return None
when an unknown ID is given.  The other uuid_* functions are fine
because they are checked ahead of time.

(cherry picked from commit 6497b4fb65)
2018-08-09 16:18:40 -04:00
Chris Lumens
b26d12e457 Add error IDs for bad state conditions.
(cherry picked from commit d76e24053a)
2018-08-09 16:18:23 -04:00
Chris Lumens
2efbd7cae5 Change the error return type for bad limit= and offset=.
Each element in the errors value is now a dict, with a msg field and an
id field.  The id field contains a value out of errors.py that can be
used by the front end to key on.  The msg field is the same as what's
been there.

The idea is to keep the number of IDs somewhat limited so there's not a
huge number of things for the front end to know.

(cherry picked from commit 9677b012da)
2018-08-09 16:18:17 -04:00
Chris Lumens
6c3b0b3cd2 Don't sort error messages.
This should make it easier to return more complex error structures.  It
also doesn't appear to matter - tests still pass without changes.

(cherry picked from commit 4c3f93e329)
2018-08-09 16:18:11 -04:00
Brian C. Lane
f557b28d19 Fix bash completion of compose info 2018-08-07 16:47:54 -07:00
Brian C. Lane
0e97afdbd6 Add + to the allowed API string character set 2018-08-07 16:47:54 -07:00
Brian C. Lane
316a829ff7 Add job_* timestamp support to compose status 2018-08-07 15:21:10 -07:00
Brian C. Lane
55dd45de6e Drop .decode from UTF8_TEST_STRING 2018-08-07 10:12:14 -07:00
Brian C. Lane
ec4c555174 Add input string checks to the branch and format arguments
Make sure no UTF8 characters are allowed and return an error if they
are.

Also includes tests to make sure the correct error is returned.

(cherry picked from commit 86d79cd8a6)
2018-08-07 10:12:02 -07:00
Brian C. Lane
f93f64a061 Add a test for invalid characters in the API route
Currently the code is not UTF8 safe, so we need to return a clear error
when invalid characters are passed in.

This also adds tests for the routes to confirm that an error is
correctly returned.

(cherry picked from commit 74f5def3d4)
2018-08-07 10:12:02 -07:00
Will Woods
2d01bec2b2 Add etc/bash_completion.d/composer-cli
Here's a bash_completion file for composer-cli! It completes known
commands, subcommands, and flags, plus arguments to flags that take
arguments, and if you've got write access to the socket it'll also
autocomplete:

* source names for commands that need source names
* blueprint names for commands that take blueprint names
* compose uuids for commands that take compose uuids
  * intelligently only suggests appropriate uuids - e.g. only
    suggests uuids of running composes for "compose cancel"

NOTE: bash completion scripts are horrible and this is no exception.
Please forgive me.
2018-08-07 13:00:56 -04:00
Will Woods
439a894947 composer-cli: clean up "list" commands
This patch does two things:

1) Add "compose list", which lists compose UUIDs and other basic info,
2) Fix up "blueprints list", "modules list", "sources list", and
   "compose types" so their output is just a plain list of identifiers
2018-08-07 13:00:56 -04:00
Brian C. Lane
741e73265c Fix logging argument 2018-08-07 09:24:11 -07:00
Brian C. Lane
e2dd59c6dc Update get_system_repo for dnf
With dnf we iterate the repos and return their names, not the filenames
like we do with yum. Also make the list test more flexible.
2018-08-07 09:22:18 -07:00
Brian C. Lane
991416adf5 Update ConfigParser usage for Py3
SafeConfigParser is now just ConfigParser
2018-08-07 09:22:18 -07:00
Brian C. Lane
fc9a469ca2 Update StringIO use for Py3
It is imported from io now.
2018-08-07 09:22:18 -07:00
Brian C. Lane
7379102923 Add a test for the pylorax.api.timestamp functions 2018-08-07 09:22:18 -07:00
Brian C. Lane
b4f9fb5c1f Fix write_timestamp for py3
toml.dumps() returns a string for write() so .encode is not needed.
2018-08-07 09:22:18 -07:00
Chris Lumens
8a2a43be99 Return a JSON error instead of a 404 on certain malformed URLs.
This handles the case where a route is requested, but without a required
parameter.  So, /blueprints/info is requested instead of
/blueprints/info/http-server.  It accomplishes this via a decorator, so
a lot of these route-related functions now have quite a few decorators
attached to them.

Typo'd URLs (/blueprints/nfo for instance) will still return a 404.  I
think this is a reasonable thing to do.

(cherry picked from commit 5daf2d416a)
2018-08-06 16:57:49 -07:00
Chris Lumens
76f714eab5 Return an error if /modules/info doesn't return anything.
Unfortunately, this isn't very useful if /modules/info is provided with
multiple modules.  yum doesn't traceback when doPackageLists is given
something that doesn't exist.  It just returns an empty list.  If
/modules/info is given just one module and yum gives us an empty list,
it's easy to say what happened.  If /modules/info is given several
modules and just one does not exist, we will not be able to detect that.

Fixing this would require doing more yum operations, which is likely to
slow things down and isn't the direction I want to be going.

(cherry picked from commit 8e948e4a4d)
2018-08-06 16:57:40 -07:00
Chris Lumens
ea7b22a8f5 Update documentation (#409).
(cherry picked from commit 6193a7c9d8)
2018-08-06 16:57:34 -07:00
Chris Lumens
022734ab01 Use constants instead of strings (#409).
(cherry picked from commit fdf4d63f3b)
2018-08-06 16:57:26 -07:00
Chris Lumens
495dcd13ec Write timestamps when important events happen during the compose (#409).
Right now, this is when the compose is queued up, when it is started by
anaconda, and when it is finished (whether that's success or not).

(cherry picked from commit 3ba9d53b8b)
2018-08-06 16:57:19 -07:00
Chris Lumens
e42b797295 Return multiple timestamps in API results (#409).
If one of the timestamps isn't present (for instance, the finished
timestamp for a job that is still running), null is returned.

(cherry picked from commit 17c40ef271)
2018-08-06 16:57:13 -07:00
Chris Lumens
ab76c6170e Add a new timestamp.py file to the API directory (#409).
This is responsible for writing out a new times.toml file, containing
important timestamps in the life of a compose.  This seems a little more
reliable than attempting to infer things from the filesystem, especially
in light of the fact that we can't ever really know when a file was
created.

(cherry picked from commit b59d59b124)
2018-08-06 16:57:07 -07:00
Brian C. Lane
ae19bc7343 Use the first enabled system repo for the test
(cherry picked from commit 3fd5e50c80)
2018-08-06 16:56:49 -07:00
Brian C. Lane
aec0e09caa Show more details when the system repo delete test fails
Works fine for me locally, but not in Travis for some reason so let's
get more info.

(cherry picked from commit 26df083bd4)
2018-08-06 16:56:03 -07:00
Brian C. Lane
18b03936c2 Add composer-cli function tests
These depend on there being a freshly installed lorax-composer API
server running, if there is no /run/weldr/api.socket they will be
skipped.

(cherry picked from commit 7700ae3135)
2018-08-06 16:53:56 -07:00
Brian C. Lane
2ce3752f90 Add a test library
This adds empty __init__.py to tests so that a lib.py library of helper
functions can be imported from the tests.

Add captured_output to use with composer-cli tests to capture stdout/err
output from the functions.

(cherry picked from commit eeae331ba0)
2018-08-06 16:53:48 -07:00