Commit Graph

1865 Commits

Author SHA1 Message Date
Brian C. Lane
48f7ad780d lorax: Only run depmod on the installed kernels
In the near-future there may be /lib/modules/ directories for older
kernels with weak dependencies listed. These may not match the installed
kernel(s) so we cannot depend on them to drive generate_module_data.

Instead use the existing findkernels() function to get the list of
installed kernels and iterate those, running depmod on them.

Resolves: rhbz#1622213
2018-10-09 14:51:05 -07:00
David Shea
2304a73676 Add virt guest agents to the qcow2 compose
(cherry picked from commit d5a1993640)
2018-10-02 12:56:56 -04:00
David Shea
de6e1d027e Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning

(cherry picked from commit 1056bfc25b)
2018-10-02 12:56:52 -04:00
David Shea
68c1a7aa96 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.

(cherry picked from commit e0c236ff36)
2018-10-02 12:56:47 -04:00
David Shea
f79fd46f1f Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.

(cherry picked from commit 18188bf6cf)
2018-10-02 12:56:33 -04:00
David Shea
09b34889bc Remove --fstype from the generated part line
Instead of specifying the fstype, just let anaconda use the default.

(cherry picked from commit 847fff4e11)
2018-10-02 12:56:03 -04:00
Brian C. Lane
7823d46747 New lorax documentation - 28.18 2018-09-06 10:41:38 -07:00
Brian C. Lane
6eb357e71e Fix /compose/cancel API documentation
It said /blueprints/cancel which is incorrect.
2018-08-29 10:08:15 -07:00
Brian C. Lane
a951bf083d Automatic commit of package [lorax] release [28.18-1].
Created by command:

/usr/bin/tito tag
2018-08-27 15:47:16 -07:00
Brian C. Lane
a286e9b3dc Fix composer-cli blueprints changes to get correct total
blueprints/changes is different, each blueprint has it's own total,
limited by the call's limit. So it needs to find the max total of all
the requested blueprints.

(cherry picked from commit 57674c9a1a)
2018-08-27 12:02:39 -07:00
Brian C. Lane
630df546e6 Fix blueprints/list and blueprints/changes to return the correct total
(cherry picked from commit ec908fcd2a)
2018-08-27 12:02:39 -07:00
Brian C. Lane
8838ee12bc Add tests for limit=0 routes
Passing ?limit=0 to the blueprints/list, blueprints/changes,
projects/list, modules/list should always return the total possible
results, not 0.

Also move the composer-cli test_diff to the end so that it will work
consistently. Do this by naming it test_z_diff.

(cherry picked from commit 972b5c4142)
2018-08-27 12:02:39 -07:00
Brian C. Lane
187cd935df Add a function to get_url_json_unlimited to retrieve the total
The blueprints/changes API is a bit different from the others, the total
that it includes is for each blueprint, not one total for all of them,
since there will be a different number of commits for each.

The function is passed the dict, and it can be used to select the total
to use for retrieving all of the results. If it isn't included it will
use data["total"] which works fine in most cases.

(cherry picked from commit 0a76d635ca)
2018-08-27 12:02:39 -07:00
Brian C. Lane
7773be2bda Fix tests related to blueprint name changes 2018-08-23 13:46:39 -07:00
Brian C. Lane
b198c6e498 Add 'example' to the example blueprint names 2018-08-23 13:45:32 -07:00
Brian C. Lane
9249d21d6f Use urllib.parse instead of urlparse
python3 moved this to a new module.

(cherry picked from commit 86d556e87c)
2018-08-21 16:50:53 -07:00
David Shea
e3ef3d9a29 In composer-cli, request all results
Add a limit argument to all potentially paginated results, equal to
whatever the composer backend is the total number of results. This still
has the potential to provide truncated data if the number of results
increases between the two HTTP requests.

Resolves: #404
(cherry picked from commit ee98d87cea)
2018-08-21 16:50:53 -07:00
David Shea
eedf1a9243 Add tests for /compose/status filter arguments
(cherry picked from commit c52ba4236a)
2018-08-21 16:50:53 -07:00
David Shea
bba0a85aaf Allow '*' as a uuid in /compose/status/<uuid>
This will display all UUIDs that match the filter arguments

(cherry picked from commit deff4d325f)
2018-08-21 16:50:53 -07:00
David Shea
5272be3127 Add filter arguments to /compose/status
This adds the following optional arguments to the /compose/status route:

  - type, matches the compose_type field
  - status, matches the queue_status field
  - blueprint, matches the blueprint field

(cherry picked from commit 40f23f093d)
2018-08-21 16:50:53 -07:00
Brian C. Lane
8694c7a7dc composer-cli should not log to a file by default
The user can pass --log /path/to/logfile.log if they want logging
enabled.

(cherry picked from commit 844ff9998b)
2018-08-21 11:59:12 -07:00
Brian C. Lane
93b207e752 Add documentation for using a DVD as the package source 2018-08-21 11:28:59 -07:00
Lars Karlitski
d55ff41870 Set TCP listen backlog for API socket to SOMAXCONN
A value of 1 is too low for heavy users of the API, such as the weldr-web
interface.

This is also systemd's default for sockets it opens. Using lorax-composer with
socket activation already results in a backlog of SOMAXCONN connections.

(cherry picked from commit be5d50e6f3)
2018-08-20 14:54:57 -07:00
Brian C. Lane
cef073d832 New lorax documentation - 28.17 2018-08-13 16:49:52 -07:00
Brian C. Lane
c2f96b3d3b Add a note about using lorax-composer.service
Update the docs to make it clear that .socket will be slow to respond to
the first request, and to use .service to speed it up.

(cherry picked from commit 83e5f3f885)
2018-08-13 16:47:12 -07:00
Brian C. Lane
64efb8b415 Ignore dnf.logging when building docs
(cherry picked from commit 07de4c371a)
2018-08-13 16:47:10 -07:00
Chris Lumens
312faf6275 Fix a little bug in running "modules list".
(cherry picked from commit 5dddef6f49)
2018-08-10 12:00:54 -04:00
Brian C. Lane
3a89828faf Automatic commit of package [lorax] release [28.17-1].
Created by command:

/usr/bin/tito tag
2018-08-09 16:38:21 -07:00
Brian C. Lane
77dc017c78 Fix bash_completion.d typo 2018-08-09 16:25:26 -07:00
Brian C. Lane
02dc3404a8 Move disklabel and UEFI support to compose.py
Currently we are making MBR disk images for qcow2 and partitioned disk,
so the UEFI packages aren't required at this point.

Move the clearpart command into compose.py so that in the futute it can
use clearpart --disklabel to create a GPT image, and add the required
packages to the package set.
2018-08-09 15:26:09 -07:00
Chris Lumens
a08c16158d Fix more tests.
These were broken due to me rebasing before sending a PR off.  They
should work now.

(cherry picked from commit 4cb15e0a0f)
2018-08-09 16:33:30 -04:00
Chris Lumens
a567157aa2 Change INVALID_NAME to INVALID_CHARS.
It's getting used in a whole lot more places now, so make it a bit more
generic sounding.

(cherry picked from commit bf0318eab6)
2018-08-09 16:33:26 -04:00
Chris Lumens
9c188b84a6 Update composer-cli for the new error return types.
(cherry picked from commit bc96f75992)
2018-08-09 16:33:19 -04:00
Chris Lumens
05e1a34687 Add default error IDs everywhere else.
The idea here is to make sure all return points have the same type for
the error cases.  There's not really all that many, so they just go in
one patch.  Some of these could potentially turn into more specialized
errors later.

(cherry picked from commit fd901c5e3f)
2018-08-09 16:33:06 -04:00
Chris Lumens
7c19be3792 Add error IDs to things that can go wrong when running a compose.
Note the exception string checking around compose_type.  I didn't really
want to introduce a new exception type just for this, but also didn't
want to duplicate strings.  I'd be open to other suggestions for how to
do this.

(cherry picked from commit b3bb438254)
2018-08-09 16:32:45 -04:00
Chris Lumens
23fe52dd70 Add error IDs for common source-related errors.
(cherry picked from commit e43adfc7af)
2018-08-09 16:32:40 -04:00
Chris Lumens
cb97c59079 Add error IDs for unknown modules and unknown projects.
(cherry picked from commit 2adcfc9563)
2018-08-09 16:32:36 -04:00
Chris Lumens
3a2716dec4 Add error IDs for when an unknown commit is requested.
(cherry picked from commit 07528a431e)
2018-08-09 16:32:27 -04:00
Chris Lumens
2a8a9bc506 Add error IDs for when an unknown blueprint is requested.
This adds some fairly redundant code to the beginning of all the
blueprint routes to attempt reading a commit from git for the
blueprint's recipe.  If it succeeds, the blueprint exists and the route
can continue.  Otherwise, return an error.  Hopefully this doesn't slow
things down too much.

(cherry picked from commit a925cc7ddb)
2018-08-09 16:32:22 -04:00
Chris Lumens
44b8b79af7 Add error IDs for when an unknown build UUID is requested.
Note that this also changes the return type of uuid_info to return None
when an unknown ID is given.  The other uuid_* functions are fine
because they are checked ahead of time.

(cherry picked from commit 6497b4fb65)
2018-08-09 16:32:05 -04:00
Chris Lumens
3e0ece3634 Add error IDs for bad state conditions.
(cherry picked from commit d76e24053a)
2018-08-09 16:31:34 -04:00
Chris Lumens
91c6c730f3 Change the error return type for bad limit= and offset=.
Each element in the errors value is now a dict, with a msg field and an
id field.  The id field contains a value out of errors.py that can be
used by the front end to key on.  The msg field is the same as what's
been there.

The idea is to keep the number of IDs somewhat limited so there's not a
huge number of things for the front end to know.

(cherry picked from commit 9677b012da)
2018-08-09 16:31:29 -04:00
Chris Lumens
31293f8460 Don't sort error messages.
This should make it easier to return more complex error structures.  It
also doesn't appear to matter - tests still pass without changes.

(cherry picked from commit 4c3f93e329)
2018-08-09 16:31:19 -04:00
Brian C. Lane
b4fa931eca Fix bash completion of compose info 2018-08-07 16:48:12 -07:00
Brian C. Lane
4307eebb49 Add + to the allowed API string character set 2018-08-07 16:48:12 -07:00
Brian C. Lane
716fb3277f Add job_* timestamp support to compose status 2018-08-07 15:21:22 -07:00
Will Woods
c44347ca77 Add etc/bash_completion.d/composer-cli
Here's a bash_completion file for composer-cli! It completes known
commands, subcommands, and flags, plus arguments to flags that take
arguments, and if you've got write access to the socket it'll also
autocomplete:

* source names for commands that need source names
* blueprint names for commands that take blueprint names
* compose uuids for commands that take compose uuids
  * intelligently only suggests appropriate uuids - e.g. only
    suggests uuids of running composes for "compose cancel"

NOTE: bash completion scripts are horrible and this is no exception.
Please forgive me.
2018-08-07 10:18:10 -07:00
Will Woods
3f8b9d7737 composer-cli: clean up "list" commands
This patch does two things:

1) Add "compose list", which lists compose UUIDs and other basic info,
2) Fix up "blueprints list", "modules list", "sources list", and
   "compose types" so their output is just a plain list of identifiers
2018-08-07 10:18:10 -07:00
Brian C. Lane
a034b205aa Drop .decode from UTF8_TEST_STRING 2018-08-07 10:07:39 -07:00
Brian C. Lane
2b0f9c4bd7 Add input string checks to the branch and format arguments
Make sure no UTF8 characters are allowed and return an error if they
are.

Also includes tests to make sure the correct error is returned.

(cherry picked from commit 86d79cd8a6)
2018-08-07 09:45:02 -07:00