Commit Graph

2098 Commits

Author SHA1 Message Date
Brian C. Lane
091820b5d6 New lorax documentation - 28.14.12
Related: rhbz#1635763
2018-10-05 09:54:44 -07:00
Brian C. Lane
b1a2d635d0 Adjust the composer-cli tests for the rename to composer
Related: rhbz#1635763
2018-10-05 08:30:34 -07:00
Lars Karlitski
2c2e3156d0 Rename composer-cli to composer
Resolves: rhbz#1635763
2018-10-05 08:30:34 -07:00
Lars Karlitski
43aedf6a15 Add and enable cloud-init for ami images
Images don't work at all on AWS without cloud-init.

Fixes #492

(cherry picked from commit 81d38b6445)

Related: rhbz#1628647
2018-10-05 11:28:17 -04:00
David Shea
c13aa84f02 Make no-virt generated images sparser
At the end of disk image installs, use fstrim on the generated filesystem to
discard any blocks that were allocated during the install and are now unused.
This will allow tools such as qemu-img to create images that do not include
deleted data.

For raw disk images that do not go through qemu-img, use fallocate --dig-holes
to create sparse holes in place of the unused blocks.

(cherry picked from commit 9717b3fd98)

Related: rhbz#1628645
Related: rhbz#1628646
Related: rhbz#1628647
Related: rhbz#1628648
2018-10-05 11:27:04 -04:00
Brian C. Lane
63a6e72a59 Automatic commit of package [lorax] release [28.14.12-1].
Created by command:

/usr/bin/tito tag
2018-10-03 15:55:10 -07:00
Brian C. Lane
212e0bcb3e Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:29:33 -07:00
Brian C. Lane
25ee5c9ee7 Automatic commit of package [lorax] release [28.14.11-1].
Created by command:

/usr/bin/tito tag
2018-10-02 17:01:23 -07:00
Brian C. Lane
de3294280b Add beakerlib to Dockerfile.test
Also kill the lorax-composer process and remove /run/weldr/api.socket
so that when this is run with podman you don't get an error about
attempting to tar up the socket.

Related: rhbz#1613058
2018-10-02 16:50:28 -07:00
Alexander Todorov
920aed16e2 New cli test covering basic compose commands
- need to specify --sharedir so lorax-composer can find its
  kickstart files

- each test script writes results into a separate directory to
  avoid a passing test overwriting the results from a failing one.
  To avoid reporting failures in case of previously failing tests
  (e.g. during development) remove the temporary directories holding
  tets results before execution!
2018-10-02 22:52:57 +02:00
Alexander Todorov
959589652b Execute bash tests for composer-cli
these are built on top of beakerlib and we use its internal
protocol to figure out the result without relying on the full
test runner that is tipically used inside of a RHEL environment!

Includes a disabled test snippet for Issue #460
2018-10-02 22:52:57 +02:00
David Shea
d94ed86cd3 Add virt guest agents to the qcow2 compose
(cherry picked from commit d5a1993640)

Resolves: rhbz#1628645
2018-10-02 13:06:59 -04:00
David Shea
1a23dc0f2b Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning

(cherry picked from commit 1056bfc25b)

Resolves: rhbz#1628646
2018-10-02 13:06:56 -04:00
David Shea
1c1f97ad70 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.

(cherry picked from commit e0c236ff36)

Resolves: rhbz#1628648
2018-10-02 13:06:10 -04:00
David Shea
253689ff49 Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.

(cherry picked from commit 18188bf6cf)

Resolves: rhbz#1628647
2018-10-02 13:05:40 -04:00
David Shea
b8a7774629 Remove --fstype from the generated part line
Instead of specifying the fstype, just let anaconda use the default.

(cherry picked from commit 847fff4e11)

Related: rhbz#1628647
Related: rhbz#1628648
2018-10-02 12:57:38 -04:00
Brian C. Lane
cec5f941bf Automatic commit of package [lorax] release [28.14.10-1].
Created by command:

/usr/bin/tito tag
2018-10-01 15:30:19 -07:00
Brian C. Lane
5eb9272c41 Add tito support for Related/Resolves to the branch
This will make it easier to generate the changelog, copied from
rhel7-branch.

Related: rhbz#1613058
2018-10-01 15:28:50 -07:00
Brian C. Lane
ca2eb38d92 Always update repo metadata when building an image
When the kickstart is handed off to Anaconda for building it will
download its own copy of the metadata and re-run the depsolve. So if the
dnf cache isn't current there will be a mismatch and the build will
fail to find some of the versions in final-kickstart.ks

This adds a new context to DNFLock, .lock_check, that will force a check
of the metadata. It also implements its own timeout and forces a
refresh of the metadata when that expires because the dnf expiration
doesn't always work as expected.

Resolves: rhbz#1631561
2018-10-01 14:50:11 -07:00
Brian C. Lane
31c1899a02 Add a test for repo metadata expiration
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
DNFLock.lock_check immediately picks up a new package.

This depends on rpmfluff which is available from Fedora or EPEL repos.

Related: rhbz#1631561
2018-10-01 14:47:49 -07:00
Brian C. Lane
e48bd5be96 Add tests for setting root password and ssh key with blueprints
Related: rhbz#1626120
2018-09-28 15:38:33 -07:00
Brian C. Lane
1c99408542 Use rootpw for setting the root password instead of user
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.

So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.

Related: rhbz#1626122
2018-09-28 15:38:02 -07:00
Brian C. Lane
85d7d3d01a Lock the root account, except on live-iso
If we leave the root account w/o a password people will use it that way,
leading to insecure images. Also if we use a default password. So lock
the root account in the templates.

Users will need to do one of these things:
 1. Use [[customizations.user]] in their blueprint to configure root or
    another user.
 2. Use [[customizations.sshkey]] to set a key for root
 2. Install a package that configures a user at install time
 3. Install a package that sets up a user at boot time (eg. cloud-init)

This also drops the auth line from the kickstart templates, allowing it
to use the default password algoritm instead of md5.

Resolves: rhbz#1626122
2018-09-28 15:34:58 -07:00
Brian C. Lane
546c8b5a62 Automatic commit of package [lorax] release [28.14.9-1].
Created by command:

/usr/bin/tito tag
2018-09-25 10:07:19 -07:00
Brian C. Lane
c355f0f203 lorax: Only run depmod on the installed kernels
In the near-future there may be /lib/modules/ directories for older
kernels with weak dependencies listed. These may not match the installed
kernel(s) so we cannot depend on them to drive generate_module_data.

Instead use the existing findkernels() function to get the list of
installed kernels and iterate those, running depmod on them.

Resolves: rhbz#1632140

(cherry picked from commit 07acd2e780)
2018-09-25 09:17:01 -07:00
Brian C. Lane
9cb0df6223 Automatic commit of package [lorax] release [28.14.8-1].
Created by command:

/usr/bin/tito tag
2018-09-18 10:25:47 -07:00
Brian C. Lane
551474ac8d Add prefixdevname support to the boot.iso
Resolves: rhbz#1623000
2018-09-17 16:19:07 -07:00
Brian C. Lane
281a005799 Automatic commit of package [lorax] release [28.14.7-1].
Created by command:

/usr/bin/tito tag
2018-09-04 11:24:53 -07:00
Brian C. Lane
a93a32a177 Ignore a pylint warning about UnquotingConfigParser get args
The args differ, but we are accepting and passing through all args so
it's ok.

Related: rhbz#1613058
2018-09-04 11:13:20 -07:00
Adam Williamson
d1aa8676ab Ditch all use of pyanaconda's simpleconfig
lorax uses pyanaconda's SimpleConfigParser in three different
places (twice with a copy that's been dumped into pylorax, once
by importing it), just to do a fairly simple job: read some
values out of /etc/os-release. The only value SimpleConfigParser
is adding over Python's own ConfigParser here is to read a file
with no section headers, and to unquote the values. The cost is
either a dependency on pyanaconda, or needing to copy the whole
of simpleparser plus some other utility bits from pyanaconda
into lorax. This seems like a bad trade-off.

This changes the approach: we copy one very simple utility
function from pyanaconda (`unquote`), and do some very simple
wrapping of ConfigParser to handle reading a file without any
section headers, and returning unquoted values. This way we can
read what we need out of os-release without needing a dep on
pyanaconda or to copy lots of things from it into pylorax.

Resolves: #449
Resolves: #450

Signed-off-by: Adam Williamson <awilliam@redhat.com>

Related: rhbz#1613058
2018-09-04 11:13:20 -07:00
Josh Boyer
97ae180677 Require python3-librepo
Newer python3-dnf no longer Requires: python3-librepo, but lorax
uses this.  Add an explicit Requires instead of getting it
transitively
2018-08-31 11:29:22 -04:00
Brian C. Lane
fc139f451a Automatic commit of package [lorax] release [28.14.6-1].
Created by command:

/usr/bin/tito tag
2018-08-29 15:54:31 -07:00
Brian C. Lane
2c28c85afe Drop mod_auth_openidc from httpd example blueprint
dnf 3.3.0 doesn't want to depsolve it, it isn't essential to the
blueprint, so drop it.

Related: rhbz#1613058
2018-08-29 10:48:02 -07:00
Brian C. Lane
8e27de29b2 Bump required dnf version to 3.2.0 for module_platform_id support 2018-08-29 10:48:02 -07:00
Brian C. Lane
3a9d39f397 Add support for DNF 3.2 module_platform_id config value
This borrows simpleconfig.py from Anaconda to make parsing os-release
easier.

It defaults to platform:el8
2018-08-29 10:48:02 -07:00
Brian C. Lane
a2375e870f Fix /compose/cancel API documentation
It said /blueprints/cancel which is incorrect.
2018-08-29 10:08:37 -07:00
Brian C. Lane
afb9ac2d59 Automatic commit of package [lorax] release [28.14.5-1].
Created by command:

/usr/bin/tito tag
2018-08-27 15:57:55 -07:00
Brian C. Lane
55d2911c7d Fix composer-cli blueprints changes to get correct total
blueprints/changes is different, each blueprint has it's own total,
limited by the call's limit. So it needs to find the max total of all
the requested blueprints.

(cherry picked from commit 57674c9a1a)
2018-08-27 12:09:27 -07:00
Brian C. Lane
d86c53527b Fix blueprints/list and blueprints/changes to return the correct total
(cherry picked from commit ec908fcd2a)
2018-08-27 12:09:27 -07:00
Brian C. Lane
fee0906118 Add tests for limit=0 routes
Passing ?limit=0 to the blueprints/list, blueprints/changes,
projects/list, modules/list should always return the total possible
results, not 0.

Also move the composer-cli test_diff to the end so that it will work
consistently. Do this by naming it test_z_diff.

(cherry picked from commit 972b5c4142)
2018-08-27 12:09:27 -07:00
Brian C. Lane
8e3d2b0252 Add a function to get_url_json_unlimited to retrieve the total
The blueprints/changes API is a bit different from the others, the total
that it includes is for each blueprint, not one total for all of them,
since there will be a different number of commits for each.

The function is passed the dict, and it can be used to select the total
to use for retrieving all of the results. If it isn't included it will
use data["total"] which works fine in most cases.

(cherry picked from commit 0a76d635ca)
2018-08-27 12:09:27 -07:00
Brian C. Lane
a567c614f1 Fix tests related to blueprint name changes 2018-08-22 10:22:36 -07:00
Brian C. Lane
d7d3dd57d6 Add 'example' to the example blueprint names 2018-08-22 10:02:34 -07:00
Brian C. Lane
a2b83bd771 Use urllib.parse instead of urlparse
python3 moved this to a new module.

(cherry picked from commit 86d556e87c)
2018-08-21 17:05:24 -07:00
David Shea
d6418246ad In composer-cli, request all results
Add a limit argument to all potentially paginated results, equal to
whatever the composer backend is the total number of results. This still
has the potential to provide truncated data if the number of results
increases between the two HTTP requests.

Resolves: #404
(cherry picked from commit ee98d87cea)
2018-08-21 17:05:24 -07:00
David Shea
42512fe560 Add tests for /compose/status filter arguments
(cherry picked from commit c52ba4236a)
2018-08-21 17:05:24 -07:00
David Shea
5057fef248 Allow '*' as a uuid in /compose/status/<uuid>
This will display all UUIDs that match the filter arguments

(cherry picked from commit deff4d325f)
2018-08-21 17:05:24 -07:00
David Shea
1938e8ef92 Add filter arguments to /compose/status
This adds the following optional arguments to the /compose/status route:

  - type, matches the compose_type field
  - status, matches the queue_status field
  - blueprint, matches the blueprint field

(cherry picked from commit 40f23f093d)
2018-08-21 17:05:24 -07:00
Brian C. Lane
792bf67ac7 Only include specific blueprints in the rpm
Related: rhbz#1613058
2018-08-21 12:30:05 -07:00
Brian C. Lane
d8750db166 composer-cli should not log to a file by default
The user can pass --log /path/to/logfile.log if they want logging
enabled.

(cherry picked from commit 844ff9998b)
2018-08-21 12:09:38 -07:00