Commit Graph

1714 Commits

Author SHA1 Message Date
David Shea
4862eccd88 Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning

(cherry picked from commit 1056bfc25b)

Resolves: rhbz#1656105
2018-12-07 13:59:45 -05:00
David Shea
202814ae9d Add a vhd compose type for Azure images
This is similar to the AMI compose type, with a handful of additional
changes specific to Azure:

* Add waagent (but leave NetworkManager enabled, despite some of the
  docs)
* Disable cloud-init
* Add Hyper-V modules into initrams.

Fixes specific for RHEL:

* Create ifcfg-eth0 required by waagent.
* Install python3 and net-tools required by waagent.

Recommended changes:

* Use recommended kernel boot args.
* Disable kdump.

(cherry picked from commit e0c236ff36)
(cherry picked from commit da0435bc90)
(cherry picked from commit b594fa99bc)

Resolves: rhbz#1656105
2018-12-07 13:59:45 -05:00
David Shea
61d4353bc7 Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.

(cherry picked from commit 18188bf6cf)
(cherry picked from commit 81d38b6445)

Resolves: rhbz#1656105
2018-12-07 13:59:45 -05:00
David Shea
1ca0393771 Remove --fstype from the generated part line
Instead of specifying the fstype, just let anaconda use the default.

(cherry picked from commit 847fff4e11)

Related: rhbz#1656105
2018-12-07 13:59:45 -05:00
Brian C. Lane
60c7ab3e91 lorax-composer: Install selinux-policy-targeted in images
This is required to ensure that SELinux is configured properly while
building. It fixes the problem with building tar, and should be
installed in the other image types for consistency.

Resolves: rhbz#1654795
2018-12-06 16:36:36 -08:00
Brian C. Lane
6b17850739 Remove setfiles from mkrootfsimage
SELinux applies the correct labels, setfiles is no longer needed.
This allows lorax to run with SELinux in Enforcing mode.

Resolves: rhbz#1654795
2018-12-06 16:36:24 -08:00
Brian C. Lane
482d6277e0 Remove SELinux Permissive checks
Anaconda, Lorax, lorax-composer, and livemedia-creator can all now run
with SELinux in Enforcing mode. It does not need to be disabled and if
there are denials they should be reported as a bug.

Log the current state of SELinux when starting, update the
documentation.

Resolves: rhbz#1654795
2018-12-06 16:36:03 -08:00
Brian C. Lane
a40fc22047 Automatic commit of package [lorax-composer] release [19.7.27-1].
Created by command:

/usr/bin/tito tag
2018-10-22 12:03:23 -07:00
Brian C. Lane
bc79b636bb Use matchPackageNames instead of searchNames
Some kickstart templates use globbing to match packages, searchNames
doesn't support this.

Add fallback code to find packages that are just a dep, matching the
behavior of yum.install()

This fixes things so that it can depsolve package names with globs in
them like grub2-efi-*-cdboot, and deps like shim and grub2-efi.

Results are still filtered by the version glob, with the highest NEVRA
selected for installation.

Resolves: rhbz#1641601
2018-10-22 11:03:40 -07:00
Brian C. Lane
e167e5d09f Automatic commit of package [lorax-composer] release [19.7.26-1].
Created by command:

/usr/bin/tito tag
2018-10-08 17:04:23 -07:00
Brian C. Lane
250e68840b Revert "Rename composer-cli to composer"
This reverts commit b8da0a8c8e.

It conflicts with the PHP dependency manager project named 'composer'

Related: rhbz#1635760
2018-10-08 16:55:57 -07:00
Brian C. Lane
b2f94675ce Automatic commit of package [lorax-composer] release [19.7.25-1].
Created by command:

/usr/bin/tito tag
2018-10-05 09:02:50 -07:00
Lars Karlitski
b8da0a8c8e Rename composer-cli to composer
Resolves: rhbz#1635760
2018-10-03 10:43:01 -07:00
Brian C. Lane
a7ab87832b Automatic commit of package [lorax-composer] release [19.7.24-1].
Created by command:

/usr/bin/tito tag
2018-10-01 13:50:24 -07:00
Brian C. Lane
0f130dd91c Add a test for repo metadata expiration
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
YumLock.lock_check immediately picks up a new package.

This depends on rpmfluff which is available from Fedora or EPEL repos.

Related: rhbz#1632962
2018-10-01 13:37:44 -07:00
Brian C. Lane
c9582a0468 Create a new YumBase object when repodata changes
The problem this solves is that yum really isn't designed to be part of\
a long running daemon. So when repodata changes upstream, even when
you force it to download the new metadata, it doesn't change in memory
so you end up with lorax-composer depsolving against old versions, and
anaconda depsolving against new versions (because it sets up its own
YumBase and cache) and then the kickstart is no longer valid.

To solve this I have
 - Added a 6h timeout to the metadata check (because yum's doesn't work
   in this situation).
 - Added a metadata check to the YumLock .lock property, but only when
   the timeout expires.
 - Added a new .lock_check property to YumLock that always checks the
   metadata and resets the timeout.

If it has changed it does its best to tear down the existing YumBase,
deleting as much as it can in hopes it doesn't leak memory. And then it
sets up a totally new YumBase with the new repodata.

Resolves: rhbz#1632962
2018-10-01 13:37:44 -07:00
Brian C. Lane
6fd0e71530 Fix projects_depsolve_with_size version globbing
Use a common _depsolve function for projects_depsolve and
projects_depsolve_with_size so that it always uses the correct version
glob support when depsolving blueprints and templates.

Resolves: rhbz#1628114
2018-10-01 11:09:20 -07:00
Brian C. Lane
f8f49159ad Add a version glob test forprojects_depsolve_with_size
Resolves: rhbz#1628114
2018-10-01 11:09:20 -07:00
Brian C. Lane
956b29eb64 Add tests for setting root password and ssh key with blueprints
Related: rhbz#1626120
2018-09-28 15:21:11 -07:00
Brian C. Lane
6e92d52c14 Use rootpw for setting the root password instead of user
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.

So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.

Related: rhbz#1626120
2018-09-28 14:30:58 -07:00
Brian C. Lane
f113766f22 Lock the root account, except on live-iso
If we leave the root account w/o a password people will use it that way,
leading to insecure images. Also if we use a default password. So lock
the root account in the templates.

Users will need to do one of these things:
 1. Use [[customizations.user]] in their blueprint to configure root or
    another user.
 2. Use [[customizations.sshkey]] to set a key for root
 2. Install a package that configures a user at install time
 3. Install a package that sets up a user at boot time (eg. cloud-init)

This also drops the auth line from the kickstart templates, allowing it
to use the default password algoritm instead of md5.

Resolves: rhbz#1626120
2018-09-28 14:30:58 -07:00
Brian C. Lane
2876211b14 Automatic commit of package [lorax-composer] release [19.7.23-1].
Created by command:

/usr/bin/tito tag
2018-09-19 08:30:34 -07:00
Brian C. Lane
2a85694c9b Fix depsolve version globbing
The blueprint version glob was being applied to the whole package NEVRA
by yum (it lacks a separate API for just globbing versions), so this
implements that in filterVersionGlob using fnmatchcase on the package
names, and the yum package verGT comparison on the versions for the
selected package.

Also includes tests.

Resolves: rhbz#1628114
2018-09-19 08:19:59 -07:00
Brian C. Lane
9685fdd7aa Fix /compose/cancel API documentation
It said /blueprints/cancel which is incorrect.
2018-08-29 10:06:25 -07:00
Brian C. Lane
b9afb2eb6b Automatic commit of package [lorax-composer] release [19.7.22-1].
Created by command:

/usr/bin/tito tag
2018-08-27 15:06:25 -07:00
Brian C. Lane
fd54205a5f Fix composer-cli blueprints changes to get correct total
blueprints/changes is different, each blueprint has it's own total,
limited by the call's limit. So it needs to find the max total of all
the requested blueprints.
2018-08-23 16:21:54 -07:00
Brian C. Lane
d17cb66195 Fix blueprints/list and blueprints/changes to return the correct total 2018-08-23 16:21:00 -07:00
Brian C. Lane
2e6f5fdf71 Add tests for limit=0 routes
Passing ?limit=0 to the blueprints/list, blueprints/changes,
projects/list, modules/list should always return the total possible
results, not 0.

Also move the composer-cli test_diff to the end so that it will work
consistently. Do this by naming it test_z_diff.
2018-08-23 16:17:33 -07:00
Brian C. Lane
e2c6ae906d Add a function to get_url_json_unlimited to retrieve the total
The blueprints/changes API is a bit different from the others, the total
that it includes is for each blueprint, not one total for all of them,
since there will be a different number of commits for each.

The function is passed the dict, and it can be used to select the total
to use for retrieving all of the results. If it isn't included it will
use data["total"] which works fine in most cases.
2018-08-23 15:29:26 -07:00
Brian C. Lane
8dd6096f5e Fix tests related to blueprint name changes 2018-08-21 15:08:58 -07:00
Brian C. Lane
3a3f358f9b Add 'example' to the example blueprint names 2018-08-21 14:02:30 -07:00
Brian C. Lane
c51000ce60 Don't include glusterfs.toml as an example blueprint 2018-08-21 12:25:57 -07:00
Brian C. Lane
a00b009f77 Add a pylorax.api.version number
lorax-composer is in a different package than lorax, so we need to use
its version number not lorax's when reporting the API /status/
2018-08-21 12:10:01 -07:00
Brian C. Lane
d0c4524253 composer-cli should not log to a file by default
The user can pass --log /path/to/logfile.log if they want logging
enabled.
2018-08-21 11:30:22 -07:00
Brian C. Lane
773613bc80 Add documentation for using a DVD as the package source 2018-08-21 11:27:55 -07:00
Lars Karlitski
a980309744 Set TCP listen backlog for API socket to SOMAXCONN
A value of 1 is too low for heavy users of the API, such as the weldr-web
interface.

This is also systemd's default for sockets it opens. Using lorax-composer with
socket activation already results in a backlog of SOMAXCONN connections.

(cherry picked from commit be5d50e6f3)
2018-08-20 15:23:53 -07:00
Brian C. Lane
3ea2946e08 Add a note about using lorax-composer.service
Update the docs to make it clear that .socket will be slow to respond to
the first request, and to use .service to speed it up.

(cherry picked from commit 83e5f3f885)
(cherry picked from commit c2f96b3d3b)
2018-08-13 16:58:15 -07:00
David Shea
c0c135426e In composer-cli, request all results
Add a limit argument to all potentially paginated results, equal to
whatever the composer backend is the total number of results. This still
has the potential to provide truncated data if the number of results
increases between the two HTTP requests.

Resolves: #404
2018-08-13 10:36:32 -04:00
Brian C. Lane
a81000529d Fix bash_completion.d typo 2018-08-10 14:35:38 -07:00
Chris Lumens
5dddef6f49 Fix a little bug in running "modules list". 2018-08-10 11:59:42 -04:00
David Shea
587f05e551 Add tests for /compose/status filter arguments 2018-08-10 10:18:27 -04:00
David Shea
c71c196157 Allow '*' as a uuid in /compose/status/<uuid>
This will display all UUIDs that match the filter arguments
2018-08-10 10:18:27 -04:00
David Shea
1b6e6254fe Add filter arguments to /compose/status
This adds the following optional arguments to the /compose/status route:

  - type, matches the compose_type field
  - status, matches the queue_status field
  - blueprint, matches the blueprint field
2018-08-10 10:18:27 -04:00
Brian C. Lane
fb85c841be Automatic commit of package [lorax-composer] release [19.7.21-1].
Created by command:

/usr/bin/tito tag
2018-08-09 15:46:45 -07:00
Brian C. Lane
acc7b811b6 Move disklabel and UEFI support to compose.py
Currently we are making MBR disk images for qcow2 and partitioned disk,
so the UEFI packages aren't required at this point.

Move the clearpart command into compose.py so that in the futute it can
use clearpart --disklabel to create a GPT image, and add the required
packages to the package set.
2018-08-09 15:13:58 -07:00
Chris Lumens
4cb15e0a0f Fix more tests.
These were broken due to me rebasing before sending a PR off.  They
should work now.
2018-08-09 16:11:32 -04:00
Chris Lumens
bf0318eab6 Change INVALID_NAME to INVALID_CHARS.
It's getting used in a whole lot more places now, so make it a bit more
generic sounding.
2018-08-09 16:11:32 -04:00
Chris Lumens
bc96f75992 Update composer-cli for the new error return types. 2018-08-09 16:11:32 -04:00
Chris Lumens
fd901c5e3f Add default error IDs everywhere else.
The idea here is to make sure all return points have the same type for
the error cases.  There's not really all that many, so they just go in
one patch.  Some of these could potentially turn into more specialized
errors later.
2018-08-09 16:11:32 -04:00
Chris Lumens
b3bb438254 Add error IDs to things that can go wrong when running a compose.
Note the exception string checking around compose_type.  I didn't really
want to introduce a new exception type just for this, but also didn't
want to duplicate strings.  I'd be open to other suggestions for how to
do this.
2018-08-09 16:11:32 -04:00