Commit Graph

2104 Commits

Author SHA1 Message Date
Brian C. Lane
f80cc97a42 New lorax documentation - 30.5 2018-10-29 13:32:54 -07:00
Brian C. Lane
7500a17f27 Build manpages for composer-cli and lorax-composer
Add manpage creation to make docs target to keep them updated.
2018-10-29 13:31:30 -07:00
Marek Marczykowski-Górecki
27e611629f Add --squashfs-only option to drop inner rootfs.img layer
Make runtime directly into squashfs image. This reduces largely
unreproducible ext4 layer, but requires anaconda's dracut module
modification to properly mount the image.
2018-10-29 13:16:38 -07:00
Brian C. Lane
df70e3d677 Update php version to 7.3.* 2018-10-24 10:22:40 -07:00
Brian C. Lane
27aff75aa3 Update the projects tests to use DNF Repo object
Stop using fake dnf object, use the real thing. This will help catch
problems with dnf returning unexpected types like VectorString.
2018-10-24 10:22:40 -07:00
Brian C. Lane
e9e5139750 dnf changed the type of gpgkey to a tuple
And in an intermediate version it returns a VectorString object which
isn't serializable by the json or toml modules.

So convert it to a list so that the type is consistent in the sources
code.
2018-10-24 10:22:40 -07:00
Brian C. Lane
e1dc54ea8c Install python3-librepo in the test container 2018-10-19 14:43:29 -07:00
Brian C. Lane
5409748e75 lorax: Log when SOURCE_DATE_EPOCH is used for the current time 2018-10-19 14:43:29 -07:00
Marek Marczykowski-Górecki
fa2158c7a9 Drop non-determinism from default templates
Some files are created in non-reproducible way, including including
random data explicitly (/etc/machine-id), timestamps (fontconfig cache,
ldconfig aux-cache, certs cache), or entries in random order (groups,
systemd catalog, package list).
Fix this by either making the files reproducible, or removing them.
2018-10-19 14:43:29 -07:00
Marek Marczykowski-Górecki
de8124366e Use SOURCE_DATE_EPOCH for volumeid of efi boot image
By default mkfs.mksdos choose volume id based on current time. If
SOURCE_DATE_EPOCH is set, use that instead.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-19 14:43:29 -07:00
Marek Marczykowski-Górecki
e7f45d333f Preserve timestamps when building fs image
Even when FS do not support owner/modes, preserve timestamps.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-19 14:43:29 -07:00
Marek Marczykowski-Górecki
876ec52215 Use SOURCE_DATE_EPOCH for metadata timestamps
This include .buildinfo, .treeinfo and .discinfo.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-19 14:43:29 -07:00
Brian C. Lane
4e0c7b3fba Automatic commit of package [lorax] release [30.5-1].
Created by command:

/usr/bin/tito tag
2018-10-12 15:56:25 -07:00
Brian C. Lane
7e0a288f5b Update depsolving with suggestions from dnf (#1636239)
The previous method worked, but wasn't exactly idiomatic. This is more
correct, and appears to work the same (templates depsolve, version globs
work, multiple repos work).

Note that this does use a private dnf attribute ._goal, but the word is
that this is going to become a public api soon, so yes it is there on
purpose.
2018-10-12 12:00:25 -07:00
Brian C. Lane
faa65bca3c Disable false context-manager pylint error 2018-10-12 12:00:25 -07:00
Brian C. Lane
d7951e3a10 Fix directory creation for blueprints
Depending on how lorax-composer is run setting up an empty blueprints
directory can fail. So this moves checking/creation until after the
other directories are created and uses make_owned_dir to make sure
ownership is correct.
2018-10-12 11:59:32 -07:00
Brian C. Lane
75644689de Update the tests for new make_dnf_dir arguments.
Use the uid and gid that the test is running as instead of hard-coding
0.
2018-10-12 11:59:32 -07:00
Brian C. Lane
65b769984b Change make_dnf_dirs to be run as root
It needs to be root in order to set the ownership and permissions on the
directories that are under /var/lib/lorax/composer/

Refactor the directory creation into a utility function, and use a umask
of 0o006 to ensure that the parent directories created do not have o+rw
set on them (makedirs behavior is different between Python 3.6 and 3.7
so umask of 0 doesn't work consistently).
2018-10-12 11:59:32 -07:00
Vendula Poncova
aafa271608 Fix vhd images
Images don't work without these fixes:

* Enable Network Manager.
* Disable cloud-init.
* Add Hyper-V modules into initramfs.

Fixes specific for RHEL:

* Create ifcfg-eth0 required by waagent.
* Install python3 and net-tools required by waagent.

Recommended changes:

* Use recommended kernel boot args.
* Disable kdump.
2018-10-12 11:44:03 -07:00
Brian C. Lane
7cc6e86299 Automatic commit of package [lorax] release [30.4-1].
Created by command:

/usr/bin/tito tag
2018-10-09 11:26:49 -07:00
Brian C. Lane
98f8b23129 Add an openstack image type
This is a qcow2 image with cloud-init in the template.
2018-10-09 10:17:14 -07:00
David Shea
ac9a2fdbc5 Add cloud-init to vhd images.
cloud-init can be used in Azure now
2018-10-09 11:33:53 -04:00
David Shea
6fab72d894 Replace /etc/machine-id with an empty file
Since these images can be used to create multiple machines, they should
not have a unique machine-id attached to them. Replace /etc/machine-id
with an empty file so that it will be regenerated at boot time.
2018-10-09 11:33:53 -04:00
Brian C. Lane
69a2835387 Automatic commit of package [lorax] release [30.3-1].
Created by command:

/usr/bin/tito tag
2018-10-08 15:38:59 -07:00
Brian C. Lane
2a7a323ddf Update cli tests to use composer-cli name 2018-10-08 09:48:34 -07:00
Brian C. Lane
c017300ad6 Revert "Rename composer-cli to composer"
This reverts commit c9be91c879.

It conflicts with the PHP dependency manager project named 'composer'
2018-10-08 09:23:47 -07:00
Brian C. Lane
9087de3c86 Automatic commit of package [lorax] release [30.2-1].
Created by command:

/usr/bin/tito tag
2018-10-05 14:41:45 -07:00
Brian C. Lane
3b41faae2c Work around dnf problem with multiple repos
If a package is in multiple repos dnf may return more than 1 of them
when using best...glob so we pick the highest NEVRA one and install
that.

Related: rhbz#1636239
2018-10-05 11:23:41 -07:00
Lars Karlitski
81d38b6445 Add and enable cloud-init for ami images
Images don't work at all on AWS without cloud-init.

Fixes #492
2018-10-05 11:24:45 -04:00
David Shea
9717b3fd98 Make no-virt generated images sparser
At the end of disk image installs, use fstrim on the generated filesystem to
discard any blocks that were allocated during the install and are now unused.
This will allow tools such as qemu-img to create images that do not include
deleted data.

For raw disk images that do not go through qemu-img, use fallocate --dig-holes
to create sparse holes in place of the unused blocks.
2018-10-05 11:24:18 -04:00
Brian C. Lane
0086362f0b New lorax documentation - 30.1 2018-10-04 16:55:20 -07:00
Brian C. Lane
c7b0644ee5 Automatic commit of package [lorax] release [30.1-1].
Created by command:

/usr/bin/tito tag
2018-10-03 16:54:30 -07:00
Brian C. Lane
6da3079349 Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
2018-10-03 16:41:49 -07:00
Lars Karlitski
6a37f80f0f cli: Clarify error message for unprivileged access
`os.path.exists("/run/weldr/api.socket")` returns False for users which have no
access. This leads to composer printing that the file does not exist, which is
misleading.

Since it's no possible to distinguish the two cases, fix this problem by
combining them and showing a single error message.
2018-10-03 16:36:51 -07:00
Brian C. Lane
e7bfab8b4a Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:30:31 -07:00
Brian C. Lane
edc5490a99 Automatic commit of package [lorax] release [30.0-1].
Created by command:

/usr/bin/tito tag --keep-version
2018-10-02 16:41:19 -07:00
Brian C. Lane
acedb3a0ed Add beakerlib to Dockerfile.test
Also kill the lorax-composer process and remove /run/weldr/api.socket
so that when this is run with podman you don't get an error about
attempting to tar up the socket.
2018-10-02 16:23:42 -07:00
Brian C. Lane
8ed443ff58 Adjust the new templates for locked root
also remote the auth line so that it uses the defaults.

Related: rhbz#1628645
Related: rhbz#1628646
Related: rhbz#1628647
Related: rhbz#1628648
2018-10-02 14:32:59 -07:00
Brian C. Lane
e4c098b7e0 Adjust projects test for DNF 3.6.1 tuple issue 2018-10-02 14:31:30 -07:00
Adam Williamson
37f264c010 Don't try to append to DNF config value that can't take it
See https://bugzilla.redhat.com/show_bug.cgi?id=1595917 and
https://github.com/rpm-software-management/dnf/pull/1200 for
more on this. Briefly, DNF before 3.0 presented this config
value as a list...and mutating it worked. DNF from 3.0 until
3.6 presented it as a list...mutating it didn't work, but also
didn't *fail*, so this has actually not been doing anything on
DNF 3.x but we haven't noticed.

In DNF 3.6 values like this are presented as tuples instead of
lists, to try and catch usages like this, and it worked! We
need to change this one.

There is an additional weirdness here. tsflags is actually, in
libdnf terms, an OptionStringListAppend option: that means that
when something tries to *set* its value, the new value is just
appended to the existing list of values. This is very weird
behaviour when you're interacting with it like this, but
happens to be quite useful, as we can just 'set' the value to
a list like this and it will actually get appended (which is
what we want), and this one syntax happens to work correctly in
DNF 2.x, 3.0 through 3.5.1, and 3.6.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-02 14:07:06 -07:00
Brian C. Lane
51c73f7570 Always update repo metadata when building an image
When the kickstart is handed off to Anaconda for building it will
download its own copy of the metadata and re-run the depsolve. So if the
dnf cache isn't current there will be a mismatch and the build will
fail to find some of the versions in final-kickstart.ks

This adds a new context to DNFLock, .lock_check, that will force a check
of the metadata. It also implements its own timeout and forces a
refresh of the metadata when that expires because the dnf expiration
doesn't always work as expected.

Resolves: rhbz#1631561
2018-10-02 13:59:42 -07:00
Brian C. Lane
7e31a2c138 Add a test for repo metadata expiration
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
DNFLock.lock_check immediately picks up a new package.

This depends on rpmfluff which is available from Fedora or EPEL repos.

Related: rhbz#1631561
2018-10-02 13:59:42 -07:00
Brian C. Lane
ef8bc70cdd Add tests for setting root password and ssh key with blueprints
Related: rhbz#1626120
2018-10-02 13:59:42 -07:00
Brian C. Lane
b2eadff77c Use rootpw for setting the root password instead of user
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.

So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.

Related: rhbz#1626122
2018-10-02 13:59:42 -07:00
Brian C. Lane
f360ef9216 Lock the root account, except on live-iso
If we leave the root account w/o a password people will use it that way,
leading to insecure images. Also if we use a default password. So lock
the root account in the templates.

Users will need to do one of these things:
 1. Use [[customizations.user]] in their blueprint to configure root or
    another user.
 2. Use [[customizations.sshkey]] to set a key for root
 2. Install a package that configures a user at install time
 3. Install a package that sets up a user at boot time (eg. cloud-init)

This also drops the auth line from the kickstart templates, allowing it
to use the default password algoritm instead of md5.

Resolves: rhbz#1626122
2018-10-02 13:59:42 -07:00
David Shea
003108b15f Add new compose types to compose sanity test 2018-10-02 11:54:41 -04:00
David Shea
d5a1993640 Add virt guest agents to the qcow2 compose 2018-10-02 11:54:41 -04:00
David Shea
1056bfc25b Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning
2018-10-02 11:54:41 -04:00
David Shea
e0c236ff36 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.
2018-10-02 11:54:41 -04:00
David Shea
18188bf6cf Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
2018-10-02 11:54:38 -04:00