Commit Graph

1927 Commits

Author SHA1 Message Date
Brian C. Lane
3b41faae2c Work around dnf problem with multiple repos
If a package is in multiple repos dnf may return more than 1 of them
when using best...glob so we pick the highest NEVRA one and install
that.

Related: rhbz#1636239
2018-10-05 11:23:41 -07:00
Lars Karlitski
81d38b6445 Add and enable cloud-init for ami images
Images don't work at all on AWS without cloud-init.

Fixes #492
2018-10-05 11:24:45 -04:00
David Shea
9717b3fd98 Make no-virt generated images sparser
At the end of disk image installs, use fstrim on the generated filesystem to
discard any blocks that were allocated during the install and are now unused.
This will allow tools such as qemu-img to create images that do not include
deleted data.

For raw disk images that do not go through qemu-img, use fallocate --dig-holes
to create sparse holes in place of the unused blocks.
2018-10-05 11:24:18 -04:00
Brian C. Lane
0086362f0b New lorax documentation - 30.1 2018-10-04 16:55:20 -07:00
Brian C. Lane
c7b0644ee5 Automatic commit of package [lorax] release [30.1-1].
Created by command:

/usr/bin/tito tag
2018-10-03 16:54:30 -07:00
Brian C. Lane
6da3079349 Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
2018-10-03 16:41:49 -07:00
Lars Karlitski
6a37f80f0f cli: Clarify error message for unprivileged access
`os.path.exists("/run/weldr/api.socket")` returns False for users which have no
access. This leads to composer printing that the file does not exist, which is
misleading.

Since it's no possible to distinguish the two cases, fix this problem by
combining them and showing a single error message.
2018-10-03 16:36:51 -07:00
Brian C. Lane
e7bfab8b4a Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:30:31 -07:00
Brian C. Lane
edc5490a99 Automatic commit of package [lorax] release [30.0-1].
Created by command:

/usr/bin/tito tag --keep-version
2018-10-02 16:41:19 -07:00
Brian C. Lane
acedb3a0ed Add beakerlib to Dockerfile.test
Also kill the lorax-composer process and remove /run/weldr/api.socket
so that when this is run with podman you don't get an error about
attempting to tar up the socket.
2018-10-02 16:23:42 -07:00
Brian C. Lane
8ed443ff58 Adjust the new templates for locked root
also remote the auth line so that it uses the defaults.

Related: rhbz#1628645
Related: rhbz#1628646
Related: rhbz#1628647
Related: rhbz#1628648
2018-10-02 14:32:59 -07:00
Brian C. Lane
e4c098b7e0 Adjust projects test for DNF 3.6.1 tuple issue 2018-10-02 14:31:30 -07:00
Adam Williamson
37f264c010 Don't try to append to DNF config value that can't take it
See https://bugzilla.redhat.com/show_bug.cgi?id=1595917 and
https://github.com/rpm-software-management/dnf/pull/1200 for
more on this. Briefly, DNF before 3.0 presented this config
value as a list...and mutating it worked. DNF from 3.0 until
3.6 presented it as a list...mutating it didn't work, but also
didn't *fail*, so this has actually not been doing anything on
DNF 3.x but we haven't noticed.

In DNF 3.6 values like this are presented as tuples instead of
lists, to try and catch usages like this, and it worked! We
need to change this one.

There is an additional weirdness here. tsflags is actually, in
libdnf terms, an OptionStringListAppend option: that means that
when something tries to *set* its value, the new value is just
appended to the existing list of values. This is very weird
behaviour when you're interacting with it like this, but
happens to be quite useful, as we can just 'set' the value to
a list like this and it will actually get appended (which is
what we want), and this one syntax happens to work correctly in
DNF 2.x, 3.0 through 3.5.1, and 3.6.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-02 14:07:06 -07:00
Brian C. Lane
51c73f7570 Always update repo metadata when building an image
When the kickstart is handed off to Anaconda for building it will
download its own copy of the metadata and re-run the depsolve. So if the
dnf cache isn't current there will be a mismatch and the build will
fail to find some of the versions in final-kickstart.ks

This adds a new context to DNFLock, .lock_check, that will force a check
of the metadata. It also implements its own timeout and forces a
refresh of the metadata when that expires because the dnf expiration
doesn't always work as expected.

Resolves: rhbz#1631561
2018-10-02 13:59:42 -07:00
Brian C. Lane
7e31a2c138 Add a test for repo metadata expiration
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
DNFLock.lock_check immediately picks up a new package.

This depends on rpmfluff which is available from Fedora or EPEL repos.

Related: rhbz#1631561
2018-10-02 13:59:42 -07:00
Brian C. Lane
ef8bc70cdd Add tests for setting root password and ssh key with blueprints
Related: rhbz#1626120
2018-10-02 13:59:42 -07:00
Brian C. Lane
b2eadff77c Use rootpw for setting the root password instead of user
Ends up you cannot use the kickstart user command on root, since it
already exists, so we have to translate that into a rootpw command.

So [[customizations.user]] with name = "root" only support key, which
will set the ssh key, and password which will use rootpw to set the
password. plain text or encrypted are supported.

Related: rhbz#1626122
2018-10-02 13:59:42 -07:00
Brian C. Lane
f360ef9216 Lock the root account, except on live-iso
If we leave the root account w/o a password people will use it that way,
leading to insecure images. Also if we use a default password. So lock
the root account in the templates.

Users will need to do one of these things:
 1. Use [[customizations.user]] in their blueprint to configure root or
    another user.
 2. Use [[customizations.sshkey]] to set a key for root
 2. Install a package that configures a user at install time
 3. Install a package that sets up a user at boot time (eg. cloud-init)

This also drops the auth line from the kickstart templates, allowing it
to use the default password algoritm instead of md5.

Resolves: rhbz#1626122
2018-10-02 13:59:42 -07:00
David Shea
003108b15f Add new compose types to compose sanity test 2018-10-02 11:54:41 -04:00
David Shea
d5a1993640 Add virt guest agents to the qcow2 compose 2018-10-02 11:54:41 -04:00
David Shea
1056bfc25b Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning
2018-10-02 11:54:41 -04:00
David Shea
e0c236ff36 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.
2018-10-02 11:54:41 -04:00
David Shea
18188bf6cf Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
2018-10-02 11:54:38 -04:00
David Shea
847fff4e11 Remove --fstype from the generated part line
Instead of specifying the fstype, just let anaconda use the default.
2018-09-28 15:27:15 -04:00
Lars Karlitski
e33b1e1636 Also run make check on travis 2018-09-25 13:49:40 +03:00
Lars Karlitski
160044ba9d Fix pylint errors and warnings
Remove `except` block which immediately raises the same exception again (it's
not a subclass of another caught exception, so this is safe).

Remove a false positive, because it is not emitted from the code base.

Disable subprocess-popen-preexec-fn in startProgram, which is not used
internally.
2018-09-25 13:49:40 +03:00
Alexander Todorov
9736dff762 New cli test covering basic compose commands
- need to specify --sharedir so lorax-composer can find its
  kickstart files

- each test script writes results into a separate directory to
  avoid a passing test overwriting the results from a failing one.
  To avoid reporting failures in case of previously failing tests
  (e.g. during development) remove the temporary directories holding
  tets results before execution!
2018-09-24 22:28:57 +03:00
Alexander Todorov
c97a6985fb Update glusterfs to 5.*
otherwise depsolve fails b/c rawhide repositories don't seem to
contain the 4.1.* versions
2018-09-24 22:28:57 +03:00
Alexander Todorov
8e0b7ba2ca Execute bash tests for composer-cli
these are built on top of beakerlib and we use its internal
protocol to figure out the result without relying on the full
test runner that is tipically used inside of a RHEL environment!

Includes a disabled test snippet for Issue #460
2018-09-20 16:07:57 +03:00
Lars Karlitski
c9be91c879 Rename composer-cli to composer 2018-09-20 12:02:03 +03:00
Brian C. Lane
3246c01443 Include python3-pyatspi on boot.iso (#1506595)
Including this makes it easier for dogtail tests to be run on the
various releases.
2018-09-19 09:04:29 -07:00
Stef Walter
e36b89e9e8 Start a HACKING.md file and document how to run the tests 2018-09-19 08:47:23 -07:00
Stef Walter
4a66b5b085 tests: Fix tests so they run on Fedora 28
Broaden the match for Samba in the glusterfs blueprint so it
can work on Fedora 28
2018-09-19 08:47:23 -07:00
Stef Walter
78717054d7 Ignore files created by tests 2018-09-19 08:47:23 -07:00
Stef Walter
ec1ebbc623 Makefile: Fix the 'make install' target
This fixes the 'make install' target to work on a typical RHEL or
Fedora system. We now by default install to a prefix of /usr instead
of /usr/local

The prefix is overridable like so:

     $ make install PREFIX=/opt/
2018-09-19 08:47:08 -07:00
Akira TAGOH
ab85ad5c86 Replace CJK fonts with Google Noto CJK 2018-09-18 09:12:09 -07:00
David Shea
c00036251e Fix a DeprecationWarning
SafeConfigParser is just a deprecated version of ConfigParser in
python3, so use ConfigParser.
2018-09-07 13:34:30 -04:00
David Shea
b4b8985caa Fix the expected versions of blueprint components 2018-09-07 13:34:30 -04:00
David Shea
a65b93c77d Switch the test container back to rawhide
Use fedora:rawhide again to run tests, but pull the test image from the
Fedora docker registry instead of the Docker docker registry.
2018-09-07 13:34:30 -04:00
Brian C. Lane
e129c4c4f5 Automatic commit of package [lorax] release [29.15-1].
Created by command:

/usr/bin/tito tag
2018-09-07 10:05:39 -07:00
Brian C. Lane
0432c6d56e Add a Makefile target for building html docs using a rawhide environment
This makes it easier to generate new documentation for
http://weldr.io/lorax/

It requires having a current welder/lorax-composer:latest image (created with
the test-in-docker target), then run docs-in-docker to rerun sphinx with
the docs/html directory mounted from the container.
2018-09-07 09:54:21 -07:00
Radek Vykydal
45e093241d Revert "Don't activate default auto connections after switchroot"
This reverts commit 044d7381a2.
2018-09-07 09:10:46 -07:00
Brian C. Lane
d514696880 Need to explicitly require python3-librepo (#1626413)
python3-dnf no longer requires it, so add it to lorax.spec
2018-09-07 09:03:30 -07:00
Brian C. Lane
330ee6c8b6 New lorax documentation - 29.14 2018-09-06 10:37:53 -07:00
Brian C. Lane
40b3eda232 Automatic commit of package [lorax] release [29.14-1].
Created by command:

/usr/bin/tito tag
2018-09-06 09:57:45 -07:00
Peter Robinson
8abf6be950 Add the create ISO component for ARMv7
The ostree compose process in pungi wants ISOs and it was the last part
of the ARMv7 components that weren't at parity with other architectures.
Add the missing functionality.

Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2018-09-06 09:10:02 -07:00
Radek Vykydal
044d7381a2 Don't activate default auto connections after switchroot
Resolves: rhbz#1555934

Add NetworkManager config file turning default auto connections off.
2018-09-04 12:04:57 -07:00
Brian C. Lane
a0aab15311 Ignore a pylint warning about UnquotingConfigParser get args
The args differ, but we are accepting and passing through all args so
it's ok.

Related: rhbz#1613058
2018-09-04 11:16:39 -07:00
Adam Williamson
5c0dfb74e6 Ditch all use of pyanaconda's simpleconfig
lorax uses pyanaconda's SimpleConfigParser in three different
places (twice with a copy that's been dumped into pylorax, once
by importing it), just to do a fairly simple job: read some
values out of /etc/os-release. The only value SimpleConfigParser
is adding over Python's own ConfigParser here is to read a file
with no section headers, and to unquote the values. The cost is
either a dependency on pyanaconda, or needing to copy the whole
of simpleparser plus some other utility bits from pyanaconda
into lorax. This seems like a bad trade-off.

This changes the approach: we copy one very simple utility
function from pyanaconda (`unquote`), and do some very simple
wrapping of ConfigParser to handle reading a file without any
section headers, and returning unquoted values. This way we can
read what we need out of os-release without needing a dep on
pyanaconda or to copy lots of things from it into pylorax.

Resolves: #449
Resolves: #450

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-30 16:21:09 -07:00
Brian C. Lane
4a5591e9a4 Automatic commit of package [lorax] release [29.13-1].
Created by command:

/usr/bin/tito tag
2018-08-29 15:46:18 -07:00