Commit Graph

1987 Commits

Author SHA1 Message Date
Brian C. Lane 010031a46c lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.

Related: rhbz#1687743
2019-03-27 11:32:19 -07:00
Brian C. Lane b399076cb0 lorax-composer: Return UnknownBlueprint errors when using deleted blueprints
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.

One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.

This also adds tests for the various operations.

(cherry picked from commit d32f477e0b)

Resolves: rhbz#1683441
2019-03-27 10:54:47 -07:00
Brian C. Lane da6e228753 lorax-composer: Delete workspace copy when deleting blueprint
Also extends the blueprint delete test to also check the workspace.

(cherry picked from commit 26bd2c1378)

Related: rhbz#1683441
2019-03-27 10:54:13 -07:00
Brian C. Lane 865fedde4e Remove 3G minimum from lorax-composer
The reason for the 3G minimum was because anaconda had a bug with how it
calculated minimum disk size when using kickstart. The fix for this has
been in Anaconda since 29.16, so we can now remove our limit and
create somewhat smaller disk images.

Resolves: rhbz#1677741
2019-03-27 10:39:58 -07:00
Brian C. Lane 8b9c2d5999 Automatic commit of package [lorax] release [28.14.24-1].
Created by command:

/bin/tito tag
2019-03-21 15:51:51 -07:00
Brian C. Lane 09d7e9ec0a Add a ppc64le template for live iso creation
Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane e90e7653df Move the package requirements for live-iso setup out of the template
In order to support iso creation on multiple arches with the templates
we need to be able to select different packages based on arch.
lorax-composer uses the arch-specific Lorax templates in order to
generate the output iso so this patch:

1. Creates a new template and type to parse it, live-install.tmpl
   which contains only installpkg commands and #if clauses for arch
2. Removes bootloader related packages from the live-iso.ks
3. Remove dracut-config-rescue exclusion because it can cause problems
   with some blueprints.
4. Switch logo requirement to system-logos which is satisfied by
   generic-logos or fedora-logos. This prevents conflicts when a blueprint
   installs fedora-release-workstation.

So in the future, if x86.tmpl, etc. need a new package to support
creating the iso it should be added to the correct section in
./share/live/live-install.tmpl

Resolves: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 21eb9c2122 Remove exclusions from lorax-composer templates
If a package is excluded in the template and later added by a blueprint
or dependency, anaconda will fail to finish the installation. So remove
the -dracut-config-rescue exclusion and instead remove the rescue
artifacts in %post

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 4e862e538b Add LiveTemplateRunner to parse per-arch live-iso package requirements
This template runner only uses the installpkg command to gather a list
of package NEVRA to be installed to support live iso creation.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 51180ad407 Move the run part of LoraxTemplateRunner into new TemplateRunner class
This will make it easier to add a new subclass that only handles
installpkg for use with livemedia-creator and lorax-composer iso
creation.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 8c94ee6ba1 lorax-composer: Use reqpart --add-boot for partitioned disk templates
Some platforms, like ppc64, require that the /boot partition be present.
It doesn't hurt to have it there on other platforms so instead of trying
to add per-arch kickstart templates just use reqpart --add-boot
everywhere.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 776bb18cee livemedia-creator: Add support for reqpart kickstart command
reqpart can be used to make kickstarts more platform agnostic, creating
needed partitions without lmc having to keep track of the arch-specific
needs. eg. ppc64 needs prepboot and /boot

This increases the size of the disk based on whether reqpart or
reqpart --add-boot is in the kickstart.

Note that this is only valid for partitioned disk output types, not
for filesystem images or live iso output.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane aa240fb591 Fix make_appliance and the libvirt.tmpl
Apparently nobody has used these since the switch to py3, xrange is now
range and it needs to read the file in binary mode when generating the
sha256.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane d37cf8d0d5 Add get_file_magic to tests/lib.py
Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 1f716641cf Clarify the ks repo only error message
This also moves the run_creator kickstart checks into check_kickstart
so that tests may be added.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 3d72aea6b2 Add tests to test_creator.py
Add a /.in-container file to the container root so that tests requiring root
and loop device support will be skipped when running in a container.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 57e3329250 Add some tests for creator.py
This is complicated by the fact that much of this module requires mount.
So for now just test the things that don't need mount.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 3c2f06899c Make the lorax-composer ks templates more generic
Some platforms do not have grub2, and some require other partitions.

Anaconda will add platform specific partitions if the 'reqpart' command
is included, and it will add bootloader specific packages to the list if
they are needed.

Related: rhbz#1673744
2019-03-21 14:48:27 -07:00
Brian C. Lane 2f8fb6fa8e Add some extra cancel_func protection to QEMUInstall
In livemedia-creator's usage of this it can never pass in None, but if
someone were to import the library and use it, it would crash with
NoneType. So add the extra checks to make sure cancel_func isn't None,
just in case.

Related: rhbz#1684316
2019-03-21 14:25:06 -07:00
Yuval Turgeman 3d376764d4 installer: make sure cancel_func has a value (#612)
When using LMC to virt-install a system to an image, cancel_func is not
provided in run_creator, causing a TypeError (NoneType object is not
callable).

Signed-off-by: Yuval Turgeman <yturgema@redhat.com>

Resolves: rhbz#1684316
2019-03-21 14:25:06 -07:00
Chris Roberts 6b4b3d98a8 Update VMware datastore location to unblock tests
Datastore switched from NFS to iSCSI

Related: rhbz#1678937
2019-03-21 11:27:12 -07:00
Alexander Todorov 551723816c Allow overriding $CLI outside test scripts
this will allow you to test against installed RPM like so:

    # export CLI="/usr/bin/composer-cli"
    # make test_images

If you already have lorax-composer running then you can directly
execute test scripts:

    # ./tests/cli/test_build_and_deploy_aws.sh

Related: rhbz#1678937
2019-03-15 11:03:38 +02:00
Alexander Todorov 05f3bf0f97 Use make ci inside test-in-copy target
this minimizes the possibility of these two to diverge over time.
make ci is the default for Jenkins and will also be used for
internal gating tests.

Related: rhbz#1678937
2019-03-15 11:03:38 +02:00
Alexander Todorov 82492a54bc New test: Build live-iso and boot with KVM
explicitly enables sshd for live-iso during testing

Related: rhbz#1653934
2019-03-04 22:08:22 +02:00
Alexander Todorov e78e0e610c New test: Build qcow2 compose and test it with QEMU-KVM
Related: rhbz#1653934
2019-03-04 13:18:35 +02:00
Jakub Vávra 5b4a2a47b8 Removed remnants of fedora branding. (#605)
Removed mentions of fedora in example live and atomic kickstarts.

Resolves: rhbz#1672583
2019-02-21 11:17:10 -08:00
Brian C. Lane 05aeb8fc24 Drop auth from the kickstart examples
System defaults should be sufficient these days.

Resolves: rhbz#1672583
2019-02-21 11:13:24 -08:00
Alexander Todorov 50febb4381 New test: Verify tar images with Docker and systemd-nspawn
- on some arches (also Fedora x86_64) systemd-nspawn may not be
  available
- delete composes from other tests in rlPhaseStartCleanup because
  we're seeing the tar compose kind of hanging in Jenkins and that
  test script is executed last so the slave may be running out of
  disk space. Be a good citizen and clean up after the previous
  tests.

Related: rhbz#1653934
2019-02-19 21:16:40 +02:00
Alexander Todorov b6e86b4216 Update OpenStack flavor and network settings in tests
b/c we've migrated to Upshift we must use different instance type,
specify the desired network to connect to and update how we get
the ip address of the launched VM.

Related: rhbz#1653934
2019-02-18 12:15:37 +02:00
Jan Stodola 6d477a403a Use existing storage account
To avoid creating a new storage account for every new VM.

Related: rhbz#1653934
2019-02-06 19:19:35 +01:00
Jan Stodola 9d34481d9b Record date/time of VM creation
Related: rhbz#1653934
2019-02-06 19:19:35 +01:00
Jiri Kortus e019ec2822 Make sure compose build tests run with SELinux in enforcing mode
Related: rhbz#1645189
2019-02-06 12:16:25 +01:00
Brian C. Lane 7d4a9ca0b1 Automatic commit of package [lorax] release [28.14.23-1].
Created by command:

/bin/tito tag
2019-01-30 10:08:44 -08:00
Brian C. Lane 6a55184c66 lorax: Move default tmp dir to /var/tmp/lorax
If systemd's tmpfiles.d timer is executed while lorax is running it will
remove any files and directories older than 30 days. This is what has
been causing the occasional error where /proc/ would seem to vanish
during the install.

Upstream has proposed this solution, https://github.com/systemd/systemd/pull/11482
but until that is released we need a work-around to protect the lorax
files.

This commit does several things:

* Move the default tmpdir from /var/tmp/ to /var/tmp/lorax/
* Add a lorax.conf tmpfiles.d file that prevents systemd-tmpfiles from
  removing anything under /var/tmp/lorax/
* Add an exit handler to lorax so that temporary directories are removed on
  exit or on a python traceback.
* Use flock to lock access to the tempdir while lorax is running.
* Remove any unlocked tempdirs named /var/tmp/lorax/lorax.* at startup

Note that the exit handler will not remove the tempdir if lorax is
killed with a signal -- those are being caught by dnf and prevent the
exit handler from running.

systemd-tmpfiles cannot clean up the tempdirs at boot time because they
contain files labeled as shadow_t, so we have to remove those when lorax
runs. It uses the flock to prevent removing any directories created by
parallel instances of lorax and only removes ones that are unlocked.
Worst case they will be around until the first run of lorax after a
reboot.

If you want to keep the working directory around for debugging purposes
use --workdir /var/tmp/lorax/my-workdir and it won't be removed by
lorax.

Resolves: rhbz#1668408
2019-01-30 09:19:24 -08:00
Jan Stodola 595f11cc10 Expand parameters as separate words
Related: rhbz#1653934
2019-01-28 12:28:23 +02:00
Alexander Todorov bee2593a13 Add /usr/local/bin to PATH for tests
pip3 will install commands there and later scripts will not be able
to find them.

Related rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov fa8a29972a Do not generate journal.xml from beakerlib
bacause this requires additional Python modules and we don't
really use it!

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov f54c0bf104 Use any tmux version
the unittest suite is currently executed on Fedora where tmux is
version 2.8 but on RHEL 8 it is 2.7 and depsolve fails when
building images.

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov b4c1e7c6a8 Make test scripts executable with chmod +x
because this is how Jenkins expects them to be

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Brian C. Lane 9caa35776b Automatic commit of package [lorax] release [28.14.22-1].
Created by command:

/bin/tito tag
2019-01-11 14:50:49 -08:00
Brian C. Lane c0c664bc5a Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.

Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov d61ba3a2cc tmux is version 2.8 on Fedora 28
Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov d72ee7bd89 Disable pylint no-member errors for 2 dnf constants
Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov 55bdb8a27a Backport cloud image tests to RHEL 8
Related: rhbz#1653934

note: for now use Fedora 28 for the Docker container until
CentOS 8 is released or we figure out how to use subscriptions
for official RHEL 8 images.
2019-01-11 14:37:01 -08:00
Brian C. Lane 7dad328424 Remove unneeded else from for/else loop. It confuses pylint
(cherry picked from commit 2950f2641b)

Related: rhbz#1664485
2019-01-09 16:26:35 -08:00
David Shea 59e2da0a96 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.

(cherry picked from commit 35ab6a1336)

Resolves: rhbz#1664485
2019-01-09 16:22:21 -08:00
Brian C. Lane 14d43a3d47 New lorax documentation - 28.14.21
Related: rhbz#1650295
2019-01-09 10:23:13 -08:00
Brian C. Lane 1a8591fdd5 Update the example livemedia-creator kickstarts for RHEL8
Rename them to rhel-*, update urls to reference BaseOS and AppStream.
rhel-livemedia will boot into a live GNOME desktop.

Resolves: rhbz#1650295
2019-01-09 09:03:27 -08:00
Brian C. Lane f3638762fe Change default releasever to 8
Related: rhbz#1650295
2019-01-09 09:03:27 -08:00
Brian C. Lane bcb8058dce Automatic commit of package [lorax] release [28.14.20-1].
Created by command:

/bin/tito tag
2019-01-08 16:15:32 -08:00