Commit Graph

1974 Commits

Author SHA1 Message Date
Jan Stodola
5a93d9bbd3 Fix path to generic.prm
Also quote ${extra_boot_args} as in the other templates

Resolves: rhbz#1712822
2019-05-28 09:59:06 -07:00
Brian C. Lane
fbcd0a6778 Add kernel to ext4-filesystem template
The filesystem was too small because Anaconda always adds the kernel,
but the template uses --nocore so it doesn't take that into account.
Add it to the template so that the filesystem size will be large enough
to hold the extra packages.

(cherry picked from commit ddb82ef598)

Resolves: rhbz#1710969
2019-05-28 09:46:55 -07:00
Brian C. Lane
76d5b0afd3 lorax-composer: Return UnknownBlueprint errors when using deleted blueprints
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.

One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.

This also adds tests for the various operations.

(cherry picked from commit b399076cb0)

Resolves: rhbz#1682992
2019-05-16 08:59:08 -07:00
Brian C. Lane
e3addd1252 lorax-composer: Delete workspace copy when deleting blueprint
Also extends the blueprint delete test to also check the workspace.

(cherry picked from commit da6e228753)

Related: rhbz#1682992
2019-05-16 08:58:11 -07:00
Brian C. Lane
dff53be616 Automatic commit of package [lorax] release [28.14.23-5].
Created by command:

/bin/tito tag --keep-version
2019-04-08 14:32:36 -07:00
Brian C. Lane
5eaa4e1449 Update anaconda-tui requirement for lorax-composer
We now depend on the ssl certificate support.

Related: rhbz#1693801
2019-04-08 14:31:37 -07:00
Brian C. Lane
3c4b838166 Automatic commit of package [lorax] release [28.14.23-4].
Created by command:

/bin/tito tag --keep-version
2019-04-08 13:06:34 -07:00
Brian C. Lane
226e50b785 Revert "lorax-composer: Add CDN repo checks to startup and compose start."
This reverts commit 6a2f574ed7.

Related: rhbz#1693801
2019-04-08 12:39:06 -07:00
Brian C. Lane
79392b91d7 Revert "lorax-composer: Check for CDN only repos"
This reverts commit f1af108e5b.

Related: rhbz#1693801
2019-04-08 12:38:39 -07:00
Brian C. Lane
5e88a9b0df Fix bz# for ssl cert patch
Resolves: rhbz#1693801
2019-04-04 08:25:57 -07:00
Brian C. Lane
de00e12dc2 Automatic commit of package [lorax] release [28.14.23-3].
Created by command:

/usr/bin/tito tag --keep-version
2019-04-04 08:18:03 -07:00
Lars Karlitski
60eee85d2e Pass ssl certificate options to anaconda
If a repository has `sslcacert`, `sslclientcert`, or `ssclientkey` set,
pass them to anaconda through the kickstart file. This is mostly the
case when using RHEL repositories that are accessed through a
subscription.

(cherry picked from commit e194b5926c)

Resolves: rhbz#1663950
2019-04-03 08:51:37 -07:00
Brian C. Lane
7d931a55e9 Automatic commit of package [lorax] release [28.14.23-2].
Created by command:

/bin/tito tag --keep-version
2019-04-01 16:32:09 -07:00
Brian C. Lane
a9ab3407b8 lorax-composer: pass customization.kernel append to extra_boot_args
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.

Also adds tests to make sure it is written to config.toml in the build
metadata.

(cherry picked from commit d7f5f3064b)

Resolves: rhbz#1690068
2019-04-01 15:37:35 -07:00
Brian C. Lane
251eb73625 Improve logging for template syntax errors
The shlex splitting can fail, resulting in error messages like:

ERROR livemedia-creator: No closing quotation

without any context in the log files. This logs the line that failed to
be split and expanded.

(cherry picked from commit f2ec60b9e4)

Related: rhbz#1690068
2019-04-01 15:37:35 -07:00
Brian C. Lane
392bb2acd6 Add extra boot args to the livemedia-creator iso templates
This adds a new livemedia-creator argument, --extra-boot-args, which can
be used to add arguments to the kernel command-line in the templates.

(cherry picked from commit a710d17294)

Related: rhbz#1690068
2019-04-01 15:37:35 -07:00
Brian C. Lane
0c3cfffff9 lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.

(cherry picked from commit 010031a46c)

Related: rhbz#1690068
2019-04-01 15:37:35 -07:00
Brian C. Lane
7d319fb2b1 Add some extra cancel_func protection to QEMUInstall
In livemedia-creator's usage of this it can never pass in None, but if
someone were to import the library and use it, it would crash with
NoneType. So add the extra checks to make sure cancel_func isn't None,
just in case.

Related: rhbz#1690507
2019-04-01 14:23:00 -07:00
Yuval Turgeman
7a90bd66db installer: make sure cancel_func has a value (#612)
When using LMC to virt-install a system to an image, cancel_func is not
provided in run_creator, causing a TypeError (NoneType object is not
callable).

Signed-off-by: Yuval Turgeman <yturgema@redhat.com>

Resolves: rhbz#1690507
2019-04-01 14:22:49 -07:00
Brian C. Lane
7d4a9ca0b1 Automatic commit of package [lorax] release [28.14.23-1].
Created by command:

/bin/tito tag
2019-01-30 10:08:44 -08:00
Brian C. Lane
6a55184c66 lorax: Move default tmp dir to /var/tmp/lorax
If systemd's tmpfiles.d timer is executed while lorax is running it will
remove any files and directories older than 30 days. This is what has
been causing the occasional error where /proc/ would seem to vanish
during the install.

Upstream has proposed this solution, https://github.com/systemd/systemd/pull/11482
but until that is released we need a work-around to protect the lorax
files.

This commit does several things:

* Move the default tmpdir from /var/tmp/ to /var/tmp/lorax/
* Add a lorax.conf tmpfiles.d file that prevents systemd-tmpfiles from
  removing anything under /var/tmp/lorax/
* Add an exit handler to lorax so that temporary directories are removed on
  exit or on a python traceback.
* Use flock to lock access to the tempdir while lorax is running.
* Remove any unlocked tempdirs named /var/tmp/lorax/lorax.* at startup

Note that the exit handler will not remove the tempdir if lorax is
killed with a signal -- those are being caught by dnf and prevent the
exit handler from running.

systemd-tmpfiles cannot clean up the tempdirs at boot time because they
contain files labeled as shadow_t, so we have to remove those when lorax
runs. It uses the flock to prevent removing any directories created by
parallel instances of lorax and only removes ones that are unlocked.
Worst case they will be around until the first run of lorax after a
reboot.

If you want to keep the working directory around for debugging purposes
use --workdir /var/tmp/lorax/my-workdir and it won't be removed by
lorax.

Resolves: rhbz#1668408
2019-01-30 09:19:24 -08:00
Jan Stodola
595f11cc10 Expand parameters as separate words
Related: rhbz#1653934
2019-01-28 12:28:23 +02:00
Alexander Todorov
bee2593a13 Add /usr/local/bin to PATH for tests
pip3 will install commands there and later scripts will not be able
to find them.

Related rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov
fa8a29972a Do not generate journal.xml from beakerlib
bacause this requires additional Python modules and we don't
really use it!

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov
f54c0bf104 Use any tmux version
the unittest suite is currently executed on Fedora where tmux is
version 2.8 but on RHEL 8 it is 2.7 and depsolve fails when
building images.

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Alexander Todorov
b4c1e7c6a8 Make test scripts executable with chmod +x
because this is how Jenkins expects them to be

Related: rhbz#1653934
2019-01-24 12:51:01 +02:00
Brian C. Lane
9caa35776b Automatic commit of package [lorax] release [28.14.22-1].
Created by command:

/bin/tito tag
2019-01-11 14:50:49 -08:00
Brian C. Lane
c0c664bc5a Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.

Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov
d61ba3a2cc tmux is version 2.8 on Fedora 28
Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov
d72ee7bd89 Disable pylint no-member errors for 2 dnf constants
Related: rhbz#1653934
2019-01-11 14:37:01 -08:00
Alexander Todorov
55bdb8a27a Backport cloud image tests to RHEL 8
Related: rhbz#1653934

note: for now use Fedora 28 for the Docker container until
CentOS 8 is released or we figure out how to use subscriptions
for official RHEL 8 images.
2019-01-11 14:37:01 -08:00
Brian C. Lane
7dad328424 Remove unneeded else from for/else loop. It confuses pylint
(cherry picked from commit 2950f2641b)

Related: rhbz#1664485
2019-01-09 16:26:35 -08:00
David Shea
59e2da0a96 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.

(cherry picked from commit 35ab6a1336)

Resolves: rhbz#1664485
2019-01-09 16:22:21 -08:00
Brian C. Lane
14d43a3d47 New lorax documentation - 28.14.21
Related: rhbz#1650295
2019-01-09 10:23:13 -08:00
Brian C. Lane
1a8591fdd5 Update the example livemedia-creator kickstarts for RHEL8
Rename them to rhel-*, update urls to reference BaseOS and AppStream.
rhel-livemedia will boot into a live GNOME desktop.

Resolves: rhbz#1650295
2019-01-09 09:03:27 -08:00
Brian C. Lane
f3638762fe Change default releasever to 8
Related: rhbz#1650295
2019-01-09 09:03:27 -08:00
Brian C. Lane
bcb8058dce Automatic commit of package [lorax] release [28.14.20-1].
Created by command:

/bin/tito tag
2019-01-08 16:15:32 -08:00
Brian C. Lane
bdc09df37b Revert "lorax-composer: Cancel running Anaconda process"
Drop running pkill. This causes problems if more than one is running on
a system (eg. in parallel using mock). It can kill off other processes
unrelated to this instance of anaconda.

This reverts commit 0ce6d3882b.

Related: rhbz#1656691
2019-01-08 14:47:25 -08:00
Brian C. Lane
0ed7167a03 Make sure cancel_func is not None
Related: rhbz#1656691

(cherry picked from commit ca2c3d9e77)
2019-01-07 14:10:11 -08:00
Brian C. Lane
47eecdfc85 Increase boot.iso rootfs to 3GiB
PPC64 runs out of space when installing into a 2GiB rootfs. Other arches
may also be getting close to running out of space.

Resolves: rhbz#1661169
2019-01-07 08:29:31 -08:00
Brian C. Lane
4f33181f2f Automatic commit of package [lorax] release [28.14.19-1].
Created by command:

/bin/tito tag
2018-12-18 15:17:10 -08:00
Brian C. Lane
efc57eae4c lorax: Save information about rootfs filesystem size and usage
Run df on the filesystem image after it has been created.
Output will be in program.log, eg:

Running... df /var/tmp/lorax.imgutils.wm04pg_v
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop0       1998672 1619508    362780  82% /var/tmp/lorax.imgutils.wm04pg_v
Return code: 0

Resolves: rhbz#1659625
2018-12-18 08:49:39 -08:00
Brian C. Lane
0ce6d3882b lorax-composer: Cancel running Anaconda process
It ends up that this isn't as easy as you'd think. Anaconda sets up some
signal handlers to handle cleanly exiting, but they are not being run
when sent a TERM after package installation has started. I think DNF
resets them causing it to get ignored.

When the cancel is sent it can take several minutes for it to have an
effect. In my testing it usually takes around 2 minutes for anaconda to
notice and exit.

This sends a TERM to the process and then waits for it to exit. When it
returns it then removed any device-mapper devices that were setup for
image installations, removes any hanging loop devices.

It then kills off any process with pyanaconda. in the cmdline, and
anaconda-bus.conf (because anaconda starts a bunch of helpers and if it
doesn't shut down cleanly they remain running).

Resolves: rhbz#1656691
2018-12-12 16:44:05 -08:00
Brian C. Lane
4b84475612 Add cancel_func to virt and novirt_install functions
In addition to monitoring the logs for errors, call a function (or
functions) that tell it to cancel the anaconda process and cleanup.

Also check for a cancel after creating the squashfs image for live-iso
since that's a long running process.

This required adding a new argument to a number of existing functions,
passing it down to QEMUInstall and novirt_install where the function is
called.

Resolves: rhbz#1656691
2018-12-11 11:37:21 -08:00
Brian C. Lane
a8f616c6da lorax-composer: Check for STATUS before deleting
When there is no run or new symlink do one last check to make sure no
STATUS file was written. If it is missing, go ahead and remove the
results directory.

Related: rhbz#1656691
2018-12-10 10:35:00 -08:00
Brian C. Lane
962d10b3b7 Check for existing CANCEL request, and exit on FINISHED
If another CANCEL request has already been made just exit from
uuid_cancel. If the build is FINISHED before it times out just exit,
don't remove the finished results.

Related: rhbz#1656691
2018-12-07 16:47:19 -08:00
Brian C. Lane
b2e1aec635 Automatic commit of package [lorax] release [28.14.18-1].
Created by command:

/bin/tito tag
2018-12-07 13:31:42 -08:00
Brian C. Lane
90b6cc7707 New lorax documentation - 28.14.18
Related: rhbz#1656642
2018-12-07 13:30:17 -08:00
David Shea
833b1bba43 Add openstack to the image type list in the docs
(cherry picked from commit d619430d6d)

Related: rhbz#1628645
2018-12-07 15:11:56 -05:00
Brian C. Lane
663a0dcd73 lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1656642
2018-12-07 11:54:02 -08:00