Commit Graph

1983 Commits

Author SHA1 Message Date
Brian C. Lane
efc57eae4c lorax: Save information about rootfs filesystem size and usage
Run df on the filesystem image after it has been created.
Output will be in program.log, eg:

Running... df /var/tmp/lorax.imgutils.wm04pg_v
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop0       1998672 1619508    362780  82% /var/tmp/lorax.imgutils.wm04pg_v
Return code: 0

Resolves: rhbz#1659625
2018-12-18 08:49:39 -08:00
Brian C. Lane
0ce6d3882b lorax-composer: Cancel running Anaconda process
It ends up that this isn't as easy as you'd think. Anaconda sets up some
signal handlers to handle cleanly exiting, but they are not being run
when sent a TERM after package installation has started. I think DNF
resets them causing it to get ignored.

When the cancel is sent it can take several minutes for it to have an
effect. In my testing it usually takes around 2 minutes for anaconda to
notice and exit.

This sends a TERM to the process and then waits for it to exit. When it
returns it then removed any device-mapper devices that were setup for
image installations, removes any hanging loop devices.

It then kills off any process with pyanaconda. in the cmdline, and
anaconda-bus.conf (because anaconda starts a bunch of helpers and if it
doesn't shut down cleanly they remain running).

Resolves: rhbz#1656691
2018-12-12 16:44:05 -08:00
Brian C. Lane
4b84475612 Add cancel_func to virt and novirt_install functions
In addition to monitoring the logs for errors, call a function (or
functions) that tell it to cancel the anaconda process and cleanup.

Also check for a cancel after creating the squashfs image for live-iso
since that's a long running process.

This required adding a new argument to a number of existing functions,
passing it down to QEMUInstall and novirt_install where the function is
called.

Resolves: rhbz#1656691
2018-12-11 11:37:21 -08:00
Brian C. Lane
a8f616c6da lorax-composer: Check for STATUS before deleting
When there is no run or new symlink do one last check to make sure no
STATUS file was written. If it is missing, go ahead and remove the
results directory.

Related: rhbz#1656691
2018-12-10 10:35:00 -08:00
Brian C. Lane
962d10b3b7 Check for existing CANCEL request, and exit on FINISHED
If another CANCEL request has already been made just exit from
uuid_cancel. If the build is FINISHED before it times out just exit,
don't remove the finished results.

Related: rhbz#1656691
2018-12-07 16:47:19 -08:00
Brian C. Lane
b2e1aec635 Automatic commit of package [lorax] release [28.14.18-1].
Created by command:

/bin/tito tag
2018-12-07 13:31:42 -08:00
Brian C. Lane
90b6cc7707 New lorax documentation - 28.14.18
Related: rhbz#1656642
2018-12-07 13:30:17 -08:00
David Shea
833b1bba43 Add openstack to the image type list in the docs
(cherry picked from commit d619430d6d)

Related: rhbz#1628645
2018-12-07 15:11:56 -05:00
Brian C. Lane
663a0dcd73 lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1656642
2018-12-07 11:54:02 -08:00
Brian C. Lane
36fb75abd2 Adjust test_drtfr_gpgkey to pass on Fedora 28 and RHEL 8
Depending on which version of dnf is installed it may return a tuple, or
may return a list. Work around this by only comparing the 1st element of
it.

Related: rhbz#1655876
2018-12-07 11:52:22 -08:00
Brian C. Lane
660e250804 Update the projects tests to use DNF Repo object
Stop using fake dnf object, use the real thing. This will help catch
problems with dnf returning unexpected types like VectorString.

(cherry picked from commit 27aff75aa3)

Related: rhbz#1655876
2018-12-07 11:52:22 -08:00
Brian C. Lane
2ad4b20a91 dnf changed the type of gpgkey to a tuple
And in an intermediate version it returns a VectorString object which
isn't serializable by the json or toml modules.

So convert it to a list so that the type is consistent in the sources
code.

(cherry picked from commit e9e5139750)

Resolves: rhbz#1655876
2018-12-07 11:52:22 -08:00
Brian C. Lane
6a2f574ed7 lorax-composer: Add CDN repo checks to startup and compose start.
This will block starting a compose if the sources only reference
cdn.redhat.com urls.

Resolves: rhbz#1655623
2018-12-07 11:50:52 -08:00
Brian C. Lane
f1af108e5b lorax-composer: Check for CDN only repos
Anaconda is currently not able to handle cdn repo urls in the kickstart.
Add a new function that checks for extra repos and returns True.

Related: rhbz#1655623
2018-12-07 11:50:52 -08:00
Brian C. Lane
15c64d9f8d There is no support for edk2-ovmf on arm/arm64
Resolves: rhbz#1655512
2018-12-07 11:48:59 -08:00
Brian C. Lane
f0bac40d7f lorax-composer: Check the queue and results at startup
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
 * Removes broken symlinks from queue/new and queue/run
 * Removes symlinks from run and sets the build to FAILED
 * Sets builds w/o a STATUS to FAILED
 * Sets builds with STATUS of RUNNING to FAILED
 * Creates missing queue/new symlinks to results with STATUS of WAITING

So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.

Resolves: rhbz#1647985
(cherry picked from commit 4dd9004d13)
2018-12-06 15:27:48 -08:00
Brian C. Lane
580e771dea Automatic commit of package [lorax] release [28.14.17-1].
Created by command:

/bin/tito tag
2018-11-29 13:21:21 -08:00
Brian C. Lane
32b3df0892 Update documentation for - 28.14.17
Regenerate the documentation for the SELinux changes.

Related: rhbz#1645189
2018-11-29 13:08:44 -08:00
Brian C. Lane
97b138c3d1 lorax-composer: Install selinux-policy-targeted in images
This is required to ensure that SELinux is configured properly while
building. It fixes the problem with building tar, and should be
installed in the other image types for consistency.

Resolves: rhbz#1645189
2018-11-29 13:05:01 -08:00
Brian C. Lane
22061ed4ab Remove setfiles from mkrootfsimage
SELinux applies the correct labels, setfiles is no longer needed.
This allows lorax to run with SELinux in Enforcing mode.

Related: rhbz#1645189
2018-11-29 13:05:01 -08:00
Brian C. Lane
14fb4a9156 Remove SELinux Permissive checks
Anaconda, Lorax, lorax-composer, and livemedia-creator can all now run
with SELinux in Enforcing mode. It does not need to be disabled and if
there are denials they should be reported as a bug.

Log the current state of SELinux when starting, update the
documentation.

Resolves: rhbz#1645189
2018-11-29 13:05:01 -08:00
Brian C. Lane
8a5176a17c New lorax documentation - 28.14.17
Related: rhbz#1639132
2018-11-27 10:03:06 -08:00
Brian C. Lane
a4881ddd6d Build manpages for composer-cli and lorax-composer
Add manpage creation to make docs target to keep them updated.

Resolves: rhbz#1639132
2018-11-27 10:00:36 -08:00
Brian C. Lane
6de44cd241 Add --no-system-repos to lorax-composer
Running lorax-composer --no-system-repos will prevent it from copying
the dnf repositories from /etc/yum.repos.d/ into the lorax-composer repo
directory. It will *only* use repositories setup using the sources api
or written to /var/lib/lorax/composer/repos.d/

If lorax-composer has previously been run without this switch the system
repos will need to be removed from the composer/repos.d/ directory. It
would also be a good idea to remove the cached metadata in
/var/tmp/composer/

Resolves: rhbz#1650363
2018-11-15 16:19:24 -08:00
Brian C. Lane
8b114327b1 Automatic commit of package [lorax] release [28.14.16-1].
Created by command:

/usr/bin/tito tag
2018-10-12 12:45:56 -07:00
Vendula Poncova
b594fa99bc Fix vhd images
Images don't work without these fixes:

* Enable Network Manager.
* Disable cloud-init.
* Add Hyper-V modules into initramfs.

Fixes specific for RHEL:

* Create ifcfg-eth0 required by waagent.
* Install python3 and net-tools required by waagent.

Recommended changes:

* Use recommended kernel boot args.
* Disable kdump.

Related: rhbz#1628648
2018-10-12 12:37:50 -07:00
Brian C. Lane
5b8341884d Update depsolving with suggestions from dnf (#1636239)
The previous method worked, but wasn't exactly idiomatic. This is more
correct, and appears to work the same (templates depsolve, version globs
work, multiple repos work).

Note that this does use a private dnf attribute ._goal, but the word is
that this is going to become a public api soon, so yes it is there on
purpose.

Resolves: rhbz#1638683
2018-10-12 10:58:04 -07:00
Brian C. Lane
ee2ec1bfe4 Automatic commit of package [lorax] release [28.14.15-1].
Created by command:

/usr/bin/tito tag
2018-10-09 12:05:32 -07:00
Brian C. Lane
58236a6b61 Add an openstack image type
This is a qcow2 image with cloud-init in the template.

(cherry picked from commit 98f8b23129)

Resolves: rhbz#1628645
2018-10-09 11:55:30 -07:00
David Shea
da0435bc90 Add cloud-init to vhd images.
cloud-init can be used in Azure now

(cherry picked from commit ac9a2fdbc5)

Related: rhbz#1628648
2018-10-09 11:35:42 -04:00
David Shea
6bcb2823d4 Replace /etc/machine-id with an empty file
Since these images can be used to create multiple machines, they should
not have a unique machine-id attached to them. Replace /etc/machine-id
with an empty file so that it will be regenerated at boot time.

(cherry picked from commit 6fab72d894)

Related: rhbz#1628645
Related: rhbz#1628646
Related: rhbz#1628647
Related: rhbz#1628648
2018-10-09 11:34:48 -04:00
Brian C. Lane
37051dbe2b Automatic commit of package [lorax] release [28.14.14-1].
Created by command:

/usr/bin/tito tag
2018-10-08 16:22:54 -07:00
Brian C. Lane
0ad0a47968 Update cli tests to use composer-cli name
Related: rhbz#1635763
2018-10-08 16:22:20 -07:00
Brian C. Lane
077582ee49 Revert "Rename composer-cli to composer"
This reverts commit 2c2e3156d0.

It conflicts with the PHP dependency manager project named 'composer'

Related: rhbz#1635763
2018-10-08 16:22:15 -07:00
Brian C. Lane
8dc43a7124 Automatic commit of package [lorax] release [28.14.13-1].
Created by command:

/usr/bin/tito tag
2018-10-05 09:56:02 -07:00
Brian C. Lane
091820b5d6 New lorax documentation - 28.14.12
Related: rhbz#1635763
2018-10-05 09:54:44 -07:00
Brian C. Lane
b1a2d635d0 Adjust the composer-cli tests for the rename to composer
Related: rhbz#1635763
2018-10-05 08:30:34 -07:00
Lars Karlitski
2c2e3156d0 Rename composer-cli to composer
Resolves: rhbz#1635763
2018-10-05 08:30:34 -07:00
Lars Karlitski
43aedf6a15 Add and enable cloud-init for ami images
Images don't work at all on AWS without cloud-init.

Fixes #492

(cherry picked from commit 81d38b6445)

Related: rhbz#1628647
2018-10-05 11:28:17 -04:00
David Shea
c13aa84f02 Make no-virt generated images sparser
At the end of disk image installs, use fstrim on the generated filesystem to
discard any blocks that were allocated during the install and are now unused.
This will allow tools such as qemu-img to create images that do not include
deleted data.

For raw disk images that do not go through qemu-img, use fallocate --dig-holes
to create sparse holes in place of the unused blocks.

(cherry picked from commit 9717b3fd98)

Related: rhbz#1628645
Related: rhbz#1628646
Related: rhbz#1628647
Related: rhbz#1628648
2018-10-05 11:27:04 -04:00
Brian C. Lane
63a6e72a59 Automatic commit of package [lorax] release [28.14.12-1].
Created by command:

/usr/bin/tito tag
2018-10-03 15:55:10 -07:00
Brian C. Lane
212e0bcb3e Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:29:33 -07:00
Brian C. Lane
25ee5c9ee7 Automatic commit of package [lorax] release [28.14.11-1].
Created by command:

/usr/bin/tito tag
2018-10-02 17:01:23 -07:00
Brian C. Lane
de3294280b Add beakerlib to Dockerfile.test
Also kill the lorax-composer process and remove /run/weldr/api.socket
so that when this is run with podman you don't get an error about
attempting to tar up the socket.

Related: rhbz#1613058
2018-10-02 16:50:28 -07:00
Alexander Todorov
920aed16e2 New cli test covering basic compose commands
- need to specify --sharedir so lorax-composer can find its
  kickstart files

- each test script writes results into a separate directory to
  avoid a passing test overwriting the results from a failing one.
  To avoid reporting failures in case of previously failing tests
  (e.g. during development) remove the temporary directories holding
  tets results before execution!
2018-10-02 22:52:57 +02:00
Alexander Todorov
959589652b Execute bash tests for composer-cli
these are built on top of beakerlib and we use its internal
protocol to figure out the result without relying on the full
test runner that is tipically used inside of a RHEL environment!

Includes a disabled test snippet for Issue #460
2018-10-02 22:52:57 +02:00
David Shea
d94ed86cd3 Add virt guest agents to the qcow2 compose
(cherry picked from commit d5a1993640)

Resolves: rhbz#1628645
2018-10-02 13:06:59 -04:00
David Shea
1a23dc0f2b Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning

(cherry picked from commit 1056bfc25b)

Resolves: rhbz#1628646
2018-10-02 13:06:56 -04:00
David Shea
1c1f97ad70 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.

(cherry picked from commit e0c236ff36)

Resolves: rhbz#1628648
2018-10-02 13:06:10 -04:00
David Shea
253689ff49 Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.

(cherry picked from commit 18188bf6cf)

Resolves: rhbz#1628647
2018-10-02 13:05:40 -04:00