Commit Graph

98 Commits

Author SHA1 Message Date
David Shea
846d839980 Add a compose type for Google Compute Engine
(cherry picked from commit bd804c271b)
2019-05-03 14:22:22 -07:00
David Shea
5308e7bfec Support compressing single files.
Modify imgutils.compress to allow the "rootdir" argument to be either a
directory or a single file to add to an archive.

(cherry picked from commit c585b91422)
2019-05-03 14:22:22 -07:00
David Shea
b2a33d5654 Add an option to align the image size to a multiplier.
If provided, round the disk image size up to a multiple of the value.
This allows for image formats with specific size-alignment requirements
(e.g., disk size must be in GiB).

(cherry picked from commit 8ef4f431d8)
2019-05-03 14:22:22 -07:00
Brian C. Lane
d5d60ebe6d Add more tests for gitrpm.py
Make sure that dotfiles are installed when not directly under /
Make sure / is not packaged in the rpm (it will cause a conflict with
the filesystem package).
Make sure that using destination="/" works

(cherry picked from commit 6fd06c6931)
2019-05-03 14:22:22 -07:00
Brian C. Lane
1d4f47e26f pylorax.ltmpl: Add a test for missing quotes
It should raise an error if a quote is missing in the template.

(cherry picked from commit df3aeee802)
2019-05-03 14:22:22 -07:00
Brian C. Lane
78f79a94ec Add tests using repos.git in blueprints
- Check final-kickstart.ks for the rpm source
- Check final-kickstart.ks for the rpm package name and version
- Make sure depsolve works
- Make sure errors from a bad repo are returned correctly
- Make sure errors from a bad reference are returned correctly

This moves _wait_for_status into a helper function so it can be shared
between the test classes.

(cherry picked from commit 8c2184d59e)
2019-05-03 14:22:22 -07:00
Brian C. Lane
136f9d877a Move git repo creation into tests/lib.py
This way it can be shared with test_server.py

(cherry picked from commit 02f757d231)
2019-05-03 14:02:50 -07:00
Brian C. Lane
087b0fe8c6 rpmgit: catch potential errors while running git
Log them and report them as RuntimeError. Also add a couple tests for
them.

(cherry picked from commit 61efa91a03)
2019-05-03 14:02:38 -07:00
Brian C. Lane
bdf5572bc6 tests: Add test for Recipe.freeze() function
The freeze function was not being tested. Add a test for it using the
repos.git test recipe.

(cherry picked from commit c26477a63c)
2019-05-03 14:02:26 -07:00
Brian C. Lane
2e596010d3 Add repos.git support to lorax-composer builds
This hooks up creation of the rpm to the build, adds it to the
kickstart, and passes the url to Anaconda. The dnf repo with the rpms is
created under the results directory so it will be included when
downloading the build's results.

(cherry picked from commit cd8c884adb)
2019-05-03 14:02:09 -07:00
Brian C. Lane
efc77c1d71 Add pylorax.api.gitrpm module and tests
This handles creating the rpm from the dictionary describing the
repository and rpm. Also adds tests for archive and rpm creation.

(cherry picked from commit f6f2308765)
2019-05-03 14:02:00 -07:00
Brian C. Lane
047f174dcf Add support for [[repos.git]] section to blueprints
This adds support, documentation, and testing for a [[repos.git]]
blueprint section that can be used to install files from a git
repository. It will create an rpm that will be added to the build,
and included in the metadata that can be downloaded. This allows you to
accurately keep track of the source of configuration files and extra
metadata that is added to the build.

The source repo and reference will be listed in the rpm's summary making
it easy to discover on the installed system.

(cherry picked from commit d7b96c8f0f)
2019-05-03 14:01:47 -07:00
Brian C. Lane
82aa9cdbc6 lorax-composer: Return UnknownBlueprint errors when using deleted blueprints
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.

One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.

This also adds tests for the various operations.

Resolves: rhbz#1682113
(cherry picked from commit d32f477e0b)
2019-05-03 13:56:52 -07:00
Brian C. Lane
5c0f127b3c lorax-composer: Delete workspace copy when deleting blueprint
Also extends the blueprint delete test to also check the workspace.

(cherry picked from commit 26bd2c1378)
2019-05-03 13:56:38 -07:00
Brian C. Lane
2861bdb95e lorax-composer: pass customization.kernel append to extra_boot_args
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.

Also adds tests to make sure it is written to config.toml in the build
metadata.

(cherry picked from commit 5dea308080)
2019-04-05 15:41:44 -07:00
Brian C. Lane
c5f4dfe113 lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.

(cherry picked from commit 59464286f9)
2019-04-05 15:41:38 -07:00
Brian C. Lane
d513747384 tests: Make it easier to update version globs
Use constants so we won't have to edit a dozen places in the test when
the package versions are bumped.

Also switch to using Fedora 31 GPG key now that it has branched for
Fedora 30.
2019-03-15 12:20:02 -07:00
Brian C. Lane
3b8de2a233 Move the package requirements for live-iso setup out of the template
In order to support iso creation on multiple arches with the templates
we need to be able to select different packages based on arch.
lorax-composer uses the arch-specific Lorax templates in order to
generate the output iso so this patch:

1. Creates a new template and type to parse it, live-install.tmpl
   which contains only installpkg commands and #if clauses for arch
2. Removes bootloader related packages from the live-iso.ks
3. Remove dracut-config-rescue exclusion because it can cause problems
   with some blueprints.
4. Switch logo requirement to system-logos which is satisfied by
   generic-logos or fedora-logos. This prevents conflicts when a blueprint
   installs fedora-release-workstation.

So in the future, if x86.tmpl, etc. need a new package to support
creating the iso it should be added to the correct section in
./share/live/live-install.tmpl
2019-02-25 13:49:12 -08:00
Brian C. Lane
b47554d716 livemedia-creator: Add support for reqpart kickstart command
reqpart can be used to make kickstarts more platform agnostic, creating
needed partitions without lmc having to keep track of the arch-specific
needs. eg. ppc64 needs prepboot and /boot

This increases the size of the disk based on whether reqpart or
reqpart --add-boot is in the kickstart.

Note that this is only valid for partitioned disk output types, not
for filesystem images or live iso output.
2019-02-25 13:49:12 -08:00
Brian C. Lane
28bd68793b tests: Fix makeFakeRPM calls 2019-02-21 10:10:11 -08:00
Brian C. Lane
2272b2b189 Update bash to 5.0.*
so that the tests will pass again on rawhide.
2019-02-21 08:55:23 -08:00
Brian C. Lane
98482e444d Remove duplicate repositories from the sources list
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.

So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
2019-01-30 08:30:40 -08:00
Brian C. Lane
3676cb65bb Clarify the ks repo only error message
This also moves the run_creator kickstart checks into check_kickstart
so that tests may be added.

This will close #164
2019-01-29 14:03:11 -08:00
Brian C. Lane
2950f2641b Remove unneeded else from for/else loop. It confuses pylint 2019-01-08 11:38:26 -08:00
David Shea
35ab6a1336 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.
2019-01-08 10:21:07 -05:00
Dan Horák
3817671587 drop ppc/ppc64 from tests 2019-01-07 13:52:46 -08:00
Brian C. Lane
d18934775c lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1656642
2018-12-06 15:48:58 -08:00
Brian C. Lane
4dd9004d13 lorax-composer: Check the queue and results at startup
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
 * Removes broken symlinks from queue/new and queue/run
 * Removes symlinks from run and sets the build to FAILED
 * Sets builds w/o a STATUS to FAILED
 * Sets builds with STATUS of RUNNING to FAILED
 * Creates missing queue/new symlinks to results with STATUS of WAITING

So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.

Resolves: rhbz#1647985
2018-12-06 15:13:28 -08:00
Alexander Todorov
cd6f4fdb67 Drop minor version from php package in blueprint
there's 7.2 in Fedora 29 (which we use in Jenkins) and 7.3 in
Rawhide which causes test jobs to fail with unsolved dependencies.
Alternatively we can switch to another recipe for cloud image tests.
2018-11-29 09:46:06 +02:00
Brian C. Lane
fb4e6f2588 Add tests for partitioned disk images
The setup for this is a bit complex, so it really ends up testing things
twice.
2018-11-12 11:13:37 -08:00
Brian C. Lane
063a1770e1 Add tests for pylorax.imgutils
Some of these can only run as root on a real system with access to loop
devices. They are skipped when running in a container.
2018-11-12 11:13:37 -08:00
Brian C. Lane
bab4b20d0d Add tests to test_creator.py
Add a /.in-container file to the container root so that tests requiring root
and loop device support will be skipped when running in a container.
2018-11-12 11:13:37 -08:00
Brian C. Lane
134a333d92 Add some tests for creator.py
This is complicated by the fact that much of this module requires mount.
So for now just test the things that don't need mount.
2018-11-12 11:13:37 -08:00
Brian C. Lane
fb87db0e1e tests: Add executils test 2018-11-12 11:13:37 -08:00
Brian C. Lane
cc69478249 tests: Add sysutils test 2018-11-12 11:13:37 -08:00
Brian C. Lane
2e36e941ce tests: Add discinfo test 2018-11-12 11:13:37 -08:00
Brian C. Lane
e8fd8a496b tests: Add treeinfo test 2018-11-12 11:13:37 -08:00
Alexander Todorov
453b1c1236 Update tmux version in tests 2018-10-29 23:15:06 +02:00
Brian C. Lane
a4783ba29f Add tests for ltmpl.py
This covers things like installing globbed package names from multiple
repos, pinned package versions, and ltmpl functions

Related: rhbz#1548586
2018-10-29 13:55:19 -07:00
Brian C. Lane
df70e3d677 Update php version to 7.3.* 2018-10-24 10:22:40 -07:00
Brian C. Lane
27aff75aa3 Update the projects tests to use DNF Repo object
Stop using fake dnf object, use the real thing. This will help catch
problems with dnf returning unexpected types like VectorString.
2018-10-24 10:22:40 -07:00
Brian C. Lane
75644689de Update the tests for new make_dnf_dir arguments.
Use the uid and gid that the test is running as instead of hard-coding
0.
2018-10-12 11:59:32 -07:00
Brian C. Lane
98f8b23129 Add an openstack image type
This is a qcow2 image with cloud-init in the template.
2018-10-09 10:17:14 -07:00
Brian C. Lane
e7bfab8b4a Write a rootpw line if no root customizations in the blueprint
Anaconda requires the root password to be set or locked, so if there
isn't anything setting it we write out 'rootpw --lock'

Also adds tests for this.

Resolves: rhbz#1626122
2018-10-03 08:30:31 -07:00
Brian C. Lane
e4c098b7e0 Adjust projects test for DNF 3.6.1 tuple issue 2018-10-02 14:31:30 -07:00
Brian C. Lane
7e31a2c138 Add a test for repo metadata expiration
This tests to make sure that the metadata timer is working (by setting
it to 10s and adding a new package to the repo), and that
DNFLock.lock_check immediately picks up a new package.

This depends on rpmfluff which is available from Fedora or EPEL repos.

Related: rhbz#1631561
2018-10-02 13:59:42 -07:00
Brian C. Lane
ef8bc70cdd Add tests for setting root password and ssh key with blueprints
Related: rhbz#1626120
2018-10-02 13:59:42 -07:00
David Shea
1056bfc25b Add a vmdk compose type.
This is similar to the AMI type, but also adds open-vm-tools and does not do
anything special to the partitioning
2018-10-02 11:54:41 -04:00
David Shea
e0c236ff36 Add a vhd compose type for Azure images
This does pretty much the same things as the AMI compose type, but also
replaces NetworkManager with the Azure linux agent.
2018-10-02 11:54:41 -04:00
David Shea
18188bf6cf Add an ami compose type for AWS images
This differs from lmc's --make-ami in that creates a full disk image instead of
an fsimage. Create a raw disk image with a / and /boot partitions, and enable
sshd, chronyd, and cockpit by default.
2018-10-02 11:54:38 -04:00