Commit Graph

123 Commits

Author SHA1 Message Date
Brian C. Lane b9d18216ef lorax-composer: Add liveimg-tar image type
This creates a tar suitable for use with the anaconda kickstart liveimg
command. It adds the kernel, grub2, and grub2-tools packages to the tar
template.
2019-08-05 14:50:32 -07:00
Brian C. Lane 361c6de2de livemedia-creator: Use --compress-arg in mksquashfs
Previously the --compress-arg option was only used for tar, this adds
support for it to the squashfs.img creation used with live isos.
2019-08-05 14:49:25 -07:00
Brian C. Lane 835c912e12 Only use repos with valid urls for test_server.py
libdnf-0.22.5-5 changed something and now the repos with fake urls are
failing when loaded by test_server.py (they still work fine with
test_projects.py) so only use the 'good' repos with the test_server.py
tests -- the others weren't needed for any of its tests anyway.
2019-08-05 14:48:23 -07:00
Brian C. Lane c46f9f034f tests: Update test_creator.py
make_squashfs has been removed, make_runtime is now used in all paths to
create the install.img

Add a tests for squashfs only and squashfs+ext4 (requires loop so only
runs as root).
2019-07-29 13:21:25 -07:00
Brian C. Lane 0708302464 Use binary mode to tail the file
In python 3 f.seek() on text doesn't work like it does in py2/C because
text is now unicode. So change read_tail to use byte mode and take
unicode into account. Also add tests for it.
2019-07-11 15:16:41 -07:00
Lars Karlitski fd57e1106e tests: set skip_if_unavailable in test repos
dnf seems to have changed the default for skip_if_unavailable. Some
mock repositories are still around in later tests, which then fail
because metadata cannot be synced.

Also expose skip_if_unavailable in dnf_repo_to_file_repo(), so that
tests checking for equality of repo files continue to pass.
2019-07-09 00:23:33 +02:00
Evan Goode 998d0140be Assert that RuntimeErrors have correct messages 2019-06-28 08:55:36 -07:00
Brian C. Lane f1733369fa tests: Add tests for recipe checking functions 2019-06-25 11:44:58 +02:00
Brian C. Lane 61059a2699 lorax-composer: Add basic recipe checker function
This makes sure that required fields are included, and that sections are
not empty. It does not check for all optional fields.

If there are errors it will gather up all of them and then raise a
RecipeError with a string of all the errors.
2019-06-25 11:44:58 +02:00
Brian C. Lane 682f8a9295 tests: rpm now returns str, drop decode() call 2019-06-24 20:15:21 +02:00
Brian C. Lane abe7df34fc Switch to new toml library
The previous library, pytoml from https://github.com/avakar/pytoml is no
longer supported. So this adds a compatibility layer on top of the
suggested replacement, toml from https://github.com/uiri/toml
2019-06-18 14:01:15 -07:00
Brian C. Lane 9011a564e8 Add support for customizations and repos.git to /blueprints/diff/
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.
2019-06-13 14:16:48 -07:00
Brian C. Lane 316257fbc0 tests: Update custom-base with customizations 2019-06-13 14:16:48 -07:00
Lars Karlitski 4bd03cb8f6 Don't send CORS headers
These are meant for web applications that are accessed by browsers, not
REST APIs.
2019-06-04 20:10:03 +02:00
Lars Karlitski 8ed910b29a composer: Set up a custom HTTP error handler
Override flask's default error handler, because that return html. Return
JSON instead with the usual { "status": false, "errors": [ ... ] }
pattern.
2019-06-04 11:50:25 +03:00
Brian C. Lane 795cc3400c tests: Update tmux version to 2.9a 2019-05-23 14:57:46 -07:00
Brian C. Lane fa4bda8317 tests: Update openssh-server to v8.* 2019-05-02 17:02:30 -07:00
Brian C. Lane 3e08389a0f Change customizations.firewall to append items instead of replace
To maintain consistency with the other options this changes firewall to
combine the existing settings from the image template with the settings
from the blueprint.

Also updated the docs, added a new test for it, and sorted the output
for consistency.
2019-05-02 16:21:36 -07:00
Brian C. Lane 1111aee92d lorax-composer: Add services support to blueprints
Add support for enabling and disabling systemd services in the
blueprint. It works like this:

    [customizations.services]
    enabled = ["sshd", "cockpit.socket", "httpd"]
    disabled = ["postfix", "telnetd"]

They are *added* to any existing settings in the kickstart templates.
2019-05-02 16:21:36 -07:00
Brian C. Lane 4d35668ab5 lorax-composer: Add firewall support to blueprints
You can now open ports in the firewall, using port numbers or service
names:

    [customizations.firewall]
    ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]

Or enable/disable services registered with firewalld:

     [customizations.firewall.services]
     enabled = ["ftp", "ntp", "dhcp"]
     disabled = ["telnet"]

If the template contains firewall --disabled it cannot be overridden,
under the assumption that it is required for the image to boot in the
selected environment.
2019-05-02 16:21:36 -07:00
Brian C. Lane e5a8700bdf lorax-composer: Add locale support to blueprints
You can now set the keyboard layout and language. Eg.

[customizations.locale]
languages = ["en_CA.utf8", "en_HK.utf8"]
keyboard = "de (dvorak)"

Existing entries in the kickstart templates are replaced with the new
ones. If there are no entries then it will default to 'keyboard us' and
'lang en_US.UTF-8'

Includes tests, and leaves the existing keyboard and lang entries in the
templates with a note that they can be replaced by the blueprint.
2019-05-02 16:21:36 -07:00
Brian C. Lane 9bdbb29662 lorax-composer: Add timezone support to blueprint
For example:

[customizations.timezone]
timezone = "US/Samoa"
ntpservers = ["0.pool.ntp.org"]

Also includes tests.

This removes the timezone kickstart command from all of the templates
except for google.ks which needs to set it's own ntp servers and timezone.

If timezone isn't included in the blueprint, and it is not already in a
template, it will be set to 'timezone UTC' by default.

If timezone is set in a template it is left as-is, under the assumption
that the image type requires it to boot correctly.
2019-05-02 16:21:36 -07:00
Brian C. Lane 65b8a2be68 tests: Cleanup on failure of in_tempdir
Otherwise other tests will also fail when they try to run from the wrong
directory.
2019-04-15 14:06:48 -07:00
David Shea 44e14176bb Add a compose type for alibaba.
This compose type creates a partitioned disk as a qcow2 file, but with
only one partition instead of using a separate /boot.
2019-04-03 13:05:31 -04:00
David Shea 434fe58c28 Add a new compose type for Hyper-V
This is based on the VHD compose type, with the following differences:

  * Use the vhdx format instead of vhd
  * No WALinuxAgent
  * Install hyperv-daemons

The hyperv-daemons are activated through udev rules, so there is no need
to add them to the services line.
2019-04-03 13:00:37 -04:00
David Shea bd804c271b Add a compose type for Google Compute Engine 2019-04-02 10:15:42 -04:00
David Shea c585b91422 Support compressing single files.
Modify imgutils.compress to allow the "rootdir" argument to be either a
directory or a single file to add to an archive.
2019-04-02 10:15:42 -04:00
David Shea 8ef4f431d8 Add an option to align the image size to a multiplier.
If provided, round the disk image size up to a multiple of the value.
This allows for image formats with specific size-alignment requirements
(e.g., disk size must be in GiB).
2019-04-02 10:15:42 -04:00
Brian C. Lane 6fd06c6931 Add more tests for gitrpm.py
Make sure that dotfiles are installed when not directly under /
Make sure / is not packaged in the rpm (it will cause a conflict with
the filesystem package).
Make sure that using destination="/" works
2019-03-29 14:05:09 -07:00
Brian C. Lane df3aeee802 pylorax.ltmpl: Add a test for missing quotes
It should raise an error if a quote is missing in the template.
2019-03-26 13:30:25 -07:00
Brian C. Lane 5dea308080 lorax-composer: pass customization.kernel append to extra_boot_args
This allows iso builds to include the extra kernel boot parameters by
passing them to the arch-specific live/*tmpl template.

Also adds tests to make sure it is written to config.toml in the build
metadata.
2019-03-26 11:06:57 -07:00
Brian C. Lane 59464286f9 lorax-composer: Add the ability to append to the kernel command-line
Sometimes it is necessary to modify the kernel command-line of the
image, this adds support for a [customizations.kernel] section to the
blueprint:

[customizations.kernel]
append = "nosmt=force"

This will be appended to the kickstart's bootloader --append argument.

Includes tests for modifying the bootloader line, the kickstart
template, and examining the final-kickstart.ks created for a compose.
2019-03-26 11:06:57 -07:00
Brian C. Lane 8c2184d59e Add tests using repos.git in blueprints
- Check final-kickstart.ks for the rpm source
- Check final-kickstart.ks for the rpm package name and version
- Make sure depsolve works
- Make sure errors from a bad repo are returned correctly
- Make sure errors from a bad reference are returned correctly

This moves _wait_for_status into a helper function so it can be shared
between the test classes.
2019-03-15 11:18:12 -07:00
Brian C. Lane 02f757d231 Move git repo creation into tests/lib.py
This way it can be shared with test_server.py
2019-03-15 11:18:12 -07:00
Brian C. Lane 61efa91a03 rpmgit: catch potential errors while running git
Log them and report them as RuntimeError. Also add a couple tests for
them.
2019-03-15 11:18:12 -07:00
Brian C. Lane c26477a63c tests: Add test for Recipe.freeze() function
The freeze function was not being tested. Add a test for it using the
repos.git test recipe.
2019-03-15 11:18:12 -07:00
Brian C. Lane cd8c884adb Add repos.git support to lorax-composer builds
This hooks up creation of the rpm to the build, adds it to the
kickstart, and passes the url to Anaconda. The dnf repo with the rpms is
created under the results directory so it will be included when
downloading the build's results.
2019-03-15 11:18:12 -07:00
Brian C. Lane f6f2308765 Add pylorax.api.gitrpm module and tests
This handles creating the rpm from the dictionary describing the
repository and rpm. Also adds tests for archive and rpm creation.
2019-03-15 11:18:12 -07:00
Brian C. Lane d7b96c8f0f Add support for [[repos.git]] section to blueprints
This adds support, documentation, and testing for a [[repos.git]]
blueprint section that can be used to install files from a git
repository. It will create an rpm that will be added to the build,
and included in the metadata that can be downloaded. This allows you to
accurately keep track of the source of configuration files and extra
metadata that is added to the build.

The source repo and reference will be listed in the rpm's summary making
it easy to discover on the installed system.
2019-03-15 11:18:12 -07:00
Brian C. Lane e39d2aec22 tests: Make it easier to update version globs
Use constants so we won't have to edit a dozen places in the test when
the package versions are bumped.

Also switch to using Fedora 31 GPG key now that it has branched for
Fedora 30.
2019-03-08 10:09:42 -08:00
Brian C. Lane d32f477e0b lorax-composer: Return UnknownBlueprint errors when using deleted blueprints
Reading a blueprint wasn't checking to see if it had been deleted so it
was returning the most recent commit before it had been deleted. This
allowed things like starting a compose with a blueprint that technically
doesn't exist.

One exception to this is the /changes/ route, it must be available so
that you can use the commit hash to undo a delete.

This also adds tests for the various operations.

Resolves: rhbz#1682113
2019-03-01 14:39:21 -08:00
Brian C. Lane 26bd2c1378 lorax-composer: Delete workspace copy when deleting blueprint
Also extends the blueprint delete test to also check the workspace.
2019-03-01 14:39:21 -08:00
Brian C. Lane 3b8de2a233 Move the package requirements for live-iso setup out of the template
In order to support iso creation on multiple arches with the templates
we need to be able to select different packages based on arch.
lorax-composer uses the arch-specific Lorax templates in order to
generate the output iso so this patch:

1. Creates a new template and type to parse it, live-install.tmpl
   which contains only installpkg commands and #if clauses for arch
2. Removes bootloader related packages from the live-iso.ks
3. Remove dracut-config-rescue exclusion because it can cause problems
   with some blueprints.
4. Switch logo requirement to system-logos which is satisfied by
   generic-logos or fedora-logos. This prevents conflicts when a blueprint
   installs fedora-release-workstation.

So in the future, if x86.tmpl, etc. need a new package to support
creating the iso it should be added to the correct section in
./share/live/live-install.tmpl
2019-02-25 13:49:12 -08:00
Brian C. Lane b47554d716 livemedia-creator: Add support for reqpart kickstart command
reqpart can be used to make kickstarts more platform agnostic, creating
needed partitions without lmc having to keep track of the arch-specific
needs. eg. ppc64 needs prepboot and /boot

This increases the size of the disk based on whether reqpart or
reqpart --add-boot is in the kickstart.

Note that this is only valid for partitioned disk output types, not
for filesystem images or live iso output.
2019-02-25 13:49:12 -08:00
Brian C. Lane 28bd68793b tests: Fix makeFakeRPM calls 2019-02-21 10:10:11 -08:00
Brian C. Lane 2272b2b189 Update bash to 5.0.*
so that the tests will pass again on rawhide.
2019-02-21 08:55:23 -08:00
Brian C. Lane 98482e444d Remove duplicate repositories from the sources list
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.

So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.
2019-01-30 08:30:40 -08:00
Brian C. Lane 3676cb65bb Clarify the ks repo only error message
This also moves the run_creator kickstart checks into check_kickstart
so that tests may be added.

This will close #164
2019-01-29 14:03:11 -08:00
Brian C. Lane 2950f2641b Remove unneeded else from for/else loop. It confuses pylint 2019-01-08 11:38:26 -08:00
David Shea 35ab6a1336 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.
2019-01-08 10:21:07 -05:00