Commit Graph

2466 Commits

Author SHA1 Message Date
Brian C. Lane f8f06ccd23 Fix some docstrings in the v1 API 2019-10-16 09:17:53 -07:00
Brian C. Lane fd55bee85d lifted: Make sure providers list is always sorted 2019-10-16 09:17:53 -07:00
Brian C. Lane c384b687ba Add /upload/providers/delete route
This allows deleting a provider's profile. Pass the provider and profile
name like:

/api/v1/upload/providers/delete/azure/test-settings

A standard json response will be returned.
2019-10-16 09:17:53 -07:00
Brian C. Lane b2fc391677 lifted: Add delete_profile function and tests
Also adds a helper to providers, _get_profile_path, so that the code
doesn't need to be repeated in all the functions.
2019-10-16 09:17:53 -07:00
Brian C. Lane 5d7aa95f2f Add support for starting a compose upload with the profile
This extends the /compose/ route to support uploading with either an
existing profile, or with one-time use settings passed in the POST.

To select a profile include the provider and profile, as returned by
`/uploads/providers`:

    "upload": {
      "image_name": "My Image",
      "provider": "azure",
      "profile": "production-azure-settings"
    }
2019-10-16 09:17:53 -07:00
Brian C. Lane bc06abceab lifted: Add a function to load the settings for a provider's profile
This returns the settings after loading them and running verify_settings
on them.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1030dd083c Fix pylint errors in lifted.upload 2019-10-16 09:17:53 -07:00
Brian C. Lane de314a0046 tests: Add yamllint of the lifted playbooks 2019-10-16 09:17:53 -07:00
Brian C. Lane 4da569442c tests: Add tests for the new lifted module
These tests cover most of the module's functions, except for the queue
monitor, and the actual execution of the playbooks.
2019-10-16 09:17:53 -07:00
Brian C. Lane 45498f2b93 All providers should have 'supported_types'
even if it is empty.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1b84f90963 lifted directories should be under share_dir and lib_dir
Otherwise passing --sharedir pointed to some other path will not use the
correct providers.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1a6bc098d9 tests: Add tests for API v1
Output from some of these are different from API v0. Instead of mixing
the two this moved v1 tests into a new class - ServerAPIV1TestCase to
make them easier to maintain, and removes the v1 tests from
ServerAPIV0TestCase
2019-10-16 09:17:53 -07:00
Brian C. Lane 3a453eaad7 Make sure V0 API doesn't return uploads information
uploads should only be included in the V1 API routes, add `api`
selection to the relevant helper functions and calls to them from v0.py

Add new V1 routes with api=1 to include the uploads information in the
results.

Also add tests to ensure that V0 requests do not include uploads.
2019-10-16 09:17:53 -07:00
Evan Goode 2692e8138c Automatically upload composed images to the cloud
Currently, Azure, vSphere, and OpenStack are supported. See
https://github.com/weldr/lorax/pull/826 for more details about this new
feature.

I've called the upload library "lifted" as a reference to Seuss'
The Lorax -- in the book, the Lorax lifts himself up by the seat of his
pants through a hole in the smog clouds, and they start calling him the
"Lifted Lorax."

This adds new features to the /compose route under API v1
2019-10-16 09:17:53 -07:00
Evan Goode 584a2d2076 Add load and dump to pylorax.api.toml 2019-10-16 09:17:53 -07:00
Martin Pitt a2c67385e4 Support CI testing against a bots project PR
* If `$COCKPIT_BOTS_REF` is set, check out that bots version instead of
   master.

 * Use git cache in $XDG_CACHE_HOME if available. Our CI uses that to
   save downloads, and it does not get in the way for local developers.

Adapted from https://github.com/cockpit-project/starter-kit/pull/233
2019-10-14 20:57:25 +02:00
Martin Pitt 29c308ae73 Makefile: Don't clobber an existing bots checkout
Commit 4643afa58 was a thinko -- for our CI we *don't* want our test to
clobber a pre-existing bots/ checkout, as we often use this to run tests
against an updated image or to validate a changes to the bots project.

On developer machines, bots may also be a symlink to an actual bots
directory in development, so don't clobber that.

Stop making "bots" a phony target, and drop the now unnecessary
`[ -d bots ]` check.

Adapted from https://github.com/cockpit-project/starter-kit/pull/232
2019-10-14 20:57:25 +02:00
Brian C. Lane 697233c14a lorax-composer: Handle RecipeError in commit_recipe_directory
A recipe that is valid TOML can still be an invalid recipe (eg. missing
the 'name' field) so this should also catch RecipeError.

Also added tests for this, as well as making sure commit_recipe_file()
raises the correct errors.

Resolves: rhbz#1755068
2019-10-04 08:07:36 -07:00
Brian C. Lane 3aac31482c test: Disable pylint subprocess check check
The return value for subprocess is already being checked, no need to use
check=True.
2019-09-30 18:11:46 -07:00
Brian C. Lane 6cebc3da88 Automatic commit of package [lorax] release [32.0-1].
Created by command:

/bin/tito tag --keep-version
2019-09-30 16:56:49 -07:00
Brian C. Lane 5efaa876e1 aarch64: Fix live-iso creation on aarch64
Drop unneeded uboot-tools, and remove iso-graft from the aarch64.tmpl

Related: rhbz#1752002
2019-09-25 11:07:20 -07:00
Jiri Kortus 5322664432 Add test for running composer with --no-system-repos option 2019-09-24 12:50:19 +03:00
Jiri Kortus f978d4b01c [tests] Use functions for starting and stopping lorax-composer 2019-09-24 12:50:19 +03:00
Sanne Raymaekers 4643afa58d Makefile: Update bots target for moved GitHub project
Cockpit bots are in their own project now.

Make the target phony so that `make bots` updates an existing checkout.
2019-09-24 10:20:56 +02:00
Martin Kolman 6f01aedc8a Keep the zramctl utility from util-linux on boot.iso
Anaconda uses zram to allow installation on low memory systems.
We used to have a custom script called "zram-stats", that can be
used to test and debug zram usage during installation.

The script no longer works & zramctl now provides much better
output than our script ever did. So we decided to decommission
the old Anaconda provided script & use zramctl instead.

So change the cleanup rule in the Lorax boot.iso template
to keep the zramctl utility.

Related: rhbz#1561773
2019-09-20 08:52:50 -07:00
Alexander Todorov 58e762f3a1 Skip kickstart tar test for fedora-30/tar scenario
b/c Fedora-30 install sources contain older anaconda version
2019-09-16 14:09:43 +03:00
Alexander Todorov 52f84adde3 Enable fedora-30/tar test scenario
Related to
https://github.com/cockpit-project/cockpit/pull/12768
2019-09-16 14:09:43 +03:00
Alexander Todorov 8af9723c37 [tests] Collect compose logs after each build
- save compose logs under /var/log/$TEST
- save qemu logs under /var/log/$TEST/qemu.log
- download everything to $TEST_ATTACHMENTS so it can be saved
  in CI results
2019-09-13 12:16:22 +03:00
Jiri Kortus 836f0ddf7a [tests] Use a function to wait for compose to finish
The function is meant to replace the duplicate pieces of code used in various
tests, the polling interval is unified to 20 seconds.
2019-09-13 12:16:22 +03:00
Alexander Todorov ea98969014 When launching AWS instances wait for the one we just launched
because the ec2.instances variable will return all that are
currently running
2019-09-11 10:46:53 +02:00
Jiri Kortus 421cc9c22f tests: Add kickstart tar installation test 2019-09-09 09:28:52 +02:00
Jiri Kortus 562eb0a15d tests: add option to disable kernel command line parameters check 2019-09-09 09:28:52 +02:00
Brian C. Lane beff4f6967 tests: Use a loop to wait for VM and sshd to start 2019-09-09 09:28:52 +02:00
Greg Turner 66cd1fdb52 creator.py: include dmsquash-live-ntfs by default
Add dmsquash-live-ntfs to the default dracut modules in livecd builds.

The omission of this is probably why:

  https://bugzilla.redhat.com/show_bug.cgi?id=1449410

didn't work out as intended (I suspect it was closed unfixed).

Without this, initramfs winds up with the ntfs-3g tools lying
around, but no hooks cause them to actually get used.

AFAICT, this is a legit bugfix, not a "proposed enhancement"
so to speak.  To test that this actually fixed the problem (well,
/a/ problem, at least), I used some scripts:

  https://github.com/gmt/test-respin-fedora-ntfsable-iso

Signed-off-by: Greg Turner <gmt@be-evil.net>
Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-09-04 11:06:26 -07:00
Alexander Todorov a0322afec3 Skip Azure test b/c misconfigured infra & creds
Also causes failures on the Cockpit side of things, see
https://github.com/cockpit-project/cockpit/pull/12645
2019-09-04 14:51:37 +02:00
Brian C. Lane 3ee07001b1 tests: Drop tito from the Dockerfile.test
It isn't needed to run the tests, and currently a package it depends on
(cheetah) is not working in rawhide.

Also bump glusterfs version to 7.*
2019-09-03 11:02:52 -07:00
Brian C. Lane 1581842132 tests: Drop sort from compose types test
The output from the API is already sorted, and 'sort' changes this
ordering in some situations.
2019-08-29 08:17:29 -07:00
Alexander Todorov 64ac615fa4 Revert "tests: Fix the order of liveimg-tar live-iso"
This reverts commit 3b6c6a35d7.
2019-08-29 08:17:29 -07:00
Alexander Todorov 114c1bbda7 New test: assert toml files in git workspace
tests: export BLUEPRINTS_DIR for use in tests

Depending on the way the tests are run the directory may be a temporary
dir, or it may be the standard /var/lib/lorax/... path.

Related: rhbz#1714103
2019-08-29 08:17:29 -07:00
Brian C. Lane 584cad72a0 Automatic commit of package [lorax] release [31.10-1].
Created by command:

/bin/tito tag
2019-08-20 17:30:10 -07:00
Brian C. Lane cda497f0cb tests: Update gpg key to fedora 32 2019-08-20 17:23:08 -07:00
Brian C. Lane 3b6c6a35d7 tests: Fix the order of liveimg-tar live-iso 2019-08-20 17:23:08 -07:00
Brian C. Lane 4da4b665ef tests: Use server-2.repo instead of single.repo
And use the working directory for the graft, not the root filesystem.
2019-08-20 17:23:08 -07:00
Brian C. Lane 6f686ff9d6 lorax-composer: Add support for dnf variables to repo sources
This loads the system dnf vars from /etc/dnf/vars at startup if
system repos are enabled, and it substitutes the values in the sources
when loaded, and when a new source is added.

Also includes tests.
2019-08-20 16:26:03 -07:00
David Lehman b8c1e706bb Use smarter multipath detection logic.
This new setting for 'find_multipaths' tries to prevent things like
LVM from going ahead and activating LVM on the individual disks/paths
until there is reasonable certainty (via a timeout) that the device
is not a component of a multipath set.

NOTE: 'smart' is supported by device-mapper-multipath v0.7.7 and later.
2019-08-20 16:23:15 -07:00
Brian C. Lane 40bb80f10f tests: Expand test coverage of the v0 and v1 sources API 2019-08-09 13:16:12 -07:00
Brian C. Lane 0935a10a0b tests: Temporarily work around rpm and pylint issues
See https://bugzilla.redhat.com/show_bug.cgi?id=1739167

Something in rpm changed and pylint isn't able to find the rpm
constants.
2019-08-09 13:16:03 -07:00
Brian C. Lane 1e88a99443 lorax-composer: Add v1 API for projects/source/
This changes the source 'name' field to match the DNF usage of it as a
descriptive string. 'id' is now used as the short name to refer to the
source. The v0 API remains unchanged.

Tests for v1 behavior have been added.

Now that the v1 API is in use the status message will return api: 1
2019-08-07 13:26:35 -07:00
Brian C. Lane 278214ff8e Add /api/v1/ handler with no routes 2019-08-06 16:08:10 -07:00
Brian C. Lane f5bb2dca8c Move common functions into pylorax.api.utils
This is so that they can be used from multiple versions of the API. eg.
v0.py, v1.py, etc.
2019-08-06 15:44:36 -07:00