Commit Graph

2327 Commits

Author SHA1 Message Date
Brian C. Lane 146560b959 Add upload status examples to compose route docstrings 2019-10-16 09:17:53 -07:00
Brian C. Lane b91d742ecb tests: Add tests for deleting unknown upload and profile 2019-10-16 09:17:53 -07:00
Brian C. Lane b23c73e52c Add docstrings to the new upload functions in pylorax.api.queue 2019-10-16 09:17:53 -07:00
Brian C. Lane d16e8f5bfc Change /compose/uploads/delete to /upload/delete
Remove the requirement to pass the compose uuid and the upload uuid --
only the upload uuid is needed.
2019-10-16 09:17:53 -07:00
Brian C. Lane 577c56ebaa tests: Add test for /compose/uploads/delete
Test deleting an upload from a compose.
2019-10-16 09:17:53 -07:00
Brian C. Lane b307ff0430 tests: Add tests for /compose/uploads/schedule
Test it with a profile, and with one-time use settings. It does not
actually upload anything, it only tests that the upload is scheduled.
2019-10-16 09:17:53 -07:00
Brian C. Lane 0eda252d61 Add profile support to /uploads/schedule/
Pass in the name of a profile in the `profile` field, or pass in
one-time use settings in the `settings` object.
2019-10-16 09:17:53 -07:00
Brian C. Lane fd2330c4b7 tests: Fix comments about V1 API results including uploads data 2019-10-16 09:17:53 -07:00
Brian C. Lane acbf63013a lifted: Make sure inputs cannot have path elements
This processes the inputs with os.path.basename to strip off any path
elements and prevent potential path traversal attacks. Also adds a test.
2019-10-16 09:17:53 -07:00
Brian C. Lane 7396c272b2 Use consistent naming for upload uuids
Since we have both compose uuids and upload uuids they need to be
clearly named. This updates the upload naming to use 'upload_uuid' in
the inputs, and 'upload_id' in the output (_id instead of _uuid for
consistency with build_id naming in the status responses).

This also adds 'upload_id' to the /upload/log response.
2019-10-16 09:17:53 -07:00
Brian C. Lane ade8945da3 tests: Add tests for new upload routes
This tests the routes for saving a profile, listing profiles, deleting
profiles, as well as composing with upload.

The composes run fake composes with upload data, one selects a profile
the other passes in the settings. No actual upload is done, but it tests
that the info, log, and cancel routes work.

This also updates the test setup to copy over the share/lifted directory
so that the providers are available to the tests.
2019-10-16 09:17:53 -07:00
Brian C. Lane f8f06ccd23 Fix some docstrings in the v1 API 2019-10-16 09:17:53 -07:00
Brian C. Lane fd55bee85d lifted: Make sure providers list is always sorted 2019-10-16 09:17:53 -07:00
Brian C. Lane c384b687ba Add /upload/providers/delete route
This allows deleting a provider's profile. Pass the provider and profile
name like:

/api/v1/upload/providers/delete/azure/test-settings

A standard json response will be returned.
2019-10-16 09:17:53 -07:00
Brian C. Lane b2fc391677 lifted: Add delete_profile function and tests
Also adds a helper to providers, _get_profile_path, so that the code
doesn't need to be repeated in all the functions.
2019-10-16 09:17:53 -07:00
Brian C. Lane 5d7aa95f2f Add support for starting a compose upload with the profile
This extends the /compose/ route to support uploading with either an
existing profile, or with one-time use settings passed in the POST.

To select a profile include the provider and profile, as returned by
`/uploads/providers`:

    "upload": {
      "image_name": "My Image",
      "provider": "azure",
      "profile": "production-azure-settings"
    }
2019-10-16 09:17:53 -07:00
Brian C. Lane bc06abceab lifted: Add a function to load the settings for a provider's profile
This returns the settings after loading them and running verify_settings
on them.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1030dd083c Fix pylint errors in lifted.upload 2019-10-16 09:17:53 -07:00
Brian C. Lane de314a0046 tests: Add yamllint of the lifted playbooks 2019-10-16 09:17:53 -07:00
Brian C. Lane 4da569442c tests: Add tests for the new lifted module
These tests cover most of the module's functions, except for the queue
monitor, and the actual execution of the playbooks.
2019-10-16 09:17:53 -07:00
Brian C. Lane 45498f2b93 All providers should have 'supported_types'
even if it is empty.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1b84f90963 lifted directories should be under share_dir and lib_dir
Otherwise passing --sharedir pointed to some other path will not use the
correct providers.
2019-10-16 09:17:53 -07:00
Brian C. Lane 1a6bc098d9 tests: Add tests for API v1
Output from some of these are different from API v0. Instead of mixing
the two this moved v1 tests into a new class - ServerAPIV1TestCase to
make them easier to maintain, and removes the v1 tests from
ServerAPIV0TestCase
2019-10-16 09:17:53 -07:00
Brian C. Lane 3a453eaad7 Make sure V0 API doesn't return uploads information
uploads should only be included in the V1 API routes, add `api`
selection to the relevant helper functions and calls to them from v0.py

Add new V1 routes with api=1 to include the uploads information in the
results.

Also add tests to ensure that V0 requests do not include uploads.
2019-10-16 09:17:53 -07:00
Evan Goode 2692e8138c Automatically upload composed images to the cloud
Currently, Azure, vSphere, and OpenStack are supported. See
https://github.com/weldr/lorax/pull/826 for more details about this new
feature.

I've called the upload library "lifted" as a reference to Seuss'
The Lorax -- in the book, the Lorax lifts himself up by the seat of his
pants through a hole in the smog clouds, and they start calling him the
"Lifted Lorax."

This adds new features to the /compose route under API v1
2019-10-16 09:17:53 -07:00
Evan Goode 584a2d2076 Add load and dump to pylorax.api.toml 2019-10-16 09:17:53 -07:00
Martin Pitt a2c67385e4 Support CI testing against a bots project PR
* If `$COCKPIT_BOTS_REF` is set, check out that bots version instead of
   master.

 * Use git cache in $XDG_CACHE_HOME if available. Our CI uses that to
   save downloads, and it does not get in the way for local developers.

Adapted from https://github.com/cockpit-project/starter-kit/pull/233
2019-10-14 20:57:25 +02:00
Martin Pitt 29c308ae73 Makefile: Don't clobber an existing bots checkout
Commit 4643afa58 was a thinko -- for our CI we *don't* want our test to
clobber a pre-existing bots/ checkout, as we often use this to run tests
against an updated image or to validate a changes to the bots project.

On developer machines, bots may also be a symlink to an actual bots
directory in development, so don't clobber that.

Stop making "bots" a phony target, and drop the now unnecessary
`[ -d bots ]` check.

Adapted from https://github.com/cockpit-project/starter-kit/pull/232
2019-10-14 20:57:25 +02:00
Brian C. Lane 697233c14a lorax-composer: Handle RecipeError in commit_recipe_directory
A recipe that is valid TOML can still be an invalid recipe (eg. missing
the 'name' field) so this should also catch RecipeError.

Also added tests for this, as well as making sure commit_recipe_file()
raises the correct errors.

Resolves: rhbz#1755068
2019-10-04 08:07:36 -07:00
Brian C. Lane 3aac31482c test: Disable pylint subprocess check check
The return value for subprocess is already being checked, no need to use
check=True.
2019-09-30 18:11:46 -07:00
Brian C. Lane 6cebc3da88 Automatic commit of package [lorax] release [32.0-1].
Created by command:

/bin/tito tag --keep-version
2019-09-30 16:56:49 -07:00
Brian C. Lane 5efaa876e1 aarch64: Fix live-iso creation on aarch64
Drop unneeded uboot-tools, and remove iso-graft from the aarch64.tmpl

Related: rhbz#1752002
2019-09-25 11:07:20 -07:00
Jiri Kortus 5322664432 Add test for running composer with --no-system-repos option 2019-09-24 12:50:19 +03:00
Jiri Kortus f978d4b01c [tests] Use functions for starting and stopping lorax-composer 2019-09-24 12:50:19 +03:00
Sanne Raymaekers 4643afa58d Makefile: Update bots target for moved GitHub project
Cockpit bots are in their own project now.

Make the target phony so that `make bots` updates an existing checkout.
2019-09-24 10:20:56 +02:00
Martin Kolman 6f01aedc8a Keep the zramctl utility from util-linux on boot.iso
Anaconda uses zram to allow installation on low memory systems.
We used to have a custom script called "zram-stats", that can be
used to test and debug zram usage during installation.

The script no longer works & zramctl now provides much better
output than our script ever did. So we decided to decommission
the old Anaconda provided script & use zramctl instead.

So change the cleanup rule in the Lorax boot.iso template
to keep the zramctl utility.

Related: rhbz#1561773
2019-09-20 08:52:50 -07:00
Alexander Todorov 58e762f3a1 Skip kickstart tar test for fedora-30/tar scenario
b/c Fedora-30 install sources contain older anaconda version
2019-09-16 14:09:43 +03:00
Alexander Todorov 52f84adde3 Enable fedora-30/tar test scenario
Related to
https://github.com/cockpit-project/cockpit/pull/12768
2019-09-16 14:09:43 +03:00
Alexander Todorov 8af9723c37 [tests] Collect compose logs after each build
- save compose logs under /var/log/$TEST
- save qemu logs under /var/log/$TEST/qemu.log
- download everything to $TEST_ATTACHMENTS so it can be saved
  in CI results
2019-09-13 12:16:22 +03:00
Jiri Kortus 836f0ddf7a [tests] Use a function to wait for compose to finish
The function is meant to replace the duplicate pieces of code used in various
tests, the polling interval is unified to 20 seconds.
2019-09-13 12:16:22 +03:00
Alexander Todorov ea98969014 When launching AWS instances wait for the one we just launched
because the ec2.instances variable will return all that are
currently running
2019-09-11 10:46:53 +02:00
Jiri Kortus 421cc9c22f tests: Add kickstart tar installation test 2019-09-09 09:28:52 +02:00
Jiri Kortus 562eb0a15d tests: add option to disable kernel command line parameters check 2019-09-09 09:28:52 +02:00
Brian C. Lane beff4f6967 tests: Use a loop to wait for VM and sshd to start 2019-09-09 09:28:52 +02:00
Greg Turner 66cd1fdb52 creator.py: include dmsquash-live-ntfs by default
Add dmsquash-live-ntfs to the default dracut modules in livecd builds.

The omission of this is probably why:

  https://bugzilla.redhat.com/show_bug.cgi?id=1449410

didn't work out as intended (I suspect it was closed unfixed).

Without this, initramfs winds up with the ntfs-3g tools lying
around, but no hooks cause them to actually get used.

AFAICT, this is a legit bugfix, not a "proposed enhancement"
so to speak.  To test that this actually fixed the problem (well,
/a/ problem, at least), I used some scripts:

  https://github.com/gmt/test-respin-fedora-ntfsable-iso

Signed-off-by: Greg Turner <gmt@be-evil.net>
Signed-off-by: Brian C. Lane <bcl@redhat.com>
2019-09-04 11:06:26 -07:00
Alexander Todorov a0322afec3 Skip Azure test b/c misconfigured infra & creds
Also causes failures on the Cockpit side of things, see
https://github.com/cockpit-project/cockpit/pull/12645
2019-09-04 14:51:37 +02:00
Brian C. Lane 3ee07001b1 tests: Drop tito from the Dockerfile.test
It isn't needed to run the tests, and currently a package it depends on
(cheetah) is not working in rawhide.

Also bump glusterfs version to 7.*
2019-09-03 11:02:52 -07:00
Brian C. Lane 1581842132 tests: Drop sort from compose types test
The output from the API is already sorted, and 'sort' changes this
ordering in some situations.
2019-08-29 08:17:29 -07:00
Alexander Todorov 64ac615fa4 Revert "tests: Fix the order of liveimg-tar live-iso"
This reverts commit 3b6c6a35d7.
2019-08-29 08:17:29 -07:00
Alexander Todorov 114c1bbda7 New test: assert toml files in git workspace
tests: export BLUEPRINTS_DIR for use in tests

Depending on the way the tests are run the directory may be a temporary
dir, or it may be the standard /var/lib/lorax/... path.

Related: rhbz#1714103
2019-08-29 08:17:29 -07:00