Commit Graph

229 Commits

Author SHA1 Message Date
Brian C. Lane b9d18216ef lorax-composer: Add liveimg-tar image type
This creates a tar suitable for use with the anaconda kickstart liveimg
command. It adds the kernel, grub2, and grub2-tools packages to the tar
template.
2019-08-05 14:50:32 -07:00
Brian C. Lane 361c6de2de livemedia-creator: Use --compress-arg in mksquashfs
Previously the --compress-arg option was only used for tar, this adds
support for it to the squashfs.img creation used with live isos.
2019-08-05 14:49:25 -07:00
Brian C. Lane 835c912e12 Only use repos with valid urls for test_server.py
libdnf-0.22.5-5 changed something and now the repos with fake urls are
failing when loaded by test_server.py (they still work fine with
test_projects.py) so only use the 'good' repos with the test_server.py
tests -- the others weren't needed for any of its tests anyway.
2019-08-05 14:48:23 -07:00
Brian C. Lane c46f9f034f tests: Update test_creator.py
make_squashfs has been removed, make_runtime is now used in all paths to
create the install.img

Add a tests for squashfs only and squashfs+ext4 (requires loop so only
runs as root).
2019-07-29 13:21:25 -07:00
Brian C. Lane 0708302464 Use binary mode to tail the file
In python 3 f.seek() on text doesn't work like it does in py2/C because
text is now unicode. So change read_tail to use byte mode and take
unicode into account. Also add tests for it.
2019-07-11 15:16:41 -07:00
Jiri Kortus 9bf8d8a2fc Use passwd --status for locked root account check 2019-07-11 12:47:18 +03:00
Brian C. Lane 3cb007a25b tests: Use liveuser account for live-iso boot check
On rawhide you cannot ssh in as root without changing PermitRootLogin,
and really we should be testing that liveuser can login not root.
2019-07-11 11:47:11 +03:00
Brian C. Lane d0e947fe3b tests: Add check to make sure the compose actually finished
Previously it was looping, waiting for FINISHED|FAILED but was not
actually failing the test if the compose failed to build.
This adds a function to check the status of the compose and calls it
after each compose.
2019-07-10 08:58:24 -07:00
Lars Karlitski fd57e1106e tests: set skip_if_unavailable in test repos
dnf seems to have changed the default for skip_if_unavailable. Some
mock repositories are still around in later tests, which then fail
because metadata cannot be synced.

Also expose skip_if_unavailable in dnf_repo_to_file_repo(), so that
tests checking for equality of repo files continue to pass.
2019-07-09 00:23:33 +02:00
Evan Goode 998d0140be Assert that RuntimeErrors have correct messages 2019-06-28 08:55:36 -07:00
Alexander Todorov e70aae69da Remove unused shell script
the file is not used anywhere, nor is the findtestfiles() function
defined inside of it.
2019-06-28 11:23:31 +03:00
Alexander Todorov e947e01331 Do not generate journal.xml from beakerlib
bacause this requires additional Python modules and we don't
really use it! Fixes
[ WARNING  ] :: cannot create journal.xml due to missing python interpreter
2019-06-25 14:38:59 +02:00
Brian C. Lane f1733369fa tests: Add tests for recipe checking functions 2019-06-25 11:44:58 +02:00
Brian C. Lane 61059a2699 lorax-composer: Add basic recipe checker function
This makes sure that required fields are included, and that sections are
not empty. It does not check for all optional fields.

If there are errors it will gather up all of them and then raise a
RecipeError with a string of all the errors.
2019-06-25 11:44:58 +02:00
Lars Karlitski 61fc4d2b4e test: Fix test_blueprint_sanity
The new toml library, introduced with abe7df34f, outputs different
whitespace from the old one. Fix the test expectation and strip()
results from toml.dumps(), because it contains superfluous newlines at
the end.
2019-06-24 23:43:08 +02:00
Brian C. Lane 682f8a9295 tests: rpm now returns str, drop decode() call 2019-06-24 20:15:21 +02:00
Brian C. Lane fb89e6f275 Update qemu arguments to work correctly with nographic
Add -monitor none to turn off the qemu monitor multiplexing.
Pass -boot d for -cdrom booting instead of 'c'.

Add 'console=ttyS0,115200n8' to the boot arguments so that kernel output
will show up on the serial port.
2019-06-18 14:28:58 -07:00
Brian C. Lane abe7df34fc Switch to new toml library
The previous library, pytoml from https://github.com/avakar/pytoml is no
longer supported. So this adds a compatibility layer on top of the
suggested replacement, toml from https://github.com/uiri/toml
2019-06-18 14:01:15 -07:00
Brian C. Lane 932ff5812c composer-cli: Update diff support for customizations and repos.git
composer-cli will now output information about changes to customizations
entries and the repos.git entries.
2019-06-13 14:16:48 -07:00
Brian C. Lane 9011a564e8 Add support for customizations and repos.git to /blueprints/diff/
This also includes extensive tests for each of the currently supported
customizations. It should be generic enough to continue working as long
as the list of dicts includes a 'name' or 'user' field in the dict.
Otherwise support for a new dict key will need to be added to the
customizations_diff function.
2019-06-13 14:16:48 -07:00
Brian C. Lane 316257fbc0 tests: Update custom-base with customizations 2019-06-13 14:16:48 -07:00
Alexander Todorov f61dfcc2c7 Increase retry count/sleep times when waiting for lorax to start
the biggest slow down is fetching data for many repositories
over a slow network. The previous retry count and sleep times
sometimes are not enough on Fedora.
2019-06-10 13:26:11 +03:00
Alexander Todorov 688f70eefa Revert "remove the check for qemu-kvm"
Partially reverts commit 303a69bcbd.
2019-06-10 13:26:11 +03:00
Alexander Todorov 22b0eb28bc Revert "remove the check for /usr/bin/docker in the setup phase"
this is still helpful when executing these test scripts manually
outside Cockpit CI or in Jenkins (for gating).

Partially reverts commit 5f5a2d5337.
2019-06-10 13:26:11 +03:00
Alexander Todorov b725eb141a [tests] Define unbound variables in test scripts 2019-06-06 14:56:40 +02:00
Alexander Todorov 649a9e2239 [tests] Handle blueprints in setup_tests/teardown_tests correctly
It's necessary to make sure the blueprints directory doesn't contain
the git/ directory before the tests are run, so that we can just simply
modify the blueprint files without using blueprints push.

Related: rhbz#1714298
2019-06-06 14:56:40 +02:00
Alexander Todorov 16fd1a2340 [tests] grep|cut for IP address in a more robust way
because sometimes these VMs may get an IPv6 address assigned
and we don't want to cut only the first part of it
2019-06-06 14:56:40 +02:00
Lars Karlitski 42c7c0691c Monkey-patch beakerlib to fail on first assert
Beakerlib upstream can't do this yet, but might at some point:

https://github.com/beakerlib/beakerlib/issues/42

This is only enabled in combination with the `--sit` option of the
`test/check-*` scripts. It leaves the system in exacly the state it was
in when an assertion failed. Finishing the test run would run cleanup as
well (such as deleting created images). It also takes longer.
2019-06-04 20:12:54 +02:00
Lars Karlitski a4dcc34396 test_cli.sh: Return beakerlib's exit code
Read the return status directly from beakerlib's TestResults file. This
is more robust when running the same test multiple times.
2019-06-04 20:12:54 +02:00
Lars Karlitski 4bd03cb8f6 Don't send CORS headers
These are meant for web applications that are accessed by browsers, not
REST APIs.
2019-06-04 20:10:03 +02:00
Lars Karlitski a164ed3d7d tests: Set BLUEPRINTS_DIR in all cases
`setup_tests()` expected BLUEPRINTS_DIR to be set, but it wasn't when
running in automated mode (with $CLI set).

Fix this and move share and blueprint dirs to function arguments.
2019-06-04 14:07:36 +03:00
Lars Karlitski ce3a277100 tests: Fail on script errors
Fixes #727
2019-06-04 14:07:36 +03:00
Lars Karlitski 8ed910b29a composer: Set up a custom HTTP error handler
Override flask's default error handler, because that return html. Return
JSON instead with the usual { "status": false, "errors": [ ... ] }
pattern.
2019-06-04 11:50:25 +03:00
Alexander Todorov b71e8f74d8 Configure $PACKAGE for beakerlib reports
this makes it more clear that we are testing the installed RPM
instead of sources
2019-05-31 12:57:23 +02:00
Alexander Todorov b67ce5379c Use cloud credentials during test if they exist 2019-05-31 12:57:23 +02:00
Jan Stodola ed8a21a627 Use ansible instead of awscli
Not all parts of the script has been switched from awscli to ansible yet,
because the ansible aws modules do not support importing s3 object as snapshots.
(https://github.com/ansible/ansible/issues/53453)
Workaround using the image_location parameter of the ec2_ami ansible module
would mean adding extra code for generating AMI manifest with pre-signed
URLs.
2019-05-24 12:44:30 +03:00
Lars Karlitski 1e5a5921e3 tests: Source lib.sh from the right directory
When testing in one of the VMs, tests are in /tests. Source the lib from
a relative directory instead.
2019-05-24 00:23:21 +02:00
Brian C. Lane 6120f74c73 Revert "Add rpmfluff temporarily"
This reverts commit 6ea1c45734.

python-rpmfluff-0.5.7.1-1 is now available.
2019-05-23 14:57:46 -07:00
Brian C. Lane 795cc3400c tests: Update tmux version to 2.9a 2019-05-23 14:57:46 -07:00
Lars Karlitski c261cec33d tests: Fail immediately when image build fails
We were checking for composer's FINISHED status only, which meant that
when a compose failed, the test ran until it timed out.

Check for failed as well. Also, always time out after 30 minutes.
2019-05-22 08:24:49 +02:00
Lars Karlitski bdffaa02f4 tests: Re-enable kvm
In 303a69bc, we stopped using kvm, because it doesn't exist on all test
builders. Enable it again on those that have it.
2019-05-21 22:45:32 +02:00
Lars Karlitski 303a69bcbd tests: Don't depend on kvm for tar and qcow2 tests
Some test runners don't have nested virtualization enabled. Because
these checks are only checking that a boot works, kvm doesn't give us
that much. Disable for now.

Also remove the check for qemu-kvm. It doesn't abort the test
prematurely anyway.
2019-05-20 20:24:01 +02:00
Lars Karlitski 77acc8972b test_compose_tar: Work around selinux policy change
A compose can change the hosts' policy, which can lead to docker
crashing if the container-selinux policy is not included. Add a
workaround and bug link.
2019-05-20 20:24:01 +02:00
Lars Karlitski ca171424eb test_compose_tar: Be less verbose
A huge chunk of the output log is tar's file list. Remove that to make
the log more readable.
2019-05-20 20:24:01 +02:00
Lars Karlitski 5f5a2d5337 test_compose_tar: Fix docker test
The docker phase always failed because `-ti` was passed even though the
the output was not a terminal.

Also remove the check for /usr/bin/docker in the setup phase. It didn't
test that the daemon was running. More importantly, it didn't abort the
test anwyay (and there doesn't seem to be a good way to do this in
beakerlib).
2019-05-20 20:24:01 +02:00
Lars Karlitski 1ba2e6fd4b tests: Extract images to /var/tmp, not /tmp
Test VMs don't have enough memory (/tmp is a tmpfs).
2019-05-20 20:24:01 +02:00
Lars Karlitski 250f49f78d Use Cockpit's test images and infrastructure
Allows to run the tests on multiple operating systems and on the
infrastructure that the Cockpit team maintains.

`make vm` downloads one of Cockpit's test images (override which one
with TEST_OS) and installs rpms build from the local checkout of lorax.
The resulting image is placed in `test/images/$TEST_OS`.

TEST_OS can be set to any of Cockpit's supported images (default:
fedora-30).

Run `make check-vm` to run the CLI checks in the VM. The bulk of the
work is done in `test/check-cli`, which uses Cockpit's `bots` library to
start the VM and run the script in it.

Also included is a `test/run` script, which is the entrypoint for
Cockpit's test infrastructure.
2019-05-20 20:24:01 +02:00
Lars Karlitski aa9c2e13b1 pylint: Remove unused false positive 2019-05-20 20:24:01 +02:00
Jiri Kortus 123b5c269d Add test for passing custom option on kernel command line 2019-05-07 12:36:32 +03:00
Jiri Kortus 596cba2255 Use verify_image function as a helper for generic tests 2019-05-07 12:36:32 +03:00