This effectively reverts 97618193 - but had to be done manually
and adjusted to maintain support for testing side tags and for
testing multiple tasks, since those features were added since
the update ISO change.
The 'scheduler injects ISOs of packages into the tests' approach
was intended to speed things up, especially for large updates,
and it did, but it had a few drawbacks. It means restarting
older tests from the web UI doesn't work as the ISOs get garbage
collected (you have to re-schedule in this case). And it has the
rather large problem that you can now only schedule tests from
the openQA server (or at least a machine with the openQA asset
share mounted), because the package download and ISO creation
just happen wherever the scheduler is running and assume that
the openQA asset share that will be used by the tests is at
/var/lib/openqa/share in that filesystem.
That's too big of a drawback to continue with this approach, IMO,
so this reverts back to the old way of doing things, with a bit
of refactoring to clean up the flow a little, and with support
for testing side tags and multiple tasks maintained.
As a follow-up I'm going to see if I can replace
_download_packages with a much more efficient downloader script
to mitigate the time this process takes on each test, especially
for large updates.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I'm attempting a new approach to the update and workaround repos.
Instead of having each update test recreate them for itself -
which is slow and wastes bandwidth - the dispatcher will create
an ISO at test schedule time and pass it as ISO_2. Then the test
just mounts the ISO. This makes the necessary adjustments on the
test side.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
At least one test (desktop_notifications_postinstall) boots from
the disk image uploaded by install_default_upload, and needs to
access the grub menu. On F29+ Workstation this is failing,
because the grub menu is now hidden by default, so when the test
boots, it never sees the bootloader screen, and fails.
I considered trying to teach it to hold down shift or hit f8 or
esc at the right time, but that seems like it might be hard. So
instead let's just try to disable the hidden menu when we're
about to upload the installed system image. This is kinda going
against the 'preserve natural system behaviour' principle we try
to use for openQA, but I think it's OK as we do have other tests
that will exercise the 'hidden boot menu' stuff to some extent.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
...as somehow a Workstation live install currently has the
desktop on tty3, I have no idea why (g-i-s not quitting right?)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
It's not really a good idea to have the comments that explain
the test_flags in *every* test, because they can go stale and
then we either have to live with them being old or update them
all. Like, now. So let's just take 'em all out. There's always
a reference in the openQA and os-autoinst docs, and those get
updated faster.
More importantly, add the new `ignore_failure` flag to relevant
tests - all the tests that don't have the 'important' or
'fatal' flag at present. Upstream killed the 'important' flag
(making all tests 'important' by default), I got it replaced
with the 'ignore_failure' flag, we now need to explicitly mark
all modules we want the 'ignore_failure' behaviour for.
We're not really *testing* shutdown here, we're just shutting
down to make sure the uploaded disk image is clean. So we don't
really mind if shutdown takes a while. It often seems to take
longer than 1 minute on KDE installs and cause a soft fail, so
let's bump the timeout to 3 minutes.
Summary:
Since we can match on multiple needles, we can drop the loop
from console_login and instead do it this way, which is simpler
and should work better on ARM (the timeouts will scale and
allow ARM to be slow here). Also move it to main_common as
there's no logical reason for it to be a class method.
Also remove the `check` arg. `check` was only set to 0 by two
tests, _console_shutdown and anacondatest's _post_fail_hook.
For _console_shutdown, I think I just wanted to give it the
best possible chance of succeeding. But we're really not going
to lose anything significant by checking, the only case where
check=>0 would've helped is if the 'good' needle had stopped
matching, and all sorts of other tests will fail in that case.
anacondatest was only using it to save a screenshot of whatever
was on the tty if it didn't reach a root console, which doesn't
seem that useful, and we'll get screenshots from check_screen
and assert_screen anyway.
Test Plan:
Run all tests, check they behave as expected and
none inappropriately fails on console login.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1016
Summary:
I believe the failures in the Server DVD chained Base tests are
happening because the VM is not cleanly shut down before the disk
image is uploaded. This adds a shutdown step to all tests that
upload a disk image (so, for now, just default_install). To keep
things simple it just runs 'shutdown' from a root console, rather
than using graphical desktop shutdown methods, as the aim is only
to make the disk state clean, not to test shutdown exactly.
I've tested this on staging; a Server DVD test run with this
change produced a full set of passed tests, as opposed to all
the Base tests failing because the system didn't boot properly.
Workstation and KDE tests seem to work fine also.
For the record, SUSE does much the same thing as this commit.
Test Plan:
Do a full test run and make sure everything that worked
before still does. Check that all default_install tests have a
_console_shutdown step added, and it works, and all chained tests
work (or fail for some unrelated reason, but make sure this
doesn't break them).
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D787
