Commit Graph

1148 Commits

Author SHA1 Message Date
Adam Williamson 5c03c435ca Stop having two 'firefox close tabs' needle tags, drop old ones
We still have a 'apps_run_firefox_stop' needle tag which is for
the same thing as 'firefox_close_tabs'. That's dumb. Get rid of
it and only have the firefox_close_tabs tag and needles. Also
clean up some old firefox_close_tabs needles that haven't matched
for months and all the 'apps_run_firefox_stop' needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-29 12:10:35 -08:00
Adam Williamson e7a7c053bd Update firefox_close_tabs needle
Seems to look different in a recent Rawhide test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-27 11:08:44 -08:00
Adam Williamson e8802935f9 Tweak text_console_login match area
On Rawhide Cloud_Base boots, there's some SSH key and network
information printed above the 'login:' prompt, so we can't
expect empty space there. Also tweak console_login() to clear
the screen after logging in, so the login prompt is cleared and
doesn't confuse things on subsequent runs (like it did first
time we tried this). And add a new user logged in needle, as it
seems after we clear the screen the tilde appears in a slightly
different position and the existing needle doesn't match.

https://openqa.fedoraproject.org/tests/489003#step/_console_wait_login/7

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-27 10:39:29 -08:00
Adam Williamson 411f0d34e7 Revert "Tweak text_console_login match area"
This reverts commit f66f510832.
Turns out this breaks lots of stuff, so it needs to go back.
I'll have to figure another way to handle the cloud login.
2019-11-27 08:36:35 -08:00
Adam Williamson f66f510832 Tweak text_console_login match area
On Rawhide Cloud_Base boots, there's some SSH key and network
information printed above the 'login:' prompt, so we can't
expect empty space there. Let's just hope not looking for the
empty space doesn't break anything else.

https://openqa.fedoraproject.org/tests/489003#step/_console_wait_login/7

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-26 08:26:27 -08:00
Adam Williamson 84a065296c Use @ not : for machine dependencies
Upstream decided @ was better, : still works for now but will
probably be taken out at some point. You need openqa
4.6-37.20191121git8fcf81f or higher for this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 12:43:18 -08:00
Adam Williamson 0b210fe44c Sleep a bit before clicking the anaconda icon on live install
KDE tests are quite frequently failing lately because anaconda
doesn't launch. I'm hoping this will help.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 09:54:47 -08:00
Adam Williamson 25cce2361c Update a KDE app test needle
Window title changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 09:13:27 -08:00
Adam Williamson dcbd7dd7c9 Add a comment explaining what autocloud.pm does
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-21 14:58:53 -08:00
Adam Williamson 54a6b21c28 Run autocloud tests on Cloud_Base qcow2 images
autocloud is dying soon. So, we want to run the tests in openQA
instead. This adds a test module called 'autocloud' and a test
suite called 'cloud_autocloud' which basically replicate what
autocloud does (download a tarball full of tests and run each
one), and the necessary template bits to run it on Cloud_Base
qcow2 images.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-21 14:29:43 -08:00
Lukáš Růžička 5a28c0c0f2 Add uefi variations of the minimal upgrade test case. 2019-11-20 19:00:13 -08:00
Adam Williamson 5c784e22cc Skip post-install actions for VNC client installs
...otherwise the VNC client tests fail on aarch64 because we try
to apply the 'console=tty0' workaround for #1661288. Fortunately
we don't really need that for the VNC install test to work, so
let's just skip it. We can make this more sophisticated later if
it turns out to be necessary.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 18:56:06 -08:00
Michel Normand bbb5d76849 install_vnc_client & install_vncconnect_client to be ARCH specific
Use  ARCH specific desktop_4 images supposed to be
generated by createhdds tool.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 14:05:28 -08:00
Adam Williamson 00c81f9a30 Drop 'anaconda' from serial console test name
It's really not useful.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:35:23 -08:00
Adam Williamson 82db1770d5 Hack around a serial console prompt check failing during install
os-autoinst has this code when doing `script_run` on a serial
terminal that does a `wait_serial` for '# ' before running the
command. For some reason when we switch to the anaconda root
console and run a couple of commands after finishing install on
ppc64, the first of these prompt checks times out, which means
the test sits there doing nothing for 90 seconds unnecessarily.
Let's try and avoid that by hacking the prompt check regex to
be empty.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson 67e7415f2f Fix a bug with stashing original serial_term_prompt setting
Whoops.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson defab3bfcb Correct renaming of the virtio serial consoles
That second string didn't work how I thought it did.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson 48b6c9d3e9 Change name we use for virtio serial consoles
There is nothing inherently 'root'-y about these so it makes no
sense to prefix their names with 'root-'. And why change from
'console' to 'terminal' compared to the naming used in the
actual qemu command and the log files? It's just confusing.
Let's be consistent (except for using - instead of _ here...
but - is easier to type!)

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 08:39:28 -08:00
Michel Normand 73ca0f2d3f Add missing groupe_name for PowerPC and AArch64 added tests
That concerns:
  test_suite => { name => "install_vnc_server" },
  test_suite => { name => "install_vnc_client" },
  test_suite => { name => "install_vncconnect_server" },
  test_suite => { name => "install_vncconnect_client" },

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 08:46:43 +01:00
Lukáš Růžička b93a197c22 Enable Anaconda Text install via serial console.
This adds the Anaconda text installation test over
serial console and FIXES #115.
2019-11-19 22:54:55 -08:00
Adam Williamson 0722cc5bcf Update Cockpit 'join domain' needle for cockpit 207
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-14 08:43:03 -08:00
Adam Williamson 3e6764dbb5 Fix French install test on aarch64 by skipping `uefi_postinstall`
The French install/boot test fails on aarch64 due to a bit of an
ordering issue:

https://openqa.stg.fedoraproject.org/tests/665124

we run `uefi_postinstall`, which does `loadkeys us`, before we
run `_console_login`, which still expects the French layout, so
it breaks. The safest way to solve this I think is to add a new
variable that lets us skip `uefi_postinstall` - I don't want to
change the ordering so we load `uefi_postinstall` after we read
`POSTINSTALL` in case that breaks things somewhere else.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:21:37 -08:00
Adam Williamson 13eb4ac2e6 Disable snapshotting on all language tests
There's no point at which the snapshot might be restored, and
this works around a difficult-to-debug crash on ppc64le (it seems
the crash only happens when resuming after the snapshot).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:00:14 -08:00
Adam Williamson 9bec7ee797 Bump timeout on exiting Firefox again...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-07 07:35:32 -08:00
Adam Williamson a8dc6c425c Use comma not space for QEMUMACHINE value
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:48:15 -08:00
Adam Williamson f2ef1f1a37 Workaround RHBZ #1769600 by using pseries-4.0 machine type
ppc64le tests are failing to boot at all with stock qemu in F31.
Using pseries-4.0 machine type (the default is pseries-4.1)
seems to work around the problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:41:15 -08:00
Adam Williamson 1c75256f9d Add new VNC tests to aarch64 and ppc64le too
Forgot to do this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 18:14:23 -08:00
Adam Williamson 4c05f19a9f Add QA:Testcase_Anaconda_User_Interface_VNC_Vncconnect tests
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 17:42:29 -08:00
Adam Williamson 4f72d4ebd3 Add one missing needle for VNC test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:48:25 -08:00
Adam Williamson b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson a0e191e097 Update cockpit login screen needle *again*
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 15:20:30 -07:00
Adam Williamson efec7010cb Handle an 'akonadi did something' notification in KDE, etc.
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:

https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34

we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 09:05:06 -07:00
Lukáš Růžička 907e27cd8f Change nodejs module version in the install test to make it pass. 2019-10-30 14:58:26 +01:00
Adam Williamson 47fe6acf73 Add a sleep to server_cockpit_autoupdate to work around #1765685
We don't know exactly what's going on with this bug yet, but as
a 15 second sleep seems to avoid the issue, let's do that for
now so we catch any future issues.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:29:24 -07:00
Adam Williamson 8c07149ec7 Workaround #1766451 in update tests
Download the fixed jss to the advisory repo.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:03:59 -07:00
Adam Williamson 764a2bdc33 Drop #1757948 workaround, shouldn't be needed any more
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 17:59:33 -07:00
Adam Williamson 81100e8a17 Add new Cockpit 'software updates' needle
Needed for the UI change in Cockpit 205.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 16:01:35 -07:00
Adam Williamson 4fb29f8bdb Add new Firefox GNOME app menu needle
Firefox icon changed, so this needs updating too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:55:52 -07:00
Adam Williamson 33e3224411 Drop stray `start_cockpit` export from utils.pm
It was moved to cockpit.pm, but this wasn't removed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:12:04 -07:00
Adam Williamson 3cdfe96a5c Add another Firefox launcher needle variant
Seems we need two as the launcher background is a different
color on different Fedora releases...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:53:52 -07:00
Adam Williamson 51d37aeeef Update Firefox launcher needles for new icon
Firefox icon changed a bit in 70.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:23:24 -07:00
Adam Williamson e313e58cdd Add new variant boot_enter_passphrase graphical needle
New plymouth has tweaked this screen's appearance.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:20:51 -07:00
Adam Williamson d4634e082c Use bypass_1691487 at the start of fedora_release
It gets bit by that problem on ppc64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-22 14:32:09 -07:00
Adam Williamson bf7d6d0149 Wait out animation in 'no user' 'not GNOME' flow
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-21 10:56:31 -07:00
Adam Williamson b12c529e34 Try and reduce timing failures in desktop_browser new tab
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-19 10:30:50 -07:00
Adam Williamson e1030118f9 Add new cockpit login needle
The login screen background changed in 205.1 (it's actually the
same image, but it's now zoomed rather than just a chunk of it
being shown on a smaller screen).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-18 10:35:14 -07:00
Lukas Ruzicka 22b185abf6 Enable tracking of executed applications in Application Start/Stop tests (FIXES #116)
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
2019-10-16 17:36:37 -07:00
Adam Williamson b455e04188 Add new cockpit needles for 205
The cockpit UI went all GRIMDARK so we need a couple of new
needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 17:10:28 -07:00
Adam Williamson 5f38971a78 Add some new variant layout indicator needles
Once again, the text is at a slightly different height to the
needles we already have. I do not know why this sometimes
happens. It's a mystery.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 16:34:55 -07:00
Adam Williamson b55782358d Don't check version or pre-release note on banners
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:

* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
  text is untranslated (e.g. 'Fedora 31') it appears on a
  different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
  area of the banner this can trigger an os-autoinst needle
  comparison bug: https://progress.opensuse.org/issues/56822

All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 11:08:24 -07:00