openQA has some problems handling needles with identical names
in different subdirectories. We haven't had the cycles to send
fixes for this yet, so for now, let's just rename all such
needles we have.
Not sure if the color change will occur in Fedora 25 nightlies
or not, so not sure if we can remove the old needles. But we
*can* remove the F24 needles now I think.
Summary:
This adds a couple of new exporter modules, renames main_common
to utils (this is a better name: openSUSE's main_common is
functions used in main.pm, utils is what they call their module
full of miscellaneous commonly-used functions), and moves a
bunch of utility functions that were previously needlessly
implemented as instance methods in base classes into the
exporter modules. That means we can get rid of all the annoying
$self-> syntax for calling them.
We get rid of `fedorabase` entirely, as it's no longer useful
for anything. Other base classes keep the 'standard' methods
(like `post_fail_hook`) and methods which actually need to be
methods (like `root_console`, whose behaviour is different in
anacondatest and installedtest).
Test Plan:
Do a full test suite run and check everything lines
up. There should be no functional differences from before at all,
this is just a re-org.
Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Subscribers: tflink
Differential Revision: https://phab.qa.fedoraproject.org/D1080
Summary:
Since 26.17, anaconda shows a warning when the user password
contains non-ASCII characters, and requires a second Done click
to confirm. This change should handle that.
On the 'catch cases where password typing went wrong and re-try'
bit: to keep that, but not re-type the password *every single
time* on the Russian install test, we'd have to make the needle
match the text of the warning. This is problematic because then
that needle will be able to break without us easily noticing;
that's why I wanted to keep the 'warning bar' needle text-free.
Unfortunately, that means we have to skip the protection for
switched-layout installs.
Note the protection was actually not working for any non-English
install anyhow, because the needle had `LANGUAGE-english` as a
tag. We never noticed that. Failed password typing is pretty
rare now, so we can live without the protection - it's just nice
to have it for the English install tests because there's so many
of them.
Test Plan:
Run the Russian install with a recent Rawhide image,
check it clicks 'Done' twice. Note, it will still fail, because
of RHBZ #1413813.
Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Subscribers: tflink
Differential Revision: https://phab.qa.fedoraproject.org/D1084
Summary:
This adds a new test, memory_check, which just does a default
package set install with `inst.debug` parameter then uploads
the memory usage file (`/tmp/memory.dat`) at the end. We can
have check-compose use the data to analyze changes in memory
usage over time.
Test Plan:
Fire off the Workstation network install image tests
and make sure the memory usage test runs and works on all three
machines. This is live on staging already.
Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames
Subscribers: tflink
Differential Revision: https://phab.qa.fedoraproject.org/D1082
`man printf` says \" is treated as a quote, but not \'. So
let's have the command use double quotes to wrap the format,
and escape any double quotes in the text. Hope this works.
We've never really done any kind of periodic merge from develop
to master, or used master for anything, so the master/develop
split seems fairly pointless. With the move to Pagure, let's
just make master the main branch.
The README looks pretty ugly on Pagure. So let's unwrap it.
Let's also move the function docs into the source files. We're
much more likely to keep them up to date that way, I think. We
should probably change over to proper perl POD documentation at
some point, but comments in-line are OK for now I think.
The F25 'desktopencrypt' images built by createhdds boot to a
console passphrase entry screen, not graphical. F24 images
built the same way boot to a graphical passphrase entry screen.
I'm not sure why, but it's not really worth spending a lot of
energy on, I don't think - let's just add a needle to cover
this case.
The old version wasn't working - it was passing even though two
services fail to start in Workstation currently. I'm really not
sure why the old approach wasn't working, but it wasn't, and I
rather hate `script_output` anyway, so here's a different way
of doing it which relies on `eval`ing `assert_script_output`
instead. (I really should send a PR for a non-fatal version of
assert_script_output...)
Without this, when there are failed services, we get an extra
column to the left of the service names with a unicode dot for
each failed service, which is awkward and screws up the parsing.
Summary:
Include some basic testing of Japanese input, and split the
input testing (including Russian) into a separate module, since
it's not really part of 'login' testing.
Test Plan:
Run the test, and the Russian and French tests too to
make sure they didn't break. Tested on staging. Note the Japanese
test soft fails, intentionally, at present, as I discovered a bug
while working on it:
https://bugzilla.gnome.org/show_bug.cgi?id=776189
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1072
Using 'host' is, for some reason, causing the problem where non-
minimal installs fail to boot. No idea why, but switching the
CPU model to Nehalem solves it.
Summary:
This isn't in the criteria, but it's commonly used, so we ought
to test this way. Require authentication for the iSCSI target
and have the test provide the appropriate auth info.
Test Plan:
Run the iscsi test and check it works (you need the
recent fixes for support_server to make *that* work). Nothing
else should be affected.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1070
Summary:
The non-English tests so far did not test that graphical login
worked as expected, which is a fairly large hole. With this
change, they should do a Workstation install and test login to
both GNOME and the console works as expected. KDE is not yet
tested.
As part of this we tweak the implementation of keyboard layout
switching in graphical environments to use a generic function
in main_common which can handle both anaconda and desktops
(just GNOME at present, but should extend easily to any desktop
with a known switcher key and a visible layout indicator),
replacing the anacondatest class method. I kinda don't like that
the test has to specifically tell the function when it's in
anaconda, but I don't think I want to start experimenting with
a global 'test phase' openQA variable or anything like that at
present.
Fixes T842.
Test Plan:
Run the French and Russian install tests and check
they work as expected. Also run an English Workstation install
if you like, and make sure that didn't break. This change is
live on staging ATM, seems to work fine.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Maniphest Tasks: T842
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1071
This should solve all those annoying "Failed to synchronize
cache for repo 'updates'" failures we've had: there's no need
for the 'updates' repository to be enabled when we've decided
we want the `repo_setup` changes to be made, and having it
enabled causes problems when we run right after the Rawhide
compose completes. We hit the awkward period where the rawhide
repo has been synced but mirrormanager has not been updated
with the new metadata checksums, so mirrormanager rejects the
metadata from dl.fp.o and DNF has to go out and hit other
mirrors until it finds one which didn't sync yet. Since the
point of `repo_setup` is specifically to hack up the config so
we only use packages from the compose *anyway*, there's no
reason at all to worry about leaving 'updates' enabled and
nerfing it like we do 'fedora' and 'rawhide', we can just turn
it off.
support_server was failing because of #1402427. I re-generated
the disk image with latest F25 so the fixed selinux-policy is
used, but even then, it seems we have to run 'restorecon' on
rpcbind manually before starting nfs.
Committing without review as this causes failures...try to make
sure we only run the AVC test when it makes sense, and fix
running it on the French install test.
Summary:
This has all console tests check for AVCs (with ausearch) and
crashes (with coredumpctl) at post-install stage. It's non-
fatal as this doesn't really mean the test failed, but we want
to spot when there are unexpected AVCs or crashes.
Test Plan:
Run some console tests, check it works right. I only
tested with one test, since so many are broken on Rawhide ATM
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1066
Committing without review as this is pretty trivial. Running
top *after* all the other collect_data tasks leads to some
meaningless fluctuations in the output; I intended to look for
activity caused by stuff running 'as usual', not activity from
the other collect_data tasks.
Summary:
I've been wanting to do this for a while; I think it'll let us
check for some significant changes between composes. This should
cause runs of a few test cases to collect and upload info on:
* installed packages
* free memory
* disk space
* active services
* 1 minute of CPU usage info (via top)
immediately after install and initial login. In some cases this
will be useful / interesting simply to look at directly, but
we can also have check-compose analyze the data and include
significant changes in its reports.
Test Plan:
Run affected tests, make sure the data collection
works.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1046
Summary:
The current installedtest post_fail_hook assumes /var/tmp/abrt
exists at all, and dies if it doesn't, leading to no /var/log
upload. We can also avoid using openQA `script_output` - which
is annoyingly indirect and slow - by using this neat `test -n`
trick I found on SO. Let's also use it in the anacondatest
post_fail_hook to avoid uploading /var/tmp when it's empty
(which we currently do). This also drops the 0 arg from a few
more script_run calls, because it's safe to wait for the run
to complete and we should probably do so to avoid later typing
errors if the commands are slow.
Test Plan:
Cause both anaconda and installed tests to fail and
check the hooks work as intended. Maybe twiddle the failures to
ensure directories do and don't exist and/or have contents and
make sure things work OK. I've tested this to some degree and
I'm pretty sure it works right.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1041
It's not always in minimal installs. This is a simple change
and needed to make the post-fail hook work for minimal installs,
so pushing without review.
Summary:
GNOME's update notification criteria are pretty braindead: it
fires the update check timer once on login then once every hour
thereafter, but only actually checks for and notifies of updates
once a day if it's after 6am(?!?!?!). So we have to do a bunch
of fiddling around to ensure we reliably get a notification.
Move the clock to 6am if it's earlier than that, and reset the
'last update check' timer to 48 hours ago, then log in to GNOME
after that.
Note: I thought this still wasn't fully reliable, but I've looked
into all the recent failures of either test on staging and
there's only one which was really 'no update notification came
up', and the logs clearly indicate PK did run an update check,
so I don't think that was a test bug (I think something went
wrong with the update check). The other failures are all 'GNOME
did something wacky', plus one case where the needle didn't quite
match because I think the match area is slightly too tall; I'll
fix that in a second.
Test Plan:
Run the tests on both KDE and GNOME and check they
work properly now (assuming nothing unrelated breaks, like KDE
crashing...)
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1039
We're not really *testing* shutdown here, we're just shutting
down to make sure the uploaded disk image is clean. So we don't
really mind if shutdown takes a while. It often seems to take
longer than 1 minute on KDE installs and cause a soft fail, so
let's bump the timeout to 3 minutes.
These keep failing because they don't work right. I have it
partially fixed on a branch but not fully fixed yet, and it's
been this way for a few weeks already, so let's disable the
tests for now until I can actually complete the fix.
