Summary:
There's a weird problem that we hit occasionally only on the
Server image, see e.g.:
https://openqa.fedoraproject.org/tests/14300
I *think* what's happening (thanks hergertme from #gtk+ IRC for
help with this) is that when the 'install done!' text and the
reboot button appears, there's a short transition for the button
so it doesn't look *exactly* like the screenshot right away. If
openQA's needle match loop happens to kick in and check the
match right after the text and button appear, while the
transition is happening, it decides that some random area of the
gradient down the left hand side of the screen matches *better*
than the button, so it 'clicks' that, and we don't get a reboot.
If the openQA needle match loop happens to kick in a millisecond
or whatever later, after the button is done transitioning, the
button is now a 100% match and the bug doesn't happen. This only
happens on Server because the left-hand side gradient differs in
each flavor, and only the Server one just happens to have a spot
which is nearly identical to the needle area.
So to avoid that, let's include a bit of the grey space next to
the button. This is a bit more fragile against GTK+ changes to
button design or padding or whatever, but it will at least avoid
the match issue.
The other option would be to try and hit the button with tab and
enter or something, I guess.
Test Plan:
bit hard since this is an intermittent bug, I guess
run install_default on a Server image a bunch of times and see if
it always works.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D824
GTK+ 3.19.10(?) has changed things in current F24/Rawhide;
weirdly, done-freetype262 seems to match sometimes but not
always. Add a new needle from a current screenshot but keep
freetype262 around for now just in case. Drop some variants
that should never be needed any more, rename the original
needle to -23 to mark that we're only keeping it around for
the F23 two-week Atomic tests. For now keep french-cantarell20
as the 'official' French needle, we'll see if it hits any
failures when the tests are working a bit better.
Let's use the same principle with DESKTOP tag as we use with
LANGUAGE tag. Where there are any GNOME-only (or KDE-only) needles, tag
them with `DESKTOP-gnome` and delete then when different `DESKTOP` value
is specified. This will help with `QA:Testcase_Package_Sets_KDE_Package_Install`
testcase - we will be able to use almost the same code and check that KDE
got installed properly.
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D670
Summary:
T617 makes some good points about the language tags; this is my
suggestion for an improvement. It requires a bit of cleverness
in unregister_prefix_tags(), but the upshot is that you don't
need to know to set any special tags when creating needles, a
needle with no language-related tags will be considered as valid
for all languages. You have to explicitly add LANGUAGE- tag(s)
to a needle for the language filtering to 'kick in' in any way.
If a needle has at least one LANGUAGE- tag, it will be filtered
unless it has the appropriate tag for the job's specified
language (default is still 'english').
With this approach, only needles which we specifically want to
*only* match their tagged language(s) need the tags, so we can
drop all those -ALL tags.
We're using LANGUAGE- instead of ENV-LANGUAGE- now because the
ENV- tag names denote tags that are treated slightly specially
by openQA, and this is not one. We cannot cleanly use
ENV-INSTLANG because openQA has a hardwired default of 'en_US'
for that.
Test Plan:
Check both English and French tests still work as
intended.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D589
Summary:
this handles Non-English European Language Install. Basically
it's a bunch of new screenshots for existing tag names, plus
a bit of configurability in _boot_to_anaconda and tweaking some
existing needles to do non-text matches. The weird 'half-the-
icon' needles are for cases where there may or may not be a
warning triangle but we want to click it either way (saves
duplicating the needle).
This also sets up a convention for tagging what languages a
needle is appropriate for. If it's specifically appropriate for
one or more languages, a tag ENV-LANGUAGE-(LANGUAGE) should be
applied for each language, where (LANGUAGE) is the install
language in upper-case ('LANGUAGE' variable, which should also
be the string that will be typed into the language selection
screen). If the needle ought to be used for *all* languages -
i.e. it's not a text match, or any text in the match is known
not to be translated - the tag ENV-INSTLANG-ALL should be
applied.
To back this, main.pm now unregisters all needles that are not
tagged with either ENV-LANGUAGE-ALL or the tag for the language
actually being used (if the LANGUAGE var is not set, we assume
english). The point of this is to check the install is actually
translated; if we allow all needles to match, the test would
pass even if no translations appeared at all.
Test Plan:
Run all tests and make sure you get the expected
results. You can schedule a run against 23 Beta TC1 to see the
French test fails 'correctly' when translations are missing.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D577
Summary:
Most of the anaconda screenshots were taken with older netinst
images, at 96.09dpi. Workstation lives use 96dpi exactly. This
led to some needles failing on the Workstation live as fonts
render slightly differently at 96 and 96.09dpi. LONG STORY
SHORT, we got a change in GTK+ which means traditional install
images now use 96dpi exactly too; so we should be consistent
between lives and trad images now, but we need to retake all
the screenshots taken at 96.09dpi. That's what this is.
I also cleaned up a couple of other things. The anaconda_error
needle was effectively a duplicate of anaconda_report_btn.
Neither is actually used atm, but I kept report_btn in case we
use it in future.
anaconda_main_hub_installdest_selected isn't needed any more,
since my tweaks to the matching in anaconda_main_hub_nonlive;
it will now match the case this needle was created for initially.
anaconda_software_select_box is no longer used by any test. The
software selection tests were changed to use key sequences and
dropped their use of this needle, but the needle wasn't removed.
anaconda_select_install_lang_english_filtered_english_selected_
as_default was never really necessary, the situation it's meant
for can be handled by simply adding the anaconda_select_install_
lang_english_filtered tag to the anaconda_select_install_lang_
english_selected needle (as this commit does). That achieves
the needed effect and doesn't break the case where the UI is in
Czech.
Most of the match area changes are just because the layout
changed a bit since the screenshot was taken. I did reduce some
over-precision in some needles, notably the multi-disk needles
- there is no reason we need to match on the disk descriptions
and sizes when we're just selecting the appropriate disks. We
aren't testing anaconda's description of the disks (which may
change unexpectedly), only that it *works* with the disk
types used in the test.
After this commit a little under half of the tests pass with
the 23 Branched 20150717 nightly. Most of the ones that fail
fail because of the anaconda tty2 bug (there's no console on
tty2) or the selinux bug (#1243168) that prevents console login
post-install with SELinux enabled. There are a few other fails
too, but none of them seems to be caused by a needle mismatch,
so I'll investigate and fix them separately.
Test Plan:
Run the full test suite on a current nightly, see that some
tests actually pass now!
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D456
This requires adding products, flavors and needles and test
cases, and tweaking some existing ones to handle the
slightly different behaviour of live images in shared tests.
To handle the different main hub screens in live and non-live,
a less stringent needle is added which is unregistered for
non-live tests, so they don't match on it before they've
finished updating repository metadata.
There are a few small bugfix tweaks in this too, like some
delays in user creation to try and avoid intermittent failures
there.
A new root_logged_in needle is also included, to handle a new
console font in Rawhide - that has nothing strictly to do with
live testing, it just happened to come up while working on
this.
Adam Williamson
Jan Sedlák
Josef Skladanka