Summary:
per details in T759, the 'unipony' updates image we use to test
the updates image features doesn't work with latest anaconda (f24
and Rawhide). I've built a new updates image which uses a neat
anaconda feature that allows you to override CSS with a file in
a special location; it sets the background for disk capacity
texts on the INSTALLATION DESTINATION spoke to be pink. This
lets us use a simple needle that just looks for a pink blob on
that spoke, on the basis that it's unlikely there'll ever be a
pink blob there for any other reason, so if there is one, the
updates image worked. There will be an accompanying tools diff
to change the updates disk image to use the new updates image.
Test Plan:
Do a test run and check the updates image tests pass
and no other tests are broken. You'll need to pull in the tools
diff and re-generate the updates disk image to check that test,
the scsi_updates_img test should work with just this diff.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D799
Some needle changes to account for latest Rawhide. There seems
to be a bug between anaconda and GTK+ 3.20 causing text in the
disk selection widget not to be styled correctly: this commit
adds needles for that which are tagged as 'workaround', meaning
openQA will complete the tests but mark them as requiring a
workaround to work ('soft fail'). The bug is #1322036. There is
also an intentional change to the volume list in custom part;
the mount point names are now in dark grey rather than black (in
fact this has always been intended, but it's been broken for a
long time, according to davidshea). I cleaned up the part select
needles at the same time; the new ones have the non-variant
names, as davidshea says the same change is coming to F24 and
the original needles haven't matched for months. The F23 Atomic
install test doesn't hit any of these needles, so we don't need
to keep the original needles around for that. The most recent
variants are kept, as they'll be needed for F24 tests until the
new anaconda build goes stable for F24. The 'freetype262'
variant of part_select_swap hasn't matched for two months and is
dropped.
Summary:
First off, this revises the anaconda crash handling needles a
bit. We ditch gtk3195 and update anaconda_error to reflect
current F24/Rawhide. We keep the old anaconda_error around for
now as anaconda_error-23, to handle crashes in the F23 two-week
Atomic nightlies. We also add an 'early' variant, which is for
when (I think) the installer crashes very early, before it's
loaded in GTK+ settings; when that happens, the dialog uses a
different font. The screenshot comes from a recent Rawhide test
that crashed.
We also restore the anaconda `post_fail_hook` code to click
the Report button when a crash happens. This was erroneously
removed in D637. Before the Report button is clicked, the
`anaconda-tb` file exists but the libreport stuff in `/var/tmp`
does not. By removing this, we lost the libreport bits from
the uploaded files, which makes it harder to report crashes. So
let's add it back.
Finally we fix the actual tarring and uploading of `/var/tmp`;
also in D637 this got broken because it was being tarred up in
whatever directory the commands happened to be running in, but
we were still trying to upload it from `/var/tmp`.
https://openqa.stg.fedoraproject.org/tests/8444 was run with
these changes, and has `/var/tmp` correctly uploaded.
Test Plan:
Run some test that crashes, make sure the crash
handling all works correctly.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D768
GTK+ 3.19.10(?) has changed things in current F24/Rawhide;
weirdly, done-freetype262 seems to match sometimes but not
always. Add a new needle from a current screenshot but keep
freetype262 around for now just in case. Drop some variants
that should never be needed any more, rename the original
needle to -23 to mark that we're only keeping it around for
the F23 two-week Atomic tests. For now keep french-cantarell20
as the 'official' French needle, we'll see if it hits any
failures when the tests are working a bit better.
mostly cantarell changes. GTK+ changes are bugs, so put 'bug'
in the name to remind us of this. There's one non-typical
change, to main_hub_installation_source; I removed the match
on the text label of the spoke entirely instead of updating it,
in line with other similar needles, it's unnecessary and only
serves to make the needle more fragile.
3.19.5 seems to change the colors of buttons slightly and also
the rendering of checkboxes/radio buttons, they're now placed
somewhat differently relative to their labels.
garretraziel took these with F23, I think. Theoretically we
could just have these ones and ditch his, as we're not running
KDE install tests on F23, but we might want to discuss that kind
of question more generally, so for now let's just add the
variant.
We may need more variants for the post-install matches, but the
test doesn't actually get that far ATM because of dependency
issues in the KDE package set, so I'll deal with that once we
can actually do a successful KDE install.
Summary:
As I figured out a while back, the seabios firmware font is
nearly the same as our console font (eurlatgr) but not quite,
significantly its @ character is different (shorter). We
already have a root_logged_in variant for the firmware font,
but we now need one for user_logged_in as well. The Atomic
installer image has had the Linux console fonts stripped from
its payload, so it's now using the firmware font, and user_
logged_in isn't matching - see
https://openqa.fedoraproject.org/tests/461 .
Test Plan:
Run the cloud_atomic_canned test (on an F23
post-release nightly, as it's not appearing for Rawhide
nightlies ATM) and confirm it now passes.
Reviewers: garretraziel, jskladan
Reviewed By: jskladan
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D684
add KDE package set testcase - this only adds needles and
new testcase configuration, because with needles cleanup from D670,
it should work without change in code.
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D673
Let's use the same principle with DESKTOP tag as we use with
LANGUAGE tag. Where there are any GNOME-only (or KDE-only) needles, tag
them with `DESKTOP-gnome` and delete then when different `DESKTOP` value
is specified. This will help with `QA:Testcase_Package_Sets_KDE_Package_Install`
testcase - we will be able to use almost the same code and check that KDE
got installed properly.
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D670
Summary:
The 'reclaim_space_first_partition' needles aren't very good.
Using the area with the filesystem means we need to duplicate
the needle for NTFS, and the needles in fact also match the
*second* partition in most cases, sometimes causing the tests
to delete the second partition instead of the first and fail.
This design should work better: a single needle should match
all filesystems, and by including some of the highlighted top
row in the match, we should never match on the second row.
Including 'da' in the needle makes this really 'first disk,
first partition', but that's all the cases we currently use
it for. We don't really need this needle to work hard to make
sure we're on the right screen, as the subsequent needles
(which match buttons that only exist on this screen) do that
job fine.
We drop some needles that should no longer be necessary: F23
should now be using the cantarell17 version of the needle,
and F24 the freetype262 needle. We don't run installer tests
for any other release.
We could tweak second_partition needle to match, but nothing
is actually using it anyway.
Test Plan:
Do a full test and run and make sure the tests
which use this needle work, and always select the correct
partition.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D681
freetype 2.6.2 changes font rendering quite significantly.
Most text needles needed redoing. Committing without review
so maybe we finally get some damn useful test results...
Summary: seems like this got lighter for F24 too.
Test Plan:
Run the shrink tests and see if they get past this
screen.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D666
Summary:
this needle seems to be fragile not because of cantarell in
fact, but because the difference between the the left edge of
the language box and the 'F' is not consistent. So, let's just
not include it in the match.
Test Plan:
Check if French install test works (see test #8847
failure in BOS).
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D661
This adds another Cantarrell 1.8 missing needle as well as
lighter variant of Done button (it's perhaps caused by displayed
warning about missing swap partition on no_swap test?). With this
in place, all tests on Rawhide passes except of KDE (but it seems
that KDE Live images are borked) and desktop upgrade tests.
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D651
Summary:
Instead of sitting there waiting 6000 seconds twice, when DNF
explicitly tells us it failed, just die.
This is why we haven't been getting proper compose check reports
lately; the upgrade tests are failing, waiting 6000 seconds to
time out, then being cloned and tried again, waiting another 6000
seconds. This is just barely going beyond check-compose's 8 hour
wait limit, as it's some time before the upgrade tests even get
started (they're low in the priority list). We're still going to
have that problem if the tests fail any other way, but this at
least catches that case.
Test Plan:
Run the upgrade tests and see that they fail quicker
(assuming the dependency problems they're dying on aren't fixed).
Maybe also do a 22-23 upgrade test and check it still succeeds
properly.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D650
Cantarell 0.0.18 showed up in Rawhide recently and changes
rendering again, so we need some more variant needles. Once
Cantarell change quiets down a bit we'll have to clean these
all up.
Since we are now logging to user account every time,
I've encountered new bug - we are missing needle for azerty
user logged in. This DR adds it.
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D640
Summary: didn't catch this before (as it depends on the image tested)
Test Plan: Test a Server ISO, the ext3 test fails without this
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D622
Summary:
So another couple of Cantarell updates hit Rawhide and changed
rendering a bit more, leading to this needle breaking. This is
for Cantarell 0.0.17.2. No other needles seem to have problems.
Test Plan:
Already in place on BOS - note the 2015-10-15 Rawhide
tests that initially failed but now pass.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D621
Summary:
T617 makes some good points about the language tags; this is my
suggestion for an improvement. It requires a bit of cleverness
in unregister_prefix_tags(), but the upshot is that you don't
need to know to set any special tags when creating needles, a
needle with no language-related tags will be considered as valid
for all languages. You have to explicitly add LANGUAGE- tag(s)
to a needle for the language filtering to 'kick in' in any way.
If a needle has at least one LANGUAGE- tag, it will be filtered
unless it has the appropriate tag for the job's specified
language (default is still 'english').
With this approach, only needles which we specifically want to
*only* match their tagged language(s) need the tags, so we can
drop all those -ALL tags.
We're using LANGUAGE- instead of ENV-LANGUAGE- now because the
ENV- tag names denote tags that are treated slightly specially
by openQA, and this is not one. We cannot cleanly use
ENV-INSTLANG because openQA has a hardwired default of 'en_US'
for that.
Test Plan:
Check both English and French tests still work as
intended.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D589
Summary:
We have these 'atomic installer' images (so far just Cloud),
and maxamillion wanted to get them tested. Turns out it's
pretty trivial - they look much like other installs. Only
little wrinkle is they have a reduced hub (no repository
needles) like live images, but are not like live images in
any other way, so I rejigged the 'small hub needle filtering'
handling a bit.
There will be an accompanying diff for tools, and also some
changes in fedfind (these images are getting built nightly
for *current stable*, and it'd be good to test those).
Because we'd like to test the 22 nightlies, I had to add some
needles for 'olddpi' versions of a few screens. See 2e4c1c2 -
the 22 Atomic installer images still have the old GTK+ code
meaning they run at 96.09dpi. I only retook the necessary
needles for the default-install test, if we add any others we
made need to retake a few more needles.
Test Plan:
Schedule jobs for a compose with the atomic installer
image. You will need the matching openqa_fedora_tools diff and
the very latest git fedfind. Check the test for that image runs,
all other tests run as usual, excessive images are not
downloaded, and the atomic installer is not used for running
universal tests.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D595
Summary:
This removes a couple of unused needles. password_required
needles were dupes of console_password_required, and no tests
actually use password_required. tmp_failed_proc just isn't
used by anything at all (I think it was at some point used in
fedup tests).
This also drops the ENV-OFW-1 tag from all needles, which I
think got in because someone based a needle on a SUSE needle
which has that tag, then it spread. It's of no use or meaning
in Fedora.
Test Plan: Check all tests still run as normal.
Reviewers: garretraziel, jskladan
Reviewed By: jskladan
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D579
Summary:
this handles Non-English European Language Install. Basically
it's a bunch of new screenshots for existing tag names, plus
a bit of configurability in _boot_to_anaconda and tweaking some
existing needles to do non-text matches. The weird 'half-the-
icon' needles are for cases where there may or may not be a
warning triangle but we want to click it either way (saves
duplicating the needle).
This also sets up a convention for tagging what languages a
needle is appropriate for. If it's specifically appropriate for
one or more languages, a tag ENV-LANGUAGE-(LANGUAGE) should be
applied for each language, where (LANGUAGE) is the install
language in upper-case ('LANGUAGE' variable, which should also
be the string that will be typed into the language selection
screen). If the needle ought to be used for *all* languages -
i.e. it's not a text match, or any text in the match is known
not to be translated - the tag ENV-INSTLANG-ALL should be
applied.
To back this, main.pm now unregisters all needles that are not
tagged with either ENV-LANGUAGE-ALL or the tag for the language
actually being used (if the LANGUAGE var is not set, we assume
english). The point of this is to check the install is actually
translated; if we allow all needles to match, the test would
pass even if no translations appeared at all.
Test Plan:
Run all tests and make sure you get the expected
results. You can schedule a run against 23 Beta TC1 to see the
French test fails 'correctly' when translations are missing.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D577
Summary:
This is a first cut which more or less works for now. Issues:
1) We're not really testing the BUILD, here. All the test does
is try and upgrade to the specified VERSION - so it'll be using
the latest 'stable' for the given VERSION at the time the test
runs. This isn't really that terrible, but especially for TC/RC
validation, we might want to make things a bit more elaborate
and set up the repo for the actual BUILD (and disable the main
repos).
2) We'd actually need --nogpgcheck for non-Rawhide, at one
specific point in the release cycle - after Branching but
before Bodhi activation (which is when we can be sure all
packages are signed). This won't matter until 24 branches, and
maybe releng will have it fixed by then...if not, I'll tweak
it.
3) We don't really test that the upgrade actually *happened*
for desktop, at the moment - the only thing in the old test
that really checked that was where we checked for the fedup
boot menu entry, but that has no analog in dnf. What we should
probably do is check that GUI login works, then switch to a
console and check /etc/fedora-release just as the minimal test
does.
Test Plan:
Run the tests. Note that creating the desktop disk
image doesn't work ATM, so I can't verify the desktop test
works, but the minimal one seems to (with D565). There'll be
a matching diff for openqa_fedora_tools to update the test
case names there.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
Summary:
To understand this change see test 1092 on BOS. When you click
Done once and then get the 'click Done again to proceed' warning,
the Done button seems to grows a little dotted line (I think
this indicates it's selected), which causes the match to drop to
94% and fail. To counter that, shrink the match area to the bit
of the button inside the dotted line, which will be the same
whether it's there or not.
Test Plan:
See if no_swap stops failing sometimes now, maybe,
at last? Yeesh. Also see if all other tests still run OK.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D549
