Revise release variable handling, prerelease checks, os-release

I started out trying to fix os-release for the recent change to
add "Prerelease" tags to the VERSION and PRETTY_NAME fields, then
things spiralled. It got me thinking about the awkward DEVELOPMENT
variable we use, so I decided to get rid of it and refactor the
few things that use it. I refactored the anaconda prerelease tag
check, and wrote a new giant comment that gives details about
exactly how anaconda decides whether to show those tags, to give
context to our choices about when to expect them. This check now
uses a new LABEL variable the scheduler now sets. I also wound up
creating new UP1REL and UP2REL vars to define the 'source' release
for upgrade tests, separate from CURRREL and PREVREL, which are
now never lies - they really are the current stable and previous
stable release, even for update upgrade tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>

VARIABLES.md

@@ -93,10 +93,12 @@ These variables should be set when tests are scheduled (when running `isos post`
 | `KOJITASK` | A Koji task ID. If set, the modified 'update testing' flow for testing scratch builds will be used: post-install tests will be run with the packages from the update, starting from the stable release base disk images |
 | `DISTRI` | contains distribution name (should be same as in WebUI, probably `fedora`) |
 | `VERSION` | contains version of distribution |
-| `DEVELOPMENT` | set for update tests if the update is for a development release (not a stable release) |
 | `FLAVOR` | indicates what type of distribution is used. Three Pungi properties, joined with `-`: `variant`, `type`, and `format`. e.g.: `Server-dvd-iso`. Special value `universal` is used to schedule the group of tests that should be run once each per arch per compose, against the 'best' available ISO |
 | `ARCH` | is set to architecture that will be used (`x86_64`, `i686`) |
 | `BUILD` | contains Pungi compose_id (something like `Fedora-24-20160121.n.3`) |
+| `LABEL` | contains Pungi compose label, if it has one (otherwise should be unset) - e.g `RC-1.5` |
 | `CURRREL` | the current stable Fedora release at the time of the test run |
 | `PREVREL` | the previous stable Fedora release at the time of the test run |
+| `UP1REL` | the source release for "1-release" upgrade tests (usually but not always same as `CURRREL`) |
+| `UP2REL` | the source release for "2-release" upgrade tests (currently always same as `PREVREL`) |
 | `LOCATION` | contains Pungi base compose location (something like `https://kojipkgs.fedoraproject.org/compose/branched/Fedora-25-20160901.n.0/compose/`) |

lib/utils.pm

@@ -488,12 +488,13 @@ sub _repo_setup_updates {
     # Check if we already ran, bail if so
     return unless script_run "test -f /etc/yum.repos.d/advisory.repo";
     my $version = get_var("VERSION");
+    my $currrel = get_var("CURRREL", "0");
     repos_mirrorlist();
     # this can be used for debugging repo config if something is wrong
 #    unless (script_run 'pushd /etc/yum.repos.d && tar czvf yumreposd.tar.gz * && popd') {
 #        upload_logs "/etc/yum.repos.d/yumreposd.tar.gz";
 #    }
-    if (get_var("DEVELOPMENT")) {
+    if ($version > $currrel) {
         # Disable updates-testing so other bad updates don't break us
         # this will do nothing on upgrade tests as we're on a stable
         # release at this point, but it won't *hurt* anything, so no
@@ -1046,38 +1047,83 @@ sub check_top_bar {
 
 sub check_prerelease {
     # This method is used by identification tests to check if
-    # Anaconda shows the PRERELEASE info for Rawhide versions.
+    # Anaconda shows the PRERELEASE tag on various screens. These are
-    # As it seems, the rules for showing the PRERELEASE are not
+    # the rules anaconda follows for deciding whether to do this, as
-    # as simple as it may look like. It seems to me, that the PRERELEASE
+    # of 2020-05-07:
-    # is shown on Rawhide intallations, which is pretty easy to test because
-    # the "VERSION" variable is Rawhide. On Final RC, the PRERELEASE should
-    # be turned off, while it should be present between before that.
-    # When Fedora is branched, the "VERSION" changes to the version number
-    # and this is not changed even with Final, so we cannot test that.
-    # Investigating the history, I noticed that all nightly builds have
-    # that prerelease turned on. And also, it is there on a Beta candidate.
-    # We can test for that. For update tests, if DEVELOPMENT is set the
-    # prerelease tags should appear, otherwise they should not.
 
-    my $beta = get_var('ISO');
+    # 1. If there's a /.buildstamp and/or /tmp/product/.buildstamp file
-    # install_pxeboot tests have ISO blanked, so we must use ISO_URL
+    # the installer environment, and/or the environment variable
-    $beta ||= get_var('ISO_URL');
+    # PRODBUILDPATH is set and points to a file that exists, it reads
-    my $nightly = get_var('BUILD');
+    # config from those file(s), in that order of precedence, and if
+    # the key 'IsFinal' exists in the section 'Main', its value is
+    # used as anaconda's `product.isFinal`. Installer images built by
+    # lorax have this buildstamp file, and it always sets IsFinal: if
+    # --isfinal was passed to lorax it is set to True, if not it is set
+    # to False. Whether lorax is run with --isfinal can be specified
+    # in the Pungi config, but there's also a heuristic: it usually
+    # defaults to False, but if the compose has a label and it's an
+    # 'RC' or 'Update' or 'SecurityFix' compose (see definition of
+    # SUPPORTED_MILESTONES in productmd.composeinfo), the default is
+    # True. AFAICS, Fedora's pungi configs don't explicitly set this,
+    # but rely on the heuristic. So for installer images, we expect
+    # isFinal to be True for RC candidate composes and post-release
+    # nightly Cloud, IoT etc. composes (these are also marked as 'RC'
+    # composes), but False for Rawhide and Branched nightly composes
+    # and Beta candidate composes. For installer images built by our
+    # own _installer_build test, we control whether --isfinal is set
+    # or not; we pass it if the update is for a stable release, we do
+    # not pass it if the update is for Branched. Live images do not
+    # have the buildstamp file.
+
+    # 2. If there's no buildstamp file, the value of the environment
+    # variable ANACONDA_ISFINAL is used as `product.isFinal`, default
+    # of False if that environment var is not set. The live installer
+    # wrapper script sets ANACONDA_ISFINAL based on the release field
+    # of whatever package provides system-release: if it starts with
+    # "0.", it sets ANACONA_ISFINAL to "false", otherwise it sets it
+    # to "true". So for live images, we expect isFinal to be True
+    # unless the fedora-release-common package release starts with 0.
+
+    # 3. If `product.isFinal` is False, the pre-release warning and
+    # tags are shown; if it is False, they are not shown.
+
+    # We don't really need to check this stuff for update tests, as
+    # the only installer images we test on updates are ones we build
+    # ourselves; there's no value to this check for those really.
+    # For compose tests, we will expect to see the pre-release tags if
+    # the compose is Rawhide, or a Beta candidate, or it's a nightly
+    # and we're checking an installer image. If it's an RC or Updates
+    # candidate, or a respin release, we expect NOT to see the tags.
+    # If it's a nightly and we're checking a live image, we don't do
+    # the check.
+
+    # bail if this is an update test
+    return if (get_var("ADVISORY OR TASK"));
+
+    # 0 means "tags MUST NOT be shown", 1 means "tags MUST be shown",
+    # any other value means we don't care
+    my $prerelease = 10;
+
+    # if this is RC or update compose we absolutely *MUST NOT* see tags
+    my $label = get_var("LABEL");
+    $prerelease = 0 if ($label =~ /^(RC|Update)-/);
+    # if it's a Beta compose we *MUST* see tags
+    $prerelease = 1 if ($label =~ /^Beta-/);
     my $version = get_var('VERSION');
-    my $development = get_var('DEVELOPMENT');
+    # if it's Rawhide we *MUST* see tags
-    # Let us see if the installed system is one that needs to have the prerelease
+    $prerelease = 1 if ($version eq "Rawhide");
-    # note shown, default is NO.
+    my $build = get_var('BUILD');
-    my $prerelease = 0;
+    # if it's a nightly installer image we should see tags
-    if ($beta =~ /Beta/ or $version eq "Rawhide" or $development) {
+    $prerelease = 1 if ($build =~ /\.n\.\d+/ && !get_var("LIVE"));
-        $prerelease = 1;
+    # if it's a respin compose we *MUST NOT* see tags
-    }
+    $prerelease = 0 if ($build =~ /Respin/);
-    # If the image is a nightly build or test build, it is not clear whether
+    # we *could* go to a console and parse fedora-release-common
-    # the prerelease should be there or not, but according to what we decided
+    # to decide if a nightly live image should have tags or not, but
-    # this is not a big issue. This piece of code can be used later, when
+    # it seems absurd as we're almost reinventing the code that
-    # we decide differently.
+    # decides whether to show the tags, at that point, and it's not
-    elsif ($nightly =~ /\.[nt]\.\d+/) {
+    # really a big deal either way whether a nightly live image has
-        $prerelease = 10;
+    # the tags or not. So we don't.
-    }
+
     # For all prerelease requiring ISOs, assert that prerelease is there.
     if ($prerelease == 1) {
         assert_screen "prerelease_note";

templates.fif.json

@@ -1873,7 +1873,7 @@
             "settings": {
                 "BOOTFROM": "c",
                 "DESKTOP": "gnome",
-                "HDD_1": "disk_f%PREVREL%_desktop_4_x86_64.img",
+                "HDD_1": "disk_f%UP2REL%_desktop_4_x86_64.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1889,7 +1889,7 @@
                 "BOOTFROM": "c",
                 "DESKTOP": "gnome",
                 "ENCRYPT_PASSWORD": "weakpassword",
-                "HDD_1": "disk_f%PREVREL%_desktopencrypt_x86_64.img",
+                "HDD_1": "disk_f%UP2REL%_desktopencrypt_x86_64.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1904,7 +1904,7 @@
             "settings": {
                 "BOOTFROM": "c",
                 "DESKTOP": "kde",
-                "HDD_1": "disk_f%PREVREL%_kde_4_x86_64.img",
+                "HDD_1": "disk_f%UP2REL%_kde_4_x86_64.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1920,7 +1920,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%PREVREL%_minimal_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP2REL%_minimal_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1933,7 +1933,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%PREVREL%_minimal-uefi_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP2REL%_minimal-uefi_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1946,7 +1946,7 @@
                 "fedora-universal-x86_64-*-64bit": 30
             },
             "settings": {
-                "+HDD_1": "disk_f%PREVREL%_server_3_%ARCH%.img",
+                "+HDD_1": "disk_f%UP2REL%_server_3_%ARCH%.img",
                 "BOOTFROM": "c",
                 "GRUB": "net.ifnames=0 biosdevname=0",
                 "GRUB_POSTINSTALL": "net.ifnames=0 biosdevname=0",
@@ -1970,7 +1970,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%PREVREL%_server_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP2REL%_server_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -1984,7 +1984,7 @@
                 "fedora-universal-x86_64-*-64bit": 30
             },
             "settings": {
-                "+HDD_1": "disk_f%PREVREL%_server_3_%ARCH%.img",
+                "+HDD_1": "disk_f%UP2REL%_server_3_%ARCH%.img",
                 "BOOTFROM": "c",
                 "GRUB": "net.ifnames=0 biosdevname=0",
                 "GRUB_POSTINSTALL": "net.ifnames=0 biosdevname=0",
@@ -2007,7 +2007,7 @@
             "settings": {
                 "BOOTFROM": "c",
                 "DESKTOP": "gnome",
-                "HDD_1": "disk_f%CURRREL%_desktop_4_x86_64.img",
+                "HDD_1": "disk_f%UP1REL%_desktop_4_x86_64.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -2023,7 +2023,7 @@
                 "BOOTFROM": "c",
                 "DESKTOP": "gnome",
                 "ENCRYPT_PASSWORD": "weakpassword",
-                "HDD_1": "disk_f%CURRREL%_desktopencrypt_x86_64.img",
+                "HDD_1": "disk_f%UP1REL%_desktopencrypt_x86_64.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -2036,7 +2036,7 @@
                 "fedora-universal-x86_64-*-64bit": 30
             },
             "settings": {
-                "+HDD_1": "disk_f%CURRREL%_kde_4_x86_64.img",
+                "+HDD_1": "disk_f%UP1REL%_kde_4_x86_64.img",
                 "BOOTFROM": "c",
                 "DESKTOP": "kde",
                 "ROOT_PASSWORD": "weakpassword",
@@ -2054,7 +2054,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%CURRREL%_minimal_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP1REL%_minimal_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -2067,7 +2067,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%CURRREL%_minimal-uefi_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP1REL%_minimal-uefi_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -2081,7 +2081,7 @@
                 "fedora-universal-x86_64-*-64bit": 30
             },
             "settings": {
-                "+HDD_1": "disk_f%CURRREL%_server_3_%ARCH%.img",
+                "+HDD_1": "disk_f%UP1REL%_server_3_%ARCH%.img",
                 "BOOTFROM": "c",
                 "GRUB": "net.ifnames=0 biosdevname=0",
                 "GRUB_POSTINSTALL": "net.ifnames=0 biosdevname=0",
@@ -2105,7 +2105,7 @@
             },
             "settings": {
                 "BOOTFROM": "c",
-                "HDD_1": "disk_f%CURRREL%_server_3_%ARCH%.img",
+                "HDD_1": "disk_f%UP1REL%_server_3_%ARCH%.img",
                 "ROOT_PASSWORD": "weakpassword",
                 "TEST_TARGET": "COMPOSE",
                 "UPGRADE": "1",
@@ -2120,7 +2120,7 @@
                 "fedora-universal-x86_64-*-64bit": 30
             },
             "settings": {
-                "+HDD_1": "disk_f%CURRREL%_server_3_%ARCH%.img",
+                "+HDD_1": "disk_f%UP1REL%_server_3_%ARCH%.img",
                 "BOOTFROM": "c",
                 "GRUB": "net.ifnames=0 biosdevname=0",
                 "GRUB_POSTINSTALL": "net.ifnames=0 biosdevname=0",

tests/_installer_build.pm

@@ -6,6 +6,7 @@ use utils;
 sub run {
     my $self = shift;
     my $version = get_var("VERSION");
+    my $currrel = get_var("CURRREL");
     my $advortask = get_var("ADVISORY_OR_TASK");
     my $arch = get_var("ARCH");
     assert_script_run "dnf -y install lorax", 90;
@@ -20,7 +21,7 @@ sub run {
     # some point, let's work around that for now:
     # https://bodhi.fedoraproject.org/updates/FEDORA-2020-1070052d10#comment-1284223
     $cmd .= " --rootfs-size 3" if ($version eq 30);
-    unless (get_var("DEVELOPMENT")) {
+    unless ($version > $currrel) {
         $cmd .= " --isfinal --repo=/etc/yum.repos.d/fedora-updates.repo";
     }
     $cmd .= " --repo=/etc/yum.repos.d/advisory.repo ./results";

tests/os_release.pm

@@ -88,6 +88,11 @@ sub run {
             $varstr = $variant;
         }
 
+        # If fedora-release-common release starts with a 0, we'll have
+        # "Prerelease" in varstr
+        my $reltag = script_output 'rpm -q fedora-release-common --qf "%{RELEASE}\n"';
+        $varstr .= " Prerelease" if (index($reltag, "0." == 0));
+
         my $version = "$version_id ($varstr)";
         # for canned variants, we need to form a different string here by using
         # the above created cannedtag. See earlier comment

tests/upgrade_preinstall.pm

@@ -11,7 +11,7 @@ sub run {
     # better way in a hurry. We want the pre-upgrade release version.
     my $testname = get_var("TEST");
     if (index($testname, "upgrade_2") != -1) {
-        $version = get_var("PREVREL");
+        $version = get_var("UP2REL");
     }
     setup_workaround_repo $version;
     assert_script_run 'dnf -y update --refresh', 1800;

tests/upgrade_run.pm

@@ -23,7 +23,7 @@ sub run {
 
     # FIXME workaround for #1811234 affecting F30 -> F31 upgrades
     # remove when bug is fixed or F30 goes EOL
-    if (get_var("UPGRADE") && $relnum eq "31" && get_var("CURRREL") eq "30") {
+    if (get_var("UPGRADE") && $relnum eq "31" && get_var("UP1REL") eq "30") {
         assert_script_run "dnf -y module reset maven", 120;
     }