os-autoinst-distri-fedora/tests
Adam Williamson ef689e75a9 use compose repository (not master repo) for most tests
Summary:
we have a long-standing problem with all the tests that hit
the repositories. The tests are triggered as soon as a compose
completes. At this point in time, the compose is not synced to
the mirrors, where the default 'fedora' repo definition looks;
the sync happens after the compose completes, and there is also
a metadata sync step that must happen after *that* before any
operation that uses the 'fedora' repository definition will
actually use the packages from the new compose. Thus all net
install tests and tests that installed packages have been
effectively testing the previous compose, not the current one.

We have some thoughts about how to fix this 'properly' (such
that the openQA tests wouldn't have to do anything special,
but their 'fedora' repository would somehow reflect the compose
under test), but none of them is in place right now or likely
to happen in the short term, so in the mean time this should
deal with most of the issues. With this change, everything but
the default_install tests for the netinst images should use
the compose-under-test's Everything tree instead of the 'fedora'
repository, and thus should install and test the correct
packages.

This relies on a corresponding change to openqa_fedora_tools
to set the LOCATION openQA setting (which is simply the base
location of the compose under test).

Test Plan:
Do a full test run, check (as far as you can) tests run sensibly
and use appropriate repositories.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D989
2016-09-01 08:22:59 -07:00
..
_boot_to_anaconda.pm add NFS tests (and DHCP/DNS in the support server) 2016-06-13 08:42:30 -07:00
_check_install_source.pm add NFS tests (and DHCP/DNS in the support server) 2016-06-13 08:42:30 -07:00
_console_shutdown.pm shutdown before uploading disk images 2016-03-22 07:19:47 -07:00
_console_wait_login.pm add a cockpit realmd FreeIPA join test 2016-06-07 13:00:39 -07:00
_do_install_and_reboot.pm drop dictionary error workaround 2016-08-30 11:18:46 -07:00
_graphical_wait_login.pm add encrypted workstation upgrade tests (current and previous) 2016-07-08 08:56:57 -07:00
_network_static.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
_software_selection.pm add comments and documentation 2015-08-05 08:23:59 +02:00
_support_server_postinstall.pm support_server: give the ISO copy a bit longer to complete 2016-08-09 17:11:53 -07:00
base_selinux.pm add a base_selinux test (follow-on from default_install) 2015-12-17 12:46:14 -08:00
base_service_manipulation.pm add base_service_manipulation test 2016-01-11 12:30:24 -08:00
base_services_start.pm workaround mcelog failing 2016-06-01 08:12:16 +02:00
desktop_browser_postinstall.pm add QA:Testcase_desktop_browser test 2016-08-03 13:22:29 -07:00
desktop_terminal_postinstall.pm add desktop_terminal test, refactor test loading a bit 2016-05-05 16:39:47 -07:00
disk_custom_btrfs.pm add ext3, btrfs and LVM thin tests, tweaks to custom methods 2015-08-10 11:01:12 -07:00
disk_custom_ext3.pm add ext3, btrfs and LVM thin tests, tweaks to custom methods 2015-08-10 11:01:12 -07:00
disk_custom_iscsi.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
disk_custom_iscsi_postinstall.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
disk_custom_lvmthin.pm add ext3, btrfs and LVM thin tests, tweaks to custom methods 2015-08-10 11:01:12 -07:00
disk_custom_no_swap.pm small fix: wait a sec between clicks of 'Done' in no_swap 2015-08-31 17:02:27 -07:00
disk_custom_software_raid.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
disk_custom_software_raid_postinstall.pm use validate_script_output instead of typing and needles matching 2015-11-04 14:38:36 +01:00
disk_custom_xfs.pm add custom partitioning xfs tests 2015-11-26 13:50:45 +01:00
disk_guided_delete_all.pm add comments and documentation 2015-08-05 08:23:59 +02:00
disk_guided_delete_partial.pm add comments and documentation 2015-08-05 08:23:59 +02:00
disk_guided_delete_partial_postinstall.pm use assert_script_run when possible 2016-01-12 09:27:14 -08:00
disk_guided_empty.pm rename BOOT_UPDATES_IMG_URL to TEST_UPDATES, add GRUBADD 2016-04-08 13:21:29 -07:00
disk_guided_encrypted.pm disk_guided_encrypted: only switch layout when appropriate 2016-05-20 11:56:48 -07:00
disk_guided_encrypted_postinstall.pm add encrypted workstation upgrade tests (current and previous) 2016-07-08 08:56:57 -07:00
disk_guided_free_space_postinstall.pm use assert_script_run when possible 2016-01-12 09:27:14 -08:00
disk_guided_multi.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
disk_guided_multi_empty_all.pm add an iscsi test, and a support_server test to support it 2016-06-09 08:43:46 -07:00
disk_guided_multi_empty_all_postinstall.pm use validate_script_output instead of typing and needles matching 2015-11-04 14:38:36 +01:00
disk_guided_multi_postinstall.pm fix server_multi postinstall for disk being virtio 2016-01-27 01:44:20 -08:00
disk_guided_shrink.pm add guided shrink test 2015-09-08 15:54:22 +02:00
disk_guided_shrink_postinstall.pm use assert_script_run when possible 2016-01-12 09:27:14 -08:00
firewall_configured_postinstall.pm add firewall kickstart tests (disabled and configured) 2016-03-23 13:52:00 -07:00
firewall_disabled_postinstall.pm add firewall kickstart tests (disabled and configured) 2016-03-23 13:52:00 -07:00
freeipa_client_postinstall.pm add FreeIPA web UI testing 2016-06-28 12:01:31 -07:00
freeipa_password_change_postinstall.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00
freeipa_webui_postinstall.pm add QA:Testcase_FreeIPA_password_change test 2016-08-03 13:21:12 -07:00
install_arm_image_deployment.pm add ARM initial-setup test 2016-05-18 14:04:45 +02:00
install_source_graphical.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00
realmd_join_cockpit_postinstall.pm add FreeIPA web UI testing 2016-06-28 12:01:31 -07:00
realmd_join_sssd_postinstall.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00
role_deploy_domain_controller.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00
server_cockpit_basic_postinstall.pm add cockpit_default and cockpit_basic tests 2016-06-01 09:05:33 -07:00
server_cockpit_default_postinstall.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00
server_firewall_default_postinstall.pm add a server default firewall test 2016-05-31 14:24:34 -07:00
uefi_postinstall.pm use assert_script_run when possible 2016-01-12 09:27:14 -08:00
upgrade_postinstall.pm add desktop_terminal test, refactor test loading a bit 2016-05-05 16:39:47 -07:00
upgrade_preinstall.pm add encrypted workstation upgrade tests (current and previous) 2016-07-08 08:56:57 -07:00
upgrade_run.pm use compose repository (not master repo) for most tests 2016-09-01 08:22:59 -07:00