srbala/os-autoinst-distri-fedora
1
0
Fork 0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-11-09 01:04:20 +00:00
Code Issues Releases Activity
a653ded877
os-autoinst-distri-fedora/templates-updates

1040 lines
49 KiB
Plaintext
Raw Normal View History

Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
#!/usr/share/openqa/script/load_templates
#
# Fedora Machines, Products, TestSuites and JobTemplates
#
# use load_templates to load the file into the database
#
{
JobTemplates => [
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "base_selinux" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_selinux" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 42,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "base_selinux" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "base_services_start" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_services_start" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 42,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "base_services_start" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "base_service_manipulation" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_service_manipulation" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 42,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "base_service_manipulation" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "base_update_cli" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_update_cli" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 42,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "base_update_cli" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "desktop_update_graphical" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 32,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "desktop_update_graphical" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "desktop_terminal" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 32,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "desktop_terminal" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "desktop_browser" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 32,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "desktop_browser" },
},
Revise desktop background checks Split this out of install_default, because it really is not a part of that test and we do not want that test to fail because the desktop background is wrong. Make it its own test module and test suite instead. Don't do it on Rawhide, because we really can't assert anything worthwhile about Rawhide at the moment at least (this means the test runs but is a no-op and will always pass on Rawhide, unfortunately). Move the needles to a more appropriate location (this has nothing to do with anaconda) and use 'background' not 'wallpaper' naming (that's the name we use elsewhere in the project, e.g. package names). Also, run the test on updates, and add an F29 needle for this purpose. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 21:30:25 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "desktop_background" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 32,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "desktop_background" },
},
Run desktop_printing test on updates too Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-07 00:33:09 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
version => "*",
},
test_suite => { name => "desktop_printing" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 32,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
version => "*",
},
test_suite => { name => "desktop_printing" },
},
Add Workstation upgrade test to update tests Since we set up this whole process to run upgrade tests for updates to run the FreeIPA ones on the Server base image, it should be easy to also run Workstation upgrade tests on the Workstation base image. So let's do that! Let's do the most complex test only, the encrypted upgrade one. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-27 23:46:56 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation-upgrade",
version => "*",
},
test_suite => { name => "upgrade_desktop_encrypted_64bit" },
},
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_role_deploy_domain_controller" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_cockpit_default" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_cockpit_basic" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "realmd_join_cockpit" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "realmd_join_sssd" },
},
Test FreeIPA upgrade on updates This adds the FreeIPA server and client upgrade tests to a new updates-server-upgrade flavor which fedora_openqa will schedule for updates. This way, we can test whether updates break FreeIPA upgrades, which is a request the FreeIPA team made to me. This has been deployed on staging for the last week or so and appears to work fine. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-20 17:12:01 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server-upgrade",
version => "*",
},
test_suite => { name => "upgrade_server_domain_controller" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 30,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server-upgrade",
version => "*",
},
test_suite => { name => "upgrade_realmd_client" },
},
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_role_deploy_database_server" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_database_client" },
},
{
Move update tests into a separate job group This makes them more visible in the web UI.
2017-03-02 05:10:41 +00:00
group_name => "Fedora Updates",
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
machine => { name => "64bit" },
prio => 40,
Also run FreeIPA replica tests on updates We just had an F30 update which broke replicas, so this seems like a good idea! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-09 01:23:51 +00:00
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_freeipa_replication_master" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_freeipa_replication_replica" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_freeipa_replication_client" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_firewall_default" },
},
Add a test of bootchain stuff for updates This adds a new test intended to just check boot chain things for updates. It doesn't run any test modules besides the stock update ones, but sets a variable, ADVISORY_BOOT_TEST, which causes _advisory_update to do some additional stuff after installing the updates but before rebooting: it forces regen of the initramfs and bootloader config, and reinstalls the bootloader on BIOS (not UEFI as it's not relevant). If the following boot fails, we probably have a bug somewhere. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 22:19:43 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "advisory_boot" },
},
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-everything-boot-iso",
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
version => "*",
},
test_suite => { name => "installer_build" },
},
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-everything-boot-iso",
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
version => "*",
},
test_suite => { name => "support_server" },
},
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-everything-boot-iso",
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
version => "*",
},
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
test_suite => { name => "install_default_update_netinst" },
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
},
Run the install_default_update test on UEFI as well Obviously an installer image may work fine for a BIOS install but fail for a UEFI install, so we should test it both ways. This requires the invention of a new (terribly named, but I couldn't think of anything better...) variable to allow us to use the 'run test on machine A after test run on machine B' mechanism without just hardcoding '64bit' as the machine name in the START_AFTER setting (which would break if we ever wanted to run update tests on any *other* arch). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 13:23:22 +00:00
{
group_name => "Fedora Updates",
machine => { name => "uefi" },
prio => 41,
product => {
arch => "x86_64",
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-everything-boot-iso",
Run the install_default_update test on UEFI as well Obviously an installer image may work fine for a BIOS install but fail for a UEFI install, so we should test it both ways. This requires the invention of a new (terribly named, but I couldn't think of anything better...) variable to allow us to use the 'run test on machine A after test run on machine B' mechanism without just hardcoding '64bit' as the machine name in the START_AFTER setting (which would break if we ever wanted to run update tests on any *other* arch). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 13:23:22 +00:00
version => "*",
},
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
test_suite => { name => "install_default_update_netinst" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation-live-iso",
version => "*",
},
test_suite => { name => "live_build" },
},
{
group_name => "Fedora Updates",
machine => { name => "64bit" },
prio => 40,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation-live-iso",
version => "*",
},
test_suite => { name => "install_default_update_live" },
},
{
group_name => "Fedora Updates",
machine => { name => "uefi" },
prio => 41,
product => {
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation-live-iso",
version => "*",
},
test_suite => { name => "install_default_update_live" },
Run the install_default_update test on UEFI as well Obviously an installer image may work fine for a BIOS install but fail for a UEFI install, so we should test it both ways. This requires the invention of a new (terribly named, but I couldn't think of anything better...) variable to allow us to use the 'run test on machine A after test run on machine B' mechanism without just hardcoding '64bit' as the machine name in the START_AFTER setting (which would break if we ever wanted to run update tests on any *other* arch). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 13:23:22 +00:00
},
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_selinux" },
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_services_start" },
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_service_manipulation" },
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_update_cli" },
},
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
flavor => "updates-server",
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "server_role_deploy_domain_controller" },
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
prio => 40,
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
product => {
arch => "ppc64le",
distri => "fedora",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
flavor => "updates-server",
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "server_cockpit_default" },
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
},
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "server_cockpit_basic" },
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "realmd_join_cockpit" },
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
prio => 30,
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "realmd_join_sssd" },
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
flavor => "updates-server-upgrade",
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "upgrade_server_domain_controller" },
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 30,
product => {
arch => "ppc64le",
distri => "fedora",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
flavor => "updates-server-upgrade",
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "upgrade_realmd_client" },
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_role_deploy_database_server" },
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_database_client" },
},
{
group_name => "Fedora PowerPC Updates",
machine => { name => "ppc64le" },
prio => 40,
product => {
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_firewall_default" },
},
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
{
group_name => "Fedora PowerPC Updates",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
machine => { name => "ppc64le" },
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
prio => 40,
product => {
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
arch => "ppc64le",
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
distri => "fedora",
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
flavor => "updates-server",
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
version => "*",
},
Add missing advisory_boot ppc64le in templates-updates and reorder ppc64 and ppc64le for updates_server* flavor to ease comparison with x86_64. Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-08 12:27:06 +00:00
test_suite => { name => "advisory_boot" },
Add PowerPC support for tests upgrade of FreeIPA This is using the two tests previously added by "Test upgrade of FreeIPA server and client deployment" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-04-27 06:56:13 +00:00
},
Fix a stray use of tabs in templates-updates EVIL EVIL TABS Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 23:44:15 +00:00
{
Add aarch64 to templates-updates
2018-03-07 03:05:04 +00:00
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_selinux" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_services_start" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_service_manipulation" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "base_update_cli" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_role_deploy_domain_controller" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_cockpit_default" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_cockpit_basic" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "realmd_join_cockpit" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 30,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "realmd_join_sssd" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_role_deploy_database_server" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_database_client" },
},
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "server_firewall_default" },
},
Add a test of bootchain stuff for updates This adds a new test intended to just check boot chain things for updates. It doesn't run any test modules besides the stock update ones, but sets a variable, ADVISORY_BOOT_TEST, which causes _advisory_update to do some additional stuff after installing the updates but before rebooting: it forces regen of the initramfs and bootloader config, and reinstalls the bootloader on BIOS (not UEFI as it's not relevant). If the following boot fails, we probably have a bug somewhere. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 22:19:43 +00:00
{
group_name => "Fedora AArch64 Updates",
machine => { name => "aarch64" },
prio => 40,
product => {
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
version => "*",
},
test_suite => { name => "advisory_boot" },
},
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
],
Products => [
{
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation",
name => "",
settings => [
{ key => "DESKTOP", value => "gnome" },
],
version => "*",
},
Add Workstation upgrade test to update tests Since we set up this whole process to run upgrade tests for updates to run the FreeIPA ones on the Server base image, it should be easy to also run Workstation upgrade tests on the Workstation base image. So let's do that! Let's do the most complex test only, the encrypted upgrade one. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-27 23:46:56 +00:00
{
arch => "x86_64",
distri => "fedora",
flavor => "updates-workstation-upgrade",
name => "",
settings => [
{ key => "DESKTOP", value => "gnome" },
],
version => "*",
},
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
{
arch => "x86_64",
distri => "fedora",
flavor => "updates-kde",
name => "",
settings => [
{ key => "DESKTOP", value => "kde" }
],
version => "*",
},
{
arch => "x86_64",
distri => "fedora",
flavor => "updates-server",
name => "",
settings => [
],
version => "*",
},
Test FreeIPA upgrade on updates This adds the FreeIPA server and client upgrade tests to a new updates-server-upgrade flavor which fedora_openqa will schedule for updates. This way, we can test whether updates break FreeIPA upgrades, which is a request the FreeIPA team made to me. This has been deployed on staging for the last week or so and appears to work fine. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-20 17:12:01 +00:00
{
arch => "x86_64",
distri => "fedora",
flavor => "updates-server-upgrade",
name => "",
settings => [
],
version => "*",
},
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
{
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
arch => "x86_64",
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-everything-boot-iso",
Add "Fedora PowerPC Updates" group and related server tests in templates-updates file Only support "updates-server" flavor do not support "updates-workstation" neither "updates-kde" Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-17 08:24:42 +00:00
name => "",
settings => [
],
version => "*",
},
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{
arch => "x86_64",
distri => "fedora",
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
flavor => "updates-workstation-live-iso",
name => "",
settings => [
{ key => "LIVE", value => "1" },
{ key => "PACKAGE_SET", value => "default" },
{ key => "DESKTOP", value => "gnome" }
],
version => "*",
},
{
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server",
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
name => "",
settings => [
],
version => "*",
},
Test FreeIPA upgrade on updates This adds the FreeIPA server and client upgrade tests to a new updates-server-upgrade flavor which fedora_openqa will schedule for updates. This way, we can test whether updates break FreeIPA upgrades, which is a request the FreeIPA team made to me. This has been deployed on staging for the last week or so and appears to work fine. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-20 17:12:01 +00:00
{
arch => "ppc64le",
distri => "fedora",
flavor => "updates-server-upgrade",
name => "",
settings => [
],
version => "*",
},
{
Add aarch64 to templates-updates
2018-03-07 03:05:04 +00:00
arch => "aarch64",
distri => "fedora",
flavor => "updates-server",
name => "",
settings => [
],
version => "*",
},
Test FreeIPA upgrade on updates This adds the FreeIPA server and client upgrade tests to a new updates-server-upgrade flavor which fedora_openqa will schedule for updates. This way, we can test whether updates break FreeIPA upgrades, which is a request the FreeIPA team made to me. This has been deployed on staging for the last week or so and appears to work fine. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-20 17:12:01 +00:00
{
arch => "aarch64",
distri => "fedora",
flavor => "updates-server-upgrade",
name => "",
settings => [
],
version => "*",
},
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
],
Add a test of bootchain stuff for updates This adds a new test intended to just check boot chain things for updates. It doesn't run any test modules besides the stock update ones, but sets a variable, ADVISORY_BOOT_TEST, which causes _advisory_update to do some additional stuff after installing the updates but before rebooting: it forces regen of the initramfs and bootloader config, and reinstalls the bootloader on BIOS (not UEFI as it's not relevant). If the following boot fails, we probably have a bug somewhere. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 22:19:43 +00:00
TestSuites => [
{
name => "advisory_boot",
settings => [
{ key => "USER_LOGIN", value => "false" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
{ key => "ADVISORY_BOOT_TEST", value => "1" },
],
},
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{
name => "installer_build",
settings => [
{ key => "POSTINSTALL", value => "_installer_build" },
{ key => "USER_LOGIN", value => "false" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
Use a second disk for the update repo in image build tests This tweaks the update live and installer image build tests to store the update repo on a separate disk. This should hopefully avoid the tests failing due to insufficient disk space when the update is huge (e.g. KDE or GNOME megaupdates). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-13 19:37:42 +00:00
{ key => "NUMDISKS", value => "2" },
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{ key => "HDD_1", value => "disk_f%VERSION%_minimal_3_%ARCH%.img" },
],
},
{
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
name => "live_build",
settings => [
{ key => "POSTINSTALL", value => "_live_build" },
Disable SELinux for live image build test Building images with SELinux in permissive mode worked fine in F29 and F30, but it seems to be broken in F31. Releng seems to have it set up so the build environment has SELinux disabled, so let's do the same. Will file a bug for this, as bcl says it should be considered a bug. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-29 22:35:51 +00:00
{ key => "GRUB_POSTINSTALL", value => "selinux=0" },
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
{ key => "USER_LOGIN", value => "false" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
{ key => "+DESKTOP", value => "" },
{ key => "+LIVE", value => "" },
Use a second disk for the update repo in image build tests This tweaks the update live and installer image build tests to store the update repo on a separate disk. This should hopefully avoid the tests failing due to insufficient disk space when the update is huge (e.g. KDE or GNOME megaupdates). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-13 19:37:42 +00:00
{ key => "NUMDISKS", value => "2" },
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
{ key => "HDD_1", value => "disk_f%VERSION%_minimal_3_%ARCH%.img" },
],
},
{
name => "install_default_update_netinst",
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
settings => [
{ key => "INSTALL", value => "1" },
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
{ key => "INSTALL_UNLOCK", value => "support_ready" },
Handle running update tests on Koji tasks We quite often want to run the update tests on a Koji task (not a Bodhi update) for some reason - usually to test a potential fix for an issue, or at a maintainer's request to test a change before it is merged upstream and officially sent out as an update. Up till now I've always hacked up utils.pm on the staging server by hand to do this, which is horrible. Together with a commit to fedora_openqa, this should allow us to do it in a nice, sane way via the CLI. It's mostly just tweaking the "updates" repo setup in utils.pm as you'd expect, but there's a bit of subtlety to it because of the installer tests that use %ADVISORY% as a variable substitution in the disk image name; you can't do something like `%ADVISORY or KOJITASK%`, sadly, so I had to have almost-redundant variables ADVISORY, KOJITASK and ADVISORY_OR_TASK (we could kinda just live with ADVISORY_OR_TASK except I didn't want to drop ADVISORY as it's an unnecessary change from previous behavior). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 09:06:16 +00:00
{ key => "ISO", value => "%ADVISORY_OR_TASK%-netinst-%ARCH%.iso" },
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
{ key => "NICTYPE", value => "tap" },
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
{ key => "PACKAGE_SET", value => "default" },
Use @ not : for machine dependencies Upstream decided @ was better, : still works for now but will probably be taken out at some point. You need openqa 4.6-37.20191121git8fcf81f or higher for this. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 20:43:18 +00:00
{ key => "PARALLEL_WITH", value => "support_server@%ARCH_BASE_MACHINE%" },
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
{ key => "ADD_REPOSITORY_VARIATION", value => "nfs://10.0.2.110:/opt/update_repo" },
Use @ not : for machine dependencies Upstream decided @ was better, : still works for now but will probably be taken out at some point. You need openqa 4.6-37.20191121git8fcf81f or higher for this. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 20:43:18 +00:00
{ key => "+START_AFTER_TEST", value => "installer_build@%ARCH_BASE_MACHINE%" },
Have update installer test install the update packages (#89) In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became obvious that there's a missing feature in the new installer test for updates: the update is both used in the image build process and built into the installer environment itself, but it is not actually included in the installed package set. This can be a problem if the update has a bug that manifests *only* at install time if it is in the install transaction (which is exactly the case there), because the test will not catch this, and nor will any other test. So this commit makes `support_server` set up the update repo and serve it out via NFS when it's run in an update context, and makes the actual update install test run parallel with it and use that repository. This way the install should include the package(s) from the update. (It also of course means the test fails if an update breaks NFS or something like that, but hey, we want to know that!) A parallel commit for fedora_openqa is necessary to add the required CURRREL setting for the updates-installer flavor. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-25 13:41:32 +00:00
{ key => "WORKER_CLASS", value => "tap" },
Add update tests to build and test a netinst image This adds a test which builds a netinst image potentially with the package(s) from the update, and uploads that image. It also adds a test which runs a default install using that image. This is intended to check whether the update breaks the creation or use of install images; particularly this will let us test anaconda etc. updates. We also update the minimal disk image name, as we have to make it bigger to accommodate this test, and making it bigger changes its name - the actual change to the disk image itself is in createhdds. We also have to redo a bunch of installer needles for F28 fonts, after I removed them a month or so back... Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 20:34:12 +00:00
],
},
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
{
name => "install_default_update_live",
settings => [
{ key => "INSTALL", value => "1" },
{ key => "ISO", value => "Fedora-%SUBVARIANT%-Live-%ARCH%-%ADVISORY_OR_TASK%.iso" },
Use @ not : for machine dependencies Upstream decided @ was better, : still works for now but will probably be taken out at some point. You need openqa 4.6-37.20191121git8fcf81f or higher for this. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 20:43:18 +00:00
{ key => "+START_AFTER_TEST", value => "live_build@%ARCH_BASE_MACHINE%" },
Add a live image build test for updates Just like the installer image build test, only...it builds a live image. This involves reimplementing quite a chunk of the Koji livemedia task. Ah, well. Also involves rethinking the flavor names a bit here, these seem...better. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 14:56:01 +00:00
],
},
Add a test of bootchain stuff for updates This adds a new test intended to just check boot chain things for updates. It doesn't run any test modules besides the stock update ones, but sets a variable, ADVISORY_BOOT_TEST, which causes _advisory_update to do some additional stuff after installing the updates but before rebooting: it forces regen of the initramfs and bootloader config, and reinstalls the bootloader on BIOS (not UEFI as it's not relevant). If the following boot fails, we probably have a bug somewhere. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 22:19:43 +00:00
],
Add support for testing updates Summary: This adds an entirely new workflow for testing distribution updates. The `ADVISORY` variable is introduced: when set, `main.pm` will load an early post-install test that sets up a repository containing the packages from the specified update, runs `dnf -y update`, and reboots. A new templates file is added, `templates-updates`, which adds two new flavors called `updates-server` and `updates-workstation`, each containing job templates for appropriate post-install tests. Scheduler is expected to post `ADVISORY=(update ID) HDD_1=(base image) FLAVOR=updates-(server|workstation)`, where (base image) is one of the stable release base disk images produced by `createhdds` and usually used for upgrade testing. This will result in the appropriate job templates being loaded. We rejig postinstall test loading and static network config a bit so that this works for both the 'compose' and 'updates' test flows: we have to ensure we bring up networking for the tap tests before we try and install the updates, but still allow later adjustment of the configuration. We take advantage of the openQA feature that was added a few months back to run the same module multiple times, so the `_advisory_update` module can reboot after installing the updates and the modules that take care of bootloader, encryption and login get run again. This looks slightly wacky in the web UI, though - it doesn't show the later runs of each module. We also use the recently added feature to specify `+HDD_1` in the test suites which use a disk image uploaded by an earlier post-install test, so the test suite value will take priority over the value POSTed by the scheduler for those tests, and we will use the uploaded disk image (and not the clean base image POSTed by the scheduler) for those tests. My intent here is to enhance the scheduler, adding a consumer which listens out for critpath updates, and runs this test flow for each one, then reports the results to ResultsDB where Bodhi could query and display them. We could also add a list of other packages to have one or both sets of update tests run on it, I guess. Test Plan: Try a post something like: HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25 FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24 Pick an appropriate `ADVISORY` (ideally, one containing some packages which might actually be involved in the tests), and matching `FLAVOR` and `HDD_1`. The appropriate tests should run, a repo with the update packages should be created and enabled (and dnf update run), and the tests should work properly. Also test a regular compose run to make sure I didn't break anything. Reviewers: jskladan, jsedlak Reviewed By: jsedlak Subscribers: tflink Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-01-25 16:16:12 +00:00
}
Reference in New Issue Copy Permalink