srbala/os-autoinst-distri-fedora
1
0
Fork 0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-12-12 14:23:09 +00:00
Code Issues Releases Activity
d3cb192675
os-autoinst-distri-fedora/templates-updates.fif.json

648 lines
23 KiB
JSON
Raw Normal View History

Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
{
"Products": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"KIWI_PROFILE": "Container-Base-Generic",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-everything-boot-iso",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-kde-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-kde",
"settings": {
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"+HDD_1": "disk_f%VERSION%_kde_5_%ARCH%.qcow2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "kde",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Revert "Bump RETRY for updates-kde to 2 while we have oom kill issues" This reverts commit 1bea573fecc2df9aa9b111c5054a16f942d486c5. We don't have OOM kill issues any more.
2023-12-04 18:07:24 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-server-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Use NUMDISKS=2 (or higher) on all update flavors We stopped doing this on Server because it caused problems with tests that use a disk image uploaded by another test, e.g. the cockpit tests - they use the `/etc/fstab` from the disk image the parent test uploaded, which says to mount the second disk as /mnt/update_repo, but since this is a new test it has a fresh, empty second disk with no filesystems to mount. This tries to fix that by making _console_shutdown.pm edit that line back out of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree flavor, which had a similar problem with the overlay and rebase tests using a disk image uploaded by the install test). We need to fix this because FEDORA-2024-9b9da603e1 is so big it causes the tests that don't use a scratch disk to run out of disk space. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 22:49:35 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Use NUMDISKS=2 (or higher) on all update flavors We stopped doing this on Server because it caused problems with tests that use a disk image uploaded by another test, e.g. the cockpit tests - they use the `/etc/fstab` from the disk image the parent test uploaded, which says to mount the second disk as /mnt/update_repo, but since this is a new test it has a fresh, empty second disk with no filesystems to mount. This tries to fix that by making _console_shutdown.pm edit that line back out of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree flavor, which had a similar problem with the overlay and rebase tests using a disk image uploaded by the install test). We need to fix this because FEDORA-2024-9b9da603e1 is so big it causes the tests that don't use a scratch disk to run out of disk space. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 22:49:35 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Bump server disk version to 5 We made it bigger. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-07-30 21:28:50 +00:00
"+HDD_1": "disk_f%VERSION%_server_5_%ARCH%.qcow2",
Use NUMDISKS=2 (or higher) on all update flavors We stopped doing this on Server because it caused problems with tests that use a disk image uploaded by another test, e.g. the cockpit tests - they use the `/etc/fstab` from the disk image the parent test uploaded, which says to mount the second disk as /mnt/update_repo, but since this is a new test it has a fresh, empty second disk with no filesystems to mount. This tries to fix that by making _console_shutdown.pm edit that line back out of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree flavor, which had a similar problem with the overlay and rebase tests using a disk image uploaded by the install test). We need to fix this because FEDORA-2024-9b9da603e1 is so big it causes the tests that don't use a scratch disk to run out of disk space. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 22:49:35 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-silverblue-dvd_ostree-iso",
"settings": {
"CANNED": "1",
"DESKTOP": "gnome",
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"DEPLOY_UPLOAD_TEST": "install_default_update_ostree",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"HDDSIZEGB": "15",
Use NUMDISKS=2 (or higher) on all update flavors We stopped doing this on Server because it caused problems with tests that use a disk image uploaded by another test, e.g. the cockpit tests - they use the `/etc/fstab` from the disk image the parent test uploaded, which says to mount the second disk as /mnt/update_repo, but since this is a new test it has a fresh, empty second disk with no filesystems to mount. This tries to fix that by making _console_shutdown.pm edit that line back out of /etc/fstab, so we can set NUMDISKS=2 again (also on the ostree flavor, which had a similar problem with the overlay and rebase tests using a disk image uploaded by the install test). We need to fix this because FEDORA-2024-9b9da603e1 is so big it causes the tests that don't use a scratch disk to run out of disk space. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-28 22:49:35 +00:00
"NUMDISKS": "2",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"PACKAGE_SET": "default",
Bump retry count on some flaky tests The silverblue tests are flaky because of https://github.com/fedora-silverblue/issue-tracker/issues/548 . The desktop_upgrade_encrypted test is flaky on Rawhide (so, when booting from F40 initially) since we switched to UEFI, not sure if it's because of UEFI somehow or just a timing coincidence. Am going to look into it, but for now this should save lots of manual restarting. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-04-05 14:59:42 +00:00
"RETRY": "2",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"SUBVARIANT": "Silverblue"
},
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-live-iso",
"settings": {
"DESKTOP": "gnome",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"LIVE": "1",
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"LIVE_BUILD_TEST": "live_build",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"RETRY": "1",
Correct casing of SUBVARIANT variables In moving these from fedora_openqa to here, I got the casing wrong, which breaks `tell_source` and causes a test to fail. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 18:55:47 +00:00
"SUBVARIANT": "Workstation"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"flavor": "updates-kde-live-iso",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"settings": {
"DESKTOP": "kde",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"KIWI_PROFILE": "KDE-Desktop-Live",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"LIVE": "1",
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"LIVE_BUILD_TEST": "kiwi_build",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"RETRY": "1",
Correct casing of SUBVARIANT variables In moving these from fedora_openqa to here, I got the casing wrong, which breaks `tell_source` and causes a test to fail. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 18:55:47 +00:00
"SUBVARIANT": "KDE"
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
},
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-upgrade",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-workstation-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"+HDD_1": "disk_f%VERSION%_desktop_5_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
}
},
"Profiles": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-container-aarch64-*"
},
"fedora-updates-container-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-container-ppc64le-*"
},
"fedora-updates-container-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-container-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-bios": {
"machine": "bios",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
"fedora-updates-kde-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-x86_64-*"
},
"fedora-updates-server-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-server-aarch64-*"
},
"fedora-updates-server-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-ppc64le-*"
},
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-upgrade-ppc64le-*"
},
"fedora-updates-server-upgrade-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-upgrade-x86_64-*"
},
"fedora-updates-server-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-x86_64-*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-kde-live-iso-x86_64-*-bios": {
"machine": "bios",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-silverblue-dvd_ostree-iso-x86_64-*"
},
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-bios": {
"machine": "bios",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"product": "fedora-updates-silverblue-dvd_ostree-iso-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-bios": {
"machine": "bios",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
Revert "D'oh, move the most important encryption test to BIOS" This reverts commit 508635ed1c1ebfcc0c91c29db34b5ee6882d662c and the fix-up follow-up, because this causes the test to have a different scenario which screws up gating. Argh. I guess we're stuck restarting it forever. Let's bump the retry number even higher instead.
2024-05-24 18:38:52 +00:00
"fedora-updates-workstation-upgrade-x86_64-*-64bit": {
"machine": "64bit",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"product": "fedora-updates-workstation-upgrade-x86_64-*"
},
"fedora-updates-workstation-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-x86_64-*"
}
},
"TestSuites": {
"advisory_boot": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"ADVISORY_BOOT_TEST": "1",
"BOOTFROM": "c",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
"base_selinux": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_service_manipulation": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_services_start": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_update_cli": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
"base_reboot_unmount": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
},
"base_system_logging": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"desktop_background": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_browser": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add several more tests to the desktop update test set There's no obvious reason we're not also running these tests on updates, so let's do it. We have to skip the advisory and UEFI post checks for desktop_login as the last step of that test is shutting down the system. We leave out desktop_login for now because of https://gitlab.gnome.org/GNOME/gjs/-/issues/647 Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-09-18 21:51:25 +00:00
"desktop_keyring": {
"profiles": {
"fedora-updates-workstation-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"desktop_printing": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add several more tests to the desktop update test set There's no obvious reason we're not also running these tests on updates, so let's do it. We have to skip the advisory and UEFI post checks for desktop_login as the last step of that test is shutting down the system. We leave out desktop_login for now because of https://gitlab.gnome.org/GNOME/gjs/-/issues/647 Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-09-18 21:51:25 +00:00
"desktop_printing_builtin": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"desktop_terminal": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_update_graphical": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"install_default_update_live": {
"profiles": {
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-kde-live-iso-x86_64-*-bios": 5,
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": 5,
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-bios": 5,
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"+START_AFTER_TEST": "%LIVE_BUILD_TEST%@%ARCH_BASE_MACHINE%",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"INSTALL": "1",
Give install_default_update_live 4G of RAM too This test seems to be struggling with 3G of RAM also. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 21:19:50 +00:00
"ISO": "Fedora-%SUBVARIANT%-Live-%ARCH%-%ADVISORY_OR_TASK%.iso",
Bump install_default_update_live retries to 5 We have what look like two different bugs(?) causing havoc here (BIOS installs getting stuck at bootloader install, UEFI installs failing to get out of grub on first boot after install). While I figure out what the heck this is all about, bump the retry count. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-29 01:08:51 +00:00
"QEMURAM": "4096",
"RETRY": "5"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"install_default_update_netinst": {
"profiles": {
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-bios": 5,
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"+START_AFTER_TEST": "installer_build@%ARCH_BASE_MACHINE%",
Support testing a side tag instead of an update or task Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-04 20:13:15 +00:00
"ADD_REPOSITORY_VARIATION": "%UPDATE_OR_TAG_REPO%",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"INSTALL": "1",
"INSTALL_UNLOCK": "support_ready",
"ISO": "%ADVISORY_OR_TASK%-netinst-%ARCH%.iso",
"NICTYPE": "tap",
Add several more tests to the desktop update test set There's no obvious reason we're not also running these tests on updates, so let's do it. We have to skip the advisory and UEFI post checks for desktop_login as the last step of that test is shutting down the system. We leave out desktop_login for now because of https://gitlab.gnome.org/GNOME/gjs/-/issues/647 Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-09-18 21:51:25 +00:00
"NO_ADVISORY_POST": "1",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"PACKAGE_SET": "default",
"PARALLEL_WITH": "support_server@%ARCH_BASE_MACHINE%",
Bump install_default_update_netinst retries to 5 This test is *also* frequently failing in current Rawhide and I don't know why. Kick up its retries till we can figure it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-30 01:46:40 +00:00
"RETRY": "5",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"WORKER_CLASS": "tap"
}
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"install_default_update_ostree": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
},
"settings": {
"INSTALL": "1",
"ISO": "%ADVISORY_OR_TASK%-%SUBVARIANT%-ostree-%ARCH%.iso",
"PACKAGE_SET": "default",
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"+START_AFTER_TEST": "ostree_build@%ARCH_BASE_MACHINE%",
"STORE_HDD_1": "disk_%FLAVOR%_%MACHINE%.qcow2"
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
}
},
updates: test Silverblue install on UEFI as well as BIOS https://bugzilla.redhat.com/show_bug.cgi?id=2268505 made it clear that this is a bit of a hole. We don't test installing the Silverblue image we build on UEFI, only on BIOS. Add this as a separate test so we don't uselessly upload a disk image we won't use for any follow-on tests. This also adds an anaconda build that fixes a bug in this path as a workaround for F40, so the test won't fail on all F40 updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-08 00:08:40 +00:00
"install_default_ostree": {
"profiles": {
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-bios": 5
updates: test Silverblue install on UEFI as well as BIOS https://bugzilla.redhat.com/show_bug.cgi?id=2268505 made it clear that this is a bit of a hole. We don't test installing the Silverblue image we build on UEFI, only on BIOS. Add this as a separate test so we don't uselessly upload a disk image we won't use for any follow-on tests. This also adds an anaconda build that fixes a bug in this path as a workaround for F40, so the test won't fail on all F40 updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-08 00:08:40 +00:00
},
"settings": {
"INSTALL": "1",
"ISO": "%ADVISORY_OR_TASK%-%SUBVARIANT%-ostree-%ARCH%.iso",
"PACKAGE_SET": "default",
"+START_AFTER_TEST": "ostree_build@%ARCH_BASE_MACHINE%"
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"installer_build": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"BOOTFROM": "c",
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"HDD_1": "disk_f%VERSION%_minimal_4_%ARCH%.qcow2",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"NUMDISKS": "2",
"POSTINSTALL": "_installer_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Move KDE live build test to Kiwi, refactor mock bits KDE live images for Rawhide are now built with Kiwi, so let's do it that way in openQA too (including for stable releases because it's awkward to do it any other way). Kiwi build steps are almost the same whatever you're building, so convert the container build test into a generic Kiwi build test and use it for both. mock setup is shared between Kiwi and LMC, so let's factor it out. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-23 19:14:50 +00:00
"kiwi_build": {
"profiles": {
"fedora-updates-container-x86_64-*-64bit": 5,
"fedora-updates-kde-live-iso-x86_64-*-64bit": 5
},
"settings": {
"+DESKTOP": "",
"+LIVE": "",
"BOOTFROM": "c",
"HDD_1": "disk_f%VERSION%_minimal_4_%ARCH%.qcow2",
"HDDSIZEGB_3": "25",
"MAX_JOB_TIME": "10800",
"+NUMDISKS": "3",
"POSTINSTALL": "_kiwi_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"live_build": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"+DESKTOP": "",
"+LIVE": "",
"BOOTFROM": "c",
"GRUB_POSTINSTALL": "selinux=0",
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"HDD_1": "disk_f%VERSION%_minimal_4_%ARCH%.qcow2",
Put /var/lib/mock on separate disk for live image build test The update live image build test keeps running out of disk space. We've bumped the minimal disk image from 12GB all the way up to 20GB so far but it keeps happening. So let's try a different strategy: use a scratch disk to mount /var/lib/mock. That's where all the space gets used. This should allow us to reduce the size of the minimal disk image again, and giving it 25GB of empty disk should avoid it running out of space again for a while. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 04:12:12 +00:00
"HDDSIZEGB_3": "25",
Give live image build even longer and bump max job time Rawhide live image builds are still taking an awful long time and often failing. I will look more into why later, but for now, let's bump the timeouts even more just to try and get through the job backlog. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-19 19:35:24 +00:00
"MAX_JOB_TIME": "10800",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"+NUMDISKS": "3",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"POSTINSTALL": "_live_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"ostree_build": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
},
"settings": {
"+DESKTOP": "",
"+CANNED": "",
"BOOTFROM": "c",
Make UEFI testing the default, add Secure Boot testing This essentially inverts the x86_64 machines so that '64bit' is UEFI and instead of a variant 'uefi' machine we have a variant 'bios' machine that is BIOS. The point is to make UEFI testing the default. We also enable Secure Boot in the UEFI testing, and add a test of UEFI fallback booting on various products. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-03-19 23:52:00 +00:00
"HDD_1": "disk_f%VERSION%_minimal_4_%ARCH%.qcow2",
Bump disk size on ostree build test to 30 GB Seems we need more space on the scratch disk used by the ostree build test, it's failing due to lack of space. Tested on lab, it worked. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-21 04:14:34 +00:00
"HDDSIZEGB_3": "30",
Give ostree_build another half hour again Another bunch of these timed out. Not sure why. Maybe it's when I run a lot of them at the same time? Let's try this, again. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-01-30 01:03:30 +00:00
"MAX_JOB_TIME": "12600",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"+NUMDISKS": "3",
"POSTINSTALL": "_ostree_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"podman": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-container-aarch64-*-aarch64": 5,
"fedora-updates-container-ppc64le-*-ppc64le": 5,
"fedora-updates-container-x86_64-*-64bit": 5
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
}
},
"podman_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-container-aarch64-*-aarch64": 5,
"fedora-updates-container-ppc64le-*-ppc64le": 5,
"fedora-updates-container-x86_64-*-64bit": 5
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"realmd_join_cockpit": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"realmd_join_cockpit_ad": {
"profiles": {
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"realmd_join_sssd": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"realmd_join_sssd_ad": {
"profiles": {
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"rpmostree_overlay": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
}
},
"rpmostree_rebase": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_cockpit_basic": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_cockpit_default": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_cockpit_updates": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_database_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_firewall_default": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_master": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_replica": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_remote_logging_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
"server_remote_logging_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_role_deploy_database_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_role_deploy_domain_controller": {
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"profiles": {
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
"server_samba_domain_controller": {
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"support_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add several more tests to the desktop update test set There's no obvious reason we're not also running these tests on updates, so let's do it. We have to skip the advisory and UEFI post checks for desktop_login as the last step of that test is shutting down the system. We leave out desktop_login for now because of https://gitlab.gnome.org/GNOME/gjs/-/issues/647 Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-09-18 21:51:25 +00:00
"toolbox": {
"profiles": {
"fedora-updates-workstation-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"upgrade_desktop_encrypted_64bit": {
"profiles": {
Revert "D'oh, move the most important encryption test to BIOS" This reverts commit 508635ed1c1ebfcc0c91c29db34b5ee6882d662c and the fix-up follow-up, because this causes the test to have a different scenario which screws up gating. Argh. I guess we're stuck restarting it forever. Let's bump the retry number even higher instead.
2024-05-24 18:38:52 +00:00
"fedora-updates-workstation-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"upgrade_realmd_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 5,
"fedora-updates-server-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"upgrade_server_domain_controller": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 5,
"fedora-updates-server-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
}
}
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
Reference in New Issue Copy Permalink