srbala/os-autoinst-distri-fedora
1
0
Fork 0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-11-17 20:13:08 +00:00
Code Issues Releases Activity
e9fd290cdc
os-autoinst-distri-fedora/templates-updates.fif.json

617 lines
22 KiB
JSON
Raw Normal View History

Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
{
"Products": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-everything-boot-iso",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-kde-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-kde",
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_kde_4_%ARCH%.qcow2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "kde",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Revert "Bump RETRY for updates-kde to 2 while we have oom kill issues" This reverts commit 1bea573fecc2df9aa9b111c5054a16f942d486c5. We don't have OOM kill issues any more.
2023-12-04 18:07:24 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-server-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_server_3_%ARCH%.qcow2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-silverblue-dvd_ostree-iso",
"settings": {
"CANNED": "1",
"DESKTOP": "gnome",
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"DEPLOY_UPLOAD_TEST": "install_default_update_ostree",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"HDDSIZEGB": "15",
"PACKAGE_SET": "default",
"RETRY": "1",
"SUBVARIANT": "Silverblue"
},
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-live-iso",
"settings": {
"DESKTOP": "gnome",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"LIVE": "1",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"RETRY": "1",
Correct casing of SUBVARIANT variables In moving these from fedora_openqa to here, I got the casing wrong, which breaks `tell_source` and causes a test to fail. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 18:55:47 +00:00
"SUBVARIANT": "Workstation"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"flavor": "updates-kde-live-iso",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"settings": {
"DESKTOP": "kde",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"LIVE": "1",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"RETRY": "1",
Correct casing of SUBVARIANT variables In moving these from fedora_openqa to here, I got the casing wrong, which breaks `tell_source` and causes a test to fail. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 18:55:47 +00:00
"SUBVARIANT": "KDE"
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
},
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-upgrade",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-workstation-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Move some update params here from fedora_openqa schedule.py We had code in fedora_openqa schedule.py to set some params for update tests depending on the flavor. I don't remember why I ever wrote it that way, but that's an odd way to do it, the obvious way to do it is just to have them here, like we already have several others and like we do it for composes. This will help with https://progress.opensuse.org/issues/119200 because the params set by fedora_openqa schedule.py go in the request URI but ones set here don't. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-28 16:11:12 +00:00
"+HDD_1": "disk_f%VERSION%_desktop_4_%ARCH%.qcow2",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
}
},
"Profiles": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-container-aarch64-*"
},
"fedora-updates-container-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-container-ppc64le-*"
},
"fedora-updates-container-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-container-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
"fedora-updates-everything-boot-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
"fedora-updates-kde-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-x86_64-*"
},
"fedora-updates-server-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-server-aarch64-*"
},
"fedora-updates-server-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-ppc64le-*"
},
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-upgrade-ppc64le-*"
},
"fedora-updates-server-upgrade-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-upgrade-x86_64-*"
},
"fedora-updates-server-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-x86_64-*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
"fedora-updates-kde-live-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-silverblue-dvd_ostree-iso-x86_64-*"
},
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-silverblue-dvd_ostree-iso-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
"fedora-updates-workstation-live-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
"fedora-updates-workstation-upgrade-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-upgrade-x86_64-*"
},
"fedora-updates-workstation-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-x86_64-*"
}
},
"TestSuites": {
"advisory_boot": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"ADVISORY_BOOT_TEST": "1",
"BOOTFROM": "c",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
"base_selinux": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_service_manipulation": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_services_start": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"base_update_cli": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
"base_reboot_unmount": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
},
"base_system_logging": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"desktop_background": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_browser": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_printing": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_terminal": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"desktop_update_graphical": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-x86_64-*-64bit": 5,
"fedora-updates-workstation-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"install_default_update_live": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": 5,
"fedora-updates-kde-live-iso-x86_64-*-uefi": 5,
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 5,
"fedora-updates-workstation-live-iso-x86_64-*-uefi": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"+START_AFTER_TEST": "live_build@%ARCH_BASE_MACHINE%",
"INSTALL": "1",
Give install_default_update_live 4G of RAM too This test seems to be struggling with 3G of RAM also. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 21:19:50 +00:00
"ISO": "Fedora-%SUBVARIANT%-Live-%ARCH%-%ADVISORY_OR_TASK%.iso",
"QEMURAM": "4096"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"install_default_update_netinst": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5,
"fedora-updates-everything-boot-iso-x86_64-*-uefi": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"+START_AFTER_TEST": "installer_build@%ARCH_BASE_MACHINE%",
Support testing a side tag instead of an update or task Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-04 20:13:15 +00:00
"ADD_REPOSITORY_VARIATION": "%UPDATE_OR_TAG_REPO%",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"INSTALL": "1",
"INSTALL_UNLOCK": "support_ready",
"ISO": "%ADVISORY_OR_TASK%-netinst-%ARCH%.iso",
"NICTYPE": "tap",
"PACKAGE_SET": "default",
"PARALLEL_WITH": "support_server@%ARCH_BASE_MACHINE%",
"WORKER_CLASS": "tap"
}
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"install_default_update_ostree": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
},
"settings": {
"INSTALL": "1",
"ISO": "%ADVISORY_OR_TASK%-%SUBVARIANT%-ostree-%ARCH%.iso",
"PACKAGE_SET": "default",
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"+START_AFTER_TEST": "ostree_build@%ARCH_BASE_MACHINE%",
"STORE_HDD_1": "disk_%FLAVOR%_%MACHINE%.qcow2"
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"installer_build": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"BOOTFROM": "c",
Update names of qcow2 base disk images We're changing these to be named `foo.qcow2` not `foo.img` due to a change in qemu and os-autoinst to do with backing file format detection. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-10 17:58:08 +00:00
"HDD_1": "disk_f%VERSION%_minimal_3_%ARCH%.qcow2",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"NUMDISKS": "2",
"POSTINSTALL": "_installer_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
"live_build": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": 5,
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"settings": {
"+DESKTOP": "",
"+LIVE": "",
"BOOTFROM": "c",
"GRUB_POSTINSTALL": "selinux=0",
Update names of qcow2 base disk images We're changing these to be named `foo.qcow2` not `foo.img` due to a change in qemu and os-autoinst to do with backing file format detection. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-10 17:58:08 +00:00
"HDD_1": "disk_f%VERSION%_minimal_3_%ARCH%.qcow2",
Put /var/lib/mock on separate disk for live image build test The update live image build test keeps running out of disk space. We've bumped the minimal disk image from 12GB all the way up to 20GB so far but it keeps happening. So let's try a different strategy: use a scratch disk to mount /var/lib/mock. That's where all the space gets used. This should allow us to reduce the size of the minimal disk image again, and giving it 25GB of empty disk should avoid it running out of space again for a while. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 04:12:12 +00:00
"HDDSIZEGB_3": "25",
Give live image build even longer and bump max job time Rawhide live image builds are still taking an awful long time and often failing. I will look more into why later, but for now, let's bump the timeouts even more just to try and get through the job backlog. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-19 19:35:24 +00:00
"MAX_JOB_TIME": "10800",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"+NUMDISKS": "3",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"POSTINSTALL": "_live_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"ostree_build": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
},
"settings": {
"+DESKTOP": "",
"+CANNED": "",
"BOOTFROM": "c",
"HDD_1": "disk_f%VERSION%_minimal_3_%ARCH%.qcow2",
Bump disk size on ostree build test to 30 GB Seems we need more space on the scratch disk used by the ostree build test, it's failing due to lack of space. Tested on lab, it worked. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-02-21 04:14:34 +00:00
"HDDSIZEGB_3": "30",
Give ostree_build another half hour again Another bunch of these timed out. Not sure why. Maybe it's when I run a lot of them at the same time? Let's try this, again. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-01-30 01:03:30 +00:00
"MAX_JOB_TIME": "12600",
Add tests to build a Silverblue installer image and install it This is like the existing tests that build network install and live images then install them, only for Silverblue. First we build an ostree, using the standard configuration for the release and subvariant but with the 'advisory' and 'workarounds' repos included, so it will contain current stable packages plus the packages from the update and any workarounds. Then we build an ostree installer image with the ostree embedded, again including advisory and workarounds repos in the installer build config so packages from them will be included in the installer environment. The image is uploaded, which completes the _ostree_build test. Then an install_default_update_ostree test runs, which does a standard install and boot from the installer image. We do make a change that affects other tests, too. We now run _advisory_post on live image install tests, as well as this new ostree install image install test. It was skipped before because of an exception that's really only needed for the netinst image install test. In that test, packages from the update won't be included in the installed system, so we can't run _advisory_post on it. But for ostree and live image build/install tests, the installed system *should* include packages from the update, so we should check and make sure that it does. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-22 19:22:19 +00:00
"+NUMDISKS": "3",
"POSTINSTALL": "_ostree_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"podman": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-container-aarch64-*-aarch64": 5,
"fedora-updates-container-ppc64le-*-ppc64le": 5,
"fedora-updates-container-x86_64-*-64bit": 5
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
}
},
"podman_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-container-aarch64-*-aarch64": 5,
"fedora-updates-container-ppc64le-*-ppc64le": 5,
"fedora-updates-container-x86_64-*-64bit": 5
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"realmd_join_cockpit": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"realmd_join_cockpit_ad": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"realmd_join_sssd": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"realmd_join_sssd_ad": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
Make rpm-ostree tests generic, run on Silverblue and CoreOS This makes the two rpm-ostree tests written for IoT - overlaying and rebasing - work across all rpm-ostree-based flavors we currently test (IoT, CoreOS and Silverblue) and runs them on all those flavors. This requires some other changes. For the Workstation ostree installer update tests, we have install_default_update_ostree upload a disk image and run these tests on that image. That means install_default_update_ostree cannot use a scratch disk (as if we boot it with two disks but only upload one, the subsequent tests fail to boot, looking for the missing second disk), but its specified disk size should be large enough for all updates. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-06 22:41:56 +00:00
"rpmostree_overlay": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
}
},
"rpmostree_rebase": {
"profiles": {
"fedora-updates-silverblue-dvd_ostree-iso-x86_64-*-64bit": 5
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_cockpit_basic": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_cockpit_default": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_cockpit_updates": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_database_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_firewall_default": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_master": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_freeipa_replication_replica": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_remote_logging_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
"server_remote_logging_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_role_deploy_database_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"server_role_deploy_domain_controller": {
Add Samba AD tests This adds a Samba AD server test, and client enrolment tests via sssd, Cockpit and kickstart. Requires the matching createhdds commit to add the kickstart to the disk_ks image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-07-19 23:41:21 +00:00
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
}
},
"server_samba_domain_controller": {
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-aarch64-*-aarch64": 5,
"fedora-updates-server-ppc64le-*-ppc64le": 5,
"fedora-updates-server-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"support_server": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"upgrade_desktop_encrypted_64bit": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-workstation-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"upgrade_realmd_client": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 5,
"fedora-updates-server-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"upgrade_server_domain_controller": {
"profiles": {
Give all update tests the same priority, ahead of compose tests It doesn't make sense for update tests to have different priorities. We gate on all update tests, so they are all equally important. With varying priorities we get a problem: if 10 updates are created quite close together, first we do all the prio 30 and prio 40 tests for all of them, and only after all of those are done do we get to the prio 41 and 42 tests for the earliest of the updates, which means they can be waiting longer than they should to have all the tests done. If all the tests have the same priority, openQA should always use creation time to order them, so it should prioritize finishing tests for earlier updates, which is what we want. Update tests are also more time-critical than compose tests, because we gate on update tests. So we should prioritize update tests over compose tests in all cases. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-11-28 16:43:30 +00:00
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 5,
"fedora-updates-server-upgrade-x86_64-*-64bit": 5
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
}
}
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
Reference in New Issue Copy Permalink