srbala/os-autoinst-distri-fedora
1
0
Fork 0
mirror of https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git synced 2024-12-20 17:33:08 +00:00
Code Issues Releases Activity
9d04235e64
os-autoinst-distri-fedora/templates-updates.fif.json

523 lines
18 KiB
JSON
Raw Normal View History

Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
{
"Products": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
"fedora-updates-container-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-container",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-everything-boot-iso",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-kde-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-kde",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "kde",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Bump RETRY for updates-kde to 2 while we have oom kill issues https://bugzilla.redhat.com/show_bug.cgi?id=2133829 is a real problem for Rawhide KDE update tests right now, despite all I've tried to do to work around it. I'm going to regenerate the base image again tomorrow, but for now, let's bump RETRY to 2 as a mitigation. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-23 16:14:35 +00:00
"RETRY": "2"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-server-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-aarch64-*": {
"arch": "aarch64",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-ppc64le-*": {
"arch": "ppc64le",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server-upgrade",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-server-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-server",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"settings": {
"RETRY": "1"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"version": "*"
},
"fedora-updates-workstation-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-live-iso",
"settings": {
"DESKTOP": "gnome",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"LIVE": "1",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-kde-live-iso",
"settings": {
"DESKTOP": "kde",
Bump HDDSIZEGB for more GNOME and KDE products We have it at 20G for Workstation live, but not for KDE live or for the update Workstation or KDE products. We just hit an issue today where anaconda thinks 10G isn't enough space for a KDE live install after a grub2 update (which I think only bumped the required space veeery slightly, but enough to throw anaconda over the limit). Let's just go up to 15G for all GNOME and KDE cases where we're not at 20G. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-08 23:53:47 +00:00
"HDDSIZEGB": "15",
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"LIVE": "1",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"PACKAGE_SET": "default",
"RETRY": "1"
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
},
"version": "*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-upgrade-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation-upgrade",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
},
"fedora-updates-workstation-x86_64-*": {
"arch": "x86_64",
"distri": "fedora",
"flavor": "updates-workstation",
"settings": {
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"DESKTOP": "gnome",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"NUMDISKS": "2",
Use new RETRY variable handling to restart update tests on fail Upstream recently implemented support for using the variable RETRY to specify how many times a test should be restarted on failure. This is something we currently handle with a downstream openQA plugin; if we switch to using this upstream feature instead, we can drop the plugin. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 17:28:57 +00:00
"RETRY": "1"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
},
"version": "*"
}
},
"Profiles": {
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"fedora-updates-container-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-container-aarch64-*"
},
"fedora-updates-container-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-container-ppc64le-*"
},
"fedora-updates-container-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-container-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-everything-boot-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
"fedora-updates-everything-boot-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-everything-boot-iso-x86_64-*"
},
"fedora-updates-kde-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-x86_64-*"
},
"fedora-updates-server-aarch64-*-aarch64": {
"machine": "aarch64",
"product": "fedora-updates-server-aarch64-*"
},
"fedora-updates-server-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-ppc64le-*"
},
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": {
"machine": "ppc64le",
"product": "fedora-updates-server-upgrade-ppc64le-*"
},
"fedora-updates-server-upgrade-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-upgrade-x86_64-*"
},
"fedora-updates-server-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-server-x86_64-*"
},
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
"fedora-updates-kde-live-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-kde-live-iso-x86_64-*"
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
"fedora-updates-workstation-live-iso-x86_64-*-uefi": {
"machine": "uefi",
"product": "fedora-updates-workstation-live-iso-x86_64-*"
},
"fedora-updates-workstation-upgrade-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-upgrade-x86_64-*"
},
"fedora-updates-workstation-x86_64-*-64bit": {
"machine": "64bit",
"product": "fedora-updates-workstation-x86_64-*"
}
},
"TestSuites": {
"advisory_boot": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
},
"settings": {
"ADVISORY_BOOT_TEST": "1",
"BOOTFROM": "c",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
"base_selinux": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
"base_service_manipulation": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
"base_services_start": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
"base_update_cli": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
"base_reboot_unmount": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
"base_system_logging": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 42,
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40,
"fedora-updates-workstation-x86_64-*-64bit": 40
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"desktop_background": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 32,
"fedora-updates-workstation-x86_64-*-64bit": 30
}
},
"desktop_browser": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 32,
"fedora-updates-workstation-x86_64-*-64bit": 30
}
},
"desktop_printing": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 32,
"fedora-updates-workstation-x86_64-*-64bit": 30
}
},
"desktop_terminal": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 32,
"fedora-updates-workstation-x86_64-*-64bit": 30
}
},
"desktop_update_graphical": {
"profiles": {
"fedora-updates-kde-x86_64-*-64bit": 32,
"fedora-updates-workstation-x86_64-*-64bit": 30
}
},
"install_default_update_live": {
"profiles": {
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": 40,
"fedora-updates-kde-live-iso-x86_64-*-uefi": 41,
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 40,
"fedora-updates-workstation-live-iso-x86_64-*-uefi": 41
},
"settings": {
"+START_AFTER_TEST": "live_build@%ARCH_BASE_MACHINE%",
"INSTALL": "1",
Give install_default_update_live 4G of RAM too This test seems to be struggling with 3G of RAM also. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-10-17 21:19:50 +00:00
"ISO": "Fedora-%SUBVARIANT%-Live-%ARCH%-%ADVISORY_OR_TASK%.iso",
"QEMURAM": "4096"
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
}
},
"install_default_update_netinst": {
"profiles": {
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 40,
"fedora-updates-everything-boot-iso-x86_64-*-uefi": 41
},
"settings": {
"+START_AFTER_TEST": "installer_build@%ARCH_BASE_MACHINE%",
Use qemu host IP 172.16.2.2 not 10.0.2.2 This is to make the infra folks happy, apparently using 10.0.x.x and 10.1.x.x is causing conflicts since our actual infra network uses those ranges too. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-17 20:42:48 +00:00
"ADD_REPOSITORY_VARIATION": "nfs://172.16.2.110:/opt/update_repo",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"INSTALL": "1",
"INSTALL_UNLOCK": "support_ready",
"ISO": "%ADVISORY_OR_TASK%-netinst-%ARCH%.iso",
"NICTYPE": "tap",
"PACKAGE_SET": "default",
"PARALLEL_WITH": "support_server@%ARCH_BASE_MACHINE%",
"WORKER_CLASS": "tap"
}
},
"installer_build": {
"profiles": {
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 40
},
"settings": {
"BOOTFROM": "c",
Update names of qcow2 base disk images We're changing these to be named `foo.qcow2` not `foo.img` due to a change in qemu and os-autoinst to do with backing file format detection. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-10 17:58:08 +00:00
"HDD_1": "disk_f%VERSION%_minimal_3_%ARCH%.qcow2",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"NUMDISKS": "2",
"POSTINSTALL": "_installer_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
"live_build": {
"profiles": {
Add KDE live image build and test for updates I hacked this up quickly on staging to test a specific update, but there's really no reason we shouldn't just do it generally. We have the capacity. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-22 23:00:16 +00:00
"fedora-updates-kde-live-iso-x86_64-*-64bit": 40,
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"fedora-updates-workstation-live-iso-x86_64-*-64bit": 40
},
"settings": {
"+DESKTOP": "",
"+LIVE": "",
"BOOTFROM": "c",
"GRUB_POSTINSTALL": "selinux=0",
Update names of qcow2 base disk images We're changing these to be named `foo.qcow2` not `foo.img` due to a change in qemu and os-autoinst to do with backing file format detection. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-09-10 17:58:08 +00:00
"HDD_1": "disk_f%VERSION%_minimal_3_%ARCH%.qcow2",
Put /var/lib/mock on separate disk for live image build test The update live image build test keeps running out of disk space. We've bumped the minimal disk image from 12GB all the way up to 20GB so far but it keeps happening. So let's try a different strategy: use a scratch disk to mount /var/lib/mock. That's where all the space gets used. This should allow us to reduce the size of the minimal disk image again, and giving it 25GB of empty disk should avoid it running out of space again for a while. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 04:12:12 +00:00
"HDDSIZEGB_3": "25",
Give live image build even longer and bump max job time Rawhide live image builds are still taking an awful long time and often failing. I will look more into why later, but for now, let's bump the timeouts even more just to try and get through the job backlog. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-19 19:35:24 +00:00
"MAX_JOB_TIME": "10800",
Make update testing more robust for very large updates openQA choked badly on https://bodhi.fedoraproject.org/updates/FEDORA-2022-6256981a71 because it's, well, huge - 87 builds including texlive, which has hundreds (thousands?) of subpackages. This exposed several frailties against such updates. First of all, we set NUMDISKS to at least 2 for *all* update tests, which should mean they all stash the RPMs from the update on a non-system disk and avoid problems with space exhaustion. After that, just extend a few timeouts in particularly fragile places, including one which is specific to texlive (as I don't know of any other source package with so many subpackages). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-08-02 16:32:47 +00:00
"+NUMDISKS": "3",
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"POSTINSTALL": "_live_build",
"ROOT_PASSWORD": "weakpassword",
"USER_LOGIN": "false"
}
},
Run podman tests on updates It has been noted that updates have broken podman in the past and this is a major issue for some users. Let's create a new update flavor and run the test in it. We'll use the server image as a base, but it's not really a server test, so I'm giving it its own flavor so it's not run on updates that we only want to run server tests on, and we can schedule just this test to run on container-y updates. As part of this, we need to install podman before running the test; for flavors we currently run it on we expect podman to be preinstalled, but that's not true for the server base image. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-21 15:38:58 +00:00
"podman": {
"profiles": {
"fedora-updates-container-aarch64-*-aarch64": 40,
"fedora-updates-container-ppc64le-*-ppc64le": 40,
"fedora-updates-container-x86_64-*-64bit": 40
}
},
"podman_client": {
"profiles": {
"fedora-updates-container-aarch64-*-aarch64": 40,
"fedora-updates-container-ppc64le-*-ppc64le": 40,
"fedora-updates-container-x86_64-*-64bit": 40
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"realmd_join_cockpit": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"realmd_join_sssd": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 30,
"fedora-updates-server-ppc64le-*-ppc64le": 30,
"fedora-updates-server-x86_64-*-64bit": 30
}
},
"server_cockpit_basic": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_cockpit_default": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_cockpit_updates": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_database_client": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_firewall_default": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_freeipa_replication_client": {
"profiles": {
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_freeipa_replication_master": {
"profiles": {
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_freeipa_replication_replica": {
"profiles": {
"fedora-updates-server-x86_64-*-64bit": 40
}
},
Add cockpit_updates and remote_logging tests to updates Again, no reason not to run these on updates. Includes adding oldcantarell versions of several needles for current cockpit, as they're needed for the tests to pass. Also tweak a couple of needles to avoid false matches (add more empty space). Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:32:55 +00:00
"server_remote_logging_client": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_remote_logging_server": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
Add a whole intermediate template format ('FIF') and tools I and @lruzicka (and I think @jskladan and @jsedlak and @michelmno and everyone else who's ever touched it...) are being gradually driven nuts by manually editing the test templates. The bigger the files get the more awkward it is to keep them straight and be sure we're doing it right. Upstream doesn't do things the same way we do (they mostly edit in the web UI and dump to file for the record), but we do still think making changes in the repo and posting to the web UI is the right way around to do it, we just wish the format was saner. Upstream has actually recently introduced a YAML-based approach to storing job templates which tries to condense things a bit, and you can dump to that format with dump-templates --json, but @lruzicka and I agree that that format is barely better for hand editing in a text editor than the older one our templates currently use. So, this commit introduces...Fedora Intermediate Format (FIF) - an alternative format for representing job templates - and some tools for working with it. It also contains our existing templates in this new format, and removes the old template files. The format is documented in the docstrings of the tools, but briefly, it keeps Machines, Products and TestSuites but improves their format a bit (by turning dicts-of-lists into dicts-of- dicts), and adds Profiles, which are combinations of Machines and Products. TestSuites can indicate which Profiles they should be run on. The intermediate format converter (`fifconverter`) converts existing template data (in JSON format; use tojson.pm to convert our perl templates to JSON) to the intermediate format and writes it out. As this was really intended only for one-time use (the idea is that after one-time conversion, we will edit the templates in the intermediate format from now on), its operation is hardcoded and relies on specific filenames. The intermediate format loader (`fifloader`) generates JobTemplates from the TestSuites and Profiles, reverses the quality-of-life improvements of the intermediate format, and produces template data compatible with the upstream loader, then can write it to disk and/or call the upstream loader directly. The check script (`fifcheck`) runs existing template data through both the converter and the loader, then checks that the result is equivalent to the input. Again this was mostly written for one- time use so is fairly rough and hard-coded, but I'm including it in the commit so others can check the work and so on. Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-23 14:20:10 +00:00
"server_role_deploy_database_server": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"server_role_deploy_domain_controller": {
"profiles": {
"fedora-updates-server-aarch64-*-aarch64": 40,
"fedora-updates-server-ppc64le-*-ppc64le": 40,
"fedora-updates-server-x86_64-*-64bit": 40
}
},
"support_server": {
"profiles": {
"fedora-updates-everything-boot-iso-x86_64-*-64bit": 40
}
},
"upgrade_desktop_encrypted_64bit": {
"profiles": {
"fedora-updates-workstation-upgrade-x86_64-*-64bit": 40
}
},
"upgrade_realmd_client": {
"profiles": {
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 30,
"fedora-updates-server-upgrade-x86_64-*-64bit": 30
}
},
"upgrade_server_domain_controller": {
"profiles": {
"fedora-updates-server-upgrade-ppc64le-*-ppc64le": 40,
"fedora-updates-server-upgrade-x86_64-*-64bit": 40
}
}
}
Add base_reboot_unmount and base_system_logging to update tests No reason not to run these on updates as well. And it's much easier with FIF! Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:29:02 +00:00
}
Reference in New Issue Copy Permalink