1
0
mirror of https://pagure.io/fedora-qa/createhdds.git synced 2024-11-24 16:03:07 +00:00
createhdds/hdds.json

214 lines
5.9 KiB
JSON
Raw Permalink Normal View History

rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
{
"guestfs" : [
{
"name" : "full",
"size" : "10G",
"labels" : ["mbr", "gpt"],
"parts" : [
{
"filesystem" : "ext4",
"type" : "p",
"start" : "4096",
"end" : "12582093"
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
},
{
"filesystem" : "ext4",
"type" : "p",
"start" : "12582094",
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
"end" : "-4097"
}
],
"writes" : [
{
"part" : "1",
"path" : "/testfile",
"content" : "Hello, world!"
},
{
"part" : "2",
"path" : "/testfile",
"content" : "Oh, hi Mark"
}
]
},
{
"name" : "freespace",
"size" : "10G",
"labels" : ["mbr", "gpt"],
"parts" : [
{
"filesystem" : "ext4",
"type" : "p",
"start" : "4096",
"end" : "2097152"
}
],
"writes" : [
{
"part" : "1",
"path" : "/testfile",
"content" : "Hello, world!"
}
]
},
{
"name" : "ks",
"size" : "100M",
"imgver" : "7",
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
"parts" : [
{
"filesystem" : "ext4",
"type" : "p",
"start" : "4096",
"end" : "-1"
}
],
"uploads" : [
{
"part" : "1",
"target" : "/root-user-crypted-net.ks",
"source" : "root-user-crypted-net.ks"
},
{
"part" : "1",
"target" : "/freeipa.ks",
"source" : "freeipa.ks"
},
{
"part" : "1",
"target" : "/freeipaclient.ks",
"source" : "freeipaclient.ks"
},
{
"part" : "1",
"target" : "/adclient.ks",
"source" : "adclient.ks"
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
}
]
},
{
"name" : "updates_img",
"size" : "100M",
"imgver" : "2",
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
"parts" : [
{
"filesystem" : "ext4",
"label" : "UPDATES_IMG",
"type" : "p",
"start" : "4096",
"end" : "-1"
}
],
"uploads" : [
{
"part" : "1",
"target" : "/updates.img",
"source" : "updates.img"
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
}
]
},
{
"name" : "shrink",
"size" : "10G",
"labels" : ["mbr"],
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
"filesystems" : ["ext4", "ntfs"],
"parts" : [
{
"type" : "p",
"start" : "4096",
"end" : "-4097"
}
],
"writes" : [
{
"part" : "1",
"path" : "/testfile",
"content" : "Hello, world!"
}
]
}
],
switch from virt-builder to virt-install (T813) Summary: We've kinda been having too much trouble with virt-builder lately, mainly SELinux related issues due to how it does image customization. It also produces images that differ in notable ways from what a 'typical' install would give. virt-install solves both these problems, and also gives us more flexibility for storage configuration and post-install customization should we need them in future. The change isn't really too drastic, and the design is similar: instead of virt-builder commands files, each image type now has a kickstart file where all its customizations can be done. There's also a single extra image dict key, 'variant', which specifies which install tree variant to use for running the install. It defaults to 'Everything' (for F24+) and 'Server' (for <F24, as Everything wasn't installable until F24) but we set it to 'Server' for the server images and 'Workstation' for the desktop images, so those installs will use the correct variant install class. We run the installs in VNC. You can do it with a serial console and log the output, but then anaconda gets clever and changes several things in the installed system based on the fact that you did the install over a serial console: it twiddles with the kernel args and doesn't set graphical.target as the default. We don't want any of that mess, so we do a VNC install. The 'size' value is just a number of gigabytes for virt-install images (as that's how the virt-install 'size' argument works). This also drops some unused 32-bit images (we don't do 32-bit KDE or Server upgrade tests, so there's no need to build those images). Test Plan: Re-generate all affected images and re-run all tests that use them, make sure they work. I am doing this on staging at present. Note: this would render D911 unnecessary. Reviewers: garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 16:29:25 +00:00
"virtinstall" : [
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
{
"name" : "minimal",
"releases" : {
"-1" : ["x86_64", "aarch64", "ppc64le"],
"-2" : ["x86_64", "aarch64", "ppc64le"],
"stable": ["x86_64", "aarch64"],
"branched": ["x86_64", "aarch64"],
"rawhide": ["x86_64"]
switch from virt-builder to virt-install (T813) Summary: We've kinda been having too much trouble with virt-builder lately, mainly SELinux related issues due to how it does image customization. It also produces images that differ in notable ways from what a 'typical' install would give. virt-install solves both these problems, and also gives us more flexibility for storage configuration and post-install customization should we need them in future. The change isn't really too drastic, and the design is similar: instead of virt-builder commands files, each image type now has a kickstart file where all its customizations can be done. There's also a single extra image dict key, 'variant', which specifies which install tree variant to use for running the install. It defaults to 'Everything' (for F24+) and 'Server' (for <F24, as Everything wasn't installable until F24) but we set it to 'Server' for the server images and 'Workstation' for the desktop images, so those installs will use the correct variant install class. We run the installs in VNC. You can do it with a serial console and log the output, but then anaconda gets clever and changes several things in the installed system based on the fact that you did the install over a serial console: it twiddles with the kernel args and doesn't set graphical.target as the default. We don't want any of that mess, so we do a VNC install. The 'size' value is just a number of gigabytes for virt-install images (as that's how the virt-install 'size' argument works). This also drops some unused 32-bit images (we don't do 32-bit KDE or Server upgrade tests, so there's no need to build those images). Test Plan: Re-generate all affected images and re-run all tests that use them, make sure they work. I am doing this on staging at present. Note: this would render D911 unnecessary. Reviewers: garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 16:29:25 +00:00
},
"size" : "10",
"imgver": "4",
"bootopts": "uefi"
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
},
{
"name" : "minimal-bios",
"releases" : {
"-1" : ["x86_64"],
"-2" : ["x86_64"],
"branched": ["x86_64"]
},
"size" : "10",
"imgver": "4"
},
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
{
"name" : "desktop",
"releases" : {
"-1" : ["ppc64le"],
"stable" : ["x86_64", "aarch64"],
"branched": ["x86_64", "aarch64"],
"rawhide": ["x86_64"]
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
},
switch from virt-builder to virt-install (T813) Summary: We've kinda been having too much trouble with virt-builder lately, mainly SELinux related issues due to how it does image customization. It also produces images that differ in notable ways from what a 'typical' install would give. virt-install solves both these problems, and also gives us more flexibility for storage configuration and post-install customization should we need them in future. The change isn't really too drastic, and the design is similar: instead of virt-builder commands files, each image type now has a kickstart file where all its customizations can be done. There's also a single extra image dict key, 'variant', which specifies which install tree variant to use for running the install. It defaults to 'Everything' (for F24+) and 'Server' (for <F24, as Everything wasn't installable until F24) but we set it to 'Server' for the server images and 'Workstation' for the desktop images, so those installs will use the correct variant install class. We run the installs in VNC. You can do it with a serial console and log the output, but then anaconda gets clever and changes several things in the installed system based on the fact that you did the install over a serial console: it twiddles with the kernel args and doesn't set graphical.target as the default. We don't want any of that mess, so we do a VNC install. The 'size' value is just a number of gigabytes for virt-install images (as that's how the virt-install 'size' argument works). This also drops some unused 32-bit images (we don't do 32-bit KDE or Server upgrade tests, so there's no need to build those images). Test Plan: Re-generate all affected images and re-run all tests that use them, make sure they work. I am doing this on staging at present. Note: this would render D911 unnecessary. Reviewers: garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 16:29:25 +00:00
"size" : "20",
"imgver": "5",
"variant": "Workstation",
"bootopts": "uefi"
},
{
"name" : "desktopencrypt",
"releases" : {
"stable" : ["x86_64", "aarch64"],
"branched" : ["x86_64", "aarch64"]
},
"size" : "20",
"imgver": "2",
"variant": "Workstation",
"bootopts": "uefi"
},
{
"name" : "server",
"releases" : {
"stable" : ["x86_64", "ppc64le", "aarch64"],
"branched": ["x86_64", "ppc64le", "aarch64"],
"rawhide": ["x86_64", "ppc64le"]
},
"size" : "7",
"imgver": "4",
"variant": "Server",
"bootopts": "uefi"
},
{
"name" : "kde",
"releases" : {
"stable" : ["x86_64"],
"branched" : ["x86_64"],
"rawhide": ["x86_64"]
},
switch from virt-builder to virt-install (T813) Summary: We've kinda been having too much trouble with virt-builder lately, mainly SELinux related issues due to how it does image customization. It also produces images that differ in notable ways from what a 'typical' install would give. virt-install solves both these problems, and also gives us more flexibility for storage configuration and post-install customization should we need them in future. The change isn't really too drastic, and the design is similar: instead of virt-builder commands files, each image type now has a kickstart file where all its customizations can be done. There's also a single extra image dict key, 'variant', which specifies which install tree variant to use for running the install. It defaults to 'Everything' (for F24+) and 'Server' (for <F24, as Everything wasn't installable until F24) but we set it to 'Server' for the server images and 'Workstation' for the desktop images, so those installs will use the correct variant install class. We run the installs in VNC. You can do it with a serial console and log the output, but then anaconda gets clever and changes several things in the installed system based on the fact that you did the install over a serial console: it twiddles with the kernel args and doesn't set graphical.target as the default. We don't want any of that mess, so we do a VNC install. The 'size' value is just a number of gigabytes for virt-install images (as that's how the virt-install 'size' argument works). This also drops some unused 32-bit images (we don't do 32-bit KDE or Server upgrade tests, so there's no need to build those images). Test Plan: Re-generate all affected images and re-run all tests that use them, make sure they work. I am doing this on staging at present. Note: this would render D911 unnecessary. Reviewers: garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 16:29:25 +00:00
"size" : "20",
"imgver": "5",
"bootopts": "uefi"
},
{
"name" : "support",
"releases" : {
"-1" : ["x86_64", "ppc64le", "aarch64"]
switch from virt-builder to virt-install (T813) Summary: We've kinda been having too much trouble with virt-builder lately, mainly SELinux related issues due to how it does image customization. It also produces images that differ in notable ways from what a 'typical' install would give. virt-install solves both these problems, and also gives us more flexibility for storage configuration and post-install customization should we need them in future. The change isn't really too drastic, and the design is similar: instead of virt-builder commands files, each image type now has a kickstart file where all its customizations can be done. There's also a single extra image dict key, 'variant', which specifies which install tree variant to use for running the install. It defaults to 'Everything' (for F24+) and 'Server' (for <F24, as Everything wasn't installable until F24) but we set it to 'Server' for the server images and 'Workstation' for the desktop images, so those installs will use the correct variant install class. We run the installs in VNC. You can do it with a serial console and log the output, but then anaconda gets clever and changes several things in the installed system based on the fact that you did the install over a serial console: it twiddles with the kernel args and doesn't set graphical.target as the default. We don't want any of that mess, so we do a VNC install. The 'size' value is just a number of gigabytes for virt-install images (as that's how the virt-install 'size' argument works). This also drops some unused 32-bit images (we don't do 32-bit KDE or Server upgrade tests, so there's no need to build those images). Test Plan: Re-generate all affected images and re-run all tests that use them, make sure they work. I am doing this on staging at present. Note: this would render D911 unnecessary. Reviewers: garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 16:29:25 +00:00
},
"size" : "11",
"imgver" : "6",
"bootopts": "uefi"
rewrite createhdds in python, make it shinier Summary: createhdds.sh was just too damn simple and understandable, so I thought I'd make it three times longer, object oriented, and hard to understand! OK, OK, that's not great sales. Alright. The main thing was to make it smarter. This rewrite lets it do these things: * Only create the images that are missing (not rebuild all) * Work out the releases to build images for * Rename images when appropriate * Rebuild images when they need rebuilding * Remove old / abandoned images It can figure out what images ought to be present - including working out the 'next' release and figuring out from that what releases it needs images for - and build only the missing ones. There's a 'version' concept for images; if the existing image is older than the version given in the data file, it'll be rebuilt. The data file can list 'rename' pairs, allowing images to be renamed (like when we move from a single image to multiple label/filesystem variants). This code uses fedfind's ability to find the current release version to figure out what releases we need virtbuilder images for (so you don't have to pass it in). And it can find image files that aren't in the 'currently expected' set and wipe them. Images can also have a 'maxage', triggering a rebuild when they exceed it - this is intended for the virtbuilder images, so we get a rebuild with the latest updates every so often (default is two weeks). The point of all this is to help with unattended deployment/ maintenance, i.e. the ansible deployment we have in infra; the idea is that we can just set that up to run the 'all' subcommand every so often, and it'll remove old images, create new ones, and rebuild ones that are outdated. I kept the ability to build a single image (or a whole image 'group'), and included the ability to just run a check without actually doing a rebuild. There's a few little weird things and holes here as it's not really the focus of the tool. Test Plan: Build all images, do a full test run, and see if it works OK. Test out all the variations of building single images / image groups, and using the 'check' command. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
}
],
"renames" : []
}