rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
{
|
|
|
|
|
"guestfs" : [
|
|
|
|
|
{
|
|
|
|
|
"name" : "full",
|
|
|
|
|
"size" : "10G",
|
|
|
|
|
"labels" : ["mbr", "gpt"],
|
|
|
|
|
"parts" : [
|
|
|
|
|
{
|
|
|
|
|
"filesystem" : "ext4",
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "4096",
|
|
|
|
|
"end" : "10485760"
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"filesystem" : "ext4",
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "10485761",
|
|
|
|
|
"end" : "-4097"
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"writes" : [
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"path" : "/testfile",
|
|
|
|
|
"content" : "Hello, world!"
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"part" : "2",
|
|
|
|
|
"path" : "/testfile",
|
|
|
|
|
"content" : "Oh, hi Mark"
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "freespace",
|
|
|
|
|
"size" : "10G",
|
|
|
|
|
"labels" : ["mbr", "gpt"],
|
|
|
|
|
"parts" : [
|
|
|
|
|
{
|
|
|
|
|
"filesystem" : "ext4",
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "4096",
|
|
|
|
|
"end" : "2097152"
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"writes" : [
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"path" : "/testfile",
|
|
|
|
|
"content" : "Hello, world!"
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "ks",
|
|
|
|
|
"size" : "100M",
|
2016-05-04 18:55:22 +00:00
|
|
|
"imgver" : "2",
|
rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
"parts" : [
|
|
|
|
|
{
|
|
|
|
|
"filesystem" : "ext4",
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "4096",
|
|
|
|
|
"end" : "-1"
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"uploads" : [
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"target" : "/root-user-crypted-net.ks",
|
|
|
|
|
"source" : "https://jskladan.fedorapeople.org/kickstarts/root-user-crypted-net.ks"
|
2016-05-04 18:55:22 +00:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"target" : "/freeipa.ks",
|
|
|
|
|
"source" : "https://www.happyassassin.net/ks/oqipas.ks"
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"target" : "/freeipaclient.ks",
|
|
|
|
|
"source" : "https://www.happyassassin.net/ks/oqipac.ks"
|
rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "updates_img",
|
|
|
|
|
"size" : "100M",
|
2016-04-01 15:01:12 +00:00
|
|
|
"imgver" : "2",
|
rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
"parts" : [
|
|
|
|
|
{
|
|
|
|
|
"filesystem" : "ext4",
|
|
|
|
|
"label" : "UPDATES_IMG",
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "4096",
|
|
|
|
|
"end" : "-1"
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"uploads" : [
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"target" : "/updates.img",
|
2016-04-01 15:01:12 +00:00
|
|
|
"source" : "https://fedorapeople.org/groups/qa/updates/updates-openqa.img"
|
rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "shrink",
|
|
|
|
|
"size" : "10G",
|
|
|
|
|
"labels" : ["mbr", "gpt"],
|
|
|
|
|
"filesystems" : ["ext4", "ntfs"],
|
|
|
|
|
"parts" : [
|
|
|
|
|
{
|
|
|
|
|
"type" : "p",
|
|
|
|
|
"start" : "4096",
|
|
|
|
|
"end" : "-4097"
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"writes" : [
|
|
|
|
|
{
|
|
|
|
|
"part" : "1",
|
|
|
|
|
"path" : "/testfile",
|
|
|
|
|
"content" : "Hello, world!"
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"virtbuilder" : [
|
|
|
|
|
{
|
|
|
|
|
"name" : "minimal",
|
|
|
|
|
"releases" : {
|
|
|
|
|
"-1" : ["x86_64"],
|
|
|
|
|
"-2" : ["x86_64"]
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "desktop",
|
|
|
|
|
"releases" : {
|
|
|
|
|
"-1" : ["x86_64", "i686"],
|
|
|
|
|
"-2" : ["x86_64", "i686"]
|
|
|
|
|
},
|
2016-05-05 23:43:10 +00:00
|
|
|
"size" : "20G",
|
|
|
|
|
"imgver": "2"
|
2016-04-25 07:59:21 +00:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "server",
|
|
|
|
|
"releases" : {
|
|
|
|
|
"-1" : ["x86_64", "i686"],
|
|
|
|
|
"-2" : ["x86_64", "i686"]
|
2016-05-05 23:43:10 +00:00
|
|
|
},
|
|
|
|
|
"imgver": "2"
|
2016-04-25 07:59:21 +00:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"name" : "kde",
|
|
|
|
|
"releases" : {
|
|
|
|
|
"-1" : ["x86_64", "i686"],
|
|
|
|
|
"-2" : ["x86_64", "i686"]
|
|
|
|
|
},
|
2016-05-05 23:43:10 +00:00
|
|
|
"size" : "20G",
|
|
|
|
|
"imgver": "2"
|
rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!
OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:
* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images
It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).
The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.
I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.
Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 21:25:30 +00:00
|
|
|
}
|
|
|
|
|
],
|
|
|
|
|
"renames" : []
|
|
|
|
|
}
|
