1
0
mirror of https://pagure.io/fedora-qa/createhdds.git synced 2024-12-23 03:03:08 +00:00
Commit Graph

119 Commits

Author SHA1 Message Date
Adam Williamson
01aa7d52b7 Goddamn JSON
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 13:39:11 -07:00
Adam Williamson
0714df1da0 Have minimal image for all stable and Branched as well
As we're gonna be running update tests off it now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-31 13:36:15 -07:00
Adam Williamson
c0a92bfd73 Maybe it's right this time...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-08 11:09:47 -07:00
Adam Williamson
4c46794638 Fix an error in previous commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-07 16:34:53 -07:00
Adam Williamson
9aeb8841c5 Try and deal with F26 being archived *already*
Grumble mumble.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-08-07 16:33:46 -07:00
Michel Normand
3f9cbd27d4 Disable plymouth for PowerPC release >f27
as bypass for pending bug
https://bugzilla.redhat.com/show_bug.cgi?id=1571860

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2018-05-04 11:05:29 +02:00
Adam Williamson
fca1c542c9 hdds.json: keep desktopencrypt around for all stable releases
This is needed now we want to run update tests on it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-05-02 15:44:14 -07:00
Adam Williamson
46d78228e9 Correct cross-arch info message again
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-15 10:37:37 -07:00
Adam Williamson
14abb9b3e1 libvirt: Undefine NVRAM when undefining domain
This is apparently necessary on aarch64, we get an error from
libvirt otherwise.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-14 13:42:01 -07:00
Adam Williamson
a886b9f4a9 Handle aarch64 being secondary for releases < 28
Sigh, this mini-fedfind is so much fun.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-14 12:24:37 -07:00
Adam Williamson
450efb8bb5 Add aarch64 support and images
We're bootstrapping aarch64 into openQA, so we need this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-06 09:06:58 -08:00
Adam Williamson
498cc306f8 Update README, make 'wrong arch' log message less scary
README was badly outdated, so give it a coat of paint. A person
trying to get up to speed on openQA was misled by the current
message into thinking something was wrong.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-06 08:54:35 -08:00
Adam Williamson
fe61724e8e Explicitly enable firstboot in the desktop kickstarts
This ensures that gnome-initial-setup will run in user mode on
first login as 'test' user for F26+, as it did for <=F25 and as
the tests expected (we can now remove the workaround I added
because this stopped happening with F26). Bump the image version
so the images get rebuilt.
2017-10-20 17:41:42 -07:00
Adam Williamson
ed7fcd75e4 Don't list images for other arches as 'missing'
...But also be careful not to treat them as 'unknown' either.
This avoids the ansible plays regenerating outdated images (we
try to avoid that and just have the cron job do it).
2017-10-16 09:51:56 -07:00
Adam Williamson
7c32272cc8 Include arch in branched virt-install URL
...*this* should be right.
2017-09-15 19:56:35 -07:00
Adam Williamson
4e2df1104d Don't include fedoradir twice in virt-install branched URL
...oops.
2017-09-15 19:53:13 -07:00
Adam Williamson
b0fd971916 Sigh, force to str when appropriate... 2017-09-04 16:08:18 -07:00
Adam Williamson
d87bafb15e Bit more rejigging, fix i386 secondary case
Broke the i386 secondary case with the previous rejig (by moving
where we fix i686 to be i386). This fixes that and again makes
readability better.
2017-09-04 16:06:44 -07:00
Adam Williamson
7e5db6a426 Re-order VirtInstallImage.create() a bit to avoid bugs
Various things were being set in odd orders and not used right
later on, this is a general clean-up that solves that and makes
things a bit easier to read (I hope).
2017-09-04 12:19:59 -07:00
Adam Williamson
12e3a050b7 Get the domain again before cleaning it up, catch errors
Whoops, that last commit just wasn't very good. We need to
re-open the libvirt connection and re-get the domain object,
and also we should catch errors while destroying and undefining
the domain...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-08-31 17:29:00 -04:00
Adam Williamson
ba1dd56896 Ensure domain is taken down when we exhaust retries
This was just an oversight - we don't destroy and undefine the
'createhdds' domain on the path where the final retry to create
an image times out, we just wipe the temp file and exit. So we
leave a qemu process sitting around until createhdds runs again
(and, on openQA production, exhausting swap space).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-08-30 17:33:03 -04:00
Adam Williamson
e112047296 Rejig supported arch stuff to avoid incorrect deletions
The way this was done, when we ran `all` on a ppc host it wiped
all the x86_64 and i686 virt-install images, and when we ran
`all` on an x86_64 host it wiped all the ppc64 virt-install
images. Doing it this way avoids that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-08-17 17:42:43 -07:00
Michel Normand
1f39c80d2b Increase memory size from 2048 to 4096 for PowerPC
as today seems to be required to avoid createhdds to hang
on creation of disk_f25_minimal_2_ppc64le.img.tmp ...[13/22]
Trying to VNC connect, show anaconda menu stuck on
"Performing post-installation setup tasks"

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-11 07:24:03 -04:00
Michel Normand
eec1f1ae6e Add PowerPC support in createhdds script and json
Assumption createhdds executed on a PowerPC ppc64le host
to create the PowerPC specific images.

Detect current CPU arch of host machine to create virt-install images
only for supported architectures. (hardcoded lists)

hdds.json specific changes for PowerPC
* no desktop or kde images

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-08-10 07:48:14 -04:00
Adam Williamson
658e082418 Allow 'stable' and 'current' as release for virt-install images
'stable' means 'all current stable releases', and is used for
desktop and server as we need to ensure we have those images
available for all stable releases (including the '-3' release
while it's not EOL) for update testing. (Currently, F24 update
tests are all failing as the images are missing).

'current' means 'the current stable release' - it's the same as
'-1', but just easier to understand. It's used for support.
2017-07-27 12:56:26 -07:00
Adam Williamson
6d997b2f80 Different fix for F26 Workstation: use Everything tree
It seems like the installer images (in os/images) for the F26
Workstation tree somehow come from the OStree installer compose
rather than the network installer compose; install.img and
boot.iso are far larger than they should be, and match the size
of the OStree installer .iso . So instead of using those images
and bumping up the memory size to 4GiB, use the Everything tree
for F26 Workstation image builds and go back to 2GiB.
2017-07-20 11:56:25 -07:00
Adam Williamson
f476e00bd6 We need 4GiB of RAM in virt-install, for F26, it seems
Running with 2GiB causes the Workstation image builds to fail.
2017-07-19 15:18:54 -07:00
Adam Williamson
d1a2869cc0 Correct fedora-secondary path 2017-07-19 13:10:04 -07:00
Adam Williamson
2f55cbede3 Handle i686 install tree location moving in F26 2017-07-19 13:08:31 -07:00
Adam Williamson
e1254ba397 Remove #1439388 workaround
The updates.img is broken with current F26, and Radek claims the
bug is fixed. Let's see. Unfortunately I don't remember what I
put in the updates.img, so if I have to recreate it things could
get fun...
2017-07-19 11:22:15 -07:00
Adam Williamson
bcc1f265e3 Workaround RHBZ #1439388 for F26 (with an updates.img)
This bug is causing havoc with image creation on openqa01, so
use an updates.img for F26 image builds until it's fixed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-04-07 20:14:33 -07:00
Adam Williamson
2d09aa6ef5 Fix recursive create() call on timeout
It wasn't passing textinst, so it'd crash on the first retry.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-04-07 10:55:20 -07:00
Adam Williamson
dfc1c216c0 We need *Server* for Branched update testing, not minimal
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-03-08 23:11:15 -08:00
Adam Williamson
9fe9b2281f Fix (hopefully) Branched base image location detection
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-03-06 17:18:16 -08:00
Adam Williamson
2940891d8f Allow logged text installs, Branched and Rawhide
This is an attempt to add features desirable for creating
Taskotron base images. It extends the 'release' handling for
virt-install images in several ways to allow requesting of
'branched' and 'rawhide' image creation. It also adds an arg
to request virt-install image creation run in text mode, not
graphical mode. Graphical mode is what we always want for
openQA (so the installed OS doesn't have kernel params intended
for serial console interaction), but for Taskotron purposes,
we want the install run in text mode.

This also adds 'branched' to the default JSON file for minimal
and desktop, as we will want branched versions of these images
for the critpath update testing workflow to work on Branched
after Bodhi activation.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-02-28 15:47:53 -08:00
Adam Williamson
80297cbc10 Rename README now project is split out 2017-02-14 15:30:05 -08:00
Adam Williamson
7a206b473c createhdds: use user-mode networking for now
There's been an annoying bug on the production openQA server
for a while now: createhdds runs to refresh the upgrade base
images don't actually work most of the time, each attempt to
run virt-install fails. I finally got time to debug this a bit
today and it seems to be some kind of network issue: the VM has
no network access so the install doesn't work. I've no idea why
that's happening, but using user-mode networking seems to work
around it for now and shouldn't have any terrible side effects
even for openQA servers not affected by this problem. So let's
do this until we can pin down the real bug.
2016-10-21 16:54:03 -07:00
Adam Williamson
359da3d896 createhdds: exit 2 if images missing, 1 if outdated
This will help us fine-tune the behaviour of the ansible plays.
2016-10-17 11:26:18 -07:00
Adam Williamson
8c0493ac35 keep upload files locally, activate network in kickstarts
Summary:
I've come to dislike the approach where we source files that
get included in the disk images remotely; it's unnecessarily
complicated and needlessly exposes us to unexpected changes in
the remote files. So simply keep the files in this git repo
instead, none of them is very big. This also simplifies the
code.

Also, change all the kickstarts to specify --device=link and
--activate on their network lines. These arguments *should*
be included, according to the documentation (the first to
specify which device the config is for, the second to specify
that it should be activated in the installer environment), and
I think the lack of the second is actually now a problem for
the FreeIPA kickstart enrolment test (not sure exactly what
changed to make this a problem where it wasn't before, but
*something* has, and this fixes it).

Test Plan:
check that all tests still run properly. The
FreeIPA enrolment kickstart test should now get somewhat
further. Before this change, for F25 and Rawhide, it fails to
enrol at all during install. Now it should enrol properly -
that's what the kickstart changes fix, it was failing because
it wasn't using the right network config - but it still fails
when trying to log in as test1, due to RHBZ #1366403 .

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D968
2016-08-15 17:02:05 -07:00
Adam Williamson
fe8a8e6665 disk images and reports for encrypted desktop upgrade tests
Summary:
Encrypted desktop upgrade tests are being added to openqa_fedora,
this generates the necessary disk image for it and handles
result reporting.

Test Plan:
Check the tests run with the generated disk images,
check result reporting generates appropriate ResTups.

Reviewers: garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D923
2016-07-08 08:58:23 -07:00
Adam Williamson
602d074708 switch from virt-builder to virt-install (T813)
Summary:
We've kinda been having too much trouble with virt-builder
lately, mainly SELinux related issues due to how it does image
customization. It also produces images that differ in notable
ways from what a 'typical' install would give. virt-install
solves both these problems, and also gives us more flexibility
for storage configuration and post-install customization should
we need them in future.

The change isn't really too drastic, and the design is similar:
instead of virt-builder commands files, each image type now has
a kickstart file where all its customizations can be done.
There's also a single extra image dict key, 'variant', which
specifies which install tree variant to use for running the
install. It defaults to 'Everything' (for F24+) and 'Server'
(for <F24, as Everything wasn't installable until F24) but we
set it to 'Server' for the server images and 'Workstation' for
the desktop images, so those installs will use the correct
variant install class.

We run the installs in VNC. You can do it with a serial console
and log the output, but then anaconda gets clever and changes
several things in the installed system based on the fact that
you did the install over a serial console: it twiddles with
the kernel args and doesn't set graphical.target as the default.
We don't want any of that mess, so we do a VNC install.

The 'size' value is just a number of gigabytes for virt-install
images (as that's how the virt-install 'size' argument works).

This also drops some unused 32-bit images (we don't do 32-bit
KDE or Server upgrade tests, so there's no need to build those
images).

Test Plan:
Re-generate all affected images and re-run all tests
that use them, make sure they work. I am doing this on staging
at present. Note: this would render D911 unnecessary.

Reviewers: garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D917
2016-07-04 09:29:25 -07:00
Adam Williamson
2f80cd8067 work around RHBZ #1349743 when generating KDE disk image 2016-06-24 00:21:35 -07:00
Adam Williamson
efd98f218d deal with #1349586 and #1349664 when building desktop images
A couple of bugs showed up with the graphical desktop images
(Workstation and KDE) for Fedora 24. Setting graphical.target
as the default by doing the symlink during image build, in the
virt-builder appliance, leaves it incorrectly labelled; in F24
this seems to stop systemd from reading it, so it falls back
to rescue.target when createhdds boots the image to try and
get the selinux autorelabel done, and relabelling never happens.
So instead, we change the default target with a firstboot
command (which will get run when createhdds boots to do the
relabel, so by the time openQA boots the image, the target will
be changed).

Also, a tricky bug in fedora-release has the ultimate effect
that if you start with a minimal install then install a desktop
environment on top - like we effectively do for these images -
the login manager service for the desktop does not get enabled,
as it should. So we work around that by explicitly enabling the
appropriate service with (again) a firstboot command.

Pushing without review as garretraziel is out this week and we
need these fixes - without them disk image generation in prod
is broken, which breaks quite a lot of tests.
2016-06-23 16:49:40 -07:00
Adam Williamson
f925eaa63d install nfs-utils and dnsmasq in support server, report results
Summary:
For NFS tests and to set up the support server to do DHCP and
DNS. Also add result reporting for the new NFS tests.

Test Plan:
Check the packages show up in the support server,
run the new tests (see openqa_fedora diff), generate ResTups
with the CLI tool, check they look OK.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D887
2016-06-13 08:42:52 -07:00
Adam Williamson
5bd815bf41 createhdds: add a disk image for use by a 'support server'
Summary:
this will be used for tests like iSCSI and NFS which need a
server end, but where (unlike e.g. FreeIPA) we don't want set
up of the server end to be considered a test and run on the
compose being tested, but instead we just consider the server
end a 'test asset' and expect it to be as reliable as possible.

For now we just set it up to provide an iSCSI target, as that's
what I want to write a test for first.

Note: I was initially writing the actual iSCSI target config
file as part of image creation, but that turns out to be a bit
tricky because we can't safely write a relative or absolute path
for the source file into the commands file. We'd have to instead
allow specifying file injections in the JSON, similar to how we
do for guestfs images, then construct an absolute path in the
Python code using SCRIPTDIR, but it was easier to just write the
file in the openQA test instead, so I did that.

Test Plan:
check that creating the support image works (and if
you're being really thorough, boot it and check the package is
there).

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D883
2016-06-08 15:45:54 -07:00
Adam Williamson
a67ee015ea bump the selinux relabel boot timeout
from 5 mins to 10 mins...we seem to hit it far too often, I
guess initial relabel takes more than 5 mins.
2016-06-07 13:27:30 -07:00
Adam Williamson
6ee0e2217d createhdds: use a temp file for virt-builder image creation
Summary:
Creating the virt-builder images directly with their final file
name means that if we're rebuilding them for age and the build
fails, we'll lose the existing image: it seems better to keep
it than have no image at all, when this happens. It also means
that while the rebuild is in process, the file might exist but
be useless and cause any tests that happen to be running at the
time to fail. So just like the guestfs images, create the file
with a .tmp extension initially and rename it after a successful
build.

Test Plan:
Do some image builds and check they work, check that
temp file is cleaned up on failure and ctrl-c if you can...

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D858
2016-05-23 06:55:17 -07:00
Adam Williamson
fc57ccdecc disable initial-setup in the KDE upgrade disk images
we don't want it (a realistic upgrade scenario would not involve
initial-setup getting in the way) and it seems to mess up boot
sometimes (i-s crashes on start if the system wasn't installed
via anaconda, and that seems to be causing the failed boots
sometimes).
2016-05-06 12:43:33 -07:00
Adam Williamson
a1bef836d7 add desktop terminal test reporting, change 'ejohn' to 'test'
Summary:
This goes with D839. It makes the necessary changes to report
the results for the desktop-terminal tests, and changes the
username in the upgrade test disk images from 'ejohn' to 'test'
to be in line with the name used in all the other tests, and
save us having to change the 'user_logged_in' needles.

Test Plan:
Run with D839, check that all tests work and results
are correctly sent to the wiki.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D840
2016-05-05 16:43:10 -07:00
Adam Williamson
731cd6e69c add FreeIPA test disks (with kickstarts) and update InstallGuide
Summary:
This goes along with the openqa_fedora commit to add the tests.
I didn't update the Docker instructions yet because I don't
quite remember how that goes. It might need a whole different
setup using some other networking...thing...

Test Plan: See https://phab.qadevel.cloud.fedoraproject.org/D831

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D832
2016-05-04 11:55:22 -07:00