560916cd83
PR #1789 improved various aspects of the ostree_container phase regarding subvariant handling and filenames, this is mainly to help us with how we want to handle bootc images (currently in the IoT compose, but the generic base bootc image may move to the Fedora compose). PR #1790 rejigs the compose phase handling so the main image build phase is not unnecessarily blocked on the ostree_install phase. This should cut 60-90 minutes out of the main Fedora compose time. (cherry picked from commit 13884fef2c199b613442af03c24b737a7e3cb057)
83 lines
3.0 KiB
Diff
83 lines
3.0 KiB
Diff
From b83ac496b5a20bd0493328999fc619711c6a512a Mon Sep 17 00:00:00 2001
|
|
From: Adam Williamson <awilliam@redhat.com>
|
|
Date: Oct 11 2024 18:18:48 +0000
|
|
Subject: Don't block main image build phase on ostree_install phase
|
|
|
|
|
|
I did a time map of a Fedora compose today, and noticed that we
|
|
spend about an hour waiting for the ostree_install phase to
|
|
complete before we start up the compose_images_phase which does
|
|
all the other image builds.
|
|
|
|
This is unnecessary. Nothing else depends on ostree_install; it
|
|
should be fine to start up the extra_phase (which contains
|
|
compose_images_phase) while the ostree stuff is still running.
|
|
|
|
This implements that by splitting the ostree phases out of the
|
|
essentials_phase which contains the real precursors to the
|
|
extra_phase. We start the essentials and ostree phases together,
|
|
but only wait for the essentials phase to complete before
|
|
kicking off extra_phase, so it can start while the ostree
|
|
phase is still running.
|
|
|
|
One tweak we have to make to accommodate this is to move
|
|
image_checksum_phase out of extra_phase, to avoid it potentially
|
|
running before all ostree installer images are built. The
|
|
checksum phase is quite fast - it takes about five minutes -
|
|
and any time benefit of running it in parallel with the osbs and
|
|
repoclosure phases seems like it must be smaller than the time
|
|
loss of waiting for ostree_install before kicking off extra.
|
|
|
|
Signed-off-by: Adam Williamson <awilliam@redhat.com>
|
|
|
|
---
|
|
|
|
diff --git a/pungi/scripts/pungi_koji.py b/pungi/scripts/pungi_koji.py
|
|
index c643d53..b2382b0 100644
|
|
--- a/pungi/scripts/pungi_koji.py
|
|
+++ b/pungi/scripts/pungi_koji.py
|
|
@@ -476,11 +476,15 @@ def run_compose(
|
|
buildinstall_phase,
|
|
(gather_phase, createrepo_phase),
|
|
extrafiles_phase,
|
|
+ )
|
|
+ ostree_schema = (
|
|
(ostree_phase, ostree_installer_phase),
|
|
ostree_container_phase,
|
|
)
|
|
essentials_phase = pungi.phases.WeaverPhase(compose, essentials_schema)
|
|
+ ostree_phase = pungi.phases.WeaverPhase(compose, ostree_schema)
|
|
essentials_phase.start()
|
|
+ ostree_phase.start()
|
|
essentials_phase.stop()
|
|
|
|
# write treeinfo before ISOs are created
|
|
@@ -507,12 +511,9 @@ def run_compose(
|
|
osbuild_phase,
|
|
kiwibuild_phase,
|
|
)
|
|
- post_image_phase = pungi.phases.WeaverPhase(
|
|
- compose, (image_checksum_phase, image_container_phase)
|
|
- )
|
|
compose_images_phase = pungi.phases.WeaverPhase(compose, compose_images_schema)
|
|
extra_phase_schema = (
|
|
- (compose_images_phase, post_image_phase),
|
|
+ (compose_images_phase, image_container_phase),
|
|
osbs_phase,
|
|
repoclosure_phase,
|
|
)
|
|
@@ -520,6 +521,12 @@ def run_compose(
|
|
|
|
extra_phase.start()
|
|
extra_phase.stop()
|
|
+ # wait for ostree phase here too - it can happily run in parallel
|
|
+ # with all of the other stuff
|
|
+ ostree_phase.stop()
|
|
+ # now we do checksums as all images are done
|
|
+ image_checksum_phase.start()
|
|
+ image_checksum_phase.stop()
|
|
|
|
pungi.metadata.write_compose_info(compose)
|
|
if not (
|
|
|