rpms/leapp-repository
5
0
Fork 1
Code Issues Pull Requests Releases Activity
d9029cec24
leapp-repository/0053-check_rhui-allow-minor-versions-in-the-RHUI-table.patch
Toshio Kuratomi d9029cec24 CTC2 candidate 1 (Release for 8.10/9.5) 
- Improve set_systemd_services_states logging
- [IPU 7 -> 8] Fix detection of bootable device on RAID
- Fix detection of valid sshd config with internal-sftp subsystem in Leapp
- Handle a false positive GPG check error when TargetUserSpaceInfo is missing
- Fix failing "update-ca-trust" command caused by missing util-linux package
- Improve report when a system is unsupported
- Fix handling of versions in RHUI configuration for ELS and SAP upgrades
- Add missing RHUI GCP config info for RHEL for SAP

- Resolves: RHEL-33902, RHEL-30573, RHEL-43978, RHEL-39046, RHEL-39047, RHEL-39049
2024-07-25 00:55:43 -07:00

410 lines
21 KiB
Diff
Raw Blame History

From 986fa6f90f68a7d3e5d9173d8496dd222d5f7884 Mon Sep 17 00:00:00 2001
From: mhecko <mhecko@redhat.com>
Date: Thu, 23 May 2024 15:28:12 +0200
Subject: [PATCH 53/92] check_rhui: allow minor versions in the RHUI table
With RHEL8.10, different target RHUI clients have to be installed,
depending on the target minor version. The RHUI_SETUPS structure
used for navigating upgrades on RHUI and associated infrastructure
did not allow to setups based on their minor version. RHUI_SETUPS
is used to both determine what RHUI variant are we running on and what
is the target setup. Therefore, any solution would have to:
- ensure that when we are on RHEL8.10, we will look for correct client
(this point is fairly easy, and it would be sufficient to just declare
another setup description in RHUI_SETUPS)
- determine what target client to install depending on whether we are
targetting RHEL8.10 or, e.g., RHEL8.8
Introducing another setup to the map causes multiple target variants
to be eligible, and, thus, leapp would error due to not being able to
pick the target setup. Therefore, a solution has to include such
conflict resolution based on target minor version.
Given the above description of the solution space, this patch allows
specifying minor versions in the RHUI_SETUPS table used to resolve
conflicts when multiple target setups matching the major version are
eligable. Identification of the source and corresponding target RHUI
variant works by using the entry with the closest (younger) minor
version. For example, when 8.4 and 8.8 are suitable target setups (based
on the installed clients), 8.4 will be used when the upgrade taget is
8.6. The minor versions in the RHUI_SETUPS table, therefore, are
in a sense a "changelog" of leapp's RHUI knowledge. If the table
contains an entry A with version 8.0, and an entry B with version
8.8, this can be understood as:
Since 8.0, we know that the setups use this client and we need to do
these things to access target content. Since 8.8 onwards, we know
that we need to use a different client etc.
---
.../libraries/checkhybridimage.py | 3 +-
.../cloud/checkrhui/libraries/checkrhui.py | 165 ++++++++++++++----
.../tests/component_test_checkrhui.py | 48 +++++
repos/system_upgrade/common/libraries/rhui.py | 44 ++++-
4 files changed, 222 insertions(+), 38 deletions(-)
diff --git a/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py b/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
index e2b7f5b2..a4eb6fa1 100644
--- a/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
+++ b/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
@@ -28,7 +28,8 @@ def is_azure_agent_installed():
agent_pkg = None
for setup in azure_setups:
- if setup.os_version == src_ver_major:
+ setup_major_ver = str(setup.os_version[0])
+ if setup_major_ver == src_ver_major:
agent_pkg = setup.extra_info.get('agent_pkg')
break
diff --git a/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py b/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
index e1c158c7..3b217917 100644
--- a/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
+++ b/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
@@ -30,8 +30,98 @@ def into_set(pkgs):
return set(pkgs)
+def fmt_matching_rhui_setups(setups):
+ def fmt_matching_rhui_setup(matching_setup):
+ if isinstance(matching_setup, MatchingSetup):
+ return '(ver={os_ver}, variant={variant}, clients={clients})'.format(
+ os_ver=matching_setup.description.os_version,
+ variant=matching_setup.family,
+ clients=matching_setup.description.clients
+ )
+ # Just a RHUISetup
+ return '(ver={os_ver}, clients={clients})'.format(
+ os_ver=matching_setup.os_version,
+ clients=matching_setup.clients
+ )
+
+ return ', '.join(fmt_matching_rhui_setup(setup) for setup in setups)
+
+
+def select_chronologically_closest_setups(matching_setups, optimal_minor_ver, minor_ver_extractor, system_role):
+ if not matching_setups:
+ return None
+
+ # Select only setups that are chronologically closest
+ highest_minor_less_than_optimal = 0
+ for setup in matching_setups:
+ setup_minor = minor_ver_extractor(setup)
+
+ less_than_src_minor = (setup_minor <= optimal_minor_ver) if optimal_minor_ver else True
+ higher_than_previous = setup_minor > highest_minor_less_than_optimal
+ if less_than_src_minor and higher_than_previous:
+ highest_minor_less_than_optimal = setup_minor
+
+ msg = 'RHUI setups matching installed clients and %s major version: %s'
+ api.current_logger().debug(msg, system_role, fmt_matching_rhui_setups(matching_setups))
+
+ chronologically_closest_setups = [
+ setup for setup in matching_setups if minor_ver_extractor(setup) == highest_minor_less_than_optimal
+ ]
+ if chronologically_closest_setups:
+ matching_setups = chronologically_closest_setups
+ msg = 'Further narrowed matching setups based on their %s minor version: %s'
+ api.current_logger().debug(msg, system_role, fmt_matching_rhui_setups(matching_setups))
+ else:
+ newest_minor = max(matching_setups, key=minor_ver_extractor).os_version[1]
+ matching_setups = [setup for setup in matching_setups if minor_ver_extractor(setup) == newest_minor]
+ api.current_logger().warning(
+ 'The %s predates any of the setups that match the installed clients. Using newest matching: %s',
+ system_role,
+ fmt_matching_rhui_setups(matching_setups)
+ )
+ return matching_setups
+
+
+def error_due_to_ambiguous_source_setups(match0, match1):
+ msg = 'Could not identify the source RHUI setup (ambiguous setup)'
+
+ variant_detail_table = {
+ rhui.RHUIVariant.ORDINARY: '',
+ rhui.RHUIVariant.SAP: ' for SAP',
+ rhui.RHUIVariant.SAP_APPS: ' for SAP Applications',
+ rhui.RHUIVariant.SAP_HA: ' for SAP HA',
+ }
+
+ variant0_detail = variant_detail_table[match0.family.variant]
+ clients0 = ' '.join(match0.description.clients)
+
+ variant1_detail = variant_detail_table[match1.family.variant]
+ clients1 = ' '.join(match1.description.clients)
+
+ details = ('Leapp uses client-based identification of the used RHUI setup in order to determine what the '
+ 'target RHEL content should be. According to the installed RHUI clients the system should be '
+ 'RHEL {os_major}{variant0_detail} ({provider0}) (identified by clients {clients0}) but also '
+ 'RHEL {os_major}{variant1_detail} ({provider1}) (identified by clients {clients1}).')
+ details = details.format(os_major=version.get_source_major_version(),
+ variant0_detail=variant0_detail, clients0=clients0, provider0=match0.family.provider,
+ variant1_detail=variant1_detail, clients1=clients1, provider1=match1.family.provider)
+
+ raise StopActorExecutionError(message=msg, details={'details': details})
+
+
+def _get_canonical_version_tuple(version):
+ ver_fragments = version.split('.')
+ major = int(ver_fragments[0])
+ try:
+ minor = int(ver_fragments[1]) if len(ver_fragments) > 1 else None
+ except ValueError as error:
+ api.current_logger().debug('Failed to convert minor version into integer: %s', error)
+ minor = None # Unlikely, the code using this can handle None as minor
+ return (major, minor)
+
+
def find_rhui_setup_matching_src_system(installed_pkgs, rhui_map):
- src_ver = version.get_source_major_version()
+ src_major_ver, src_minor_ver = _get_canonical_version_tuple(version.get_source_version())
arch = api.current_actor().configuration.architecture
matching_setups = []
@@ -40,8 +130,9 @@ def find_rhui_setup_matching_src_system(installed_pkgs, rhui_map):
continue
for setup in family_setups:
- if setup.os_version != src_ver:
+ if setup.os_version[0] != src_major_ver:
continue
+
if setup.clients.issubset(installed_pkgs):
matching_setups.append(MatchingSetup(family=rhui_family, description=setup))
@@ -50,50 +141,54 @@ def find_rhui_setup_matching_src_system(installed_pkgs, rhui_map):
# In case that a RHUI variant uses a combination of clients identify the maximal client set
matching_setups_by_size = sorted(matching_setups, key=lambda match: -len(match.description.clients))
+ max_client_cnt = len(matching_setups_by_size[0].description.clients)
+ matching_setups = tuple(
+ setup for setup in matching_setups if len(setup.description.clients) == max_client_cnt
+ )
+ msg = 'Identified RHUI setups with the largest installed client sets: %s'
+ api.current_logger().debug(msg, fmt_matching_rhui_setups(matching_setups))
- match = matching_setups_by_size[0] # Matching setup with the highest number of clients
- if len(matching_setups) == 1:
- return match
+ if not matching_setups:
+ return None
- if len(matching_setups_by_size[0].description.clients) == len(matching_setups_by_size[1].description.clients):
- # Should not happen as no cloud providers use multi-client setups (at the moment)
- msg = 'Could not identify the source RHUI setup (ambiguous setup)'
+ # Since we allow minor versions in RHUI table, we might have multiple entries that are identified by the
+ # same clients. E.g.:
+ # RHEL8.4 with client X
+ # RHEL8.9 with client X (but with some modified setup info)
+ # If upgrading from 8.6, select 8.4. If upgrading from 8.10, select 8.9
+ matching_setups = select_chronologically_closest_setups(matching_setups,
+ src_minor_ver,
+ lambda setup: setup.description.os_version[1],
+ 'source')
- variant_detail_table = {
- rhui.RHUIVariant.ORDINARY: '',
- rhui.RHUIVariant.SAP: ' for SAP',
- rhui.RHUIVariant.SAP_APPS: ' for SAP Applications',
- rhui.RHUIVariant.SAP_HA: ' for SAP HA',
- }
+ # If we fail to identify chronologically proper setup, we always return a nonempty list
- match0 = matching_setups_by_size[0]
- variant0_detail = variant_detail_table[match0.family.variant]
- clients0 = ' '.join(match0.description.clients)
+ match = matching_setups[0] # Matching setup with the highest number of clients
+ if len(matching_setups) == 1:
+ return match
- match1 = matching_setups_by_size[1]
- variant1_detail = variant_detail_table[match1.family.variant]
- clients1 = ' '.join(match1.description.clients)
+ other_match = matching_setups[1]
+ error_due_to_ambiguous_source_setups(match, other_match)
+ return None # Unreachable
- details = ('Leapp uses client-based identification of the used RHUI setup in order to determine what the '
- 'target RHEL content should be. According to the installed RHUI clients the system should be '
- 'RHEL {os_major}{variant0_detail} ({provider0}) (identified by clients {clients0}) but also '
- 'RHEL {os_major}{variant1_detail} ({provider1}) (identified by clients {clients1}).')
- details = details.format(os_major=version.get_source_major_version(),
- variant0_detail=variant0_detail, clients0=clients0, provider0=match0.family.provider,
- variant1_detail=variant1_detail, clients1=clients1, provider1=match1.family.provider)
- raise StopActorExecutionError(message=msg, details={'details': details})
+def determine_target_setup_desc(cloud_map, rhui_family):
+ variant_setups = cloud_map[rhui_family]
- return match
+ target_major, target_minor = _get_canonical_version_tuple(version.get_target_version())
+ matching_setups = [setup for setup in variant_setups if setup.os_version[0] == target_major]
+ msg = 'Identified target RHUI setups matching target major: %s'
+ api.current_logger().debug(msg, fmt_matching_rhui_setups(matching_setups))
-def determine_target_setup_desc(cloud_map, rhui_family):
- variant_setups = cloud_map[rhui_family]
- target_major = version.get_target_major_version()
+ matching_setups = select_chronologically_closest_setups(matching_setups,
+ target_minor,
+ lambda setup: setup.os_version[1],
+ 'target')
+
+ if matching_setups:
+ return next(iter(matching_setups))
- for setup in variant_setups:
- if setup.os_version == target_major:
- return setup
return None
diff --git a/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py b/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
index aa0089b6..27e70eea 100644
--- a/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
+++ b/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
@@ -326,3 +326,51 @@ def test_unknown_target_rhui_setup(monkeypatch, is_target_setup_known):
else:
with pytest.raises(StopActorExecutionError):
checkrhui_lib.process()
+
+
+@pytest.mark.parametrize(
+ ('setups', 'desired_minor', 'expected_setup'),
+ [
+ (
+ [
+ mk_rhui_setup(clients={'A'}, os_version='8.4', leapp_pkg='leapp-A'),
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ],
+ 8,
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ),
+ (
+ [
+ mk_rhui_setup(clients={'A'}, os_version='8.4', leapp_pkg='leapp-A'),
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ],
+ 6,
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ),
+ (
+ [
+ mk_rhui_setup(clients={'A'}, os_version='8.4', leapp_pkg='leapp-A'),
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ],
+ 5,
+ mk_rhui_setup(clients={'A'}, os_version='8.4', leapp_pkg='leapp-A'),
+ ),
+ (
+ [
+ mk_rhui_setup(clients={'A'}, os_version='8.4', leapp_pkg='leapp-A'),
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ ],
+ 3,
+ mk_rhui_setup(clients={'A'}, os_version='8.6', leapp_pkg='leapp-B'),
+ )
+ ]
+)
+def test_select_chronologically_closest(monkeypatch, setups, desired_minor, expected_setup):
+ setups = checkrhui_lib.select_chronologically_closest_setups(setups,
+ desired_minor,
+ lambda setup: setup.os_version[1],
+ 'source')
+ assert len(setups) == 1
+ setup = setups[0]
+
+ assert setup == expected_setup
diff --git a/repos/system_upgrade/common/libraries/rhui.py b/repos/system_upgrade/common/libraries/rhui.py
index ab6a27f8..a0669ec4 100644
--- a/repos/system_upgrade/common/libraries/rhui.py
+++ b/repos/system_upgrade/common/libraries/rhui.py
@@ -101,8 +101,15 @@ class RHUIFamily(object):
def mk_rhui_setup(clients=None, leapp_pkg='', mandatory_files=None, optional_files=None,
- extra_info=None, os_version='7', arch=arch.ARCH_X86_64, content_channel=ContentChannel.GA,
+ extra_info=None, os_version='7.0', arch=arch.ARCH_X86_64, content_channel=ContentChannel.GA,
files_supporting_client_operation=None):
+
+ os_version_fragments = os_version.split('.')
+ if len(os_version_fragments) == 1:
+ os_version_tuple = (int(os_version), 0)
+ else:
+ os_version_tuple = (int(os_version_fragments[0]), int(os_version_fragments[1]))
+
clients = clients or set()
mandatory_files = mandatory_files or []
extra_info = extra_info or {}
@@ -115,7 +122,7 @@ def mk_rhui_setup(clients=None, leapp_pkg='', mandatory_files=None, optional_fil
return RHUISetup(clients=clients, leapp_pkg=leapp_pkg, mandatory_files=mandatory_files, arch=arch,
content_channel=content_channel, optional_files=optional_files, extra_info=extra_info,
- os_version=os_version, files_supporting_client_operation=files_supporting_client_operation)
+ os_version=os_version_tuple, files_supporting_client_operation=files_supporting_client_operation)
# This will be the new "cloud map". Essentially a directed graph with edges defined implicitly by OS versions +
@@ -186,6 +193,8 @@ RHUI_SETUPS = {
mandatory_files=[
('rhui-client-config-server-8-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
('rhui-client-config-server-8-sap-bundle.key', RHUI_PKI_DIR),
+ ('content-rhel8-sap-bundle-e4s.crt', RHUI_PKI_PRODUCT_DIR),
+ ('content-rhel8-sap-bundle-e4s.key', RHUI_PKI_DIR),
(AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
('leapp-aws-sap-e4s.repo', YUM_REPOS_PATH)
],
@@ -195,6 +204,21 @@ RHUI_SETUPS = {
('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
('content-rhel8-sap.crt', RHUI_PKI_PRODUCT_DIR)
], os_version='8', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-sap-bundle'}, leapp_pkg='leapp-rhui-aws-sap-e4s',
+ mandatory_files=[
+ ('rhui-client-config-server-8-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-8-sap-bundle.key', RHUI_PKI_DIR),
+ ('content-rhel8-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
+ ('content-rhel8-sap-bundle.key', RHUI_PKI_DIR),
+ (AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
+ ('leapp-aws-sap.repo', YUM_REPOS_PATH)
+ ],
+ files_supporting_client_operation=[AWS_DNF_PLUGIN_NAME],
+ optional_files=[
+ ('content-rhel8-sap.key', RHUI_PKI_DIR),
+ ('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
+ ('content-rhel8-sap.crt', RHUI_PKI_PRODUCT_DIR)
+ ], os_version='8.10'),
mk_rhui_setup(clients={'rh-amazon-rhui-client-sap-bundle-e4s'}, leapp_pkg='leapp-rhui-aws-sap-e4s',
mandatory_files=[
('rhui-client-config-server-9-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
@@ -237,6 +261,14 @@ RHUI_SETUPS = {
],
extra_info={'agent_pkg': 'WALinuxAgent'},
os_version='8', content_channel=ContentChannel.EUS),
+ mk_rhui_setup(clients={'rhui-azure-rhel8-base-sap-apps'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-apps.repo', YUM_REPOS_PATH)],
+ optional_files=[
+ ('key-sapapps.pem', RHUI_PKI_DIR),
+ ('content-sapapps.crt', RHUI_PKI_PRODUCT_DIR)
+ ],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='8.10', content_channel=ContentChannel.GA),
mk_rhui_setup(clients={'rhui-azure-rhel9-sapapps'}, leapp_pkg='leapp-rhui-azure-sap',
mandatory_files=[('leapp-azure-sap-apps.repo', YUM_REPOS_PATH)],
optional_files=[
@@ -256,6 +288,14 @@ RHUI_SETUPS = {
],
extra_info={'agent_pkg': 'WALinuxAgent'},
os_version='8', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rhui-azure-rhel8-base-sap-ha'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)],
+ optional_files=[
+ ('key-sap-ha.pem', RHUI_PKI_DIR),
+ ('content-sap-ha.crt', RHUI_PKI_PRODUCT_DIR)
+ ],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='8.10'),
mk_rhui_setup(clients={'rhui-azure-rhel9-sap-ha'}, leapp_pkg='leapp-rhui-azure-sap',
mandatory_files=[('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)],
optional_files=[
--
2.42.0
Reference in New Issue View Git Blame Copy Permalink