ykohut/leapp-repository
1
0
Fork 0
forked from rpms/leapp-repository
Code Pull Requests Activity
48c8fb0fa7
leapp-repository/0026-rhui-bootstrap-target-rhui-clients-in-scratch-contai.patch
Petr Stodulka 75c9028095 RHEL 8.10: CTC1 candidate 
- Enable new upgrade path for RHEL 8.10 -> RHEL 9.4 (including RHEL with SAP HANA)
- Introduce generic transition of systemd services states during the IPU
- Introduce possibility to upgrade with local repositories
- Improve possibilities of upgrade when a proxy is configured in DNF configutation file
- Fix handling of symlinks under /etc/pki when managing certificates
- Fix the upgrade with custom https repositories
- Default to the NO_RHSM mode when subscription-manager is not installed
- Detect customized configuration of dynamic linker
- Drop the invalid `tuv` target channel for the --channel option
- Fix the issue of going out of bounds in the isccfg parser
- Fix traceback when saving the rhsm facts results and the /etc/rhsm/facts directory doesn’t exist yet
- Load all rpm repository substitutions that dnf knows about, not just "releasever" only
- Simplify handling of upgrades on systems using RHUI, reducing the maintenance burden for cloud providers
- Detect possible unexpected RPM GPG keys has been installed during RPM transaction
- Resolves: RHEL-16729
2023-11-16 20:15:43 +01:00

1739 lines
82 KiB
Diff
Raw Blame History

From bbed72d18dabb9c47aed4f2e760ee637decc30f1 Mon Sep 17 00:00:00 2001
From: Michal Hecko <mhecko@redhat.com>
Date: Wed, 8 Mar 2023 12:41:03 +0100
Subject: [PATCH 26/38] rhui: bootstrap target rhui clients in scratch
container
In order to upgrade a RHUI system leapp uses custom `leapp-rhui-
X` packages providing leapp with necessary repository definitions as
well as certs and keys to access these repositories. The content of
the `leapp-rhui-X` packages is therefore almost identical to the RHUI
client(s) found on the target systems, implying that leapp's rhui
packages must actively mirror any changes to the target client packages.
This patch modifies leapp so that leapp uses the `leapp-rhui-X` package
only to provide a definion of the repository where a target RHUI client
can be found. The current RHUI client and target RHUI client is then
(bootstrapped) atomically swapped in the scratch container, allowing the
upgrade process to access target content. This change thus minimizes
the effort put into maintaining leapp-rhui-X.
This patch also does redesigns the "cloud map" to contain declarative
descriptions of setups, allowing to produce different the client
bootstrap steps if desired (not implemented). The new map also contains
information about content channel used on known rhui systems, laying the
necessary foundation for better error messages when the user forgets to
run leapp with --channel.
Finally, the RHUI-handling logic has been mostly isolated into a fully
unit-tested actor, whereas the implemented userspacegen modifications
have the nature of somehow blindly following the instructions produced
by the RHUI actor.
Jira: OAMG-8599
---
.../tests/test_checketcreleasever.py | 36 +-
.../libraries/checkhybridimage.py | 17 +-
.../common/actors/cloud/checkrhui/actor.py | 93 +----
.../cloud/checkrhui/libraries/checkrhui.py | 250 +++++++++++++
.../tests/component_test_checkrhui.py | 339 ++++++++++++++++--
.../libraries/pes_events_scanner.py | 5 +-
.../actors/redhatsignedrpmscanner/actor.py | 21 +-
.../tests/test_setetcreleasever.py | 25 +-
.../libraries/setuptargetrepos.py | 7 +-
.../libraries/userspacegen.py | 196 +++++++---
.../tests/unit_test_targetuserspacecreator.py | 7 +-
repos/system_upgrade/common/libraries/rhui.py | 266 ++++++++++++--
.../system_upgrade/common/models/rhuiinfo.py | 52 ++-
13 files changed, 1066 insertions(+), 248 deletions(-)
create mode 100644 repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
diff --git a/repos/system_upgrade/common/actors/checketcreleasever/tests/test_checketcreleasever.py b/repos/system_upgrade/common/actors/checketcreleasever/tests/test_checketcreleasever.py
index 82eb0847..1310ace2 100644
--- a/repos/system_upgrade/common/actors/checketcreleasever/tests/test_checketcreleasever.py
+++ b/repos/system_upgrade/common/actors/checketcreleasever/tests/test_checketcreleasever.py
@@ -4,13 +4,16 @@ import pytest
from leapp import reporting
from leapp.libraries.actor import checketcreleasever
-from leapp.libraries.common.testutils import (
- create_report_mocked,
- CurrentActorMocked,
- logger_mocked
-)
+from leapp.libraries.common.testutils import create_report_mocked, CurrentActorMocked, logger_mocked
from leapp.libraries.stdlib import api
-from leapp.models import PkgManagerInfo, Report, RHUIInfo
+from leapp.models import (
+ PkgManagerInfo,
+ Report,
+ RHUIInfo,
+ TargetRHUIPostInstallTasks,
+ TargetRHUIPreInstallTasks,
+ TargetRHUISetupInfo
+)
@pytest.mark.parametrize('exists', [True, False])
@@ -55,9 +58,24 @@ def test_etc_releasever_empty(monkeypatch):
assert api.current_logger.dbgmsg
+def mk_rhui_info():
+ preinstall_tasks = TargetRHUIPreInstallTasks()
+ postinstall_tasks = TargetRHUIPostInstallTasks()
+ setup_info = TargetRHUISetupInfo(preinstall_tasks=preinstall_tasks, postinstall_tasks=postinstall_tasks)
+ rhui_info = RHUIInfo(provider='aws',
+ src_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_setup_info=setup_info)
+ return rhui_info
+
+
@pytest.mark.parametrize('is_rhui', [True, False])
def test_etc_releasever_rhui(monkeypatch, is_rhui):
- rhui_msg = [RHUIInfo(provider='aws')] if is_rhui else []
+ if is_rhui:
+ rhui_msg = [mk_rhui_info()]
+ else:
+ rhui_msg = []
+
expected_rel_ver = '6.10'
mocked_report = create_report_mocked()
@@ -92,7 +110,9 @@ def test_etc_releasever_neither(monkeypatch):
def test_etc_releasever_both(monkeypatch):
- msgs = [RHUIInfo(provider='aws'), PkgManagerInfo(etc_releasever='7.7')]
+ rhui_info = mk_rhui_info()
+
+ msgs = [rhui_info, PkgManagerInfo(etc_releasever='7.7')]
expected_rel_ver = '6.10'
mocked_report = create_report_mocked()
diff --git a/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py b/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
index e894683b..e2b7f5b2 100644
--- a/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
+++ b/repos/system_upgrade/common/actors/cloud/checkhybridimage/libraries/checkhybridimage.py
@@ -2,6 +2,7 @@ import os
from leapp import reporting
from leapp.libraries.common import rhui
+from leapp.libraries.common.config.version import get_source_major_version
from leapp.libraries.common.rpms import has_package
from leapp.libraries.stdlib import api
from leapp.models import FirmwareFacts, HybridImage, InstalledRPM
@@ -20,8 +21,20 @@ def is_grubenv_symlink_to_efi():
def is_azure_agent_installed():
"""Check whether 'WALinuxAgent' package is installed."""
- upg_path = rhui.get_upg_path()
- agent_pkg = rhui.RHUI_CLOUD_MAP[upg_path].get('azure', {}).get('agent_pkg', '')
+ src_ver_major = get_source_major_version()
+
+ family = rhui.RHUIFamily(rhui.RHUIProvider.AZURE)
+ azure_setups = rhui.RHUI_SETUPS.get(family, [])
+
+ agent_pkg = None
+ for setup in azure_setups:
+ if setup.os_version == src_ver_major:
+ agent_pkg = setup.extra_info.get('agent_pkg')
+ break
+
+ if not agent_pkg:
+ return False
+
return has_package(InstalledRPM, agent_pkg)
diff --git a/repos/system_upgrade/common/actors/cloud/checkrhui/actor.py b/repos/system_upgrade/common/actors/cloud/checkrhui/actor.py
index 9cf69dad..593e73e5 100644
--- a/repos/system_upgrade/common/actors/cloud/checkrhui/actor.py
+++ b/repos/system_upgrade/common/actors/cloud/checkrhui/actor.py
@@ -1,11 +1,5 @@
-import os
-
-from leapp import reporting
from leapp.actors import Actor
-from leapp.libraries.common import rhsm, rhui
-from leapp.libraries.common.config.version import get_source_major_version
-from leapp.libraries.common.rpms import has_package
-from leapp.libraries.stdlib import api
+from leapp.libraries.actor import checkrhui as checkrhui_lib
from leapp.models import (
CopyFile,
DNFPluginTask,
@@ -16,7 +10,7 @@ from leapp.models import (
RpmTransactionTasks,
TargetUserSpacePreupgradeTasks
)
-from leapp.reporting import create_report, Report
+from leapp.reporting import Report
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
@@ -40,85 +34,4 @@ class CheckRHUI(Actor):
tags = (FactsPhaseTag, IPUWorkflowTag)
def process(self):
- upg_path = rhui.get_upg_path()
- for provider, info in rhui.RHUI_CLOUD_MAP[upg_path].items():
- if has_package(InstalledRPM, info['src_pkg']):
- # we need to do this workaround in order to overcome our RHUI handling limitation
- # in case there are more client packages on the source system
- # @Note(mhecko): Azure has changed the structure of their images to not use a pair of RHUI clients and
- # # instead they started to use a single package. However, it could happen that a user
- # # does not run `dnf upgrade` and thus has both packages installed.
- if 'azure' in info['src_pkg']:
- azure_sap_variants = ['azure-sap-ha', 'azure-sap-apps']
- for azure_sap_variant in azure_sap_variants:
- sap_variant_info = rhui.RHUI_CLOUD_MAP[upg_path][azure_sap_variant]
- if has_package(InstalledRPM, sap_variant_info['src_pkg']):
- info = sap_variant_info
- provider = azure_sap_variant
-
- if provider.startswith('google'):
- rhui_dir = api.get_common_folder_path('rhui')
- repofile = os.path.join(rhui_dir, provider, 'leapp-{}.repo'.format(provider))
- api.produce(
- TargetUserSpacePreupgradeTasks(
- copy_files=[CopyFile(src=repofile, dst='/etc/yum.repos.d/leapp-google-copied.repo')]
- )
- )
-
- if not rhsm.skip_rhsm():
- create_report([
- reporting.Title('Upgrade initiated with RHSM on public cloud with RHUI infrastructure'),
- reporting.Summary(
- 'Leapp detected this system is on public cloud with RHUI infrastructure '
- 'but the process was initiated without "--no-rhsm" command line option '
- 'which implies RHSM usage (valid subscription is needed).'
- ),
- reporting.Severity(reporting.Severity.INFO),
- reporting.Groups([reporting.Groups.PUBLIC_CLOUD]),
- ])
- return
-
- # When upgrading with RHUI we cannot switch certs and let RHSM provide us repos for target OS content.
- # Instead, Leapp's provider-specific package containing target OS certs and repos has to be installed.
- if not has_package(InstalledRPM, info['leapp_pkg']):
- create_report([
- reporting.Title('Package "{}" is missing'.format(info['leapp_pkg'])),
- reporting.Summary(
- 'On {} using RHUI infrastructure, a package "{}" is needed for '
- 'in-place upgrade'.format(provider.upper(), info['leapp_pkg'])
- ),
- reporting.Severity(reporting.Severity.HIGH),
- reporting.RelatedResource('package', info['leapp_pkg']),
- reporting.Groups([reporting.Groups.INHIBITOR]),
- reporting.Groups([reporting.Groups.PUBLIC_CLOUD, reporting.Groups.RHUI]),
- reporting.Remediation(commands=[['yum', 'install', '-y', info['leapp_pkg']]])
- ])
- return
-
- # there are several "variants" related to the *AWS* provider (aws, aws-sap)
- if provider.startswith('aws'):
- # We have to disable Amazon-id plugin in the initramdisk phase as the network
- # is down at the time
- self.produce(DNFPluginTask(name='amazon-id', disable_in=['upgrade']))
-
- # If source OS and target OS packages differ we must remove the source pkg, and install the target pkg.
- # If the packages do not differ, it is sufficient to upgrade them during the upgrade
- if info['src_pkg'] != info['target_pkg']:
- self.produce(RpmTransactionTasks(to_install=[info['target_pkg']]))
- self.produce(RpmTransactionTasks(to_remove=[info['src_pkg']]))
-
- # Although SAP systems on Azure should not rely on a pair of RHUI clients, it is still possible
- # that the source system has both clients installed, and it is safer to remove both of them.
- azure_nonsap_pkg = None
- if provider == 'azure-sap-ha':
- azure_nonsap_pkg = rhui.RHUI_CLOUD_MAP[upg_path]['azure']['src_pkg']
- elif provider == 'azure-sap-apps':
- # SAP Apps systems have EUS content channel from RHEL8+
- src_rhel_content_type = 'azure' if get_source_major_version() == '7' else 'azure-eus'
- azure_nonsap_pkg = rhui.RHUI_CLOUD_MAP[upg_path][src_rhel_content_type]['src_pkg']
- if azure_nonsap_pkg and has_package(InstalledRPM, azure_nonsap_pkg):
- self.produce(RpmTransactionTasks(to_remove=[azure_nonsap_pkg]))
-
- self.produce(RHUIInfo(provider=provider))
- self.produce(RequiredTargetUserspacePackages(packages=[info['target_pkg']]))
- return
+ checkrhui_lib.process()
diff --git a/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py b/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
new file mode 100644
index 00000000..84ab40e3
--- /dev/null
+++ b/repos/system_upgrade/common/actors/cloud/checkrhui/libraries/checkrhui.py
@@ -0,0 +1,250 @@
+import itertools
+import os
+from collections import namedtuple
+
+from leapp import reporting
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.common import rhsm, rhui
+from leapp.libraries.common.config import version
+from leapp.libraries.stdlib import api
+from leapp.models import (
+ CopyFile,
+ DNFPluginTask,
+ InstalledRPM,
+ RHUIInfo,
+ RpmTransactionTasks,
+ TargetRHUIPostInstallTasks,
+ TargetRHUIPreInstallTasks,
+ TargetRHUISetupInfo,
+ TargetUserSpacePreupgradeTasks
+)
+
+MatchingSetup = namedtuple('MatchingSetup', ['family', 'description'])
+
+
+def into_set(pkgs):
+ if isinstance(pkgs, set):
+ return pkgs
+ if isinstance(pkgs, str):
+ return {pkgs}
+ return set(pkgs)
+
+
+def find_rhui_setup_matching_src_system(installed_pkgs, rhui_map):
+ src_ver = version.get_source_major_version()
+ arch = api.current_actor().configuration.architecture
+
+ matching_setups = []
+ for rhui_family, family_setups in rhui_map.items():
+ if rhui_family.arch != arch:
+ continue
+
+ for setup in family_setups:
+ if setup.os_version != src_ver:
+ continue
+ if setup.clients.issubset(installed_pkgs):
+ matching_setups.append(MatchingSetup(family=rhui_family, description=setup))
+
+ if not matching_setups:
+ return None
+
+ # In case that a RHUI variant uses a combination of clients identify the maximal client set
+ matching_setups_by_size = sorted(matching_setups, key=lambda match: -len(match.description.clients))
+
+ match = matching_setups_by_size[0] # Matching setup with the highest number of clients
+ if len(matching_setups) == 1:
+ return match
+
+ if len(matching_setups_by_size[0].description.clients) == len(matching_setups_by_size[1].description.clients):
+ # Should not happen as no cloud providers use multi-client setups (at the moment)
+ msg = 'Could not identify the source RHUI setup (ambiguous setup)'
+
+ variant_detail_table = {
+ rhui.RHUIVariant.ORDINARY: '',
+ rhui.RHUIVariant.SAP: ' for SAP',
+ rhui.RHUIVariant.SAP_APPS: ' for SAP Applications',
+ rhui.RHUIVariant.SAP_HA: ' for SAP HA',
+ }
+
+ match0 = matching_setups_by_size[0]
+ variant0_detail = variant_detail_table[match0.family.variant]
+ clients0 = ' '.join(match0.description.clients)
+
+ match1 = matching_setups_by_size[1]
+ variant1_detail = variant_detail_table[match1.family.variant]
+ clients1 = ' '.join(match1.description.clients)
+
+ details = ('Leapp uses client-based identification of the used RHUI setup in order to determine what the '
+ 'target RHEL content should be. According to the installed RHUI clients the system should be '
+ 'RHEL {os_major}{variant0_detail} ({provider0}) (identified by clients {clients0}) but also '
+ 'RHEL {os_major}{variant1_detail} ({provider1}) (identified by clients {clients1}).')
+ details = details.format(os_major=version.get_source_major_version(),
+ variant0_detail=variant0_detail, clients0=clients0, provider0=match0.family.provider,
+ variant1_detail=variant1_detail, clients1=clients1, provider1=match1.family.provider)
+
+ raise StopActorExecutionError(message=msg, details={'details': details})
+
+ return match
+
+
+def determine_target_setup_desc(cloud_map, rhui_family):
+ variant_setups = cloud_map[rhui_family]
+ target_major = version.get_target_major_version()
+
+ for setup in variant_setups:
+ if setup.os_version == target_major:
+ return setup
+ return None
+
+
+def inhibit_if_leapp_pkg_to_access_target_missing(installed_pkgs, rhui_family, target_setup_desc):
+ pkg_name = target_setup_desc.leapp_pkg
+
+ if pkg_name not in installed_pkgs:
+ summary = 'On {provider} the "{pkg}" is required to perform an in-place upgrade'
+ summary = summary.format(provider=rhui_family.provider, pkg=pkg_name)
+ reporting.create_report([
+ reporting.Title('Package "{}" is not installed'.format(pkg_name)),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.RelatedResource('package', pkg_name),
+ reporting.Groups([reporting.Groups.INHIBITOR]),
+ reporting.Groups([reporting.Groups.PUBLIC_CLOUD, reporting.Groups.RHUI]),
+ reporting.Remediation(commands=[['yum', 'install', '-y', pkg_name]])
+ ])
+ return True
+ return False
+
+
+def stop_due_to_unknown_target_system_setup(rhui_family):
+ msg = 'Failed to identify target RHUI setup'
+ variant_detail = ' ({rhui_family.variant})' if rhui_family.variant != rhui.RHUIVariant.ORDINARY else ''
+ details = ('Leapp successfully identified the current RHUI setup as a system provided by '
+ '{provider}{variant_detail}, but it failed to determine'
+ ' equivalent RHUI setup for the target OS.')
+ details = details.format(provider=rhui_family.provider, variant_detail=variant_detail)
+ raise StopActorExecutionError(message=msg, details={'details': details})
+
+
+def customize_rhui_setup_for_gcp(rhui_family, setup_info):
+ if not rhui_family.provider == rhui.RHUIProvider.GOOGLE:
+ return
+
+ # The google-cloud.repo repofile provides the repository containing the target clients. However, its repoid is the
+ # same across all rhel versions, therefore, we need to remove the source google-cloud.repo to enable
+ # correct target one.
+ setup_info.preinstall_tasks.files_to_remove.append('/etc/yum.repos.d/google-cloud.repo')
+
+
+def customize_rhui_setup_for_aws(rhui_family, setup_info):
+ if rhui_family.provider != rhui.RHUIProvider.AWS:
+ return
+
+ target_version = version.get_target_major_version()
+ if target_version == '8':
+ return # The rhel8 plugin is packed into leapp-rhui-aws as we need python2 compatible client
+
+ amazon_plugin_copy_task = CopyFile(src='/usr/lib/python3.9/site-packages/dnf-plugins/amazon-id.py',
+ dst='/usr/lib/python3.6/site-packages/dnf-plugins/')
+ setup_info.postinstall_tasks.files_to_copy.append(amazon_plugin_copy_task)
+
+
+def produce_rhui_info_to_setup_target(rhui_family, source_setup_desc, target_setup_desc):
+ rhui_files_location = os.path.join(api.get_common_folder_path('rhui'), rhui_family.client_files_folder)
+
+ files_to_access_target_client_repo = []
+ for filename, target_path in target_setup_desc.mandatory_files:
+ src_path = os.path.join(rhui_files_location, filename)
+ files_to_access_target_client_repo.append(CopyFile(src=src_path, dst=target_path))
+
+ for filename, target_path in target_setup_desc.optional_files:
+ src_path = os.path.join(rhui_files_location, filename)
+
+ if not os.path.exists(src_path):
+ msg = "Optional file {} is present, will be used to setup target RHUI."
+ api.current_logger().debug(msg.format(src_path))
+ continue
+
+ files_to_access_target_client_repo.append(CopyFile(src=src_path, dst=target_path))
+
+ preinstall_tasks = TargetRHUIPreInstallTasks(files_to_copy_into_overlay=files_to_access_target_client_repo)
+
+ files_supporting_client_operation = sorted(
+ os.path.join(rhui_files_location, file) for file in target_setup_desc.files_supporting_client_operation
+ )
+
+ target_client_setup_info = TargetRHUISetupInfo(
+ preinstall_tasks=preinstall_tasks,
+ postinstall_tasks=TargetRHUIPostInstallTasks(),
+ files_supporting_client_operation=files_supporting_client_operation
+ )
+
+ customize_rhui_setup_for_gcp(rhui_family, target_client_setup_info)
+ customize_rhui_setup_for_aws(rhui_family, target_client_setup_info)
+
+ rhui_info = RHUIInfo(
+ provider=rhui_family.provider.lower(),
+ variant=rhui_family.variant,
+ src_client_pkg_names=sorted(source_setup_desc.clients),
+ target_client_pkg_names=sorted(target_setup_desc.clients),
+ target_client_setup_info=target_client_setup_info
+ )
+ api.produce(rhui_info)
+
+
+def produce_rpms_to_install_into_target(source_setup, target_setup):
+ to_install = sorted(target_setup.clients - source_setup.clients)
+ to_remove = sorted(source_setup.clients - target_setup.clients)
+
+ api.produce(TargetUserSpacePreupgradeTasks(install_rpms=sorted(target_setup.clients)))
+ if to_install or to_remove:
+ api.produce(RpmTransactionTasks(to_install=to_install, to_remove=to_remove))
+
+
+def inform_about_upgrade_with_rhui_without_no_rhsm():
+ if not rhsm.skip_rhsm():
+ reporting.create_report([
+ reporting.Title('Upgrade initiated with RHSM on public cloud with RHUI infrastructure'),
+ reporting.Summary(
+ 'Leapp detected this system is on public cloud with RHUI infrastructure '
+ 'but the process was initiated without "--no-rhsm" command line option '
+ 'which implies RHSM usage (valid subscription is needed).'
+ ),
+ reporting.Severity(reporting.Severity.INFO),
+ reporting.Groups([reporting.Groups.PUBLIC_CLOUD]),
+ ])
+ return True
+ return False
+
+
+def process():
+ installed_rpm = itertools.chain(*[installed_rpm_msg.items for installed_rpm_msg in api.consume(InstalledRPM)])
+ installed_pkgs = {rpm.name for rpm in installed_rpm}
+
+ src_rhui_setup = find_rhui_setup_matching_src_system(installed_pkgs, rhui.RHUI_SETUPS)
+ if not src_rhui_setup:
+ return
+ api.current_logger().debug("The RHUI family of the source system is {}".format(src_rhui_setup.family))
+
+ target_setup_desc = determine_target_setup_desc(rhui.RHUI_SETUPS, src_rhui_setup.family)
+
+ if not target_setup_desc:
+ # We know that we are on RHUI because we have identified what RHUI variant it is, but we don't know how does
+ # the target system look like. Likely, our knowledge of what RHUI setups are there (RHUI_SETUPS) is incomplete.
+ stop_due_to_unknown_target_system_setup(src_rhui_setup.family)
+ return
+
+ if inform_about_upgrade_with_rhui_without_no_rhsm():
+ return
+
+ if inhibit_if_leapp_pkg_to_access_target_missing(installed_pkgs, src_rhui_setup.family, target_setup_desc):
+ return
+
+ # Instruction on how to access the target content
+ produce_rhui_info_to_setup_target(src_rhui_setup.family, src_rhui_setup.description, target_setup_desc)
+
+ produce_rpms_to_install_into_target(src_rhui_setup.description, target_setup_desc)
+
+ if src_rhui_setup.family.provider == rhui.RHUIProvider.AWS:
+ # We have to disable Amazon-id plugin in the initramdisk phase as there is no network
+ api.produce(DNFPluginTask(name='amazon-id', disable_in=['upgrade']))
diff --git a/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py b/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
index fde5ea72..93f13a00 100644
--- a/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
+++ b/repos/system_upgrade/common/actors/cloud/checkrhui/tests/component_test_checkrhui.py
@@ -1,60 +1,329 @@
from collections import namedtuple
+from enum import Enum
import pytest
-from leapp.libraries.common import rhsm
-from leapp.libraries.common.config import mock_configs
+from leapp import reporting
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.actor import checkrhui as checkrhui_lib
+from leapp.libraries.common import rhsm, rhui
+from leapp.libraries.common.config import mock_configs, version
+from leapp.libraries.common.rhui import mk_rhui_setup, RHUIFamily
+from leapp.libraries.common.testutils import create_report_mocked, CurrentActorMocked, produce_mocked
+from leapp.libraries.stdlib import api
from leapp.models import (
+ CopyFile,
InstalledRedHatSignedRPM,
InstalledRPM,
RequiredTargetUserspacePackages,
RHUIInfo,
- RPM
+ RPM,
+ RpmTransactionTasks,
+ TargetRHUIPostInstallTasks,
+ TargetRHUIPreInstallTasks,
+ TargetRHUISetupInfo,
+ TargetUserSpacePreupgradeTasks
)
from leapp.reporting import Report
from leapp.snactor.fixture import current_actor_context
RH_PACKAGER = 'Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>'
-NO_RHUI = [
- RPM(name='yolo', version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER, arch='noarch',
- pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51'),
-]
-ON_AWS_WITHOUT_LEAPP_PKG = [
- RPM(name='rh-amazon-rhui-client', version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER,
- arch='noarch', pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51'),
-]
+def mk_pkg(name):
+ return RPM(name=name, version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51')
-ON_AWS_WITH_LEAPP_PKG = [
- RPM(name='rh-amazon-rhui-client', version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER,
- arch='noarch', pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51'),
- RPM(name='leapp-rhui-aws', version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER,
- arch='noarch', pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51')
-]
+def mk_setup_info():
+ pre_tasks = TargetRHUIPreInstallTasks()
+ post_tasks = TargetRHUIPostInstallTasks()
+ return TargetRHUISetupInfo(preinstall_tasks=pre_tasks, postinstall_tasks=post_tasks)
-def create_modulesfacts(installed_rpm):
- return InstalledRPM(items=installed_rpm)
+def iter_known_rhui_setups():
+ for upgrade_path, providers in rhui.RHUI_CLOUD_MAP.items():
+ for provider_variant, variant_description in providers.items():
+ src_clients = variant_description['src_pkg']
+ if isinstance(src_clients, str):
+ src_clients = {src_clients, }
-msgs_received = namedtuple('MsgsReceived', ['report', 'rhui_info', 'req_target_userspace'])
+ yield provider_variant, upgrade_path, src_clients
-@pytest.mark.parametrize('skip_rhsm, msgs_received, installed_rpms', [
- (False, msgs_received(False, False, False), NO_RHUI),
- (True, msgs_received(True, False, False), ON_AWS_WITHOUT_LEAPP_PKG),
- (True, msgs_received(False, True, True), ON_AWS_WITH_LEAPP_PKG),
- (False, msgs_received(True, False, False), ON_AWS_WITH_LEAPP_PKG)
-])
-def test_check_rhui_actor(
- monkeypatch, current_actor_context, skip_rhsm, msgs_received, installed_rpms
-):
+def mk_cloud_map(variants):
+ upg_path = {}
+ for variant_desc in variants:
+ provider, desc = next(iter(variant_desc.items()))
+ upg_path[provider] = desc
+ return upg_path
+
+
+@pytest.mark.parametrize(
+ ('extra_pkgs', 'rhui_setups', 'expected_result'),
+ [
+ (
+ ['client'],
+ {RHUIFamily('provider'): [mk_rhui_setup(clients={'client'})]},
+ RHUIFamily('provider')
+ ),
+ (
+ ['client'],
+ {RHUIFamily('provider'): [mk_rhui_setup(clients={'missing_client'})]},
+ None
+ ),
+ (
+ ['clientA', 'clientB'],
+ {RHUIFamily('provider'): [mk_rhui_setup(clients={'clientB'})]},
+ RHUIFamily('provider')
+ ),
+ (
+ ['clientA', 'clientB'],
+ {
+ RHUIFamily('provider'): [mk_rhui_setup(clients={'clientA'})],
+ RHUIFamily('provider+'): [mk_rhui_setup(clients={'clientA', 'clientB'})],
+ },
+ RHUIFamily('provider+')
+ ),
+ (
+ ['client'],
+ {
+ RHUIFamily('providerA'): [mk_rhui_setup(clients={'client'})],
+ RHUIFamily('providerB'): [mk_rhui_setup(clients={'client'})],
+ },
+ StopActorExecutionError
+ ),
+ ]
+)
+def test_determine_rhui_src_variant(monkeypatch, extra_pkgs, rhui_setups, expected_result):
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(src_ver='7.9'))
+ installed_pkgs = {'zip', 'zsh', 'bash', 'grubby'}.union(set(extra_pkgs))
+
+ if expected_result and not isinstance(expected_result, RHUIFamily): # An exception
+ with pytest.raises(expected_result) as err:
+ checkrhui_lib.find_rhui_setup_matching_src_system(installed_pkgs, rhui_setups)
+ assert 'ambiguous' in str(err)
+ return
+
+ variant_setup_pair = checkrhui_lib.find_rhui_setup_matching_src_system(installed_pkgs, rhui_setups)
+ if not expected_result:
+ assert variant_setup_pair == expected_result
+ else:
+ variant = variant_setup_pair[0]
+ assert variant == expected_result
+
+
+@pytest.mark.parametrize(
+ ('extra_pkgs', 'target_rhui_setup', 'should_inhibit'),
+ [
+ (['pkg'], mk_rhui_setup(leapp_pkg='pkg'), False),
+ ([], mk_rhui_setup(leapp_pkg='pkg'), True),
+ ]
+)
+def test_inhibit_on_missing_leapp_rhui_pkg(monkeypatch, extra_pkgs, target_rhui_setup, should_inhibit):
+ installed_pkgs = set(['bash', 'zsh', 'zip'] + extra_pkgs)
+ monkeypatch.setattr(reporting, 'create_report', create_report_mocked())
+ checkrhui_lib.inhibit_if_leapp_pkg_to_access_target_missing(installed_pkgs,
+ RHUIFamily('rhui-variant'),
+ target_rhui_setup)
+ assert bool(reporting.create_report.called) == should_inhibit
+
+
+def are_setup_infos_eq(actual, expected):
+ eq = True
+ eq &= actual.enable_only_repoids_in_copied_files == expected.enable_only_repoids_in_copied_files
+ eq &= actual.files_supporting_client_operation == expected.files_supporting_client_operation
+ eq &= actual.preinstall_tasks.files_to_remove == expected.preinstall_tasks.files_to_remove
+ eq &= actual.preinstall_tasks.files_to_copy_into_overlay == expected.preinstall_tasks.files_to_copy_into_overlay
+ eq &= actual.postinstall_tasks.files_to_copy == expected.postinstall_tasks.files_to_copy
+ return eq
+
+
+@pytest.mark.parametrize(
+ ('provider', 'should_mutate'),
+ [
+ (RHUIFamily(rhui.RHUIProvider.GOOGLE), True),
+ (RHUIFamily(rhui.RHUIProvider.GOOGLE, variant=rhui.RHUIVariant.SAP), True),
+ (RHUIFamily('azure'), False),
+ ]
+)
+def test_google_specific_customization(provider, should_mutate):
+ setup_info = mk_setup_info()
+ checkrhui_lib.customize_rhui_setup_for_gcp(provider, setup_info)
+
+ if should_mutate:
+ assert setup_info != mk_setup_info()
+ else:
+ assert setup_info == mk_setup_info()
+
+
+@pytest.mark.parametrize(
+ ('rhui_family', 'target_major', 'should_mutate'),
+ [
+ (RHUIFamily(rhui.RHUIProvider.AWS), '8', False),
+ (RHUIFamily(rhui.RHUIProvider.AWS), '9', True),
+ (RHUIFamily(rhui.RHUIProvider.AWS, variant=rhui.RHUIVariant.SAP), '9', True),
+ (RHUIFamily('azure'), '9', False),
+ ]
+)
+def test_aws_specific_customization(monkeypatch, rhui_family, target_major, should_mutate):
+ dst_ver = '{major}.0'.format(major=target_major)
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(dst_ver=dst_ver))
+
+ setup_info = mk_setup_info()
+ checkrhui_lib.customize_rhui_setup_for_aws(rhui_family, setup_info)
+
+ was_mutated = not are_setup_infos_eq(setup_info, mk_setup_info())
+ assert should_mutate == was_mutated
+
+
+def produce_rhui_info_to_setup_target(monkeypatch):
+ source_rhui_setup = mk_rhui_setup(
+ clients={'src_pkg'},
+ leapp_pkg='leapp_pkg',
+ mandatory_files=[('src_file1', '/etc'), ('src_file2', '/var')],
+ )
+
+ target_rhui_setup = mk_rhui_setup(
+ clients={'target_pkg'},
+ leapp_pkg='leapp_pkg',
+ mandatory_files=[('target_file1', '/etc'), ('target_file2', '/var')],
+ )
+
+ monkeypatch.setattr(api, 'get_common_folder_path', lambda dummy: 'common_folder')
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+
+ checkrhui_lib.produce_rhui_info_to_setup_target('provider', source_rhui_setup, target_rhui_setup)
+
+ assert len(api.produce.model_instances) == 1
+
+ rhui_info = api.produce.model_instances[0]
+ assert rhui_info.provider == 'provider'
+ assert rhui_info.src_client_pkg_names == ['src_pkg']
+ assert rhui_info.target_client_pkg_names == ['target_pkg']
+
+ setup_info = rhui_info.target_client_setup_info
+
+ expected_copies = {
+ ('common_folder/provider/target_file1', '/etc'),
+ ('common_folder/provider/target_file2', '/var')
+ }
+ actual_copies = {(instr.src, instr.dst) for instr in setup_info.preinstall_tasks.files_to_copy_in}
+
+ assert expected_copies == actual_copies
+
+ assert not setup_info.postinstall_tasks.files_to_copy
+
+
+def test_produce_rpms_to_install_into_target(monkeypatch):
+ source_rhui_setup = mk_rhui_setup(clients={'src_pkg'}, leapp_pkg='leapp_pkg')
+ target_rhui_setup = mk_rhui_setup(clients={'target_pkg'}, leapp_pkg='leapp_pkg')
+
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+
+ checkrhui_lib.produce_rpms_to_install_into_target(source_rhui_setup, target_rhui_setup)
+
+ assert len(api.produce.model_instances) == 2
+ userspace_tasks, target_rpm_tasks = api.produce.model_instances[0], api.produce.model_instances[1]
+
+ if isinstance(target_rpm_tasks, TargetUserSpacePreupgradeTasks):
+ userspace_tasks, target_rpm_tasks = target_rpm_tasks, userspace_tasks
+
+ assert 'target_pkg' in target_rpm_tasks.to_install
+ assert 'src_pkg' in target_rpm_tasks.to_remove
+ assert 'target_pkg' in userspace_tasks.install_rpms
+
+
+@pytest.mark.parametrize('skip_rhsm', (True, False))
+def test_inform_about_upgrade_with_rhui_without_no_rhsm(monkeypatch, skip_rhsm):
+ monkeypatch.setattr(rhsm, 'skip_rhsm', lambda: skip_rhsm)
+ monkeypatch.setattr(reporting, 'create_report', create_report_mocked())
+
+ checkrhui_lib.inform_about_upgrade_with_rhui_without_no_rhsm()
+
+ assert bool(reporting.create_report.called) is not skip_rhsm
+
+
+class ExpectedAction(Enum):
+ NOTHING = 1 # Actor should not produce anything
+ INHIBIT = 2
+ PRODUCE = 3 # Actor should produce RHUI related info
+
+
+# Scenarios to cover:
+# 1. source client + NO_RHSM -> RPMs are produced, and setup info is produced
+# 2. source client -> inhibit
+# 3. leapp pkg missing -> inhibit
+@pytest.mark.parametrize(
+ ('extra_installed_pkgs', 'skip_rhsm', 'expected_action'),
+ [
+ (['src_pkg', 'leapp_pkg'], True, ExpectedAction.PRODUCE), # Everything OK
+ (['src_pkg', 'leapp_pkg'], False, ExpectedAction.INHIBIT), # No --no-rhsm
+ (['src_pkg'], True, ExpectedAction.INHIBIT), # Missing leapp-rhui package
+ ([], True, ExpectedAction.NOTHING) # Not a RHUI system
+ ]
+)
+def test_process(monkeypatch, extra_installed_pkgs, skip_rhsm, expected_action):
+ known_setups = {
+ RHUIFamily('rhui-variant'): [
+ mk_rhui_setup(clients={'src_pkg'}, os_version='7'),
+ mk_rhui_setup(clients={'target_pkg'}, os_version='8', leapp_pkg='leapp_pkg',
+ mandatory_files=[('file1', '/etc'), ('file2', '/var')]),
+ ]
+ }
+
+ installed_pkgs = {'zip', 'kernel-core', 'python'}.union(set(extra_installed_pkgs))
+ installed_pkgs = [mk_pkg(pkg_name) for pkg_name in installed_pkgs]
+ installed_rpms = InstalledRPM(items=installed_pkgs)
+
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(src_ver='7.9', msgs=[installed_rpms]))
+ monkeypatch.setattr(reporting, 'create_report', create_report_mocked())
monkeypatch.setattr(rhsm, 'skip_rhsm', lambda: skip_rhsm)
+ monkeypatch.setattr(rhui, 'RHUI_SETUPS', known_setups)
+
+ checkrhui_lib.process()
+
+ if expected_action == ExpectedAction.NOTHING:
+ assert not api.produce.called
+ assert not reporting.create_report.called
+ elif expected_action == ExpectedAction.INHIBIT:
+ assert not api.produce.called
+ assert len(reporting.create_report.reports) == 1
+ else: # expected_action = ExpectedAction.PRODUCE
+ assert not reporting.create_report.called
+ assert len(api.produce.model_instances) == 3
+ assert any(isinstance(pkg, RpmTransactionTasks) for pkg in api.produce.model_instances)
+ assert any(isinstance(pkg, RHUIInfo) for pkg in api.produce.model_instances)
+ assert any(isinstance(pkg, TargetUserSpacePreupgradeTasks) for pkg in api.produce.model_instances)
+
+
+@pytest.mark.parametrize('is_target_setup_known', (False, True))
+def test_unknown_target_rhui_setup(monkeypatch, is_target_setup_known):
+ rhui_family = RHUIFamily('rhui-variant')
+ known_setups = {
+ rhui_family: [
+ mk_rhui_setup(clients={'src_pkg'}, os_version='7'),
+ ]
+ }
+
+ if is_target_setup_known:
+ target_setup = mk_rhui_setup(clients={'target_pkg'}, os_version='8', leapp_pkg='leapp_pkg')
+ known_setups[rhui_family].append(target_setup)
+
+ installed_pkgs = {'zip', 'kernel-core', 'python', 'src_pkg', 'leapp_pkg'}
+ installed_pkgs = [mk_pkg(pkg_name) for pkg_name in installed_pkgs]
+ installed_rpms = InstalledRPM(items=installed_pkgs)
+
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(src_ver='7.9', msgs=[installed_rpms]))
+ monkeypatch.setattr(reporting, 'create_report', create_report_mocked())
+ monkeypatch.setattr(rhsm, 'skip_rhsm', lambda: True)
+ monkeypatch.setattr(rhui, 'RHUI_SETUPS', known_setups)
- current_actor_context.feed(create_modulesfacts(installed_rpm=installed_rpms))
- current_actor_context.run(config_model=mock_configs.CONFIG)
- assert bool(current_actor_context.consume(Report)) is msgs_received.report
- assert bool(current_actor_context.consume(RHUIInfo)) is msgs_received.rhui_info
- assert bool(current_actor_context.consume(
- RequiredTargetUserspacePackages)) is msgs_received.req_target_userspace
+ if is_target_setup_known:
+ checkrhui_lib.process()
+ assert api.produce.called
+ else:
+ with pytest.raises(StopActorExecutionError):
+ checkrhui_lib.process()
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
index 01457f2a..f8d8dcfc 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
@@ -355,9 +355,10 @@ def get_pesid_to_repoid_map(target_pesids):
details={'Problem': 'Did not receive a message with mapped repositories'}
)
- rhui_info = next(api.consume(RHUIInfo), RHUIInfo(provider=''))
+ rhui_info = next(api.consume(RHUIInfo), None)
+ cloud_provider = rhui_info.provider if rhui_info else ''
- repomap = peseventsscanner_repomap.RepoMapDataHandler(repositories_map_msg, cloud_provider=rhui_info.provider)
+ repomap = peseventsscanner_repomap.RepoMapDataHandler(repositories_map_msg, cloud_provider=cloud_provider)
# NOTE: We have to calculate expected target repositories like in the setuptargetrepos actor.
# It's planned to handle this in different a way in future...
diff --git a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py b/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
index 1085beee..41f9d343 100644
--- a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
+++ b/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
@@ -54,26 +54,7 @@ class RedHatSignedRpmScanner(Actor):
"""Whitelist the katello package."""
return pkg.name.startswith('katello-ca-consumer')
- upg_path = rhui.get_upg_path()
- # AWS RHUI packages do not have to be whitelisted because they are signed by RedHat
- whitelisted_cloud_flavours = (
- 'azure',
- 'azure-eus',
- 'azure-sap-ha',
- 'azure-sap-apps',
- 'google',
- 'google-sap',
- 'alibaba'
- )
- whitelisted_cloud_pkgs = {
- rhui.RHUI_CLOUD_MAP[upg_path].get(flavour, {}).get('src_pkg') for flavour in whitelisted_cloud_flavours
- }
- whitelisted_cloud_pkgs.update(
- rhui.RHUI_CLOUD_MAP[upg_path].get(flavour, {}).get('target_pkg') for flavour in whitelisted_cloud_flavours
- )
- whitelisted_cloud_pkgs.update(
- rhui.RHUI_CLOUD_MAP[upg_path].get(flavour, {}).get('leapp_pkg') for flavour in whitelisted_cloud_flavours
- )
+ whitelisted_cloud_pkgs = rhui.get_all_known_rhui_pkgs_for_current_upg()
for rpm_pkgs in self.consume(InstalledRPM):
for pkg in rpm_pkgs.items:
diff --git a/repos/system_upgrade/common/actors/setetcreleasever/tests/test_setetcreleasever.py b/repos/system_upgrade/common/actors/setetcreleasever/tests/test_setetcreleasever.py
index d86ac926..a14dd2b8 100644
--- a/repos/system_upgrade/common/actors/setetcreleasever/tests/test_setetcreleasever.py
+++ b/repos/system_upgrade/common/actors/setetcreleasever/tests/test_setetcreleasever.py
@@ -3,13 +3,15 @@ import os
import pytest
from leapp.libraries.actor import setetcreleasever
-from leapp.libraries.common.testutils import (
- create_report_mocked,
- CurrentActorMocked,
- logger_mocked
-)
+from leapp.libraries.common.testutils import create_report_mocked, CurrentActorMocked, logger_mocked
from leapp.libraries.stdlib import api
-from leapp.models import PkgManagerInfo, RHUIInfo
+from leapp.models import (
+ PkgManagerInfo,
+ RHUIInfo,
+ TargetRHUIPostInstallTasks,
+ TargetRHUIPreInstallTasks,
+ TargetRHUISetupInfo
+)
CUR_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -33,8 +35,15 @@ class mocked_set_releasever(object):
def test_set_releasever(monkeypatch, current_actor_context):
-
- msgs = [RHUIInfo(provider='aws'), PkgManagerInfo(etc_releasever='7.7')]
+ preinstall_tasks = TargetRHUIPreInstallTasks()
+ postinstall_tasks = TargetRHUIPostInstallTasks()
+ setup_info = TargetRHUISetupInfo(preinstall_tasks=preinstall_tasks, postinstall_tasks=postinstall_tasks)
+ rhui_info = RHUIInfo(provider='aws',
+ src_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_setup_info=setup_info)
+
+ msgs = [rhui_info, PkgManagerInfo(etc_releasever='7.7')]
expected_rel_ver = '8.0'
monkeypatch.setattr(setetcreleasever, '_set_releasever', mocked_set_releasever())
diff --git a/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py b/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
index 4b8405d0..2b14a29a 100644
--- a/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
+++ b/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
@@ -85,8 +85,11 @@ def process():
# Setup repomap handler
repo_mappig_msg = next(api.consume(RepositoriesMapping), RepositoriesMapping())
- rhui_info = next(api.consume(RHUIInfo), RHUIInfo(provider=''))
- repomap = setuptargetrepos_repomap.RepoMapDataHandler(repo_mappig_msg, cloud_provider=rhui_info.provider)
+
+ rhui_info = next(api.consume(RHUIInfo), None)
+ cloud_provider = rhui_info.provider if rhui_info else ''
+
+ repomap = setuptargetrepos_repomap.RepoMapDataHandler(repo_mappig_msg, cloud_provider=cloud_provider)
# Filter set of repoids from installed packages so that it contains only repoids with mapping
repoids_from_installed_packages_with_mapping = _get_mapped_repoids(repomap, repoids_from_installed_packages)
diff --git a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
index 0982a796..039b99a5 100644
--- a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
+++ b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
@@ -6,7 +6,7 @@ import shutil
from leapp import reporting
from leapp.exceptions import StopActorExecution, StopActorExecutionError
from leapp.libraries.actor import constants
-from leapp.libraries.common import dnfplugin, mounting, overlaygen, repofileutils, rhsm, rhui, utils
+from leapp.libraries.common import dnfplugin, mounting, overlaygen, repofileutils, rhsm, utils
from leapp.libraries.common.config import get_env, get_product_type
from leapp.libraries.common.config.version import get_target_major_version
from leapp.libraries.stdlib import api, CalledProcessError, config, run
@@ -282,25 +282,11 @@ def prepare_target_userspace(context, userspace_dir, enabled_repos, packages):
raise StopActorExecutionError(message=message, details=details)
-def _get_all_rhui_pkgs():
- """
- Return the list of rhui packages
-
- Currently, do not care about what rhui we have, release, etc.
- Just take all packages. We need them just for the purpose of filtering
- what files we have to remove (see _prep_repository_access) and it's ok
- for us to use whatever rhui rpms (the relevant rpms catch the problem,
- the others are just taking bytes in memory...). It's a hot-fix. We are going
- to refactor the library later completely..
- """
- upg_path = rhui.get_upg_path()
- pkgs = []
- for rhui_map in rhui.RHUI_CLOUD_MAP[upg_path].values():
- for key in rhui_map.keys():
- if not key.endswith('pkg'):
- continue
- pkgs.append(rhui_map[key])
- return pkgs
+def _query_rpm_for_pkg_files(context, pkgs):
+ files_owned_by_rpm = set()
+ rpm_query_result = context.call(['rpm', '-ql'] + pkgs, split=True)
+ files_owned_by_rpm.update(rpm_query_result['stdout'])
+ return files_owned_by_rpm
def _get_files_owned_by_rpms(context, dirpath, pkgs=None, recursive=False):
@@ -405,42 +391,30 @@ def _prep_repository_access(context, target_userspace):
if not rhsm.skip_rhsm():
run(['rm', '-rf', os.path.join(target_etc, 'rhsm')])
context.copytree_from('/etc/rhsm', os.path.join(target_etc, 'rhsm'))
- # NOTE: we cannot just remove the original target yum.repos.d dir
- # as e.g. in case of RHUI a special RHUI repofiles are installed by a pkg
- # when the target userspace container is created. Removing these files we loose
- # RHUI target repositories. So ...->
- # -> detect such a files...
+
+ # NOTE: We cannot just remove the target yum.repos.d dir and replace it with yum.repos.d from the scratch
+ # # that we've used to obtain the new DNF stack and install it into the target userspace. Although
+ # # RHUI clients are being installed in both scratch and target containers, users can request their package
+ # # to be installed into target userspace that might add some repos to yum.repos.d that are not in scratch.
+
+ # Detect files that are owned by some RPM - these cannot be deleted
with mounting.NspawnActions(base_dir=target_userspace) as target_context:
files_owned_by_rpms = _get_files_owned_by_rpms(target_context, '/etc/yum.repos.d')
- # -> backup the orig dir & install the new one
+ # Backup the target yum.repos.d so we can always copy the files installed by some RPM back into yum.repos.d
+ # when we modify it
run(['mv', target_yum_repos_d, backup_yum_repos_d])
- context.copytree_from('/etc/yum.repos.d', target_yum_repos_d)
- # -> find old rhui repo files (we have to remove these as they cause duplicates)
- rhui_pkgs = _get_all_rhui_pkgs()
- old_files_owned_by_rhui_rpms = _get_files_owned_by_rpms(context, '/etc/yum.repos.d', rhui_pkgs)
- for fname in old_files_owned_by_rhui_rpms:
- api.current_logger().debug('Remove the old repofile: {}'.format(fname))
- run(['rm', '-f', os.path.join(target_yum_repos_d, fname)])
- # .. continue: remove our leapp rhui repo file (do not care if we are on rhui or not)
- for rhui_map in rhui.gen_rhui_files_map().values():
- for item in rhui_map:
- if item[1] != rhui.YUM_REPOS_PATH:
- continue
- target_leapp_repofile = os.path.join(target_yum_repos_d, item[0])
- if not os.path.isfile(target_leapp_repofile):
- continue
- # we found it!!
- run(['rm', '-f', target_leapp_repofile])
- break
+ # Copy the yum.repos.d from scratch - preserve any custom repositories. No need to clean-up old RHUI clients,
+ # we swap them for the new RHUI client in scratch (so the old one is not installed).
+ context.copytree_from('/etc/yum.repos.d', target_yum_repos_d)
- # -> copy expected files back
+ # Copy back files owned by some RPM
for fname in files_owned_by_rpms:
api.current_logger().debug('Copy the backed up repo file: {}'.format(fname))
run(['mv', os.path.join(backup_yum_repos_d, fname), os.path.join(target_yum_repos_d, fname)])
- # -> remove the backed up dir
+ # Cleanup - remove the backed up dir
run(['rm', '-rf', backup_yum_repos_d])
@@ -637,22 +611,71 @@ def _get_rhui_available_repoids(context, cloud_repo):
return set(repoids)
+def get_copy_location_from_copy_in_task(context, copy_task):
+ basename = os.path.basename(copy_task.src)
+ dest_in_container = context.full_path(copy_task.dst)
+ if os.path.isdir(dest_in_container):
+ return os.path.join(copy_task.dst, basename)
+ return copy_task.dst
+
+
def _get_rh_available_repoids(context, indata):
"""
RH repositories are provided either by RHSM or are stored in the expected repo file provided by
RHUI special packages (every cloud provider has itw own rpm).
"""
- upg_path = rhui.get_upg_path()
-
rh_repoids = _get_rhsm_available_repoids(context)
+ # If we are upgrading a RHUI system, check what repositories are provided by the (already installed) target clients
if indata and indata.rhui_info:
- cloud_repo = os.path.join(
- '/etc/yum.repos.d/', rhui.RHUI_CLOUD_MAP[upg_path][indata.rhui_info.provider]['leapp_pkg_repo']
+ files_provided_by_clients = _query_rpm_for_pkg_files(context, indata.rhui_info.target_client_pkg_names)
+
+ def is_repofile(path):
+ return os.path.dirname(path) == '/etc/yum.repos.d' and os.path.basename(path).endswith('.repo')
+
+ def extract_repoid_from_line(line):
+ return line.split(':', 1)[1].strip()
+
+ target_ver = api.current_actor().configuration.version.target
+ setup_tasks = indata.rhui_info.target_client_setup_info.preinstall_tasks.files_to_copy_into_overlay
+
+ yum_repos_d = context.full_path('/etc/yum.repos.d')
+ all_repofiles = {os.path.join(yum_repos_d, path) for path in os.listdir(yum_repos_d) if path.endswith('.repo')}
+ client_repofiles = {context.full_path(path) for path in files_provided_by_clients if is_repofile(path)}
+
+ # Exclude repofiles used to setup the target rhui access as on some platforms the repos provided by
+ # the client are not sufficient to install the client into target userspace (GCP)
+ rhui_setup_repofile_tasks = [task for task in setup_tasks if task.src.endswith('repo')]
+ rhui_setup_repofiles = (
+ get_copy_location_from_copy_in_task(context, copy_task) for copy_task in rhui_setup_repofile_tasks
)
- rhui_repoids = _get_rhui_available_repoids(context, cloud_repo)
- rh_repoids.update(rhui_repoids)
+ rhui_setup_repofiles = {context.full_path(repofile) for repofile in rhui_setup_repofiles}
+
+ foreign_repofiles = all_repofiles - client_repofiles - rhui_setup_repofiles
+
+ # Rename non-client repofiles so they will not be recognized when running dnf repolist
+ for foreign_repofile in foreign_repofiles:
+ os.rename(foreign_repofile, '{0}.back'.format(foreign_repofile))
+
+ try:
+ dnf_cmd = ['dnf', 'repolist', '--releasever', target_ver, '-v']
+ repolist_result = context.call(dnf_cmd)['stdout']
+ repoid_lines = [line for line in repolist_result.split('\n') if line.startswith('Repo-id')]
+ rhui_repoids = {extract_repoid_from_line(line) for line in repoid_lines}
+ rh_repoids.update(rhui_repoids)
+
+ except CalledProcessError as err:
+ details = {'err': err.stderr, 'details': str(err)}
+ raise StopActorExecutionError(
+ message='Failed to retrieve repoids provided by target RHUI clients.',
+ details=details
+ )
+
+ finally:
+ # Revert the renaming of non-client repofiles
+ for foreign_repofile in foreign_repofiles:
+ os.rename('{0}.back'.format(foreign_repofile), foreign_repofile)
return rh_repoids
@@ -790,8 +813,7 @@ def _gather_target_repositories(context, indata, prod_cert_path):
"""
rhsm.set_container_mode(context)
rhsm.switch_certificate(context, indata.rhsm_info, prod_cert_path)
- if indata.rhui_info:
- rhui.copy_rhui_data(context, indata.rhui_info.provider)
+
_install_custom_repofiles(context, indata.custom_repofiles)
return gather_target_repositories(context, indata)
@@ -834,6 +856,69 @@ def _create_target_userspace(context, packages, files, target_repoids):
rhsm.set_container_mode(target_context)
+def install_target_rhui_client_if_needed(context, indata):
+ if not indata.rhui_info:
+ return
+
+ target_major_version = get_target_major_version()
+ userspace_dir = _get_target_userspace()
+ _create_target_userspace_directories(userspace_dir)
+
+ setup_info = indata.rhui_info.target_client_setup_info
+ if setup_info.preinstall_tasks:
+ preinstall_tasks = setup_info.preinstall_tasks
+
+ for file_to_remove in preinstall_tasks.files_to_remove:
+ context.remove(file_to_remove)
+
+ for copy_info in preinstall_tasks.files_to_copy_into_overlay:
+ context.makedirs(os.path.dirname(copy_info.dst), exists_ok=True)
+ context.copy_to(copy_info.src, copy_info.dst)
+
+ cmd = ['dnf', '-y']
+
+ if setup_info.enable_only_repoids_in_copied_files and setup_info.preinstall_tasks:
+ copy_tasks = setup_info.preinstall_tasks.files_to_copy_into_overlay
+ copied_repofiles = [copy.src for copy in copy_tasks if copy.src.endswith('.repo')]
+ copied_repoids = set()
+ for repofile in copied_repofiles:
+ repofile_contents = repofileutils.parse_repofile(repofile)
+ copied_repoids.update(entry.repoid for entry in repofile_contents.data)
+
+ cmd += ['--disablerepo', '*']
+ for copied_repoid in copied_repoids:
+ cmd.extend(('--enablerepo', copied_repoid))
+
+ src_client_remove_steps = ['remove {0}'.format(client) for client in indata.rhui_info.src_client_pkg_names]
+ target_client_install_steps = ['install {0}'.format(client) for client in indata.rhui_info.target_client_pkg_names]
+
+ dnf_transaction_steps = src_client_remove_steps + target_client_install_steps + ['transaction run']
+
+ cmd += [
+ '--setopt=module_platform_id=platform:el{}'.format(target_major_version),
+ '--setopt=keepcache=1',
+ '--releasever', api.current_actor().configuration.version.target,
+ '--disableplugin', 'subscription-manager',
+ 'shell'
+ ]
+
+ context.call(cmd, callback_raw=utils.logging_handler, stdin='\n'.join(dnf_transaction_steps))
+
+ if setup_info.postinstall_tasks:
+ for copy_info in setup_info.postinstall_tasks.files_to_copy:
+ context.makedirs(os.path.dirname(copy_info.dst), exists_ok=True)
+ context.call(['cp', copy_info.src, copy_info.dst])
+
+ # Do a cleanup so there are not duplicit repoids
+ files_owned_by_clients = _query_rpm_for_pkg_files(context, indata.rhui_info.target_client_pkg_names)
+
+ for copy_task in setup_info.preinstall_tasks.files_to_copy_into_overlay:
+ dest = get_copy_location_from_copy_in_task(context, copy_task)
+ can_be_cleaned_up = copy_task.src not in setup_info.files_supporting_client_operation
+ if dest not in files_owned_by_clients and can_be_cleaned_up:
+ context.remove(dest)
+
+
@suppress_deprecation(TMPTargetRepositoriesFacts)
def perform():
# NOTE: this one action is out of unit-tests completely; we do not use
@@ -853,6 +938,9 @@ def perform():
# Mount the ISO into the scratch container
target_iso = next(api.consume(TargetOSInstallationImage), None)
with mounting.mount_upgrade_iso_to_root_dir(overlay.target, target_iso):
+
+ install_target_rhui_client_if_needed(context, indata)
+
target_repoids = _gather_target_repositories(context, indata, prod_cert_path)
_create_target_userspace(context, indata.packages, indata.files, target_repoids)
# TODO: this is tmp solution as proper one needs significant refactoring
diff --git a/repos/system_upgrade/common/actors/targetuserspacecreator/tests/unit_test_targetuserspacecreator.py b/repos/system_upgrade/common/actors/targetuserspacecreator/tests/unit_test_targetuserspacecreator.py
index a519275e..cc684c7d 100644
--- a/repos/system_upgrade/common/actors/targetuserspacecreator/tests/unit_test_targetuserspacecreator.py
+++ b/repos/system_upgrade/common/actors/targetuserspacecreator/tests/unit_test_targetuserspacecreator.py
@@ -85,7 +85,12 @@ def _gen_packages_msgs():
_PACKAGES_MSGS = _gen_packages_msgs()
_RHSMINFO_MSG = models.RHSMInfo(attached_skus=['testing-sku'])
-_RHUIINFO_MSG = models.RHUIInfo(provider='aws')
+_RHUIINFO_MSG = models.RHUIInfo(provider='aws',
+ src_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_pkg_names=['rh-amazon-rhui-client'],
+ target_client_setup_info=models.TargetRHUISetupInfo(
+ preinstall_tasks=models.TargetRHUIPreInstallTasks(),
+ postinstall_tasks=models.TargetRHUIPostInstallTasks()))
_XFS_MSG = models.XFSPresence()
_STORAGEINFO_MSG = models.StorageInfo()
_CTRF_MSGS = [
diff --git a/repos/system_upgrade/common/libraries/rhui.py b/repos/system_upgrade/common/libraries/rhui.py
index 14a91c42..aa40b597 100644
--- a/repos/system_upgrade/common/libraries/rhui.py
+++ b/repos/system_upgrade/common/libraries/rhui.py
@@ -1,9 +1,12 @@
import os
+from collections import namedtuple
import six
-from leapp.libraries.common.config.version import get_target_major_version
+from leapp.libraries.common.config import architecture as arch
+from leapp.libraries.common.config.version import get_source_major_version, get_target_major_version
from leapp.libraries.stdlib import api
+from leapp.utils.deprecation import deprecated
# when on AWS and upgrading from RHEL 7, we need also Python2 version of "Amazon-id" dnf
# plugin which is served by "leapp-rhui-aws" rpm package (please note this package is not
@@ -18,10 +21,233 @@ RHUI_PKI_PRIVATE_DIR = os.path.join(RHUI_PKI_DIR, 'private')
AWS_DNF_PLUGIN_NAME = 'amazon-id.py'
+class ContentChannel(object):
+ GA = 'ga'
+ TUV = 'tuv'
+ E4S = 'e4s'
+ EUS = 'eus'
+ AUS = 'aus'
+ BETA = 'beta'
+
+
+class RHUIVariant(object):
+ ORDINARY = 'ordinary' # Special value - not displayed in report/errors
+ SAP = 'sap'
+ SAP_APPS = 'sap-apps'
+ SAP_HA = 'sap-ha'
+
+
+class RHUIProvider(object):
+ GOOGLE = 'Google'
+ AZURE = 'Azure'
+ AWS = 'AWS'
+ ALIBABA = 'Alibaba'
+
+
# The files in 'files_map' are provided by special Leapp rpms (per cloud) and
# are delivered into "repos/system_upgrade/common/files/rhui/<PROVIDER>
+RHUISetup = namedtuple(
+ 'RHUISetup',
+ ('clients', 'leapp_pkg', 'mandatory_files', 'optional_files', 'extra_info', 'os_version',
+ 'arch', 'content_channel', 'files_supporting_client_operation')
+)
+"""RHUI-Setup-specific details used during IPU
+.. py:attribute:: clients
+ A set of RHUI clients present on the system.
+.. py:attribute:: leapp_pkg
+ The name of leapp's rhui-specific pkg providing repofiles, certs and keys to access package of the setup.
+.. py:attribute:: mandatory_files
+ Mandatory files and their destinations to copy into target userspace container required to access the target OS
+ content. If not present, an exception will be raised.
+.. py:attribute:: optional_files
+ Optional files and their destinations to copy into target userspace container required to access the target OS
+ content. Nonexistence of any of these files is ignored.
+.. py:attribute:: extra_info
+ Extra information about the setup.
+.. py:attribute:: os_version
+ The major OS version of the RHUI system.
+.. py:attribute:: content_channel
+ Content channel used by the RHUI setup.
+.. py:attribute:: files_supporting_client_operation
+ A subset of files from ``mandatory_files`` that are necessary for client to work (cannot be cleaned up).
+"""
+
+
+class RHUIFamily(object):
+ def __init__(self, provider, client_files_folder='', variant=RHUIVariant.ORDINARY, arch=arch.ARCH_X86_64,):
+ self.provider = provider
+ self.client_files_folder = client_files_folder
+ self.variant = variant
+ self.arch = arch
+
+ def __hash__(self):
+ return hash((self.provider, self.variant, self.arch))
+
+ def __eq__(self, other):
+ if not isinstance(other, RHUIFamily):
+ return False
+ self_repr = (self.provider, self.variant, self.arch)
+ other_repr = (other.provider, other.variant, other.arch)
+ return self_repr == other_repr
+
+ def full_eq(self, other):
+ partial_eq = self == other
+ return partial_eq and self.client_files_folder == other.client_files_folder
+
+ def __str__(self):
+ template = 'RHUIFamily(provider={provider}, variant={variant}, arch={arch})'
+ return template.format(provider=self.provider, variant=self.variant, arch=self.arch)
+
+
+def mk_rhui_setup(clients=None, leapp_pkg='', mandatory_files=None, optional_files=None,
+ extra_info=None, os_version='7', arch=arch.ARCH_X86_64, content_channel=ContentChannel.GA,
+ files_supporting_client_operation=None):
+ clients = clients or set()
+ mandatory_files = mandatory_files or []
+ extra_info = extra_info or {}
+ files_supporting_client_operation = files_supporting_client_operation or []
+ # Since the default optional files are not [], we cannot use the same construction as above
+ # to allow the caller to specify empty optional files
+ default_opt_files = [('content-leapp.crt', RHUI_PKI_PRODUCT_DIR), ('key-leapp.pem', RHUI_PKI_DIR)]
+ optional_files = default_opt_files if optional_files is None else optional_files
+
+ return RHUISetup(clients=clients, leapp_pkg=leapp_pkg, mandatory_files=mandatory_files, arch=arch,
+ content_channel=content_channel, optional_files=optional_files, extra_info=extra_info,
+ os_version=os_version, files_supporting_client_operation=files_supporting_client_operation)
+
+
+# This will be the new "cloud map". Essentially a directed graph with edges defined implicitly by OS versions +
+# setup family identification. In theory, we can make the variant be part of rhui setups, but this way we don't
+# have to repeatedly write it to every known setup there is (a sort of compression). Furthermore, it limits
+# the search for target equivalent to setups sharing the same family, and thus reducing a chance of error.
+RHUI_SETUPS = {
+ RHUIFamily(RHUIProvider.AWS, client_files_folder='aws'): [
+ mk_rhui_setup(clients={'rh-amazon-rhui-client'}, optional_files=[], os_version='7'),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client'}, leapp_pkg='leapp-rhui-aws',
+ mandatory_files=[
+ ('rhui-client-config-server-8.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-8.key', RHUI_PKI_DIR),
+ (AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
+ ('leapp-aws.repo', YUM_REPOS_PATH)
+ ],
+ files_supporting_client_operation=[AWS_DNF_PLUGIN_NAME],
+ optional_files=[], os_version='8'),
+ # @Note(mhecko): We don't need to deal with AWS_DNF_PLUGIN_NAME here as on rhel8+ we can use the plugin
+ # # provided by the target client - there is no Python2 incompatibility issue there.
+ mk_rhui_setup(clients={'rh-amazon-rhui-client'}, leapp_pkg='leapp-rhui-aws',
+ mandatory_files=[
+ ('rhui-client-config-server-9.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-9.key', RHUI_PKI_DIR),
+ ('leapp-aws.repo', YUM_REPOS_PATH)
+ ],
+ optional_files=[], os_version='9'),
+ ],
+ RHUIFamily(RHUIProvider.AWS, arch=arch.ARCH_ARM64, client_files_folder='aws'): [
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-arm'}, optional_files=[], os_version='7', arch=arch.ARCH_ARM64),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-arm'}, leapp_pkg='leapp-rhui-aws',
+ mandatory_files=[
+ ('rhui-client-config-server-8.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-8.key', RHUI_PKI_DIR),
+ (AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
+ ('leapp-aws.repo', YUM_REPOS_PATH)
+ ],
+ files_supporting_client_operation=[AWS_DNF_PLUGIN_NAME],
+ optional_files=[], os_version='8', arch=arch.ARCH_ARM64),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-arm'}, leapp_pkg='leapp-rhui-aws',
+ mandatory_files=[
+ ('rhui-client-config-server-9.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-9.key', RHUI_PKI_DIR),
+ ('leapp-aws.repo', YUM_REPOS_PATH)
+ ],
+ optional_files=[], os_version='9', arch=arch.ARCH_ARM64),
+ ],
+ RHUIFamily(RHUIProvider.AWS, variant=RHUIVariant.SAP, client_files_folder='aws-sap-e4s'): [
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-sap-bundle'}, optional_files=[], os_version='7',
+ content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-sap-bundle-e4s'}, leapp_pkg='leapp-rhui-aws-sap-e4s',
+ mandatory_files=[
+ ('rhui-client-config-server-8-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-8-sap-bundle.key', RHUI_PKI_DIR),
+ (AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
+ ('leapp-aws-sap-e4s.repo', YUM_REPOS_PATH)
+ ],
+ files_supporting_client_operation=[AWS_DNF_PLUGIN_NAME],
+ optional_files=[], os_version='8', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rh-amazon-rhui-client-sap-bundle-e4s'}, leapp_pkg='leapp-rhui-aws-sap-e4s',
+ mandatory_files=[
+ ('rhui-client-config-server-9-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
+ ('rhui-client-config-server-9-sap-bundle.key', RHUI_PKI_DIR),
+ ('leapp-aws-sap-e4s.repo', YUM_REPOS_PATH)
+ ],
+ optional_files=[], os_version='9', content_channel=ContentChannel.E4S),
+ ],
+ RHUIFamily(RHUIProvider.AZURE, client_files_folder='azure'): [
+ mk_rhui_setup(clients={'rhui-azure-rhel7'}, os_version='7',
+ extra_info={'agent_pkg': 'WALinuxAgent'}),
+ mk_rhui_setup(clients={'rhui-azure-rhel8'}, leapp_pkg='leapp-rhui-azure',
+ mandatory_files=[('leapp-azure.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='8'),
+ mk_rhui_setup(clients={'rhui-azure-rhel9'}, leapp_pkg='leapp-rhui-azure',
+ mandatory_files=[('leapp-azure.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='9'),
+ ],
+ RHUIFamily(RHUIProvider.AZURE, variant=RHUIVariant.SAP_APPS, client_files_folder='azure-sap-apps'): [
+ mk_rhui_setup(clients={'rhui-azure-rhel7-base-sap-apps'}, os_version='7', content_channel=ContentChannel.EUS),
+ mk_rhui_setup(clients={'rhui-azure-rhel8-sapapps'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-apps.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='8', content_channel=ContentChannel.EUS),
+ mk_rhui_setup(clients={'rhui-azure-rhel9-sapapps'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-apps.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='9', content_channel=ContentChannel.EUS),
+ ],
+ RHUIFamily(RHUIProvider.AZURE, variant=RHUIVariant.SAP_HA, client_files_folder='azure-sap-ha'): [
+ mk_rhui_setup(clients={'rhui-azure-rhel7-base-sap-ha'}, os_version='7', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rhui-azure-rhel8-sap-ha'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='8', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'rhui-azure-rhel9-sap-ha'}, leapp_pkg='leapp-rhui-azure-sap',
+ mandatory_files=[('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)],
+ extra_info={'agent_pkg': 'WALinuxAgent'},
+ os_version='9', content_channel=ContentChannel.E4S),
+ ],
+ RHUIFamily(RHUIProvider.GOOGLE, client_files_folder='google'): [
+ mk_rhui_setup(clients={'google-rhui-client-rhel7'}, os_version='7'),
+ mk_rhui_setup(clients={'google-rhui-client-rhel8'}, leapp_pkg='leapp-rhui-google',
+ mandatory_files=[('leapp-google.repo', YUM_REPOS_PATH)],
+ files_supporting_client_operation=['leapp-google.repo'],
+ os_version='8'),
+ mk_rhui_setup(clients={'google-rhui-client-rhel9'}, leapp_pkg='leapp-rhui-google',
+ mandatory_files=[('leapp-google.repo', YUM_REPOS_PATH)],
+ files_supporting_client_operation=['leapp-google.repo'],
+ os_version='9'),
+ ],
+ RHUIFamily(RHUIProvider.GOOGLE, variant=RHUIVariant.SAP, client_files_folder='google-sap'): [
+ mk_rhui_setup(clients={'google-rhui-client-rhel79-sap'}, os_version='7', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'google-rhui-client-rhel8-sap'}, leapp_pkg='leapp-rhui-google-sap',
+ mandatory_files=[('leapp-google-sap.repo', YUM_REPOS_PATH)],
+ files_supporting_client_operation=['leapp-google-sap.repo'],
+ os_version='8', content_channel=ContentChannel.E4S),
+ mk_rhui_setup(clients={'google-rhui-client-rhel9-sap'}, leapp_pkg='leapp-rhui-google-sap',
+ mandatory_files=[('leapp-google-sap.repo', YUM_REPOS_PATH)],
+ files_supporting_client_operation=['leapp-google-sap.repo'],
+ os_version='9', content_channel=ContentChannel.E4S),
+ ],
+ RHUIFamily(RHUIProvider.ALIBABA, client_files_folder='alibaba'): [
+ mk_rhui_setup(clients={'client-rhel7'}, os_version='7'),
+ mk_rhui_setup(clients={'aliyun_rhui_rhel8'}, leapp_pkg='leapp-rhui-alibaba',
+ mandatory_files=[('leapp-alibaba.repo', YUM_REPOS_PATH)], os_version='8'),
+ ]
+}
+
+
+# DEPRECATED, use RHUI_SETUPS instead
RHUI_CLOUD_MAP = {
'7to8': {
'aws': {
@@ -32,8 +258,6 @@ RHUI_CLOUD_MAP = {
'files_map': [
('rhui-client-config-server-8.crt', RHUI_PKI_PRODUCT_DIR),
('rhui-client-config-server-8.key', RHUI_PKI_DIR),
- ('content-rhel8.crt', RHUI_PKI_PRODUCT_DIR),
- ('content-rhel8.key', RHUI_PKI_DIR),
('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
(AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
('leapp-aws.repo', YUM_REPOS_PATH)
@@ -47,8 +271,6 @@ RHUI_CLOUD_MAP = {
'files_map': [
('rhui-client-config-server-8-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
('rhui-client-config-server-8-sap-bundle.key', RHUI_PKI_DIR),
- ('content-rhel8-sap.crt', RHUI_PKI_PRODUCT_DIR),
- ('content-rhel8-sap.key', RHUI_PKI_DIR),
('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
(AWS_DNF_PLUGIN_NAME, DNF_PLUGIN_PATH_PY2),
('leapp-aws-sap-e4s.repo', YUM_REPOS_PATH)
@@ -61,8 +283,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure',
'leapp_pkg_repo': 'leapp-azure.repo',
'files_map': [
- ('content.crt', RHUI_PKI_PRODUCT_DIR),
- ('key.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure.repo', YUM_REPOS_PATH)
],
},
@@ -73,8 +293,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure-sap',
'leapp_pkg_repo': 'leapp-azure-sap-apps.repo',
'files_map': [
- ('content-sapapps.crt', RHUI_PKI_PRODUCT_DIR),
- ('key-sapapps.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure-sap-apps.repo', YUM_REPOS_PATH),
],
},
@@ -85,8 +303,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure-sap',
'leapp_pkg_repo': 'leapp-azure-sap-ha.repo',
'files_map': [
- ('content-sap-ha.crt', RHUI_PKI_PRODUCT_DIR),
- ('key-sap-ha.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)
],
},
@@ -133,8 +349,6 @@ RHUI_CLOUD_MAP = {
'files_map': [
('rhui-client-config-server-9.crt', RHUI_PKI_PRODUCT_DIR),
('rhui-client-config-server-9.key', RHUI_PKI_DIR),
- ('content-rhel9.crt', RHUI_PKI_PRODUCT_DIR),
- ('content-rhel9.key', RHUI_PKI_DIR),
('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
('leapp-aws.repo', YUM_REPOS_PATH)
],
@@ -147,8 +361,6 @@ RHUI_CLOUD_MAP = {
'files_map': [
('rhui-client-config-server-9-sap-bundle.crt', RHUI_PKI_PRODUCT_DIR),
('rhui-client-config-server-9-sap-bundle.key', RHUI_PKI_DIR),
- ('content-rhel9-sap-bundle-e4s.crt', RHUI_PKI_PRODUCT_DIR),
- ('content-rhel9-sap-bundle-e4s.key', RHUI_PKI_DIR),
('cdn.redhat.com-chain.crt', RHUI_PKI_DIR),
('leapp-aws-sap-e4s.repo', YUM_REPOS_PATH)
],
@@ -160,8 +372,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure',
'leapp_pkg_repo': 'leapp-azure.repo',
'files_map': [
- ('content.crt', RHUI_PKI_PRODUCT_DIR),
- ('key.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure.repo', YUM_REPOS_PATH)
],
},
@@ -178,8 +388,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure-eus',
'leapp_pkg_repo': 'leapp-azure.repo',
'files_map': [
- ('content.crt', RHUI_PKI_PRODUCT_DIR),
- ('key.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure.repo', YUM_REPOS_PATH)
],
},
@@ -190,8 +398,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure-sap',
'leapp_pkg_repo': 'leapp-azure-sap-ha.repo',
'files_map': [
- ('content-sap-ha.crt', RHUI_PKI_PRODUCT_DIR),
- ('key-sap-ha.pem', RHUI_PKI_DIR),
('leapp-azure-sap-ha.repo', YUM_REPOS_PATH)
],
},
@@ -202,8 +408,6 @@ RHUI_CLOUD_MAP = {
'leapp_pkg': 'leapp-rhui-azure-sap',
'leapp_pkg_repo': 'leapp-azure-sap-apps.repo',
'files_map': [
- ('content-sapapps.crt', RHUI_PKI_PRODUCT_DIR),
- ('key-sapapps.pem', RHUI_PKI_PRIVATE_DIR),
('leapp-azure-sap-apps.repo', YUM_REPOS_PATH)
],
},
@@ -240,6 +444,7 @@ def get_upg_path():
return '7to8' if get_target_major_version() == '8' else '8to9'
+@deprecated(since='2023-07-27', message='This functionality has been replaced with the RHUIInfo message.')
def gen_rhui_files_map():
"""
Generate RHUI files map based on architecture and upgrade path
@@ -256,6 +461,7 @@ def gen_rhui_files_map():
return files_map
+@deprecated(since='2023-07-27', message='This functionality has been integrated into target_userspace_creator.')
def copy_rhui_data(context, provider):
"""
Copy relevant RHUI certificates and key into the target userspace container
@@ -268,3 +474,17 @@ def copy_rhui_data(context, provider):
for path_ in gen_rhui_files_map().get(provider, ()):
context.copy_to(os.path.join(data_dir, path_[0]), path_[1])
+
+
+def get_all_known_rhui_pkgs_for_current_upg():
+ upg_major_versions = (get_source_major_version(), get_target_major_version())
+
+ known_pkgs = set()
+ for setup_family in RHUI_SETUPS.values():
+ for setup in setup_family:
+ if setup.os_version not in upg_major_versions:
+ continue
+ known_pkgs.update(setup.clients)
+ known_pkgs.add(setup.leapp_pkg)
+
+ return known_pkgs
diff --git a/repos/system_upgrade/common/models/rhuiinfo.py b/repos/system_upgrade/common/models/rhuiinfo.py
index 0b518928..3eaa4826 100644
--- a/repos/system_upgrade/common/models/rhuiinfo.py
+++ b/repos/system_upgrade/common/models/rhuiinfo.py
@@ -1,12 +1,58 @@
-from leapp.models import fields, Model
+from leapp.models import CopyFile, fields, Model
from leapp.topics import SystemInfoTopic
+class TargetRHUIPreInstallTasks(Model):
+ """Tasks required to be executed before target RHUI clients are installed"""
+ topic = SystemInfoTopic
+
+ files_to_remove = fields.List(fields.String(), default=[])
+ """Files to remove from the source system in order to setup target RHUI access"""
+
+ files_to_copy_into_overlay = fields.List(fields.Model(CopyFile), default=[])
+ """Files to copy into the scratch (overlayfs) container in order to setup target RHUI access"""
+
+
+class TargetRHUIPostInstallTasks(Model):
+ """Tasks required to be executed after target RHUI clients are installed to facilitate access to target content."""
+ topic = SystemInfoTopic
+
+ files_to_copy = fields.List(fields.Model(CopyFile), default=[])
+ """Source and destination are paths inside the container"""
+
+
+class TargetRHUISetupInfo(Model):
+ topic = SystemInfoTopic
+
+ enable_only_repoids_in_copied_files = fields.Boolean(default=True)
+ """If True (default) only the repoids from copied files will be enabled during client installation"""
+
+ preinstall_tasks = fields.Model(TargetRHUIPreInstallTasks)
+ """Tasks that must be performed before attempting to install the target client(s)"""
+
+ postinstall_tasks = fields.Model(TargetRHUIPostInstallTasks)
+ """Tasks that must be performed after the target client is installed (before any other content is accessed)"""
+
+ files_supporting_client_operation = fields.List(fields.String(), default=[])
+ """A subset of files copied in preinstall tasks that should not be cleaned up."""
+
+
class RHUIInfo(Model):
"""
- Facts about public cloud provider and RHUI infrastructure
+ Facts about public cloud variant and RHUI infrastructure
"""
topic = SystemInfoTopic
provider = fields.String()
- """ Provider name """
+ """Provider name"""
+
+ variant = fields.StringEnum(['ordinary', 'sap', 'sap-apps', 'sap-ha'], default='ordinary')
+ """Variant of the system"""
+
+ src_client_pkg_names = fields.List(fields.String())
+ """Names of the RHUI client packages providing repofiles to the source system"""
+
+ target_client_pkg_names = fields.List(fields.String())
+ """Names of the RHUI client packages providing repofiles to the target system"""
+
+ target_client_setup_info = fields.Model(TargetRHUISetupInfo)
--
2.41.0
View Git Blame Copy Permalink