- Requires leapp-framework 6.6+ - Initial implementation of upgrades on systems with configured Software RAID - Include multipath related configuration in the upgrade environment - Fix upgrades on systems with multiple LUKS devices - Fix upgrades for systems with /boot/efi on a Software RAID - Fix upgrade crashing when dnf repofiles contain URL encoded characters - Fix mount failures for FSTAB entries with the `nofail` option specified - Fix upgrade incorrectly resuming during SELinux relabeling - Fix AVC SELinux warnings - LiveMode: Fix upgrade getting stuck when the `.leapp_upgrade_failed` file exists - LiveMode: Fix system version determination in the upgrade environment on CentOS Stream - Fix target system initramfs containing outdated configuration files in some circumstances - Inhibit the upgrade when network devices configured with ifcfg files could depend on NM - Ensure SELinux is set to permissive mode during the upgrade even when enforcing=1 is set on the kernel cmdline - Fix source and target distribution names in /etc/migration-results - Fix upgrades with the RealTime kernel on CentOS Stream - Install the correct kernel for the source system kernel page size on ARM systems - Introduce rhui.obsolete_gpg_keys configuration option for the removal of obsoleted RPM GPG keys of RHUI Cloud providers - Detect misconfigured kernel & systemd API mountpoints in FSTAB - Detect misconfigured /var/run on the source system - Unify behaviour of upgrades on systems with enabled CRB repositories and explicitly required CRB repositories by user during the upgrade - Drop the inconsistent report about the use of CRB (unsupported by Red Hat) repositories - Check and migrate the multipath configuration when possible - Fix unwanted suspend mode after 15min of inactivity after the upgrade on systems with graphical environment - Resolves: RHEL-111860, RHEL-115867, RHEL-121205, RHEL-147438, RHEL-148697, RHEL-151509, RHEL-154369, RHEL-154370, RHEL-156580, RHEL-161560, RHEL-161684, RHEL-174874
753 lines
35 KiB
Diff
753 lines
35 KiB
Diff
From 4ff499df1e88ef3a7af57aca5ac58a648ddd4707 Mon Sep 17 00:00:00 2001
|
|
From: Petr Stodulka <pstodulk@redhat.com>
|
|
Date: Thu, 4 Jun 2026 07:20:14 +0200
|
|
Subject: [PATCH 098/105] [2/9] targetcontentresolver: Init message consumption
|
|
centralization
|
|
|
|
Centralize consumption of RepositoriesFacts, RepositoriesSetupTasks,
|
|
and CustomTargetRepository into a new InputData class in
|
|
targetcontentresolver. This deduplicates logic previously scattered
|
|
across repositoriesblocklist and pes_events_scanner, and validates
|
|
that required messages are present before processing begins.
|
|
|
|
This is initial set of changes for future refactoring in individual
|
|
libraries of the actor.
|
|
|
|
Some details about individual changes:
|
|
* setuptargetrepos:
|
|
* Drop the rename of get_enabled_repoids to a private functions
|
|
when importing.
|
|
|
|
* targetcontentresolver:
|
|
* Introduce InputData class that consumes and validates required
|
|
messages, warns about unexpected duplicates, and raises
|
|
StopActorExecutionError when required messages are missing.
|
|
* Introduce ExternalRepoSetupTasks named tuple to collect all
|
|
external repository requests (to_enable, to_block, custom)
|
|
from RepositoriesSetupTasks and CustomTargetRepository messages.
|
|
|
|
* pes_events_scanner:
|
|
* Pass enabled_repoids as parameter through scan_pes_events,
|
|
get_pesid_to_repoid_map, and replace_pesids_with_repoids_in_packages
|
|
instead of fetching it internally via get_enabled_repoids().
|
|
* Drop default parameter for blacklisted_repoids in scan_pes_events
|
|
(must be always specified).
|
|
|
|
* repositoriesblocklist.compute_blocklist:
|
|
* Accept external_tasks (ExternalRepoSetupTasks) instead of consuming
|
|
RepositoriesSetupTasks directly.
|
|
* Return only the set of blocklisted repoids instead of a tuple of
|
|
(blocklisted, enabled).
|
|
* Move RepositoriesFacts missing-message check to InputData.
|
|
* Update docstring to describe expected precedence order for
|
|
conflicting requests (implementation deferred to next commit).
|
|
* Remove misused FAILURE group from the CRB exclusion report.
|
|
|
|
Unit tests adjusted with assistance of AI.
|
|
|
|
Jira: RHEL-115867
|
|
Assisted-by: Claude Code (Opus 4.6)
|
|
Co-authored-by: Matej Matuska <mmatuska@redhat.com>
|
|
Signed-off-by: Petr Stodulka <pstodulk@redhat.com>
|
|
---
|
|
.../libraries/pes_event_parsing.py | 1 +
|
|
.../libraries/pes_events_scanner.py | 18 +--
|
|
.../libraries/repositoriesblocklist.py | 83 +++++++------
|
|
.../libraries/setuptargetrepos.py | 4 +-
|
|
.../libraries/targetcontentresolver.py | 93 ++++++++++++--
|
|
.../tests/test_pes_event_scanner.py | 12 +-
|
|
.../tests/test_repositoriesblocklist.py | 49 +++-----
|
|
.../tests/test_targetcontentresolver.py | 114 +++++++++++++++++-
|
|
8 files changed, 279 insertions(+), 95 deletions(-)
|
|
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_event_parsing.py b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_event_parsing.py
|
|
index f24dda68..febf8578 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_event_parsing.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_event_parsing.py
|
|
@@ -109,6 +109,7 @@ def get_pes_events(pes_json_directory, pes_json_filename):
|
|
reporting.Groups([reporting.Groups.SANITY, reporting.Groups.INHIBITOR]),
|
|
reporting.RelatedResource('file', os.path.join(pes_json_directory, pes_json_filename))
|
|
])
|
|
+
|
|
raise StopActorExecution()
|
|
|
|
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_events_scanner.py b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_events_scanner.py
|
|
index a03071ae..cbc99adf 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_events_scanner.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/pes_events_scanner.py
|
|
@@ -320,7 +320,7 @@ def report_skipped_packages(title, message, skipped_pkgs, remediation=None):
|
|
api.current_logger().info(summary)
|
|
|
|
|
|
-def remove_new_packages_from_blacklisted_repos(source_pkgs, target_pkgs, blacklisted_repoids=None):
|
|
+def remove_new_packages_from_blacklisted_repos(source_pkgs, target_pkgs, blacklisted_repoids):
|
|
"""
|
|
Remove newly installed packages from blacklisted repositories that were computed to be on the target system.
|
|
|
|
@@ -357,7 +357,7 @@ def get_enabled_repoids():
|
|
return enabled_repoids
|
|
|
|
|
|
-def get_pesid_to_repoid_map(target_pesids, repositories_map_msg):
|
|
+def get_pesid_to_repoid_map(target_pesids, repositories_map_msg, enabled_repoids):
|
|
"""
|
|
Get a dictionary mapping all PESID repositories to their corresponding repoid.
|
|
|
|
@@ -373,7 +373,6 @@ def get_pesid_to_repoid_map(target_pesids, repositories_map_msg):
|
|
# NOTE: We have to calculate expected target repositories like in the setuptargetrepos actor.
|
|
# It's planned to handle this in different a way in future...
|
|
|
|
- enabled_repoids = get_enabled_repoids()
|
|
default_channels = repomap_calc.get_default_repository_channels(repomap_handler, enabled_repoids)
|
|
repomap_handler.set_default_channels(default_channels)
|
|
|
|
@@ -433,7 +432,7 @@ def get_pesid_to_repoid_map(target_pesids, repositories_map_msg):
|
|
return repositories_mapping
|
|
|
|
|
|
-def replace_pesids_with_repoids_in_packages(packages, source_pkgs_repoids, repositories_map_msg):
|
|
+def replace_pesids_with_repoids_in_packages(packages, source_pkgs_repoids, repositories_map_msg, enabled_repoids):
|
|
"""Replace packages with PESID in their .repository field with ones that have repoid providing the package."""
|
|
# We want to map only PESIDs - if some package had no events, it will its repository set to source system repoid
|
|
packages_with_pesid = {pkg for pkg in packages if pkg.repository not in source_pkgs_repoids}
|
|
@@ -441,7 +440,7 @@ def replace_pesids_with_repoids_in_packages(packages, source_pkgs_repoids, repos
|
|
|
|
required_target_pesids = {pkg.repository for pkg in packages_with_pesid}
|
|
|
|
- pesid_to_target_repoid_map = get_pesid_to_repoid_map(required_target_pesids, repositories_map_msg)
|
|
+ pesid_to_target_repoid_map = get_pesid_to_repoid_map(required_target_pesids, repositories_map_msg, enabled_repoids)
|
|
|
|
packages_with_unknown_target_repoid = {
|
|
pkg
|
|
@@ -555,7 +554,7 @@ def include_instructions_from_transaction_configuration(rpm_tasks, transaction_c
|
|
modules_to_reset=modules_to_reset)
|
|
|
|
|
|
-def scan_pes_events(repositories_map_msg, blacklisted_repoids=None):
|
|
+def scan_pes_events(repositories_map_msg, blacklisted_repoids, enabled_repoids):
|
|
"""
|
|
Process PES events and compute RPM transaction tasks.
|
|
|
|
@@ -586,7 +585,12 @@ def scan_pes_events(repositories_map_msg, blacklisted_repoids=None):
|
|
events, releases)
|
|
|
|
# Packages coming out of the events have PESID as their repository, however, we need real repoid
|
|
- target_pkgs = replace_pesids_with_repoids_in_packages(target_pkgs, repoids_of_source_pkgs, repositories_map_msg)
|
|
+ target_pkgs = replace_pesids_with_repoids_in_packages(
|
|
+ target_pkgs,
|
|
+ repoids_of_source_pkgs,
|
|
+ repositories_map_msg,
|
|
+ enabled_repoids
|
|
+ )
|
|
|
|
# Apply the desired repository blacklisting
|
|
blacklisted_repoids, target_pkgs = remove_new_packages_from_blacklisted_repos(pkgs_to_begin_computation_with,
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/repositoriesblocklist.py b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/repositoriesblocklist.py
|
|
index 48656187..216bea52 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/repositoriesblocklist.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/repositoriesblocklist.py
|
|
@@ -1,14 +1,7 @@
|
|
from leapp import reporting
|
|
-from leapp.exceptions import StopActorExecutionError
|
|
from leapp.libraries.common.config.version import get_source_major_version, get_target_major_version
|
|
from leapp.libraries.stdlib import api
|
|
-from leapp.models import (
|
|
- CustomTargetRepository,
|
|
- RepositoriesBlacklisted,
|
|
- RepositoriesBlocklisted,
|
|
- RepositoriesFacts,
|
|
- RepositoriesSetupTasks
|
|
-)
|
|
+from leapp.models import CustomTargetRepository, RepositoriesBlacklisted, RepositoriesBlocklisted, RepositoriesFacts
|
|
from leapp.utils.deprecation import suppress_deprecation
|
|
|
|
# {OS_MAJOR_VERSION: PESID}
|
|
@@ -47,7 +40,6 @@ def _report_excluded_repos(repos):
|
|
),
|
|
reporting.Severity(reporting.Severity.INFO),
|
|
reporting.Groups([reporting.Groups.REPOSITORY]),
|
|
- reporting.Groups([reporting.Groups.FAILURE]),
|
|
reporting.Remediation(
|
|
hint=(
|
|
'If some of excluded repositories are still required to be used'
|
|
@@ -136,7 +128,7 @@ def get_blocklisted_repoids(repo_mapping):
|
|
|
|
Conditions to exclude:
|
|
- there are not such repositories already enabled on the source system
|
|
- (e.g. "Optional" repositories)
|
|
+ (e.g. "CRB" repositories)
|
|
- such repositories are not required for the upgrade explicitly by the user
|
|
(e.g. via the --enablerepo option or via the /etc/leapp/files/leapp_upgrade_repositories.repo file)
|
|
|
|
@@ -146,13 +138,9 @@ def get_blocklisted_repoids(repo_mapping):
|
|
:returns: Set of repoids to blocklist on the target system.
|
|
:rtype: set
|
|
"""
|
|
+ # TODO(pstodulk): replace it by enabled_repos
|
|
repos_facts = next(api.consume(RepositoriesFacts), None)
|
|
|
|
- if not repos_facts:
|
|
- raise StopActorExecutionError('Actor didn\'t receive required messages: {0}'.format(
|
|
- 'RepositoriesFacts'
|
|
- ))
|
|
-
|
|
if not _are_optional_repos_disabled(repo_mapping, repos_facts):
|
|
# nothing to do - an optional repository is enabled
|
|
return set()
|
|
@@ -172,34 +160,51 @@ def get_blocklisted_repoids(repo_mapping):
|
|
|
|
|
|
@suppress_deprecation(RepositoriesBlacklisted)
|
|
-def compute_blocklist(repo_mapping):
|
|
+def compute_blocklist(repo_mapping, external_tasks):
|
|
"""
|
|
- Compute the full blocklist and extract external repoid requests.
|
|
+ Create the blocklist of repositories that should be blocked during the upgrade.
|
|
+
|
|
+ The content in the CRB repository is considered as unsupported. Hence
|
|
+ we do not want to enable such a repository by default on the target system
|
|
+ unless
|
|
+ * it is explicitely requested, or
|
|
+ * it is already enabled on the source system
|
|
+ This is important as some functionality originally supported on the source
|
|
+ source system can be moved to the CRB repository on the target system.
|
|
+ In such a case, we do not want to install automatically such a content
|
|
+ that lost a support.
|
|
+
|
|
+ For this reason, compute the list of repositories that should be blocklisted
|
|
+ and take into account also external requests.
|
|
+ In case that multiple requests are raised for a specific repository, the
|
|
+ priority order is following:
|
|
+ internal block rq > external enable rq > external block rq > external custom rq
|
|
+
|
|
+ The external custom request has the highest priority as this comes from
|
|
+ the system configuration or from the `--enablerepo` option used when executing
|
|
+ leapp - so it's understood as decision made by user explicitely for
|
|
+ the in-place upgrade purpose. Currently there is no explicit custom
|
|
+ configuration to disable a repo.
|
|
+
|
|
+ Once the list is calculated, the RepositoriesBlocklisted and
|
|
+ RepositoriesBlacklisted messages are produced.
|
|
|
|
- Merges the internally determined blocklist (unsupported repos like CRB)
|
|
- with external blocklist requests from ``RepositoriesSetupTasks.to_block``.
|
|
- When the resulting blocklist is non-empty, produces both
|
|
- ``RepositoriesBlocklisted`` and the deprecated ``RepositoriesBlacklisted``
|
|
- messages.
|
|
+ :param repo_mapping: RepositoriesMapping message.
|
|
+ :type repo_mapping: RepositoriesMapping
|
|
+ :param external_tasks: External repositories tasks represented by object with following fields:
|
|
|
|
- Also extracts ``to_enable`` repoids from the same external tasks so that
|
|
- the caller does not need to re-consume the messages.
|
|
+ * ``to_enable`` - repositories that should be enabled
|
|
+ * ``to_block`` - repositories that should be blocklisted
|
|
+ * ``custom`` - repositories explicitely requested by user to be used during the upgrade
|
|
|
|
- :param repo_mapping: RepositoriesMapping message.
|
|
- :returns: Tuple of (full_blocklist_repoids, external_repoids_requests).
|
|
- :rtype: tuple[set, set]
|
|
+ :type external_tasks: targetcontentresolver.ExternalRepoSetupTasks
|
|
+ :returns: List of blocklisted repositories
|
|
+ :rtype: List[str]
|
|
"""
|
|
internal_blocklist = get_blocklisted_repoids(repo_mapping)
|
|
+ full_blocklist = internal_blocklist.union(external_tasks.to_block)
|
|
+ if full_blocklist:
|
|
+ api.produce(RepositoriesBlocklisted(repoids=sorted(full_blocklist)))
|
|
+ api.produce(RepositoriesBlacklisted(repoids=sorted(full_blocklist)))
|
|
|
|
- external_blocklist_repoids = set()
|
|
- external_repoids_requests = set()
|
|
- for task in api.consume(RepositoriesSetupTasks):
|
|
- external_blocklist_repoids.update(task.to_block)
|
|
- external_repoids_requests.update(task.to_enable)
|
|
-
|
|
- full_blocklist_repoids = internal_blocklist.union(external_blocklist_repoids)
|
|
- if full_blocklist_repoids:
|
|
- api.produce(RepositoriesBlocklisted(repoids=sorted(full_blocklist_repoids)))
|
|
- api.produce(RepositoriesBlacklisted(repoids=sorted(full_blocklist_repoids)))
|
|
-
|
|
- return full_blocklist_repoids, external_repoids_requests
|
|
+ return full_blocklist
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/setuptargetrepos.py b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/setuptargetrepos.py
|
|
index b9e5a1c4..d6bfb91d 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/setuptargetrepos.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/setuptargetrepos.py
|
|
@@ -1,5 +1,5 @@
|
|
from leapp.libraries.actor import repomap_calc
|
|
-from leapp.libraries.actor.pes_events_scanner import get_enabled_repoids as _get_enabled_repoids
|
|
+from leapp.libraries.actor.pes_events_scanner import get_enabled_repoids
|
|
from leapp.libraries.common.config import get_source_distro_id, get_target_distro_id
|
|
from leapp.libraries.common.config.version import get_source_major_version, get_source_version
|
|
from leapp.libraries.stdlib import api
|
|
@@ -70,7 +70,7 @@ def setup_target_repos(repositories_map_msg, pes_requested_repoids=None,
|
|
"""
|
|
# Load relevant data from messages
|
|
used_repoids_dict = _get_used_repo_dict()
|
|
- enabled_repoids = _get_enabled_repoids()
|
|
+ enabled_repoids = get_enabled_repoids()
|
|
excluded_repoids = blacklisted_repoids if blacklisted_repoids is not None else set()
|
|
custom_repos = _get_custom_target_repos()
|
|
repoids_from_installed_packages = _get_repoids_from_installed_packages()
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/targetcontentresolver.py b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/targetcontentresolver.py
|
|
index 4db9003d..c1b46916 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/libraries/targetcontentresolver.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/libraries/targetcontentresolver.py
|
|
@@ -1,5 +1,83 @@
|
|
+from collections import namedtuple
|
|
+
|
|
+from leapp.exceptions import StopActorExecutionError
|
|
from leapp.libraries.actor import pes_events_scanner, repositoriesblocklist, setuptargetrepos
|
|
from leapp.libraries.actor.repomap_loader import scan_repositories
|
|
+from leapp.libraries.stdlib import api
|
|
+from leapp.models import CustomTargetRepository, RepositoriesFacts, RepositoriesSetupTasks
|
|
+
|
|
+ExternalRepoSetupTasks = namedtuple('ExternalRepoSetupTasks', ('to_enable', 'to_block', 'custom'))
|
|
+
|
|
+
|
|
+class InputData():
|
|
+ """
|
|
+ Provide data from consumed messages
|
|
+ """
|
|
+
|
|
+ def __init__(self):
|
|
+ self._missing_messages = []
|
|
+
|
|
+ self._get_enabled_repoids()
|
|
+ self._get_external_reposetup_tasks()
|
|
+
|
|
+ if self._missing_messages:
|
|
+ # NOTE(pstodulk): This would be an invalid object,
|
|
+ # so let's end the story here.
|
|
+ raise StopActorExecutionError(
|
|
+ message='Cannot calculate target content requirements',
|
|
+ details={
|
|
+ 'details': 'Some of required leapp messages are missing.',
|
|
+ 'missing': [i.__name__ for i in self._missing_messages]
|
|
+ }
|
|
+ )
|
|
+
|
|
+ def _treat_consume_msg(self, model):
|
|
+ msgs = api.consume(model)
|
|
+ msg = next(msgs, None)
|
|
+ if list(msgs):
|
|
+ w_msg = f'Unexpectedly received more than one {model.__name__} message.'
|
|
+ api.current_logger().warning(w_msg)
|
|
+ if not msg:
|
|
+ self._missing_messages.append(model)
|
|
+ return None
|
|
+ return msg
|
|
+
|
|
+ def _get_external_reposetup_tasks(self):
|
|
+ """
|
|
+ Collect repository related tasks from other actors (or configs in future).
|
|
+
|
|
+ This includes consumption of RepositoriesSetupTasks and CustomTargetRepository.
|
|
+ The second one should be understood as task as well based on the definition,
|
|
+ even when the name does not suggest it's actually task as well.
|
|
+
|
|
+ Return named tuple with `to_enable`, `to_block`, and `custom` fields.
|
|
+ The last one is based on reposids in CustomTargetRepository messages.
|
|
+ """
|
|
+ self.external_tasks = ExternalRepoSetupTasks(set(), set(), set())
|
|
+ for task in api.consume(RepositoriesSetupTasks):
|
|
+ self.external_tasks.to_block.update(task.to_block)
|
|
+ self.external_tasks.to_enable.update(task.to_enable)
|
|
+
|
|
+ self.external_tasks.custom.update({repo.repoid for repo in api.consume(CustomTargetRepository)})
|
|
+
|
|
+ def _get_enabled_repoids(self):
|
|
+ """
|
|
+ Return set of repository IDs enabled on the source system.
|
|
+
|
|
+ The information is consumed from the RepositoriesFacts message.
|
|
+
|
|
+ :return: Set of all enabled repository IDs present on the source system.
|
|
+ :rtype: Set[str]
|
|
+ """
|
|
+ self.enabled_repoids = set()
|
|
+ repos_facts = self._treat_consume_msg(RepositoriesFacts)
|
|
+ if repos_facts is None:
|
|
+ return
|
|
+
|
|
+ for repo_file in repos_facts.repositories:
|
|
+ for repo in repo_file.data:
|
|
+ if repo.enabled:
|
|
+ self.enabled_repoids.add(repo.repoid)
|
|
|
|
|
|
def process():
|
|
@@ -23,17 +101,18 @@ def process():
|
|
source repos, installed package repos, and custom/blacklisted repo
|
|
configuration.
|
|
"""
|
|
+ indata = InputData()
|
|
repositories_map_msg = scan_repositories()
|
|
-
|
|
- full_blocklist_repoids, external_repoids_requests = (
|
|
- repositoriesblocklist.compute_blocklist(repositories_map_msg)
|
|
+ blocklisted_repoids = repositoriesblocklist.compute_blocklist(repositories_map_msg, indata.external_tasks)
|
|
+ pes_requested_repoids = pes_events_scanner.scan_pes_events(
|
|
+ repositories_map_msg,
|
|
+ blocklisted_repoids,
|
|
+ indata.enabled_repoids
|
|
)
|
|
|
|
- pes_requested_repoids = pes_events_scanner.scan_pes_events(repositories_map_msg, full_blocklist_repoids)
|
|
-
|
|
setuptargetrepos.setup_target_repos(
|
|
repositories_map_msg,
|
|
pes_requested_repoids=pes_requested_repoids,
|
|
- blacklisted_repoids=full_blocklist_repoids,
|
|
- external_repoids_requests=external_repoids_requests,
|
|
+ blacklisted_repoids=blocklisted_repoids,
|
|
+ external_repoids_requests=indata.external_tasks.to_enable,
|
|
)
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_pes_event_scanner.py b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_pes_event_scanner.py
|
|
index f872a947..a1f3a6e1 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_pes_event_scanner.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_pes_event_scanner.py
|
|
@@ -270,7 +270,7 @@ def test_actor_performs(monkeypatch):
|
|
monkeypatch.setattr(api, 'produce', produced_messages)
|
|
monkeypatch.setattr(reporting, 'create_report', created_report)
|
|
|
|
- pes_events_scanner.scan_pes_events(repositories_mapping)
|
|
+ pes_events_scanner.scan_pes_events(repositories_mapping, set(), {'rhel8-repo'})
|
|
|
|
assert produced_messages.called
|
|
|
|
@@ -341,13 +341,13 @@ def test_blacklisted_repoid_is_not_produced(monkeypatch):
|
|
monkeypatch.setattr(pes_events_scanner, 'get_pes_events', lambda folder, filename: events)
|
|
monkeypatch.setattr(pes_events_scanner, 'apply_transaction_configuration', lambda pkgs, transaction_cfg: pkgs)
|
|
monkeypatch.setattr(pes_events_scanner, 'replace_pesids_with_repoids_in_packages',
|
|
- lambda pkgs, src_pkgs_repoids, repo_map_msg=None: pkgs)
|
|
+ lambda pkgs, src_pkgs_repoids, repo_map_msg, enabled_repoids: pkgs)
|
|
|
|
monkeypatch.setattr(api, 'current_actor', CurrentActorMocked())
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
monkeypatch.setattr(reporting, 'create_report', create_report_mocked())
|
|
|
|
- setup_tasks = pes_events_scanner.scan_pes_events(None, blacklisted_repoids={'blacklisted-rhel9'})
|
|
+ setup_tasks = pes_events_scanner.scan_pes_events(None, {'blacklisted-rhel9'}, set())
|
|
|
|
assert not reporting.create_report.called
|
|
|
|
@@ -497,10 +497,10 @@ def test_transaction_configuration_is_applied(monkeypatch):
|
|
monkeypatch.setattr(pes_events_scanner, 'remove_undesired_events', lambda events, releases: events)
|
|
monkeypatch.setattr(pes_events_scanner, '_get_enabled_modules', lambda *args: [])
|
|
monkeypatch.setattr(pes_events_scanner, 'replace_pesids_with_repoids_in_packages',
|
|
- lambda target_pkgs, repoids_of_source_pkgs, repo_map_msg=None: target_pkgs)
|
|
+ lambda target_pkgs, repoids_of_source_pkgs, repo_map_msg, enabled_repoids: target_pkgs)
|
|
monkeypatch.setattr(pes_events_scanner,
|
|
'remove_new_packages_from_blacklisted_repos',
|
|
- lambda source_pkgs, target_pkgs, bl=None: (set(), target_pkgs))
|
|
+ lambda source_pkgs, target_pkgs, bl: (set(), target_pkgs))
|
|
|
|
msgs = [
|
|
RpmTransactionTasks(to_remove=['pkg-not-in-events']),
|
|
@@ -513,7 +513,7 @@ def test_transaction_configuration_is_applied(monkeypatch):
|
|
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- setup_tasks = pes_events_scanner.scan_pes_events(None)
|
|
+ setup_tasks = pes_events_scanner.scan_pes_events(None, set(), set())
|
|
|
|
assert api.produce.called == 1
|
|
assert setup_tasks is not None
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_repositoriesblocklist.py b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_repositoriesblocklist.py
|
|
index 5ffa0990..fc657824 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_repositoriesblocklist.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_repositoriesblocklist.py
|
|
@@ -2,6 +2,7 @@ import pytest
|
|
|
|
from leapp import reporting
|
|
from leapp.libraries.actor import repositoriesblocklist
|
|
+from leapp.libraries.actor.targetcontentresolver import ExternalRepoSetupTasks
|
|
from leapp.libraries.common.testutils import create_report_mocked, CurrentActorMocked, produce_mocked
|
|
from leapp.libraries.stdlib import api
|
|
from leapp.models import (
|
|
@@ -12,7 +13,6 @@ from leapp.models import (
|
|
RepositoriesBlocklisted,
|
|
RepositoriesFacts,
|
|
RepositoriesMapping,
|
|
- RepositoriesSetupTasks,
|
|
RepositoryData,
|
|
RepositoryFile
|
|
)
|
|
@@ -228,21 +228,19 @@ def _get_produced_blacklisted():
|
|
def test_compute_blocklist_merges_internal_and_external(monkeypatch):
|
|
"""
|
|
compute_blocklist merges internal blocklist with external
|
|
- RepositoriesSetupTasks.to_block and produces both messages.
|
|
+ ExternalRepoSetupTasks.to_block and produces both messages.
|
|
"""
|
|
- external_tasks = [
|
|
- RepositoriesSetupTasks(to_enable=['some-repo'], to_block=['ext-blocked-1', 'ext-blocked-2']),
|
|
- ]
|
|
+ external_tasks = ExternalRepoSetupTasks(
|
|
+ to_enable={'some-repo'}, to_block={'ext-blocked-1', 'ext-blocked-2'}, custom=set()
|
|
+ )
|
|
monkeypatch.setattr(
|
|
repositoriesblocklist, 'get_blocklisted_repoids', lambda _repo_map: {'crb-repo-1'}
|
|
)
|
|
- monkeypatch.setattr(api, 'consume', lambda _model: iter(external_tasks))
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- full_blocklist, external_repoids = repositoriesblocklist.compute_blocklist(None)
|
|
+ full_blocklist = repositoriesblocklist.compute_blocklist(None, external_tasks)
|
|
|
|
assert full_blocklist == {'crb-repo-1', 'ext-blocked-1', 'ext-blocked-2'}
|
|
- assert external_repoids == {'some-repo'}
|
|
|
|
blocklisted_msgs = _get_produced_blocklisted()
|
|
assert len(blocklisted_msgs) == 1
|
|
@@ -255,16 +253,15 @@ def test_compute_blocklist_merges_internal_and_external(monkeypatch):
|
|
|
|
def test_compute_blocklist_no_messages_when_empty(monkeypatch):
|
|
"""No blocklist messages are produced when blocklist is empty."""
|
|
+ external_tasks = ExternalRepoSetupTasks(to_enable=set(), to_block=set(), custom=set())
|
|
monkeypatch.setattr(
|
|
repositoriesblocklist, 'get_blocklisted_repoids', lambda _repo_map: set()
|
|
)
|
|
- monkeypatch.setattr(api, 'consume', lambda _model: iter([]))
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- full_blocklist, external_repoids = repositoriesblocklist.compute_blocklist(None)
|
|
+ full_blocklist = repositoriesblocklist.compute_blocklist(None, external_tasks)
|
|
|
|
assert full_blocklist == set()
|
|
- assert external_repoids == set()
|
|
assert len(_get_produced_blocklisted()) == 0
|
|
assert len(_get_produced_blacklisted()) == 0
|
|
|
|
@@ -272,53 +269,47 @@ def test_compute_blocklist_no_messages_when_empty(monkeypatch):
|
|
@suppress_deprecation(RepositoriesBlacklisted)
|
|
def test_compute_blocklist_only_internal(monkeypatch):
|
|
"""Only internal blocklist when no external tasks are present."""
|
|
+ external_tasks = ExternalRepoSetupTasks(to_enable=set(), to_block=set(), custom=set())
|
|
monkeypatch.setattr(
|
|
repositoriesblocklist, 'get_blocklisted_repoids', lambda _repo_map: {'crb-repo-1'}
|
|
)
|
|
- monkeypatch.setattr(api, 'consume', lambda _model: iter([]))
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- full_blocklist, external_repoids = repositoriesblocklist.compute_blocklist(None)
|
|
+ full_blocklist = repositoriesblocklist.compute_blocklist(None, external_tasks)
|
|
|
|
assert full_blocklist == {'crb-repo-1'}
|
|
- assert external_repoids == set()
|
|
assert len(_get_produced_blocklisted()) == 1
|
|
|
|
|
|
-def test_compute_blocklist_multiple_external_tasks(monkeypatch):
|
|
- """Blocklist and to_enable from multiple external tasks are aggregated."""
|
|
- external_tasks = [
|
|
- RepositoriesSetupTasks(to_block=['ext-1'], to_enable=['repo-a']),
|
|
- RepositoriesSetupTasks(to_block=['ext-2', 'ext-3'], to_enable=['repo-b']),
|
|
- ]
|
|
+def test_compute_blocklist_aggregated_external_tasks(monkeypatch):
|
|
+ """Blocklist from pre-aggregated external tasks is merged with internal."""
|
|
+ external_tasks = ExternalRepoSetupTasks(
|
|
+ to_enable={'repo-a', 'repo-b'}, to_block={'ext-1', 'ext-2', 'ext-3'}, custom=set()
|
|
+ )
|
|
monkeypatch.setattr(
|
|
repositoriesblocklist, 'get_blocklisted_repoids', lambda _repo_map: set()
|
|
)
|
|
- monkeypatch.setattr(api, 'consume', lambda _model: iter(external_tasks))
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- full_blocklist, external_repoids = repositoriesblocklist.compute_blocklist(None)
|
|
+ full_blocklist = repositoriesblocklist.compute_blocklist(None, external_tasks)
|
|
|
|
assert full_blocklist == {'ext-1', 'ext-2', 'ext-3'}
|
|
- assert external_repoids == {'repo-a', 'repo-b'}
|
|
|
|
|
|
@suppress_deprecation(RepositoriesBlacklisted)
|
|
def test_compute_blocklist_overlapping_internal_and_external(monkeypatch):
|
|
"""Overlapping repoids between internal and external blocklists are deduplicated."""
|
|
- external_tasks = [
|
|
- RepositoriesSetupTasks(to_block=['shared-repo', 'ext-only']),
|
|
- ]
|
|
+ external_tasks = ExternalRepoSetupTasks(
|
|
+ to_enable=set(), to_block={'shared-repo', 'ext-only'}, custom=set()
|
|
+ )
|
|
monkeypatch.setattr(
|
|
repositoriesblocklist, 'get_blocklisted_repoids', lambda _repo_map: {'shared-repo', 'internal-only'}
|
|
)
|
|
- monkeypatch.setattr(api, 'consume', lambda _model: iter(external_tasks))
|
|
monkeypatch.setattr(api, 'produce', produce_mocked())
|
|
|
|
- full_blocklist, external_repoids = repositoriesblocklist.compute_blocklist(None)
|
|
+ full_blocklist = repositoriesblocklist.compute_blocklist(None, external_tasks)
|
|
|
|
assert full_blocklist == {'shared-repo', 'internal-only', 'ext-only'}
|
|
- assert external_repoids == set()
|
|
|
|
blocklisted_msgs = _get_produced_blocklisted()
|
|
assert len(blocklisted_msgs) == 1
|
|
diff --git a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_targetcontentresolver.py b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_targetcontentresolver.py
|
|
index 4e425da6..98f19ea8 100644
|
|
--- a/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_targetcontentresolver.py
|
|
+++ b/repos/system_upgrade/common/actors/targetcontentresolver/tests/test_targetcontentresolver.py
|
|
@@ -1,4 +1,17 @@
|
|
+import pytest
|
|
+
|
|
+from leapp.exceptions import StopActorExecutionError
|
|
from leapp.libraries.actor import targetcontentresolver
|
|
+from leapp.libraries.actor.targetcontentresolver import ExternalRepoSetupTasks, InputData
|
|
+from leapp.libraries.common.testutils import CurrentActorMocked, logger_mocked
|
|
+from leapp.libraries.stdlib import api
|
|
+from leapp.models import (
|
|
+ CustomTargetRepository,
|
|
+ RepositoriesFacts,
|
|
+ RepositoriesSetupTasks,
|
|
+ RepositoryData,
|
|
+ RepositoryFile
|
|
+)
|
|
|
|
|
|
def test_process_orchestration(monkeypatch):
|
|
@@ -10,22 +23,33 @@ def test_process_orchestration(monkeypatch):
|
|
|
|
fake_repo_map = object()
|
|
fake_blocklist = frozenset({'blocked-repo'})
|
|
- fake_external_repoids = frozenset({'ext-repo'})
|
|
+ fake_external_tasks = ExternalRepoSetupTasks(
|
|
+ to_enable=frozenset({'ext-repo'}), to_block=frozenset(), custom=frozenset()
|
|
+ )
|
|
+ fake_enabled_repoids = frozenset({'enabled-repo'})
|
|
fake_pes_repoids = frozenset({'pes-repo'})
|
|
|
|
+ class FakeInputData:
|
|
+ def __init__(self):
|
|
+ call_log.append('InputData')
|
|
+ self.external_tasks = fake_external_tasks
|
|
+ self.enabled_repoids = fake_enabled_repoids
|
|
+
|
|
def mock_scan_repositories():
|
|
call_log.append('scan_repositories')
|
|
return fake_repo_map
|
|
|
|
- def mock_compute_blocklist(repo_mapping):
|
|
+ def mock_compute_blocklist(repo_mapping, external_tasks):
|
|
call_log.append('compute_blocklist')
|
|
assert repo_mapping is fake_repo_map
|
|
- return fake_blocklist, fake_external_repoids
|
|
+ assert external_tasks is fake_external_tasks
|
|
+ return fake_blocklist
|
|
|
|
- def mock_scan_pes_events(repo_mapping, blacklisted_repoids):
|
|
+ def mock_scan_pes_events(repo_mapping, blacklisted_repoids, enabled_repoids):
|
|
call_log.append('scan_pes_events')
|
|
assert repo_mapping is fake_repo_map
|
|
assert blacklisted_repoids is fake_blocklist
|
|
+ assert enabled_repoids is fake_enabled_repoids
|
|
return fake_pes_repoids
|
|
|
|
def mock_setup_target_repos(repo_mapping, pes_requested_repoids=None,
|
|
@@ -34,8 +58,12 @@ def test_process_orchestration(monkeypatch):
|
|
assert repo_mapping is fake_repo_map
|
|
assert pes_requested_repoids is fake_pes_repoids
|
|
assert blacklisted_repoids is fake_blocklist
|
|
- assert external_repoids_requests is fake_external_repoids
|
|
+ assert external_repoids_requests is fake_external_tasks.to_enable
|
|
|
|
+ monkeypatch.setattr(
|
|
+ 'leapp.libraries.actor.targetcontentresolver.InputData',
|
|
+ FakeInputData,
|
|
+ )
|
|
monkeypatch.setattr(
|
|
'leapp.libraries.actor.targetcontentresolver.scan_repositories',
|
|
mock_scan_repositories,
|
|
@@ -56,8 +84,84 @@ def test_process_orchestration(monkeypatch):
|
|
targetcontentresolver.process()
|
|
|
|
assert call_log == [
|
|
+ 'InputData',
|
|
'scan_repositories',
|
|
'compute_blocklist',
|
|
'scan_pes_events',
|
|
'setup_target_repos',
|
|
]
|
|
+
|
|
+
|
|
+def _make_repo_facts(repoids_enabled=None, repoids_disabled=None):
|
|
+ repos_data = []
|
|
+ for repoid in (repoids_enabled or []):
|
|
+ repos_data.append(RepositoryData(repoid=repoid, name=repoid, enabled=True))
|
|
+ for repoid in (repoids_disabled or []):
|
|
+ repos_data.append(RepositoryData(repoid=repoid, name=repoid, enabled=False))
|
|
+ return RepositoriesFacts(
|
|
+ repositories=[RepositoryFile(file='/etc/yum.repos.d/test.repo', data=repos_data)]
|
|
+ )
|
|
+
|
|
+
|
|
+def test_inputdata_collects_enabled_repoids(monkeypatch):
|
|
+ repo_facts = _make_repo_facts(
|
|
+ repoids_enabled=['repo-a', 'repo-b'],
|
|
+ repoids_disabled=['repo-c'],
|
|
+ )
|
|
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(msgs=[repo_facts]))
|
|
+
|
|
+ indata = InputData()
|
|
+
|
|
+ assert indata.enabled_repoids == {'repo-a', 'repo-b'}
|
|
+
|
|
+
|
|
+def test_inputdata_raises_when_repofacts_missing(monkeypatch):
|
|
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(msgs=[]))
|
|
+
|
|
+ with pytest.raises(StopActorExecutionError):
|
|
+ InputData()
|
|
+
|
|
+
|
|
+def test_inputdata_aggregates_external_tasks(monkeypatch):
|
|
+ repo_facts = _make_repo_facts(repoids_enabled=['repo-a'])
|
|
+ setup_tasks_1 = RepositoriesSetupTasks(to_enable=['en-1'], to_block=['bl-1'])
|
|
+ setup_tasks_2 = RepositoriesSetupTasks(to_enable=['en-2'], to_block=['bl-2', 'bl-3'])
|
|
+ custom_1 = CustomTargetRepository(repoid='custom-1')
|
|
+ custom_2 = CustomTargetRepository(repoid='custom-2')
|
|
+
|
|
+ monkeypatch.setattr(
|
|
+ api, 'current_actor',
|
|
+ CurrentActorMocked(msgs=[repo_facts, setup_tasks_1, setup_tasks_2, custom_1, custom_2])
|
|
+ )
|
|
+
|
|
+ indata = InputData()
|
|
+
|
|
+ assert indata.external_tasks.to_enable == {'en-1', 'en-2'}
|
|
+ assert indata.external_tasks.to_block == {'bl-1', 'bl-2', 'bl-3'}
|
|
+ assert indata.external_tasks.custom == {'custom-1', 'custom-2'}
|
|
+
|
|
+
|
|
+def test_inputdata_empty_external_tasks(monkeypatch):
|
|
+ repo_facts = _make_repo_facts(repoids_enabled=['repo-a'])
|
|
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(msgs=[repo_facts]))
|
|
+
|
|
+ indata = InputData()
|
|
+
|
|
+ assert indata.external_tasks.to_enable == set()
|
|
+ assert indata.external_tasks.to_block == set()
|
|
+ assert indata.external_tasks.custom == set()
|
|
+
|
|
+
|
|
+def test_inputdata_warns_on_duplicate_repofacts(monkeypatch):
|
|
+ repo_facts_1 = _make_repo_facts(repoids_enabled=['repo-a'])
|
|
+ repo_facts_2 = _make_repo_facts(repoids_enabled=['repo-b'])
|
|
+ monkeypatch.setattr(
|
|
+ api, 'current_actor',
|
|
+ CurrentActorMocked(msgs=[repo_facts_1, repo_facts_2])
|
|
+ )
|
|
+ monkeypatch.setattr(api, 'current_logger', logger_mocked())
|
|
+
|
|
+ indata = InputData()
|
|
+
|
|
+ assert indata.enabled_repoids == {'repo-a'}
|
|
+ assert any('more than one' in msg for msg in api.current_logger.warnmsg)
|
|
--
|
|
2.54.0
|
|
|