ykohut/leapp-repository
1
0
Fork 0
forked from rpms/leapp-repository
Code Pull Requests Activity

RHEL 8.10: CTC2 candidate - 0

Browse Source 
- Add detection of possible usage of OpenSSL IBMCA engine on IBM Z machines
- Add detection of modified /etc/pki/tls/openssl.cnf file
- Update the leapp upgrade data files
- Fix handling of symlinks under /etc/pki with relative paths specified
- Report custom actors and modifications of the upgrade tooling
- Requires xfsprogs and e2fsprogs to ensure that Ext4 and XFS tools are installed
- Bump leapp-repository-dependencies to 10
- Resolves: RHEL-1774, RHEL-16729
This commit is contained in:
Petr Stodulka 2024-01-12 20:45:10 +01:00
parent 75c9028095
commit e5599cfda4
23 changed files with 279952 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -3,3 +3,4 @@ SOURCES/leapp-repository-0.18.0.tar.gz
/deps-pkgs-9.tar.gz
/leapp-repository-0.18.0.tar.gz
/leapp-repository-0.19.0.tar.gz
/deps-pkgs-10.tar.gz

38
0042-BZ-2250254-force-removal-of-tomcat-during-the-upgrad.patch Normal file
View File

@ -0,0 +1,38 @@
From 28a5cc0d49451592f5184c25d155f5e7be81f17e Mon Sep 17 00:00:00 2001
From: Evgeni Golov <evgeni@golov.de>
Date: Mon, 20 Nov 2023 14:35:03 +0100
Subject: [PATCH 42/60] BZ#2250254 - force removal of tomcat during the upgrade
We need pki-servlet-engine, which we depend on, but tomcat conflicts
with.
---
.../el7toel8/actors/satellite_upgrade_facts/actor.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/actor.py b/repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/actor.py
index 01e63465..3cd9d9da 100644
--- a/repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/actor.py
+++ b/repos/system_upgrade/el7toel8/actors/satellite_upgrade_facts/actor.py
@@ -42,6 +42,7 @@ class SatelliteUpgradeFacts(Actor):
postgresql_contrib = has_package(InstalledRPM, 'rh-postgresql12-postgresql-contrib')
postgresql_evr = has_package(InstalledRPM, 'rh-postgresql12-postgresql-evr')
+ # SCL-related packages
to_remove = ['tfm-runtime', 'tfm-pulpcore-runtime', 'rh-redis5-runtime', 'rh-ruby27-runtime',
'rh-python38-runtime']
to_install = ['rubygem-foreman_maintain']
@@ -54,6 +55,11 @@ class SatelliteUpgradeFacts(Actor):
# enable modules that are needed for Pulpcore
modules_to_enable.append(Module(name='python38', stream='3.8'))
to_install.append('katello')
+ # Force removal of tomcat
+ # PES data indicates tomcat.el7 can be upgraded to tomcat.el8 since EL 8.8,
+ # but we need pki-servlet-engine from the module instead which will be pulled in via normal
+ # package dependencies
+ to_remove.extend(['tomcat', 'tomcat-lib'])
if has_package(InstalledRPM, 'rh-redis5-redis'):
modules_to_enable.append(Module(name='redis', stream='5'))
--
2.43.0

64
0043-Add-79to88-and-79to89-aws-upgrade-paths.patch Normal file
View File

@ -0,0 +1,64 @@
From de4d8cb60e05ffe7d2ce90282b1884a7d345461c Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Tue, 14 Nov 2023 11:57:58 +0100
Subject: [PATCH 43/60] Add 79to88 and 79to89 aws upgrade paths
Thanks to the detailed downstream review by mmoran
I have realised that upstream upgrade paths have to
be revised and updated as well.
Also change identifier for the dot notation in the upgrade
paths.
---
.packit.yaml | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/.packit.yaml b/.packit.yaml
index 2e606a40..a307cc75 100644
--- a/.packit.yaml
+++ b/.packit.yaml
@@ -123,7 +123,7 @@ jobs:
targets:
epel-7-x86_64:
distros: [RHEL-7.9-rhui]
- identifier: sanity-7to8-aws-e2e
+ identifier: sanity-7.9to8.6-aws-e2e
# NOTE(ivasilev) Unfortunately to use yaml templates we need to rewrite the whole tf_extra_params dict
# to use plan_filter (can't just specify one section test.tmt.plan_filter, need to specify environments.* as well)
tf_extra_params:
@@ -145,6 +145,24 @@ jobs:
RHUI: "aws"
LEAPPDATA_BRANCH: "upstream"
+- &sanity-79to88-aws
+ <<: *sanity-79to86-aws
+ identifier: sanity-7.9to8.8-aws-e2e
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.8"
+ RHUI: "aws"
+ LEAPPDATA_BRANCH: "upstream"
+
+- &sanity-79to89-aws
+ <<: *sanity-79to86-aws
+ identifier: sanity-7.9to8.9-aws-e2e
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.9"
+ RHUI: "aws"
+ LEAPPDATA_BRANCH: "upstream"
+
# On-demand minimal beaker tests
- &beaker-minimal-79to86
<<: *sanity-79to86
@@ -487,7 +505,7 @@ jobs:
targets:
epel-8-x86_64:
distros: [RHEL-8.6-rhui]
- identifier: sanity-8to9-aws-e2e
+ identifier: sanity-8.6to9.0-aws-e2e
tf_extra_params:
test:
tmt:
--
2.43.0

169
0044-Add-7.9to8.10-and-8.10to9.4-upgrade-paths.patch Normal file
View File

@ -0,0 +1,169 @@
From 2340bd5322d3d083c33be065858148e1b32f3d7b Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Mon, 20 Nov 2023 13:03:48 +0100
Subject: [PATCH 44/60] Add 7.9to8.10 and 8.10to9.4 upgrade paths
---
.packit.yaml | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 116 insertions(+), 2 deletions(-)
diff --git a/.packit.yaml b/.packit.yaml
index a307cc75..acbd2b86 100644
--- a/.packit.yaml
+++ b/.packit.yaml
@@ -163,6 +163,15 @@ jobs:
RHUI: "aws"
LEAPPDATA_BRANCH: "upstream"
+- &sanity-79to810-aws
+ <<: *sanity-79to86-aws
+ identifier: sanity-7.9to8.10-aws-e2e
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.10"
+ RHUI: "aws"
+ LEAPPDATA_BRANCH: "upstream"
+
# On-demand minimal beaker tests
- &beaker-minimal-79to86
<<: *sanity-79to86
@@ -274,6 +283,40 @@ jobs:
TARGET_RELEASE: "8.9"
LEAPPDATA_BRANCH: "upstream"
+- &sanity-79to810
+ <<: *sanity-79to86
+ identifier: sanity-7.9to8.10
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.10"
+ LEAPPDATA_BRANCH: "upstream"
+
+# On-demand minimal beaker tests
+- &beaker-minimal-79to810
+ <<: *beaker-minimal-79to86
+ labels:
+ - beaker-minimal
+ - beaker-minimal-7.9to8.10
+ - 7.9to8.10
+ identifier: sanity-7.9to8.10-beaker-minimal
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.10"
+ LEAPPDATA_BRANCH: "upstream"
+
+# On-demand kernel-rt tests
+- &kernel-rt-79to810
+ <<: *kernel-rt-79to88
+ labels:
+ - kernel-rt
+ - kernel-rt-7.9to8.10
+ - 7.9to8.10
+ identifier: sanity-7.9to8.10-kernel-rt
+ env:
+ SOURCE_RELEASE: "7.9"
+ TARGET_RELEASE: "8.10"
+ LEAPPDATA_BRANCH: "upstream"
+
- &sanity-86to90
<<: *sanity-79to86
targets:
@@ -445,7 +488,6 @@ jobs:
env:
SOURCE_RELEASE: "8.9"
TARGET_RELEASE: "9.3"
- RHSM_REPOS: "rhel-8-for-x86_64-appstream-beta-rpms,rhel-8-for-x86_64-baseos-beta-rpms"
LEAPPDATA_BRANCH: "upstream"
LEAPP_DEVEL_TARGET_RELEASE: "9.3"
@@ -475,7 +517,6 @@ jobs:
env:
SOURCE_RELEASE: "8.9"
TARGET_RELEASE: "9.3"
- RHSM_REPOS: "rhel-8-for-x86_64-appstream-beta-rpms,rhel-8-for-x86_64-baseos-beta-rpms"
LEAPPDATA_BRANCH: "upstream"
LEAPP_DEVEL_TARGET_RELEASE: "9.3"
@@ -500,6 +541,79 @@ jobs:
tags:
BusinessUnit: sst_upgrades@leapp_upstream_test
+- &sanity-810to94
+ <<: *sanity-88to92
+ targets:
+ epel-8-x86_64:
+ distros: [RHEL-8.10.0-Nightly]
+ identifier: sanity-8.10to9.4
+ tf_extra_params:
+ test:
+ tmt:
+ plan_filter: 'tag:sanity & tag:8to9'
+ environments:
+ - tmt:
+ context:
+ distro: "rhel-8.10"
+ settings:
+ provisioning:
+ tags:
+ BusinessUnit: sst_upgrades@leapp_upstream_test
+ env:
+ SOURCE_RELEASE: "8.10"
+ TARGET_RELEASE: "9.4"
+ RHSM_REPOS: "rhel-8-for-x86_64-appstream-beta-rpms,rhel-8-for-x86_64-baseos-beta-rpms"
+ LEAPPDATA_BRANCH: "upstream"
+
+# On-demand minimal beaker tests
+- &beaker-minimal-810to94
+ <<: *beaker-minimal-88to92
+ labels:
+ - beaker-minimal
+ - beaker-minimal-8.10to9.4
+ - 8.10to9.4
+ targets:
+ epel-8-x86_64:
+ distros: [RHEL-8.10.0-Nightly]
+ identifier: sanity-8.10to9.4-beaker-minimal
+ tf_extra_params:
+ test:
+ tmt:
+ plan_filter: 'tag:partitioning & tag:8to9'
+ environments:
+ - tmt:
+ context:
+ distro: "rhel-8.10"
+ settings:
+ provisioning:
+ tags:
+ BusinessUnit: sst_upgrades@leapp_upstream_test
+ env:
+ SOURCE_RELEASE: "8.10"
+ TARGET_RELEASE: "9.4"
+ LEAPPDATA_BRANCH: "upstream"
+
+# On-demand kernel-rt tests
+- &kernel-rt-810to94
+ <<: *beaker-minimal-810to94
+ labels:
+ - kernel-rt
+ - kernel-rt-8.10to9.4
+ - 8.10to9.4
+ identifier: sanity-8.10to9.4-kernel-rt
+ tf_extra_params:
+ test:
+ tmt:
+ plan_filter: 'tag:kernel-rt & tag:8to9'
+ environments:
+ - tmt:
+ context:
+ distro: "rhel-8.10"
+ settings:
+ provisioning:
+ tags:
+ BusinessUnit: sst_upgrades@leapp_upstream_test
+
- &sanity-86to90-aws
<<: *sanity-79to86-aws
targets:
--
2.43.0

66
0045-Utilize-get_target_major_version-in-no-enabled-targe.patch Normal file
View File

@ -0,0 +1,66 @@
From d9af1f2a19ec3352a4eff596bcb13e7ad073d763 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Andrea=20Waltlov=C3=A1?= <awaltlov@redhat.com>
Date: Sun, 26 Nov 2023 19:31:44 +0100
Subject: [PATCH 45/60] Utilize get_target_major_version in no enabled target
repositories report (#1151)
* Utilize get_target_major_version in no enabled target repositories report
so the shortened URL in the report points to the right documentation based
based on the target OS major version.
* Add expected docs URLs to comments for easier grep
Signed-off-by: Andrea Waltlova <awaltlov@redhat.com>
---
.../libraries/userspacegen.py | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
index d605ba0e..c1d34f18 100644
--- a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
+++ b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
@@ -678,8 +678,10 @@ def _get_rhsm_available_repoids(context):
).format(target_major_version)),
reporting.ExternalLink(
- # TODO: How to handle different documentation links for each version?
- url='https://red.ht/preparing-for-upgrade-to-rhel8',
+ # https://red.ht/preparing-for-upgrade-to-rhel8
+ # https://red.ht/preparing-for-upgrade-to-rhel9
+ # https://red.ht/preparing-for-upgrade-to-rhel10
+ url='https://red.ht/preparing-for-upgrade-to-rhel{}'.format(target_major_version),
title='Preparing for the upgrade')
])
raise StopActorExecution()
@@ -812,6 +814,7 @@ def gather_target_repositories(context, indata):
missing_custom_repoids.append(custom_repo.repoid)
api.current_logger().debug("Gathered target repositories: {}".format(', '.join(target_repoids)))
if not target_repoids:
+ target_major_version = get_target_major_version()
reporting.create_report([
reporting.Title('There are no enabled target repositories'),
reporting.Summary(
@@ -833,8 +836,10 @@ def gather_target_repositories(context, indata):
' Finally, verify that the "/etc/leapp/files/repomap.json" file is up-to-date.'
).format(version=api.current_actor().configuration.version.target)),
reporting.ExternalLink(
- # TODO: How to handle different documentation links for each version?
- url='https://red.ht/preparing-for-upgrade-to-rhel8',
+ # https://red.ht/preparing-for-upgrade-to-rhel8
+ # https://red.ht/preparing-for-upgrade-to-rhel9
+ # https://red.ht/preparing-for-upgrade-to-rhel10
+ url='https://red.ht/preparing-for-upgrade-to-rhel{}'.format(target_major_version),
title='Preparing for the upgrade'),
reporting.RelatedResource("file", "/etc/leapp/files/repomap.json"),
reporting.RelatedResource("file", "/etc/yum.repos.d/")
@@ -854,7 +859,7 @@ def gather_target_repositories(context, indata):
reporting.Groups([reporting.Groups.INHIBITOR]),
reporting.Severity(reporting.Severity.HIGH),
reporting.ExternalLink(
- # TODO: How to handle different documentation links for each version?
+ # NOTE: Article covers both RHEL 7 to RHEL 8 and RHEL 8 to RHEL 9
url='https://access.redhat.com/articles/4977891',
title='Customizing your Red Hat Enterprise Linux in-place upgrade'),
reporting.Remediation(hint=(
--
2.43.0

174
0046-Workaround-tft-issue-with-listing-disabled-plans.patch Normal file
View File

@ -0,0 +1,174 @@
From 677e5e63829aecf023b01747848e5e1b712350f8 Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Tue, 12 Dec 2023 11:27:19 +0100
Subject: [PATCH 46/60] Workaround tft issue with listing disabled plans
Until TFT-2298 is resolved a mandatory enabled:true tests
filtering won't hurt as we do have some tests that are disabled
for particular distros.
OAMG-10177
---
.packit.yaml | 34 +++++++++++++++++-----------------
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/.packit.yaml b/.packit.yaml
index acbd2b86..1d0b6433 100644
--- a/.packit.yaml
+++ b/.packit.yaml
@@ -101,7 +101,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:sanity'
+ plan_filter: 'tag:sanity & enabled:true'
environments:
- tmt:
context:
@@ -129,7 +129,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:e2e'
+ plan_filter: 'tag:e2e & enabled:true'
environments:
- tmt:
context:
@@ -184,7 +184,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:partitioning & tag:7to8'
+ plan_filter: 'tag:partitioning & tag:7to8 & enabled:true'
environments:
- tmt:
context:
@@ -205,7 +205,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:kernel-rt & tag:7to8'
+ plan_filter: 'tag:kernel-rt & tag:7to8 & enabled:true'
environments:
- tmt:
context:
@@ -326,7 +326,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:sanity & tag:8to9'
+ plan_filter: 'tag:sanity & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -355,7 +355,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:partitioning & tag:8to9'
+ plan_filter: 'tag:partitioning & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -381,7 +381,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:kernel-rt & tag:8to9'
+ plan_filter: 'tag:kernel-rt & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -400,7 +400,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:sanity & tag:8to9'
+ plan_filter: 'tag:sanity & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -430,7 +430,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:partitioning & tag:8to9'
+ plan_filter: 'tag:partitioning & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -457,7 +457,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:kernel-rt & tag:8to9'
+ plan_filter: 'tag:kernel-rt & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -476,7 +476,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:sanity & tag:8to9'
+ plan_filter: 'tag:sanity & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -505,7 +505,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:partitioning & tag:8to9'
+ plan_filter: 'tag:partitioning & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -531,7 +531,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:kernel-rt & tag:8to9'
+ plan_filter: 'tag:kernel-rt & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -550,7 +550,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:sanity & tag:8to9'
+ plan_filter: 'tag:sanity & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -579,7 +579,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:partitioning & tag:8to9'
+ plan_filter: 'tag:partitioning & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -604,7 +604,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:kernel-rt & tag:8to9'
+ plan_filter: 'tag:kernel-rt & tag:8to9 & enabled:true'
environments:
- tmt:
context:
@@ -623,7 +623,7 @@ jobs:
tf_extra_params:
test:
tmt:
- plan_filter: 'tag:e2e'
+ plan_filter: 'tag:e2e & enabled:true'
environments:
- tmt:
context:
--
2.43.0

418
0047-Distribution-agnostick-check-of-signed-packages-1-2.patch Normal file
View File

@ -0,0 +1,418 @@
From 1778818611efc961eda1e44894132689543cfcbe Mon Sep 17 00:00:00 2001
From: Evgeni Golov <evgeni@golov.de>
Date: Mon, 11 Dec 2023 10:45:22 +0100
Subject: [PATCH 47/60] Distribution agnostick check of signed packages [1/2]
The original detection covered only RHEL system, requiring rpms
to be signed by Red Hat (hardcoded). Also the model
InstalledRedHatSignedRPM didn't provide to much space for detection
of other distros.
The new solution checks RPMs signatures based on the detected
distribution ID (currently: rhel, centos). Fingerprints of GPG keys
and the packager string are stored under
repos/system_upgrade/common/files/distro/<distro>/signatures.json
where <distro> is the distribution id.
RedHatSignedRPMScanner is deprecated, replaced by DistributionSignedRPM
message. The original RedHatSignedRPMScanner will contain till the
removal just packages signed by RH.
The update of all other actors to consume DistributionSignedRPM is
covered in the next commit for the easier reading.
jira: OAMG-9824
Co-authored-by: Petr Stodulka <pstodulk@redhat.com>
---
.../distributionsignedrpmscanner/actor.py | 94 +++++++++++++++++++
.../test_distributionsignedrpmscanner.py} | 73 ++++++++++++++
.../actors/redhatsignedrpmscanner/actor.py | 75 ---------------
.../files/distro/centos/gpg-signatures.json | 8 ++
.../files/distro/rhel/gpg-signatures.json | 10 ++
.../common/models/installedrpm.py | 6 ++
6 files changed, 191 insertions(+), 75 deletions(-)
create mode 100644 repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
rename repos/system_upgrade/common/actors/{redhatsignedrpmscanner/tests/test_redhatsignedrpmscanner.py => distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py} (68%)
delete mode 100644 repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
create mode 100644 repos/system_upgrade/common/files/distro/centos/gpg-signatures.json
create mode 100644 repos/system_upgrade/common/files/distro/rhel/gpg-signatures.json
diff --git a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
new file mode 100644
index 00000000..5772cb25
--- /dev/null
+++ b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
@@ -0,0 +1,94 @@
+import json
+import os
+
+from leapp.actors import Actor
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.common import rhui
+from leapp.libraries.common.config import get_env
+from leapp.libraries.stdlib import api
+from leapp.models import DistributionSignedRPM, InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+from leapp.utils.deprecation import suppress_deprecation
+
+
+@suppress_deprecation(InstalledRedHatSignedRPM)
+class DistributionSignedRpmScanner(Actor):
+ """Provide data about installed RPM Packages signed by the distribution.
+
+ After filtering the list of installed RPM packages by signature, a message
+ with relevant data will be produced.
+ """
+
+ name = 'distribution_signed_rpm_scanner'
+ consumes = (InstalledRPM,)
+ produces = (DistributionSignedRPM, InstalledRedHatSignedRPM, InstalledUnsignedRPM,)
+ tags = (IPUWorkflowTag, FactsPhaseTag)
+
+ def process(self):
+ # TODO(pstodulk): refactor this function
+ # - move it to the private library
+ # - split it into several functions (so the main function stays small)
+ # FIXME(pstodulk): gpg-pubkey is handled wrong; it's not a real package
+ # and create FP report about unsigned RPMs. Keeping the fix for later.
+ distribution = self.configuration.os_release.release_id
+ distributions_path = api.get_common_folder_path('distro')
+
+ distribution_config = os.path.join(distributions_path, distribution, 'gpg-signatures.json')
+ if os.path.exists(distribution_config):
+ with open(distribution_config) as distro_config_file:
+ distro_config_json = json.load(distro_config_file)
+ distribution_keys = distro_config_json.get('keys', [])
+ distribution_packager = distro_config_json.get('packager', 'not-available')
+ else:
+ raise StopActorExecutionError(
+ 'Cannot find distribution signature configuration.',
+ details={'Problem': 'Distribution {} was not found in {}.'.format(distribution, distributions_path)})
+
+ signed_pkgs = DistributionSignedRPM()
+ rh_signed_pkgs = InstalledRedHatSignedRPM()
+ unsigned_pkgs = InstalledUnsignedRPM()
+
+ all_signed = get_env('LEAPP_DEVEL_RPMS_ALL_SIGNED', '0') == '1'
+
+ def has_distributionsig(pkg):
+ return any(key in pkg.pgpsig for key in distribution_keys)
+
+ def is_gpg_pubkey(pkg):
+ """
+ Check if gpg-pubkey pkg exists or LEAPP_DEVEL_RPMS_ALL_SIGNED=1
+
+ gpg-pubkey is not signed as it would require another package
+ to verify its signature
+ """
+ return ( # pylint: disable-msg=consider-using-ternary
+ pkg.name == 'gpg-pubkey'
+ and pkg.packager.startswith(distribution_packager)
+ or all_signed
+ )
+
+ def has_katello_prefix(pkg):
+ """Whitelist the katello package."""
+ return pkg.name.startswith('katello-ca-consumer')
+
+ whitelisted_cloud_pkgs = rhui.get_all_known_rhui_pkgs_for_current_upg()
+
+ for rpm_pkgs in self.consume(InstalledRPM):
+ for pkg in rpm_pkgs.items:
+ if any(
+ [
+ has_distributionsig(pkg),
+ is_gpg_pubkey(pkg),
+ has_katello_prefix(pkg),
+ pkg.name in whitelisted_cloud_pkgs,
+ ]
+ ):
+ signed_pkgs.items.append(pkg)
+ if distribution == 'rhel':
+ rh_signed_pkgs.items.append(pkg)
+ continue
+
+ unsigned_pkgs.items.append(pkg)
+
+ self.produce(signed_pkgs)
+ self.produce(rh_signed_pkgs)
+ self.produce(unsigned_pkgs)
diff --git a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/tests/test_redhatsignedrpmscanner.py b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
similarity index 68%
rename from repos/system_upgrade/common/actors/redhatsignedrpmscanner/tests/test_redhatsignedrpmscanner.py
rename to repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
index 6652142e..a15ae173 100644
--- a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/tests/test_redhatsignedrpmscanner.py
+++ b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
@@ -3,12 +3,14 @@ import mock
from leapp.libraries.common import rpms
from leapp.libraries.common.config import mock_configs
from leapp.models import (
+ DistributionSignedRPM,
fields,
InstalledRedHatSignedRPM,
InstalledRPM,
InstalledUnsignedRPM,
IPUConfig,
Model,
+ OSRelease,
RPM
)
@@ -30,6 +32,7 @@ class MockModel(Model):
def test_no_installed_rpms(current_actor_context):
current_actor_context.run(config_model=mock_configs.CONFIG)
+ assert current_actor_context.consume(DistributionSignedRPM)
assert current_actor_context.consume(InstalledRedHatSignedRPM)
assert current_actor_context.consume(InstalledUnsignedRPM)
@@ -57,12 +60,74 @@ def test_actor_execution_with_signed_unsigned_data(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG)
+ assert current_actor_context.consume(DistributionSignedRPM)
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 5
assert current_actor_context.consume(InstalledRedHatSignedRPM)
assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 5
assert current_actor_context.consume(InstalledUnsignedRPM)
assert len(current_actor_context.consume(InstalledUnsignedRPM)[0].items) == 4
+def test_actor_execution_with_signed_unsigned_data_centos(current_actor_context):
+ CENTOS_PACKAGER = 'CentOS BuildSystem <http://bugs.centos.org>'
+ config = mock_configs.CONFIG
+
+ config.os_release = OSRelease(
+ release_id='centos',
+ name='CentOS Linux',
+ pretty_name='CentOS Linux 7 (Core)',
+ version='7 (Core)',
+ version_id='7'
+ )
+
+ installed_rpm = [
+ RPM(name='sample01', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 24c6a8a7f4a80eb5'),
+ RPM(name='sample02', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='SOME_OTHER_SIG_X'),
+ RPM(name='sample03', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 05b555b38483c65d'),
+ RPM(name='sample04', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='SOME_OTHER_SIG_X'),
+ RPM(name='sample05', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 4eb84e71f2ee9d55'),
+ RPM(name='sample06', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='SOME_OTHER_SIG_X'),
+ RPM(name='sample07', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID fd372689897da07a'),
+ RPM(name='sample08', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='SOME_OTHER_SIG_X'),
+ RPM(name='sample09', version='0.1', release='1.sm01', epoch='1', packager=CENTOS_PACKAGER, arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 45689c882fa658e0')]
+
+ current_actor_context.feed(InstalledRPM(items=installed_rpm))
+ current_actor_context.run(config_model=config)
+ assert current_actor_context.consume(DistributionSignedRPM)
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 3
+ assert current_actor_context.consume(InstalledRedHatSignedRPM)
+ assert not current_actor_context.consume(InstalledRedHatSignedRPM)[0].items
+ assert current_actor_context.consume(InstalledUnsignedRPM)
+ assert len(current_actor_context.consume(InstalledUnsignedRPM)[0].items) == 6
+
+
+def test_actor_execution_with_unknown_distro(current_actor_context):
+ config = mock_configs.CONFIG
+
+ config.os_release = OSRelease(
+ release_id='myos',
+ name='MyOS Linux',
+ pretty_name='MyOS Linux 7 (Core)',
+ version='7 (Core)',
+ version_id='7'
+ )
+
+ current_actor_context.feed(InstalledRPM(items=[]))
+ current_actor_context.run(config_model=config)
+ assert not current_actor_context.consume(DistributionSignedRPM)
+ assert not current_actor_context.consume(InstalledRedHatSignedRPM)
+ assert not current_actor_context.consume(InstalledUnsignedRPM)
+
+
def test_all_rpms_signed(current_actor_context):
installed_rpm = [
RPM(name='sample01', version='0.1', release='1.sm01', epoch='1', packager=RH_PACKAGER, arch='noarch',
@@ -77,6 +142,8 @@ def test_all_rpms_signed(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG_ALL_SIGNED)
+ assert current_actor_context.consume(DistributionSignedRPM)
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 4
assert current_actor_context.consume(InstalledRedHatSignedRPM)
assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 4
assert not current_actor_context.consume(InstalledUnsignedRPM)[0].items
@@ -95,6 +162,8 @@ def test_katello_pkg_goes_to_signed(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG_ALL_SIGNED)
+ assert current_actor_context.consume(DistributionSignedRPM)
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 1
assert current_actor_context.consume(InstalledRedHatSignedRPM)
assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 1
assert not current_actor_context.consume(InstalledUnsignedRPM)[0].items
@@ -110,6 +179,8 @@ def test_gpg_pubkey_pkg(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG)
+ assert current_actor_context.consume(DistributionSignedRPM)
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 1
assert current_actor_context.consume(InstalledRedHatSignedRPM)
assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 1
assert current_actor_context.consume(InstalledUnsignedRPM)
@@ -165,6 +236,8 @@ def test_has_package(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG)
+ assert rpms.has_package(DistributionSignedRPM, 'sample01', context=current_actor_context)
+ assert not rpms.has_package(DistributionSignedRPM, 'nosuchpackage', context=current_actor_context)
assert rpms.has_package(InstalledRedHatSignedRPM, 'sample01', context=current_actor_context)
assert not rpms.has_package(InstalledRedHatSignedRPM, 'nosuchpackage', context=current_actor_context)
assert rpms.has_package(InstalledUnsignedRPM, 'sample02', context=current_actor_context)
diff --git a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py b/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
deleted file mode 100644
index 41f9d343..00000000
--- a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
+++ /dev/null
@@ -1,75 +0,0 @@
-from leapp.actors import Actor
-from leapp.libraries.common import rhui
-from leapp.models import InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM
-from leapp.tags import FactsPhaseTag, IPUWorkflowTag
-
-
-class RedHatSignedRpmScanner(Actor):
- """Provide data about installed RPM Packages signed by Red Hat.
-
- After filtering the list of installed RPM packages by signature, a message
- with relevant data will be produced.
- """
-
- name = 'red_hat_signed_rpm_scanner'
- consumes = (InstalledRPM,)
- produces = (InstalledRedHatSignedRPM, InstalledUnsignedRPM,)
- tags = (IPUWorkflowTag, FactsPhaseTag)
-
- def process(self):
- RH_SIGS = ['199e2f91fd431d51',
- '5326810137017186',
- '938a80caf21541eb',
- 'fd372689897da07a',
- '45689c882fa658e0']
-
- signed_pkgs = InstalledRedHatSignedRPM()
- unsigned_pkgs = InstalledUnsignedRPM()
-
- env_vars = self.configuration.leapp_env_vars
- # if we start upgrade with LEAPP_DEVEL_RPMS_ALL_SIGNED=1, we consider
- # all packages to be signed
- all_signed = [
- env
- for env in env_vars
- if env.name == 'LEAPP_DEVEL_RPMS_ALL_SIGNED' and env.value == '1'
- ]
-
- def has_rhsig(pkg):
- return any(key in pkg.pgpsig for key in RH_SIGS)
-
- def is_gpg_pubkey(pkg):
- """Check if gpg-pubkey pkg exists or LEAPP_DEVEL_RPMS_ALL_SIGNED=1
-
- gpg-pubkey is not signed as it would require another package
- to verify its signature
- """
- return ( # pylint: disable-msg=consider-using-ternary
- pkg.name == 'gpg-pubkey'
- and pkg.packager.startswith('Red Hat, Inc.')
- or all_signed
- )
-
- def has_katello_prefix(pkg):
- """Whitelist the katello package."""
- return pkg.name.startswith('katello-ca-consumer')
-
- whitelisted_cloud_pkgs = rhui.get_all_known_rhui_pkgs_for_current_upg()
-
- for rpm_pkgs in self.consume(InstalledRPM):
- for pkg in rpm_pkgs.items:
- if any(
- [
- has_rhsig(pkg),
- is_gpg_pubkey(pkg),
- has_katello_prefix(pkg),
- pkg.name in whitelisted_cloud_pkgs,
- ]
- ):
- signed_pkgs.items.append(pkg)
- continue
-
- unsigned_pkgs.items.append(pkg)
-
- self.produce(signed_pkgs)
- self.produce(unsigned_pkgs)
diff --git a/repos/system_upgrade/common/files/distro/centos/gpg-signatures.json b/repos/system_upgrade/common/files/distro/centos/gpg-signatures.json
new file mode 100644
index 00000000..30e329ee
--- /dev/null
+++ b/repos/system_upgrade/common/files/distro/centos/gpg-signatures.json
@@ -0,0 +1,8 @@
+{
+ "keys": [
+ "24c6a8a7f4a80eb5",
+ "05b555b38483c65d",
+ "4eb84e71f2ee9d55"
+ ],
+ "packager": "CentOS"
+}
diff --git a/repos/system_upgrade/common/files/distro/rhel/gpg-signatures.json b/repos/system_upgrade/common/files/distro/rhel/gpg-signatures.json
new file mode 100644
index 00000000..eccf0106
--- /dev/null
+++ b/repos/system_upgrade/common/files/distro/rhel/gpg-signatures.json
@@ -0,0 +1,10 @@
+{
+ "keys": [
+ "199e2f91fd431d51",
+ "5326810137017186",
+ "938a80caf21541eb",
+ "fd372689897da07a",
+ "45689c882fa658e0"
+ ],
+ "packager": "Red Hat, Inc."
+}
diff --git a/repos/system_upgrade/common/models/installedrpm.py b/repos/system_upgrade/common/models/installedrpm.py
index 5a632b03..cc9fd508 100644
--- a/repos/system_upgrade/common/models/installedrpm.py
+++ b/repos/system_upgrade/common/models/installedrpm.py
@@ -1,5 +1,6 @@
from leapp.models import fields, Model
from leapp.topics import SystemInfoTopic
+from leapp.utils.deprecation import deprecated
class RPM(Model):
@@ -21,6 +22,11 @@ class InstalledRPM(Model):
items = fields.List(fields.Model(RPM), default=[])
+class DistributionSignedRPM(InstalledRPM):
+ pass
+
+
+@deprecated(since='2024-01-31', message='Replaced by DistributionSignedRPM')
class InstalledRedHatSignedRPM(InstalledRPM):
pass
--
2.43.0

2267
0048-Distribution-agnostick-check-of-signed-packages-2-2.patch Normal file
View File

File diff suppressed because it is too large Load Diff

25
0049-Pylint-fix-superfluous-parens-in-the-code.patch Normal file
View File

@ -0,0 +1,25 @@
From 0776bc34b9b3dc323c98ddb446a5444ea7176970 Mon Sep 17 00:00:00 2001
From: Petr Stodulka <pstodulk@redhat.com>
Date: Wed, 6 Dec 2023 18:25:44 +0100
Subject: [PATCH 49/60] Pylint: fix superfluous-parens in the code
---
.../libraries/checkinstalleddebugkernels.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/repos/system_upgrade/el7toel8/actors/kernel/checkinstalleddebugkernels/checkinstalleddebugkernels/libraries/checkinstalleddebugkernels.py b/repos/system_upgrade/el7toel8/actors/kernel/checkinstalleddebugkernels/checkinstalleddebugkernels/libraries/checkinstalleddebugkernels.py
index 889196ea..15b7b79e 100644
--- a/repos/system_upgrade/el7toel8/actors/kernel/checkinstalleddebugkernels/checkinstalleddebugkernels/libraries/checkinstalleddebugkernels.py
+++ b/repos/system_upgrade/el7toel8/actors/kernel/checkinstalleddebugkernels/checkinstalleddebugkernels/libraries/checkinstalleddebugkernels.py
@@ -26,7 +26,7 @@ def process():
title = 'Multiple debug kernels installed'
summary = ('DNF cannot produce a valid upgrade transaction when'
' multiple kernel-debug packages are installed.')
- hint = ('Remove all but one kernel-debug packages before running Leapp again.')
+ hint = 'Remove all but one kernel-debug packages before running Leapp again.'
all_but_latest_kernel_debug = pkgs[:-1]
packages = ['{n}-{v}-{r}'.format(n=pkg.name, v=pkg.version, r=pkg.release)
for pkg in all_but_latest_kernel_debug]
--
2.43.0

246
0050-distributionsignedrpmscanner-refactoring-gpg-pubkey-.patch Normal file
View File

@ -0,0 +1,246 @@
From 4968bec73947fb83aeb2d89fe7e919fba2ca2776 Mon Sep 17 00:00:00 2001
From: Petr Stodulka <pstodulk@redhat.com>
Date: Mon, 11 Dec 2023 18:00:40 +0100
Subject: [PATCH 50/60] distributionsignedrpmscanner: refactoring + gpg-pubkey
fix
We have decided to refactor the code in the actor (coming history
time ago) to make it more readable.
Also it's fixing an old issue with gpg-pubkey detection as unsigned
rpm. gpg-pubkey is not a real package and it's just an entry in RPM DB
about the imported RPM GPG keys. Originally it has been checked whether
the packager is vendor/authority of the installed distribution and if
not, such a package (key) has been mared as unsigned.
This led to false positive reports, that we do not know what will
happen with gpg-pubkey packages (reported even multiple times..)
and that they might be removed or do another problems with the upgrade
transaction - which has not been true.
So I dropped the check for the packager and mark gpg-pubkey always
as signed. It's a question whether we should not ignore this package
always and do not put it to any signed/unsigned list. Handling it
in this way for now.
---
.../distributionsignedrpmscanner/actor.py | 94 ++++---------------
.../libraries/distributionsignedrpmscanner.py | 72 ++++++++++++++
.../test_distributionsignedrpmscanner.py | 6 +-
3 files changed, 92 insertions(+), 80 deletions(-)
create mode 100644 repos/system_upgrade/common/actors/distributionsignedrpmscanner/libraries/distributionsignedrpmscanner.py
diff --git a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
index 5772cb25..56016513 100644
--- a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
+++ b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/actor.py
@@ -1,11 +1,5 @@
-import json
-import os
-
from leapp.actors import Actor
-from leapp.exceptions import StopActorExecutionError
-from leapp.libraries.common import rhui
-from leapp.libraries.common.config import get_env
-from leapp.libraries.stdlib import api
+from leapp.libraries.actor import distributionsignedrpmscanner
from leapp.models import DistributionSignedRPM, InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
from leapp.utils.deprecation import suppress_deprecation
@@ -13,10 +7,22 @@ from leapp.utils.deprecation import suppress_deprecation
@suppress_deprecation(InstalledRedHatSignedRPM)
class DistributionSignedRpmScanner(Actor):
- """Provide data about installed RPM Packages signed by the distribution.
+ """
+ Provide data about distribution signed & unsigned RPM packages.
+
+ For various checks and actions done during the upgrade it's important to
+ know what packages are signed by GPG keys of the installed linux system
+ distribution. RPMs that are not provided in the distribution could have
+ different versions, different behaviour, and also it could be completely
+ different application just with the same RPM name.
+
+ For that reasons, various actors rely on the DistributionSignedRPM message
+ to check whether particular package is installed, to be sure it provides
+ valid data. Fingerprints of distribution GPG keys are stored under
+ common/files/distro/<distro>/gpg_signatures.json
+ where <distro> is distribution ID of the installed system (e.g. centos, rhel).
- After filtering the list of installed RPM packages by signature, a message
- with relevant data will be produced.
+ If the file for the installed distribution is not find, end with error.
"""
name = 'distribution_signed_rpm_scanner'
@@ -25,70 +31,4 @@ class DistributionSignedRpmScanner(Actor):
tags = (IPUWorkflowTag, FactsPhaseTag)
def process(self):
- # TODO(pstodulk): refactor this function
- # - move it to the private library
- # - split it into several functions (so the main function stays small)
- # FIXME(pstodulk): gpg-pubkey is handled wrong; it's not a real package
- # and create FP report about unsigned RPMs. Keeping the fix for later.
- distribution = self.configuration.os_release.release_id
- distributions_path = api.get_common_folder_path('distro')
-
- distribution_config = os.path.join(distributions_path, distribution, 'gpg-signatures.json')
- if os.path.exists(distribution_config):
- with open(distribution_config) as distro_config_file:
- distro_config_json = json.load(distro_config_file)
- distribution_keys = distro_config_json.get('keys', [])
- distribution_packager = distro_config_json.get('packager', 'not-available')
- else:
- raise StopActorExecutionError(
- 'Cannot find distribution signature configuration.',
- details={'Problem': 'Distribution {} was not found in {}.'.format(distribution, distributions_path)})
-
- signed_pkgs = DistributionSignedRPM()
- rh_signed_pkgs = InstalledRedHatSignedRPM()
- unsigned_pkgs = InstalledUnsignedRPM()
-
- all_signed = get_env('LEAPP_DEVEL_RPMS_ALL_SIGNED', '0') == '1'
-
- def has_distributionsig(pkg):
- return any(key in pkg.pgpsig for key in distribution_keys)
-
- def is_gpg_pubkey(pkg):
- """
- Check if gpg-pubkey pkg exists or LEAPP_DEVEL_RPMS_ALL_SIGNED=1
-
- gpg-pubkey is not signed as it would require another package
- to verify its signature
- """
- return ( # pylint: disable-msg=consider-using-ternary
- pkg.name == 'gpg-pubkey'
- and pkg.packager.startswith(distribution_packager)
- or all_signed
- )
-
- def has_katello_prefix(pkg):
- """Whitelist the katello package."""
- return pkg.name.startswith('katello-ca-consumer')
-
- whitelisted_cloud_pkgs = rhui.get_all_known_rhui_pkgs_for_current_upg()
-
- for rpm_pkgs in self.consume(InstalledRPM):
- for pkg in rpm_pkgs.items:
- if any(
- [
- has_distributionsig(pkg),
- is_gpg_pubkey(pkg),
- has_katello_prefix(pkg),
- pkg.name in whitelisted_cloud_pkgs,
- ]
- ):
- signed_pkgs.items.append(pkg)
- if distribution == 'rhel':
- rh_signed_pkgs.items.append(pkg)
- continue
-
- unsigned_pkgs.items.append(pkg)
-
- self.produce(signed_pkgs)
- self.produce(rh_signed_pkgs)
- self.produce(unsigned_pkgs)
+ distributionsignedrpmscanner.process()
diff --git a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/libraries/distributionsignedrpmscanner.py b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/libraries/distributionsignedrpmscanner.py
new file mode 100644
index 00000000..0bc71bfa
--- /dev/null
+++ b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/libraries/distributionsignedrpmscanner.py
@@ -0,0 +1,72 @@
+import json
+import os
+
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.common import rhui
+from leapp.libraries.common.config import get_env
+from leapp.libraries.stdlib import api
+from leapp.models import DistributionSignedRPM, InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM
+
+
+def get_distribution_data(distribution):
+ distributions_path = api.get_common_folder_path('distro')
+
+ distribution_config = os.path.join(distributions_path, distribution, 'gpg-signatures.json')
+ if os.path.exists(distribution_config):
+ with open(distribution_config) as distro_config_file:
+ distro_config_json = json.load(distro_config_file)
+ distro_keys = distro_config_json.get('keys', [])
+ # distro_packager = distro_config_json.get('packager', 'not-available')
+ else:
+ raise StopActorExecutionError(
+ 'Cannot find distribution signature configuration.',
+ details={'Problem': 'Distribution {} was not found in {}.'.format(distribution, distributions_path)})
+ return distro_keys
+
+
+def is_distro_signed(pkg, distro_keys):
+ return any(key in pkg.pgpsig for key in distro_keys)
+
+
+def is_exceptional(pkg, allowlist):
+ """
+ Some packages should be marked always as signed
+
+ tl;dr; gpg-pubkey, katello packages, and rhui packages
+
+ gpg-pubkey is not real RPM. It's just an entry representing
+ gpg key imported inside the RPM DB. For that same reason, it cannot be
+ signed. Note that it cannot affect the upgrade transaction, so ignore
+ who vendored the key. Total majority of all machines have imported third
+ party gpg keys.
+
+ Katello packages have various names and are created on a Satellite server.
+
+ The allowlist is now used for any other package names that should be marked
+ always as signed for the particular upgrade.
+ """
+ return pkg.name == 'gpg-pubkey' or pkg.name.startswith('katello-ca-consumer') or pkg.name in allowlist
+
+
+def process():
+ distribution = api.current_actor().configuration.os_release.release_id
+ distro_keys = get_distribution_data(distribution)
+ all_signed = get_env('LEAPP_DEVEL_RPMS_ALL_SIGNED', '0') == '1'
+ rhui_pkgs = rhui.get_all_known_rhui_pkgs_for_current_upg()
+
+ signed_pkgs = DistributionSignedRPM()
+ rh_signed_pkgs = InstalledRedHatSignedRPM()
+ unsigned_pkgs = InstalledUnsignedRPM()
+
+ for rpm_pkgs in api.consume(InstalledRPM):
+ for pkg in rpm_pkgs.items:
+ if all_signed or is_distro_signed(pkg, distro_keys) or is_exceptional(pkg, rhui_pkgs):
+ signed_pkgs.items.append(pkg)
+ if distribution == 'rhel':
+ rh_signed_pkgs.items.append(pkg)
+ continue
+ unsigned_pkgs.items.append(pkg)
+
+ api.produce(signed_pkgs)
+ api.produce(rh_signed_pkgs)
+ api.produce(unsigned_pkgs)
diff --git a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
index a15ae173..f138bcb2 100644
--- a/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
+++ b/repos/system_upgrade/common/actors/distributionsignedrpmscanner/tests/test_distributionsignedrpmscanner.py
@@ -180,11 +180,11 @@ def test_gpg_pubkey_pkg(current_actor_context):
current_actor_context.feed(InstalledRPM(items=installed_rpm))
current_actor_context.run(config_model=mock_configs.CONFIG)
assert current_actor_context.consume(DistributionSignedRPM)
- assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 1
+ assert len(current_actor_context.consume(DistributionSignedRPM)[0].items) == 2
assert current_actor_context.consume(InstalledRedHatSignedRPM)
- assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 1
+ assert len(current_actor_context.consume(InstalledRedHatSignedRPM)[0].items) == 2
assert current_actor_context.consume(InstalledUnsignedRPM)
- assert len(current_actor_context.consume(InstalledUnsignedRPM)[0].items) == 1
+ assert not current_actor_context.consume(InstalledUnsignedRPM)[0].items
def test_create_lookup():
--
2.43.0

255
0051-Introduce-two-functions-for-listing-which-packages-a.patch Normal file
View File

@ -0,0 +1,255 @@
From 118133a734987e4d2c01ab9775525b0152adc780 Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Thu, 14 Dec 2023 14:03:25 +0100
Subject: [PATCH 51/60] Introduce two functions for listing which packages are
leapp packages.
* rpms.get_leapp_packages() library function returns the leapp packages which ship the leapp
components.
* rpms.get_leapp_deps_packages() library function returns the leapp deps meta packages which list
the requirements of the associated leapp packages.
This function can be used as leapp-installation-packages-getter.
Refactoring of other actors using it will be done later.
---
repos/system_upgrade/common/libraries/rpms.py | 139 ++++++++++++++++++
.../common/libraries/tests/test_rpms.py | 67 ++++++++-
2 files changed, 205 insertions(+), 1 deletion(-)
diff --git a/repos/system_upgrade/common/libraries/rpms.py b/repos/system_upgrade/common/libraries/rpms.py
index 6a5e8637..2890240f 100644
--- a/repos/system_upgrade/common/libraries/rpms.py
+++ b/repos/system_upgrade/common/libraries/rpms.py
@@ -1,7 +1,47 @@
from leapp.libraries import stdlib
+from leapp.libraries.common.config.version import get_source_major_version
from leapp.models import InstalledRPM
+class LeappComponents(object):
+ """
+ Supported component values to be used with get_packages_function:
+ * FRAMEWORK - the core of the leapp project: the leapp executable and
+ associated leapp libraries
+ * REPOSITORY - the leapp-repository project
+ * COCKPIT - the cockpit-leapp project
+ * TOOLS - miscellaneous tooling like snactor
+ """
+ FRAMEWORK = 'framework'
+ REPOSITORY = 'repository'
+ COCKPIT = 'cockpit'
+ TOOLS = 'tools'
+
+
+_LEAPP_PACKAGES_MAP = {
+ LeappComponents.FRAMEWORK: {'7': {'pkgs': ['leapp', 'python2-leapp'],
+ 'deps': ['leapp-deps']},
+ '8': {'pkgs': ['leapp', 'python3-leapp'],
+ 'deps': ['leapp-deps']}
+ },
+ LeappComponents.REPOSITORY: {'7': {'pkgs': ['leapp-upgrade-el7toel8'],
+ 'deps': ['leapp-upgrade-el7toel8-deps']},
+ '8': {'pkgs': ['leapp-upgrade-el8toel9'],
+ 'deps': ['leapp-upgrade-el8toel9-deps']}
+ },
+ LeappComponents.COCKPIT: {'7': {'pkgs': ['cockpit-leapp']},
+ '8': {'pkgs': ['cockpit-leapp']}
+ },
+ LeappComponents.TOOLS: {'7': {'pkgs': ['snactor']},
+ '8': {'pkgs': ['snactor']}
+ }
+ }
+
+GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS = frozenset((LeappComponents.FRAMEWORK,
+ LeappComponents.REPOSITORY,
+ LeappComponents.TOOLS))
+
+
def get_installed_rpms():
rpm_cmd = [
'/bin/rpm',
@@ -114,3 +154,102 @@ def check_file_modification(config):
"""
output = _read_rpm_modifications(config)
return _parse_config_modification(output, config)
+
+
+def _get_leapp_packages_of_type(major_version, component, type_='pkgs'):
+ """
+ Private implementation of get_leapp_packages() and get_leapp_deps_packages().
+
+ :param major_version: Same as for :func:`get_leapp_packages` and
+ :func:`get_leapp_deps_packages`
+ :param component: Same as for :func:`get_leapp_packages` and :func:`get_leapp_deps_packages`
+ :param type_: Either "pkgs" or "deps". Determines which set of packages we're looking for.
+ Corresponds to the keys in the `_LEAPP_PACKAGES_MAP`.
+
+ Retrieving the set of leapp and leapp-deps packages only differs in which key is used to
+ retrieve the packages from _LEAPP_PACKAGES_MAP. This function abstracts that difference.
+ """
+ res = set()
+
+ major_versions = [major_version] if isinstance(major_version, str) else major_version
+ if not major_versions:
+ # No major_version of interest specified -> treat as if only current source system version
+ # requested
+ major_versions = [get_source_major_version()]
+
+ components = [component] if isinstance(component, str) else component
+ if not components:
+ error_msg = ("At least one component must be specified when calling this"
+ " function, available choices are {choices}".format(
+ choices=sorted(_LEAPP_PACKAGES_MAP.keys()))
+ )
+ raise ValueError(error_msg)
+
+ for comp in components:
+ for a_major_version in major_versions:
+ if comp not in _LEAPP_PACKAGES_MAP:
+ error_msg = "The requested component {comp} is unknown, available choices are {choices}".format(
+ comp=component, choices=sorted(_LEAPP_PACKAGES_MAP.keys()))
+ raise ValueError(error_msg)
+
+ if a_major_version not in _LEAPP_PACKAGES_MAP[comp]:
+ error_msg = "The requested major_version {ver} is unknown, available choices are {choices}".format(
+ ver=a_major_version, choices=sorted(_LEAPP_PACKAGES_MAP[comp].keys()))
+ raise ValueError(error_msg)
+
+ # All went well otherwise, get the data
+ res.update(_LEAPP_PACKAGES_MAP[comp][a_major_version].get(type_, []))
+
+ return sorted(res)
+
+
+def get_leapp_packages(major_version=None, component=GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS):
+ """
+ Get list of leapp packages.
+
+ :param major_version: a list or string specifying major_versions. If not defined then current
+ system_version will be used.
+ :param component: a list or a single enum value specifying leapp components
+ (use enum :class: LeappComponents) If defined then only packages related to the specific
+ component(s) will be returned.
+ The default set of components is in `GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS` and
+ simple modifications of the default can be achieved with code like:
+
+ .. code-block:: python
+ get_leapp_packages(
+ component=GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS.difference(
+ [LeappComponents.TOOLS]
+ ))
+
+ :raises ValueError: if a requested component or major_version doesn't exist.
+
+ .. note::
+ Call :func:`get_leapp_dep_packages` as well if you also need the deps metapackages.
+ Those packages determine which RPMs need to be installed for leapp to function.
+ They aren't just Requires on the base leapp and leapp-repository RPMs because they
+ need to be switched from the old system_version's to the new ones at a different
+ point in the upgrade than the base RPMs.
+ """
+ return _get_leapp_packages_of_type(major_version, component, type_="pkgs")
+
+
+def get_leapp_dep_packages(major_version=None, component=GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS):
+ """
+ Get list of leapp dep metapackages.
+
+ :param major_version: a list or string specifying major_versions. If not defined then current
+ system_version will be used.
+ :param component: a list or a single enum value specifying leapp components
+ (use enum :class: LeappComponents) If defined then only packages related to the specific
+ component(s) will be returned.
+ The default set of components is in `GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS` and
+ simple modifications of the default can be achieved with code like:
+
+ .. code-block:: python
+ get_leapp_packages(
+ component=GET_LEAPP_PACKAGES_DEFAULT_COMPONENTS.difference(
+ [LeappComponents.TOOLS]
+ ))
+ :raises ValueError: if a requested component or major_version doesn't exist.
+ """
+ return _get_leapp_packages_of_type(major_version, component, type_="deps")
diff --git a/repos/system_upgrade/common/libraries/tests/test_rpms.py b/repos/system_upgrade/common/libraries/tests/test_rpms.py
index 39a32dcb..955ab05c 100644
--- a/repos/system_upgrade/common/libraries/tests/test_rpms.py
+++ b/repos/system_upgrade/common/libraries/tests/test_rpms.py
@@ -1,4 +1,8 @@
-from leapp.libraries.common.rpms import _parse_config_modification
+import pytest
+
+from leapp.libraries.common.rpms import _parse_config_modification, get_leapp_dep_packages, get_leapp_packages
+from leapp.libraries.common.testutils import CurrentActorMocked
+from leapp.libraries.stdlib import api
def test_parse_config_modification():
@@ -30,3 +34,64 @@ def test_parse_config_modification():
"S.5....T. c /etc/ssh/sshd_config",
]
assert _parse_config_modification(data, "/etc/ssh/sshd_config")
+
+
+@pytest.mark.parametrize('major_version,component,result', [
+ (None, None, ['leapp', 'python3-leapp', 'leapp-upgrade-el8toel9', 'snactor']),
+ ('7', None, ['leapp', 'python2-leapp', 'leapp-upgrade-el7toel8', 'snactor']),
+ (['7', '8'], None, ['leapp', 'python2-leapp', 'leapp-upgrade-el7toel8',
+ 'python3-leapp', 'leapp-upgrade-el8toel9', 'snactor']),
+ ('8', 'framework', ['leapp', 'python3-leapp']),
+ ])
+def test_get_leapp_packages(major_version, component, result, monkeypatch):
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(arch='x86_64', src_ver='8.9', dst_ver='9.3'))
+
+ kwargs = {}
+ if major_version:
+ kwargs["major_version"] = major_version
+ if component:
+ kwargs["component"] = component
+
+ assert set(get_leapp_packages(** kwargs)) == set(result)
+
+
+@pytest.mark.parametrize('major_version,component,result', [
+ ('8', 'nosuchcomponent',
+ (ValueError,
+ r"component nosuchcomponent is unknown, available choices are \['cockpit', 'framework', 'repository', 'tools']")
+ ),
+ ('nosuchversion', "framework",
+ (ValueError, r"major_version nosuchversion is unknown, available choices are \['7', '8']")),
+ ('nosuchversion', False,
+ (ValueError, r"At least one component must be specified when calling this function,"
+ r" available choices are \['cockpit', 'framework', 'repository', 'tools']")),
+])
+def test_get_leapp_packages_errors(major_version, component, result, monkeypatch):
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(arch='x86_64', src_ver='8.9', dst_ver='9.3'))
+
+ kwargs = {}
+ if major_version:
+ kwargs["major_version"] = major_version
+ if component is not None:
+ kwargs["component"] = component
+
+ exc_type, exc_msg = result
+ with pytest.raises(exc_type, match=exc_msg):
+ get_leapp_packages(**kwargs)
+
+
+@pytest.mark.parametrize('major_version,component,result', [
+ (None, None, ['leapp-deps', 'leapp-upgrade-el8toel9-deps']),
+ ('8', 'framework', ['leapp-deps']),
+ ('7', 'tools', []),
+])
+def test_get_leapp_dep_packages(major_version, component, result, monkeypatch):
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(arch='x86_64', src_ver='8.9', dst_ver='9.3'))
+
+ kwargs = {}
+ if major_version:
+ kwargs["major_version"] = major_version
+ if component:
+ kwargs["component"] = component
+
+ assert frozenset(get_leapp_dep_packages(**kwargs)) == frozenset(result)
--
2.43.0

28
0052-Switch-test-repo-branch-to-main.patch Normal file
View File

@ -0,0 +1,28 @@
From c627a0be13bf2170df0089cd5516e7615a97eb34 Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Wed, 10 Jan 2024 13:34:44 +0100
Subject: [PATCH 52/60] Switch test repo branch to main
As default branch in tmt tests repo has been changed
from master to main, we have to address this in
packit configuration.
---
.packit.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.packit.yaml b/.packit.yaml
index 1d0b6433..383f5314 100644
--- a/.packit.yaml
+++ b/.packit.yaml
@@ -88,7 +88,7 @@ jobs:
- &sanity-79to86
job: tests
fmf_url: "https://gitlab.cee.redhat.com/oamg/leapp-tests"
- fmf_ref: "master"
+ fmf_ref: "main"
use_internal_tf: True
trigger: pull_request
labels:
--
2.43.0

98
0053-Update-dependencies-require-xfsprogs-and-e2fsprogs.patch Normal file
View File

@ -0,0 +1,98 @@
From 7a819fb293340b2ed22b6d5e2816dd9c39fefdc9 Mon Sep 17 00:00:00 2001
From: Petr Stodulka <pstodulk@redhat.com>
Date: Tue, 5 Dec 2023 13:53:52 +0100
Subject: [PATCH 53/60] Update dependencies: require xfsprogs and e2fsprogs
To be able to format our OVL disk images with XFS or Ext4, we need
the required tools present on the system. However, on systems with
XFS file systems only, it's not needed to have tools for ext4,
and vice versa. So on such systems, users can remove these packages
manually. In that cases, we get into a problems, especially when
XFS is the default FS in our case.
To resolve that, we add dependencies for xfsprogs and e2fsprogs rpms
into the spec file, so we are sure these are always present on the
system.
In case of Ext4 it is a little bit "redundant" - as use of Ext4 is
optional. However we expect actually that many people will do it
(many == not a small amount of people -> not uncommon use..).
So keeping this the least effort, let's add the requirement for both
as the actual installation stack is not big.
Packaging:
* Requires xfsprogs, e2fsprogs
* Bump leapp-repository-dependencies to 10
jira: RHEL-10847
---
packaging/leapp-repository.spec | 12 +++++++++++-
packaging/other_specs/leapp-el7toel8-deps.spec | 15 ++++++++++++++-
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/packaging/leapp-repository.spec b/packaging/leapp-repository.spec
index 937a738e..2b0a80d4 100644
--- a/packaging/leapp-repository.spec
+++ b/packaging/leapp-repository.spec
@@ -2,7 +2,7 @@
%global repositorydir %{leapp_datadir}/repositories
%global custom_repositorydir %{leapp_datadir}/custom-repositories
-%define leapp_repo_deps 9
+%define leapp_repo_deps 10
%if 0%{?rhel} == 7
%define leapp_python_sitelib %{python2_sitelib}
@@ -149,6 +149,16 @@ Provides: leapp-repository-dependencies = %{leapp_repo_deps}
##################################################
Requires: dnf >= 4
Requires: pciutils
+
+# required to be able to format disk images with XFS file systems (default)
+Requires: xfsprogs
+
+# required to be able to format disk images with Ext4 file systems
+# NOTE: this is not happening by default, but we can expact that many customers
+# will want to / need to do this - especially on RHEL 7 now. Adding this deps
+# as the best trade-off to resolve this problem.
+Requires: e2fsprogs
+
%if 0%{?rhel} && 0%{?rhel} == 7
# Required to gather system facts about SELinux
Requires: libselinux-python
diff --git a/packaging/other_specs/leapp-el7toel8-deps.spec b/packaging/other_specs/leapp-el7toel8-deps.spec
index 4a181ee1..c4e0dd90 100644
--- a/packaging/other_specs/leapp-el7toel8-deps.spec
+++ b/packaging/other_specs/leapp-el7toel8-deps.spec
@@ -9,7 +9,7 @@
%endif
-%define leapp_repo_deps 9
+%define leapp_repo_deps 10
%define leapp_framework_deps 5
# NOTE: the Version contains the %{rhel} macro just for the convenience to
@@ -68,6 +68,19 @@ Requires: cpio
# just to be sure that /etc/modprobe.d is present
Requires: kmod
+# required to be able to format disk images with XFS file systems (default)
+# NOTE: this is really needed on the source system, but keep it for the target
+# one too
+Requires: xfsprogs
+
+# required to be able to format disk images with Ext4 file systems
+# NOTE: this is not happening by default, but we can expact that many customers
+# will want to / need to do this - especially on RHEL 7 now. Adding this deps
+# as the best trade-off to resolve this problem.
+# NOTE: this is really needed on the source system, but keep it for the target
+# one too
+Requires: e2fsprogs
+
%description -n %{lrdname}
%{summary}
--
2.43.0

106
0054-Several-enhancements-to-the-Makefile.patch Normal file
View File

@ -0,0 +1,106 @@
From 50b4fc016befd855094bdba4d7187bf690c4b2ad Mon Sep 17 00:00:00 2001
From: Toshio Kuratomi <a.badger@gmail.com>
Date: Thu, 11 Jan 2024 11:00:43 -0800
Subject: [PATCH 54/60] Several enhancements to the Makefile
* Allow arbitrary user supplied arguments for pytest, pylint, and flake8. This can be used, for
instance, to select specific tests in pytest (PYTEST_ARGS="-k 'perform_ok'"), or to "disable" a
single linter: (PYLINT_ARGS='--version').
* Better document how to determine the proper value for ACTOR=<actor>.
---
Makefile | 33 +++++++++++++++++++++------------
1 file changed, 21 insertions(+), 12 deletions(-)
diff --git a/Makefile b/Makefile
index b504a854..0de2a86a 100644
--- a/Makefile
+++ b/Makefile
@@ -16,6 +16,12 @@ REPOSITORIES ?= $(shell ls $(_SYSUPG_REPOS) | xargs echo | tr " " ",")
SYSUPG_TEST_PATHS=$(shell echo $(REPOSITORIES) | sed -r "s|(,\\|^)| $(_SYSUPG_REPOS)/|g")
TEST_PATHS:=commands repos/common $(SYSUPG_TEST_PATHS)
+# Several commands can take arbitrary user supplied arguments from environment
+# variables as well:
+PYTEST_ARGS ?=
+PYLINT_ARGS ?=
+FLAKE8_ARGS ?=
+
# python version to run test with
_PYTHON_VENV=$${PYTHON_VENV:-python2.7}
@@ -131,10 +137,13 @@ help:
@echo " test_container_all_no_lint run tests without linting in all available containers"
@echo " clean_containers clean all testing and building container images (to force a rebuild for example)"
@echo ""
- @echo "Targets test, lint and test_no_lint support environment variables ACTOR and"
- @echo "TEST_LIBS."
- @echo "If ACTOR=<actor> is specified, targets are run against the specified actor."
- @echo "If TEST_LIBS=y is specified, targets are run against shared libraries."
+ @echo "* Targets test, lint and test_no_lint support environment variables ACTOR and"
+ @echo " TEST_LIBS."
+ @echo "* If ACTOR=<actor> is specified, targets are run against the specified actor."
+ @echo " <actor> must be the name attribute defined in actor.py."
+ @echo "* If TEST_LIBS=y is specified, targets are run against shared libraries."
+ @echo "* Command line options can be added to pytest, pylint, and flake8 by setting"
+ @echo " the PYTEST_ARGS, PYLINT_ARGS, and FLAKE8_ARGS environment variables."
@echo ""
@echo "Envars affecting actions with COPR (optional):"
@echo " COPR_REPO specify COPR repository, e,g. @oamg/leapp"
@@ -323,15 +332,15 @@ lint:
SEARCH_PATH="$(TEST_PATHS)" && \
echo "Using search path '$${SEARCH_PATH}'" && \
echo "--- Running pylint ---" && \
- bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && find $${SEARCH_PATH} -name '*.py' | sort -u | xargs pylint -j0" && \
+ bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && find $${SEARCH_PATH} -name '*.py' | sort -u | xargs pylint -j0 $(PYLINT_ARGS)" && \
echo "--- Running flake8 ---" && \
- bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && flake8 $${SEARCH_PATH}"
+ bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && flake8 $${SEARCH_PATH} $(FLAKE8_ARGS)"
if [[ "$(_PYTHON_VENV)" == "python2.7" ]] ; then \
. $(VENVNAME)/bin/activate; \
echo "--- Checking py3 compatibility ---" && \
SEARCH_PATH=$(REPOS_PATH) && \
- bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && find $${SEARCH_PATH} -name '*.py' | sort -u | xargs pylint --py3k" && \
+ bash -c "[[ ! -z '$${SEARCH_PATH}' ]] && find $${SEARCH_PATH} -name '*.py' | sort -u | xargs pylint --py3k $(PYLINT_ARGS)" && \
echo "--- Linting done. ---"; \
fi
@@ -358,7 +367,7 @@ test_no_lint:
cd repos/system_upgrade/el7toel8/; \
snactor workflow sanity-check ipu && \
cd - && \
- $(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(TEST_PATHS) $(LIBRARY_PATH)
+ $(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(TEST_PATHS) $(LIBRARY_PATH) $(PYTEST_ARGS)
test: lint test_no_lint
@@ -474,14 +483,14 @@ fast_lint:
@. $(VENVNAME)/bin/activate; \
FILES_TO_LINT="$$(git diff --name-only $(MASTER_BRANCH) --diff-filter AMR | grep '\.py$$')"; \
if [[ -n "$$FILES_TO_LINT" ]]; then \
- pylint -j 0 $$FILES_TO_LINT && \
- flake8 $$FILES_TO_LINT; \
+ pylint -j 0 $$FILES_TO_LINT $(PYLINT_ARGS) && \
+ flake8 $$FILES_TO_LINT $(FLAKE8_ARG); \
LINT_EXIT_CODE="$$?"; \
if [[ "$$LINT_EXIT_CODE" != "0" ]]; then \
exit $$LINT_EXIT_CODE; \
fi; \
if [[ "$(_PYTHON_VENV)" == "python2.7" ]] ; then \
- pylint --py3k $$FILES_TO_LINT; \
+ pylint --py3k $$FILES_TO_LINT $(PYLINT_ARGS); \
fi; \
else \
echo "No files to lint."; \
@@ -489,7 +498,7 @@ fast_lint:
dev_test_no_lint:
. $(VENVNAME)/bin/activate; \
- $(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(APPROX_TEST_PATHS) $(LIBRARY_PATH)
+ $(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(APPROX_TEST_PATHS) $(LIBRARY_PATH) $(PYTEST_ARGS)
dashboard_data:
. $(VENVNAME)/bin/activate; \
--
2.43.0

122
0055-pes_events_scanner-Ignore-Leapp-related-PES-events.patch Normal file
View File

@ -0,0 +1,122 @@
From e414f7c6572af4293cacadd810154677892c4028 Mon Sep 17 00:00:00 2001
From: Matej Matuska <mmatuska@redhat.com>
Date: Thu, 23 Nov 2023 17:38:35 +0100
Subject: [PATCH 55/60] pes_events_scanner: Ignore Leapp related PES events
When PES events are added for all the Leapp related packages, we need to
make sure to ignore them in `pes_events_scanner` to make sure they are
*not* taken into account during the RPM upgrade transaction as we don't
want to upgrade them or want to handle their upgrade in a different way.
Jira: OAMG-5645
---
.../libraries/pes_events_scanner.py | 16 +++++
.../tests/test_pes_event_scanner.py | 61 +++++++++++++++++++
2 files changed, 77 insertions(+)
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
index 72dd34ec..75c3ea89 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
@@ -480,6 +480,21 @@ def apply_transaction_configuration(source_pkgs):
return source_pkgs_with_conf_applied
+def remove_leapp_related_events(events):
+ leapp_pkgs = [
+ 'leapp', 'leapp-deps', 'leapp-upgrade-el7toel8', 'leapp-upgrade-el8toel9',
+ 'leapp-upgrade-el7toel8-deps', 'leapp-upgrade-el8toel9-deps', 'python2-leapp',
+ 'python3-leapp', 'snactor'
+ ]
+ res = []
+ for event in events:
+ if not any(pkg.name in leapp_pkgs for pkg in event.in_pkgs):
+ res.append(event)
+ else:
+ api.current_logger().debug('Filtered out leapp related event, event id: {}'.format(event.id))
+ return res
+
+
def process():
# Retrieve data - installed_pkgs, transaction configuration, pes events
events = get_pes_events('/etc/leapp/files', 'pes-events.json')
@@ -494,6 +509,7 @@ def process():
# packages of the target system, so we can distinguish what needs to be repomapped
repoids_of_source_pkgs = {pkg.repository for pkg in source_pkgs}
+ events = remove_leapp_related_events(events)
events = remove_undesired_events(events, releases)
# Apply events - compute what packages should the target system have
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py b/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
index 243f85c4..8150c164 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
@@ -402,3 +402,64 @@ def test_pkgs_are_demodularized_when_crossing_major_version(monkeypatch):
}
assert demodularized_pkgs == {Package('demodularized', 'repo', ('module-demodularized', 'stream'))}
assert target_pkgs == expected_target_pkgs
+
+
+def test_remove_leapp_related_events():
+ # these are just hypothetical and not necessarily correct
+ package_set_two_leapp = {Package('leapp-upgrade-el7toel8', 'repoid-rhel7', None),
+ Package('leapp-upgrade-el7toel8-deps', 'repoid-rhel7', None)}
+ package_set_one_leapp = {Package('leapp-upgrade-el7toel8', 'repoid-rhel7', None),
+ Package('other', 'repoid-rhel7', None)}
+ in_events = [
+ Event(1, Action.PRESENT, {Package('leapp', 'repoid-rhel7', None)},
+ {Package('leapp', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+
+ Event(1, Action.RENAMED, {Package('leapp-deps', 'repoid-rhel7', None)},
+ {Package('leapp-deps', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+ Event(1, Action.RENAMED, {Package('leapp-upgrade-el7toel8', 'repoid-rhel7', None)},
+ {Package('leapp-upgrade-el8toel9', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+ Event(2, Action.RENAMED, {Package('leapp-upgrade-el7toel8-deps', 'repoid-rhel7', None)},
+ {Package('leapp-upgrade-el8toel9-deps', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+ Event(2, Action.PRESENT, {Package('snactor', 'repoid-rhel7', None)},
+ {Package('snactor', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+ Event(2, Action.REPLACED, {Package('python2-leapp', 'repoid-rhel7', None)},
+ {Package('python3-leapp', 'repoid-rhel8', None)},
+ (7, 0), (8, 0), []),
+
+ Event(1, Action.DEPRECATED, {Package('leapp-upgrade-el8toel9', 'repoid-rhel8', None)},
+ {Package('leapp-upgrade-el8toel9', 'repoid-rhel9', None)}, (8, 0), (9, 0), []),
+ Event(2, Action.REMOVED, {Package('leapp-upgrade-el8toel9-deps', 'repoid-rhel8', None)},
+ {}, (8, 0), (9, 0), []),
+ Event(1, Action.RENAMED, {Package('leapp-deps', 'repoid-rhel8', None)},
+ {Package('leapp-deps', 'repoid-rhel9', None)}, (8, 0), (9, 0), []),
+ Event(2, Action.PRESENT, {Package('snactor', 'repoid-rhel8', None)},
+ {Package('snactor', 'repoid-rhel9', None)}, (8, 0), (9, 0), []),
+ Event(2, Action.REMOVED, {Package('python3-leapp', 'repoid-rhel8', None)},
+ {Package('snactor', 'repoid-rhel9', None)}, (8, 0), (9, 0), []),
+
+ Event(2, Action.PRESENT, {Package('other-pkg', 'repoid-rhel8', None)},
+ {Package('other-pkg', 'repoid-rhel9', None)}, (7, 0), (8, 0), []),
+ Event(2, Action.PRESENT, {Package('other-pkg-with-leapp-in-the-name', 'repoid-rhel7', None)},
+ {Package('other-pkg-with-leapp-in-the-name', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+
+ # multiple leapp packages in in_pkgs
+ Event(1, Action.MERGED, package_set_two_leapp, {Package('leapp-upgrade-el7toel8', 'repoid-rhel8', None)},
+ (7, 0), (8, 0), []),
+
+ # multiple leapp packages in out_pkgs
+ Event(1, Action.SPLIT, {Package('leapp-upgrade-el7toel8', 'repoid-rhel7', None)},
+ package_set_two_leapp, (7, 0), (8, 0), []),
+
+ # leapp and other pkg in in_pkgs
+ Event(1, Action.MERGED, package_set_one_leapp, {Package('leapp', 'repoid-rhel8', None)},
+ (7, 0), (8, 0), []),
+ ]
+ expected_out_events = [
+ Event(2, Action.PRESENT, {Package('other-pkg', 'repoid-rhel8', None)},
+ {Package('other-pkg', 'repoid-rhel9', None)}, (7, 0), (8, 0), []),
+ Event(2, Action.PRESENT, {Package('other-pkg-with-leapp-in-the-name', 'repoid-rhel7', None)},
+ {Package('other-pkg-with-leapp-in-the-name', 'repoid-rhel8', None)}, (7, 0), (8, 0), []),
+ ]
+
+ out_events = pes_events_scanner.remove_leapp_related_events(in_events)
+ assert out_events == expected_out_events
--
2.43.0

61
0056-Use-library-functions-for-getting-leapp-packages.patch Normal file
View File

@ -0,0 +1,61 @@
From 14667eef1fbec335780f995af89e0c0fb8dc25ba Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Thu, 11 Jan 2024 14:27:56 +0100
Subject: [PATCH 56/60] Use library functions for getting leapp packages
Instead of harcoded list of leapp packages let's rely on
native leapp library functions that were introduced a few
commits ago.
OAMG-5645
---
.../peseventsscanner/libraries/pes_events_scanner.py | 9 ++++-----
.../peseventsscanner/tests/test_pes_event_scanner.py | 5 ++++-
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
index 75c3ea89..f9411dfe 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/libraries/pes_events_scanner.py
@@ -5,6 +5,7 @@ from leapp import reporting
from leapp.exceptions import StopActorExecutionError
from leapp.libraries.actor import peseventsscanner_repomap
from leapp.libraries.actor.pes_event_parsing import Action, get_pes_events, Package
+from leapp.libraries.common import rpms
from leapp.libraries.common.config import version
from leapp.libraries.stdlib import api
from leapp.libraries.stdlib.config import is_verbose
@@ -481,11 +482,9 @@ def apply_transaction_configuration(source_pkgs):
def remove_leapp_related_events(events):
- leapp_pkgs = [
- 'leapp', 'leapp-deps', 'leapp-upgrade-el7toel8', 'leapp-upgrade-el8toel9',
- 'leapp-upgrade-el7toel8-deps', 'leapp-upgrade-el8toel9-deps', 'python2-leapp',
- 'python3-leapp', 'snactor'
- ]
+ # NOTE(ivasilev) Need to revisit this once rhel9->rhel10 upgrades become a thing
+ leapp_pkgs = rpms.get_leapp_dep_packages(
+ major_version=['7', '8']) + rpms.get_leapp_packages(major_version=['7', '8'])
res = []
for event in events:
if not any(pkg.name in leapp_pkgs for pkg in event.in_pkgs):
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py b/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
index 8150c164..7cdcf820 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/tests/test_pes_event_scanner.py
@@ -404,7 +404,10 @@ def test_pkgs_are_demodularized_when_crossing_major_version(monkeypatch):
assert target_pkgs == expected_target_pkgs
-def test_remove_leapp_related_events():
+def test_remove_leapp_related_events(monkeypatch):
+ # NOTE(ivasilev) That's required to use leapp library functions that rely on calls to
+ # get_source/target_system_version functions
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(arch='x86_64', src_ver='7.9', dst_ver='8.8'))
# these are just hypothetical and not necessarily correct
package_set_two_leapp = {Package('leapp-upgrade-el7toel8', 'repoid-rhel7', None),
Package('leapp-upgrade-el7toel8-deps', 'repoid-rhel7', None)}
--
2.43.0

233
0057-Introduce-TrackedFilesInfoSource-message-and-new-act.patch Normal file
View File

@ -0,0 +1,233 @@
From 1afd0fb1a0ed7354e7ed525bf0de3b883eddff8e Mon Sep 17 00:00:00 2001
From: Petr Stodulka <pstodulk@redhat.com>
Date: Thu, 19 Oct 2023 18:44:06 +0200
Subject: [PATCH 57/60] Introduce TrackedFilesInfoSource message and new actor
We hit already several times a situation that an actor needed an
information about specific file (whether exists, has been changed,...).
And for that purpose extra scanner actor needed to be created, with
an associated message and Model.
To cover such cases, we are introducing new model
TrackedFilesInfoSource and actor scansourcefiles. So in future, when
any actor needs such a piece of information and do something based
on it, developer can just update lists in the introduced actor's
library, so the information about particular file will be provided.
Another benefit is saving a time on writting new unit tests and code
for the scan, as updating a list of files to be tracked does not
affect the algorithm.
---
.../common/actors/scansourcefiles/actor.py | 32 ++++++++
.../libraries/scansourcefiles.py | 79 +++++++++++++++++++
.../tests/unit_test_scansourcefiles.py | 5 ++
.../common/models/trackedfiles.py | 60 ++++++++++++++
4 files changed, 176 insertions(+)
create mode 100644 repos/system_upgrade/common/actors/scansourcefiles/actor.py
create mode 100644 repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
create mode 100644 repos/system_upgrade/common/actors/scansourcefiles/tests/unit_test_scansourcefiles.py
create mode 100644 repos/system_upgrade/common/models/trackedfiles.py
diff --git a/repos/system_upgrade/common/actors/scansourcefiles/actor.py b/repos/system_upgrade/common/actors/scansourcefiles/actor.py
new file mode 100644
index 00000000..b368fc88
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scansourcefiles/actor.py
@@ -0,0 +1,32 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import scansourcefiles
+from leapp.models import TrackedFilesInfoSource
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+
+
+class ScanSourceFiles(Actor):
+ """
+ Scan files (explicitly specified) of the source system.
+
+ If an actor require information about a file, like whether it's installed,
+ modified, etc. It can be added to the list of files to be tracked, so no
+ extra actor is required to be created to provide just that one information.
+
+ The scan of all changed files tracked by RPMs is very expensive. So we rather
+ provide this possibility to simplify the work for others.
+
+ See lists defined in the private library.
+ """
+ # TODO(pstodulk): in some cases could be valuable to specify an rpm name
+ # and provide information about all changed files instead. Both approaches
+ # have a little bit different use-cases and expectations. In the second
+ # case it would be good solution regarding track of leapp-repository
+ # changed files.
+
+ name = 'scan_source_files'
+ consumes = ()
+ produces = (TrackedFilesInfoSource,)
+ tags = (IPUWorkflowTag, FactsPhaseTag)
+
+ def process(self):
+ scansourcefiles.process()
diff --git a/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py b/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
new file mode 100644
index 00000000..33e0275f
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
@@ -0,0 +1,79 @@
+import os
+
+from leapp.libraries.common.config.version import get_source_major_version
+from leapp.libraries.stdlib import api, CalledProcessError, run
+from leapp.models import FileInfo, TrackedFilesInfoSource
+
+# TODO(pstodulk): make linter happy about this
+# common -> Files supposed to be scanned on all system versions.
+# '8' (etc..) -> files supposed to be scanned when particular major version of OS is used
+TRACKED_FILES = {
+ 'common': [
+ ],
+ '8': [
+ ],
+ '9': [
+ ],
+}
+
+# TODO(pstodulk)?: introduce possibility to discover files under a dir that
+# are not tracked by any rpm or a specified rpm? Currently I have only one
+# use case for that in my head, so possibly it will be better to skip a generic
+# solution and just introduce a new actor and msg for that (check whether
+# actors not owned by our package(s) are present).
+
+
+def _get_rpm_name(input_file):
+ try:
+ rpm_names = run(['rpm', '-qf', '--queryformat', r'%{NAME}\n', input_file], split=True)['stdout']
+ except CalledProcessError:
+ # is not owned by any rpm
+ return ''
+
+ if len(rpm_names) > 1:
+ # this is very seatbelt; could happen for directories, but we do
+ # not expect here directories specified at all. if so, we should
+ # provide list instead of string
+ api.current_logger().warning(
+ 'The {} file is owned by multiple rpms: {}.'
+ .format(input_file, ', '.join(rpm_names))
+ )
+ return rpm_names[0]
+
+
+def is_modified(input_file):
+ """
+ Return True if checksum has been changed (or removed).
+
+ Ignores mode, user, type, ...
+ """
+ result = run(['rpm', '-Vf', '--nomtime', input_file], checked=False)
+ if not result['exit_code']:
+ return False
+ status = result['stdout'].split()[0]
+ return status == 'missing' or '5' in status
+
+
+def scan_file(input_file):
+ data = {
+ 'path': input_file,
+ 'exists': os.path.exists(input_file),
+ 'rpm_name': _get_rpm_name(input_file),
+ }
+
+ if data['rpm_name']:
+ data['is_modified'] = is_modified(input_file)
+ else:
+ # it's not tracked by any rpm at all, so always False
+ data['is_modified'] = False
+
+ return FileInfo(**data)
+
+
+def scan_files(files):
+ return [scan_file(fname) for fname in files]
+
+
+def process():
+ files = scan_files(TRACKED_FILES['common'] + TRACKED_FILES.get(get_source_major_version(), []))
+ api.produce(TrackedFilesInfoSource(files=files))
diff --git a/repos/system_upgrade/common/actors/scansourcefiles/tests/unit_test_scansourcefiles.py b/repos/system_upgrade/common/actors/scansourcefiles/tests/unit_test_scansourcefiles.py
new file mode 100644
index 00000000..6a6b009a
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scansourcefiles/tests/unit_test_scansourcefiles.py
@@ -0,0 +1,5 @@
+def test_scansourcefiles():
+ # TODO(pstodulk): keeping unit tests for later after I check the idea
+ # of this actor with the team.
+ # JIRA: OAMG-10367
+ pass
diff --git a/repos/system_upgrade/common/models/trackedfiles.py b/repos/system_upgrade/common/models/trackedfiles.py
new file mode 100644
index 00000000..f7c2c809
--- /dev/null
+++ b/repos/system_upgrade/common/models/trackedfiles.py
@@ -0,0 +1,60 @@
+from leapp.models import fields, Model
+from leapp.topics import SystemInfoTopic
+
+
+class FileInfo(Model):
+ """
+ Various data about a file.
+
+ This model is not supposed to be used as a message directly.
+ See e.g. :class:`TrackedSourceFilesInfo` instead.
+ """
+ topic = SystemInfoTopic
+
+ path = fields.String()
+ """
+ Canonical path to the file.
+ """
+
+ exists = fields.Boolean()
+ """
+ True if the file is present on the system.
+ """
+
+ rpm_name = fields.String(default="")
+ """
+ Name of the rpm that owns the file. Otherwise empty string if not owned
+ by any rpm.
+ """
+
+ # NOTE(pstodulk): I have been thinking about the "state"/"modified" field
+ # instead. Which could contain enum list, where could be specified what has
+ # been changed (checksum, type, owner, ...). But currently we do not have
+ # use cases for that and do not want to implement it now. So starting simply
+ # with this one.
+ is_modified = fields.Boolean()
+ """
+ True if the checksum of the file has been changed (includes the missing state).
+
+ The field is valid only for a file tracked by rpm - excluding ghost files.
+ In such a case the value is always false.
+ """
+
+
+class TrackedFilesInfoSource(Model):
+ """
+ Provide information about files on the source system explicitly defined
+ in the actor to be tracked.
+
+ Search an actor producing this message to discover the list where you
+ could add the file into the list to be tracked.
+
+ This particular message is expected to be produced only once by the
+ specific actor. Do not produce multiple messages of this model.
+ """
+ topic = SystemInfoTopic
+
+ files = fields.List(fields.Model(FileInfo), default=[])
+ """
+ List of :class:`FileInfo`.
+ """
--
2.43.0

581
0058-Add-actors-for-OpenSSL-conf-and-IBMCA.patch Normal file
View File

@ -0,0 +1,581 @@
From c8321a9da33ecfb71d4f6ebd03c4b334f9e91dcc Mon Sep 17 00:00:00 2001
From: Petr Stodulka <pstodulk@redhat.com>
Date: Fri, 20 Oct 2023 16:40:09 +0200
Subject: [PATCH 58/60] Add actors for OpenSSL conf and IBMCA
* The openssl-ibmca needs to be reconfigured manually after the upgrade.
Report it to the user if the package is installed.
* The openssl configuration file (/etc/pki/tls/openssl.cnf) is not
100% compatible between major verions of RHEL due to different
versions of OpenSSL. Also the configuration is supposed to be
done via system wide crypto policies instead, so it's expected
to not modify this file anymore. If the content of the file has
been modified, report to user what will happen during the upgrade
and what they should do after it.
* If the openssl config file is modified (rpm -Vf <file>) and
*.rpmnew file exists, back up the file with .leappsave suffix
and replace it by the *.rpmsave one.
---
.../actors/openssl/checkopensslconf/actor.py | 33 ++++
.../libraries/checkopensslconf.py | 135 ++++++++++++++++
.../tests/unit_test_checkopensslconf.py | 102 ++++++++++++
.../openssl/migrateopensslconf/actor.py | 26 ++++
.../libraries/migrateopensslconf.py | 54 +++++++
.../tests/unit_test_migrateopensslconf.py | 145 ++++++++++++++++++
.../libraries/scansourcefiles.py | 1 +
7 files changed, 496 insertions(+)
create mode 100644 repos/system_upgrade/common/actors/openssl/checkopensslconf/actor.py
create mode 100644 repos/system_upgrade/common/actors/openssl/checkopensslconf/libraries/checkopensslconf.py
create mode 100644 repos/system_upgrade/common/actors/openssl/checkopensslconf/tests/unit_test_checkopensslconf.py
create mode 100644 repos/system_upgrade/common/actors/openssl/migrateopensslconf/actor.py
create mode 100644 repos/system_upgrade/common/actors/openssl/migrateopensslconf/libraries/migrateopensslconf.py
create mode 100644 repos/system_upgrade/common/actors/openssl/migrateopensslconf/tests/unit_test_migrateopensslconf.py
diff --git a/repos/system_upgrade/common/actors/openssl/checkopensslconf/actor.py b/repos/system_upgrade/common/actors/openssl/checkopensslconf/actor.py
new file mode 100644
index 00000000..dd05db9c
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/checkopensslconf/actor.py
@@ -0,0 +1,33 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import checkopensslconf
+from leapp.models import DistributionSignedRPM, Report, TrackedFilesInfoSource
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+
+
+class CheckOpenSSLConf(Actor):
+ """
+ Check whether the openssl configuration and openssl-IBMCA.
+
+ See the report messages for more details. The summary is that since RHEL 8
+ it's expected to configure OpenSSL via crypto policies. Also, OpenSSL has
+ different versions between major versions of RHEL:
+ * RHEL 7: 1.0,
+ * RHEL 8: 1.1,
+ * RHEL 9: 3.0
+ So OpenSSL configuration from older system does not have to be 100%
+ compatible with the new system. In some cases, the old configuration could
+ make the system inaccessible remotely. So new approach is to ensure the
+ upgraded system will use always new default /etc/pki/tls/openssl.cnf
+ configuration file (the original one will be backed up if modified by user).
+
+ Similar for OpenSSL-IBMCA, when it's expected to configure it again on
+ each newer system.
+ """
+
+ name = 'check_openssl_conf'
+ consumes = (DistributionSignedRPM, TrackedFilesInfoSource)
+ produces = (Report,)
+ tags = (IPUWorkflowTag, ChecksPhaseTag)
+
+ def process(self):
+ checkopensslconf.process()
diff --git a/repos/system_upgrade/common/actors/openssl/checkopensslconf/libraries/checkopensslconf.py b/repos/system_upgrade/common/actors/openssl/checkopensslconf/libraries/checkopensslconf.py
new file mode 100644
index 00000000..06a30fa1
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/checkopensslconf/libraries/checkopensslconf.py
@@ -0,0 +1,135 @@
+from leapp import reporting
+from leapp.libraries.common.config import architecture, version
+from leapp.libraries.common.rpms import has_package
+from leapp.libraries.stdlib import api
+from leapp.models import DistributionSignedRPM, TrackedFilesInfoSource
+
+DEFAULT_OPENSSL_CONF = '/etc/pki/tls/openssl.cnf'
+URL_8_CRYPTOPOLICIES = 'https://red.ht/rhel-8-system-wide-crypto-policies'
+URL_9_CRYPTOPOLICIES = 'https://red.ht/rhel-9-system-wide-crypto-policies'
+
+
+def check_ibmca():
+ if not architecture.matches_architecture(architecture.ARCH_S390X):
+ # not needed check really, but keeping it to make it clear
+ return
+ if not has_package(DistributionSignedRPM, 'openssl-ibmca'):
+ return
+ # In RHEL 9 has been introduced new technology: openssl providers. The engine
+ # is deprecated, so keep proper teminology to not confuse users.
+ dst_tech = 'engine' if version.get_target_major_version() == '8' else 'providers'
+ summary = (
+ 'The presence of openssl-ibmca package suggests that the system may be configured'
+ ' to use the IBMCA OpenSSL engine.'
+ ' Due to major changes in OpenSSL and libica between RHEL {source} and RHEL {target} it is not'
+ ' possible to migrate OpenSSL configuration files automatically. Therefore,'
+ ' it is necessary to enable IBMCA {tech} in the OpenSSL config file manually'
+ ' after the system upgrade.'
+ .format(
+ source=version.get_source_major_version(),
+ target=version.get_target_major_version(),
+ tech=dst_tech
+ )
+ )
+
+ hint = (
+ 'Configure the IBMCA {tech} manually after the upgrade.'
+ ' Please, be aware that it is not recommended to configure the system default'
+ ' {fpath}. Instead, it is recommended to configure a copy of'
+ ' that file and use this copy only for particular applications that are supposed'
+ ' to utilize the IBMCA {tech}. The location of the OpenSSL configuration file'
+ ' can be specified using the OPENSSL_CONF environment variable.'
+ .format(tech=dst_tech, fpath=DEFAULT_OPENSSL_CONF)
+ )
+
+ reporting.create_report([
+ reporting.Title('Detected possible use of IBMCA in OpenSSL'),
+ reporting.Summary(summary),
+ reporting.Remediation(hint=hint),
+ reporting.Severity(reporting.Severity.MEDIUM),
+ reporting.Groups([
+ reporting.Groups.POST,
+ reporting.Groups.ENCRYPTION
+ ]),
+ ])
+
+
+def _is_openssl_modified():
+ tracked_files = next(api.consume(TrackedFilesInfoSource), None)
+ if not tracked_files:
+ # unexpected at all, skipping testing, but keeping the log just in case
+ api.current_logger.warning('The TrackedFilesInfoSource message is missing! Skipping check of openssl config.')
+ return False
+ for finfo in tracked_files.files:
+ if finfo.path == DEFAULT_OPENSSL_CONF:
+ return finfo.is_modified
+ return False
+
+
+def check_default_openssl():
+ if not _is_openssl_modified():
+ return
+
+ crypto_url = URL_8_CRYPTOPOLICIES if version.get_target_major_version == '8' else URL_9_CRYPTOPOLICIES
+
+ # TODO(pstodulk): Needs in future some rewording, as OpenSSL engines are
+ # deprecated since "RHEL 8" and people should use OpenSSL providers instead.
+ # (IIRC, they are required to use OpenSSL providers since RHEL 9.) The
+ # current wording could be inaccurate.
+ summary = (
+ 'The OpenSSL configuration file ({fpath}) has been'
+ ' modified on the system. RHEL 8 (and newer) systems provide a crypto-policies'
+ ' mechanism ensuring usage of system-wide secure cryptography algorithms.'
+ ' Also the target system uses newer version of OpenSSL that is not fully'
+ ' compatible with the current one.'
+ ' To ensure the upgraded system uses crypto-policies as expected,'
+ ' the new version of the openssl configuration file must be installed'
+ ' during the upgrade. This will be done automatically.'
+ ' The original configuration file will be saved'
+ ' as "{fpath}.leappsave".'
+ '\n\nNote this can affect the ability to connect to the system after'
+ ' the upgrade if it depends on the current OpenSSL configuration.'
+ ' Such a problem may be caused by using a particular OpenSSL engine, as'
+ ' OpenSSL engines built for the'
+ ' RHEL {source} system are not compatible with RHEL {target}.'
+ .format(
+ fpath=DEFAULT_OPENSSL_CONF,
+ source=version.get_source_major_version(),
+ target=version.get_target_major_version()
+ )
+ )
+ if version.get_target_major_version() == '9':
+ # NOTE(pstodulk): that a try to make things with engine/providers a
+ # little bit better (see my TODO note above)
+ summary += (
+ '\n\nNote the legacy ENGINE API is deprecated since RHEL 8 and'
+ ' it is required to use the new OpenSSL providers API instead on'
+ ' RHEL 9 systems.'
+ )
+ hint = (
+ 'Check that your ability to login to the system does not depend on'
+ ' the OpenSSL configuration. After the upgrade, review the system configuration'
+ ' and configure the system as needed.'
+ ' Please, be aware that it is not recommended to configure the system default'
+ ' {fpath}. Instead, it is recommended to copy the file and use this copy'
+ ' to configure particular applications.'
+ ' The default OpenSSL configuration file should be modified only'
+ ' when it is really necessary.'
+ )
+ reporting.create_report([
+ reporting.Title('The /etc/pki/tls/openssl.cnf file is modified and will be replaced during the upgrade.'),
+ reporting.Summary(summary),
+ reporting.Remediation(hint=hint),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Groups([reporting.Groups.POST, reporting.Groups.SECURITY]),
+ reporting.RelatedResource('file', DEFAULT_OPENSSL_CONF),
+ reporting.ExternalLink(
+ title='Using system-wide cryptographic policies.',
+ url=crypto_url
+ )
+ ])
+
+
+def process():
+ check_ibmca()
+ check_default_openssl()
diff --git a/repos/system_upgrade/common/actors/openssl/checkopensslconf/tests/unit_test_checkopensslconf.py b/repos/system_upgrade/common/actors/openssl/checkopensslconf/tests/unit_test_checkopensslconf.py
new file mode 100644
index 00000000..541ff75d
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/checkopensslconf/tests/unit_test_checkopensslconf.py
@@ -0,0 +1,102 @@
+import pytest
+
+from leapp import reporting
+from leapp.libraries.actor import checkopensslconf
+from leapp.libraries.common.config import architecture
+from leapp.libraries.common.testutils import create_report_mocked, CurrentActorMocked, logger_mocked
+from leapp.libraries.stdlib import api
+from leapp.models import DistributionSignedRPM, FileInfo, RPM, TrackedFilesInfoSource
+
+_DUMP_PKG_NAMES = ['random', 'pkgs', 'openssl-ibmca-nope', 'ibmca', 'nope-openssl-ibmca']
+_SSL_CONF = checkopensslconf.DEFAULT_OPENSSL_CONF
+
+
+def _msg_pkgs(pkgnames):
+ rpms = []
+ for pname in pkgnames:
+ rpms.append(RPM(
+ name=pname,
+ epoch='0',
+ version='1.0',
+ release='1',
+ arch='noarch',
+ pgpsig='RSA/SHA256, Mon 01 Jan 1970 00:00:00 AM -03, Key ID 199e2f91fd431d51',
+ packager='Red Hat, Inc. (auxiliary key 2) <security@redhat.com>'
+
+ ))
+ return DistributionSignedRPM(items=rpms)
+
+
+@pytest.mark.parametrize('arch,pkgnames,ibmca_report', (
+ (architecture.ARCH_S390X, [], False),
+ (architecture.ARCH_S390X, _DUMP_PKG_NAMES, False),
+ (architecture.ARCH_S390X, ['openssl-ibmca'], True),
+ (architecture.ARCH_S390X, _DUMP_PKG_NAMES + ['openssl-ibmca'], True),
+ (architecture.ARCH_S390X, ['openssl-ibmca'] + _DUMP_PKG_NAMES, True),
+
+ # stay false for non-IBM-z arch - invalid scenario basically
+ (architecture.ARCH_X86_64, ['openssl-ibmca'], False),
+ (architecture.ARCH_PPC64LE, ['openssl-ibmca'], False),
+ (architecture.ARCH_ARM64, ['openssl-ibmca'], False),
+
+))
+@pytest.mark.parametrize('src_maj_ver', ('7', '8', '9'))
+def test_check_ibmca(monkeypatch, src_maj_ver, arch, pkgnames, ibmca_report):
+ monkeypatch.setattr(reporting, "create_report", create_report_mocked())
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(
+ arch=arch,
+ msgs=[_msg_pkgs(pkgnames)],
+ src_ver='{}.6'.format(src_maj_ver),
+ dst_ver='{}.0'.format(int(src_maj_ver) + 1)
+ ))
+ checkopensslconf.check_ibmca()
+
+ if not ibmca_report:
+ assert not reporting.create_report.called, 'IBMCA report created when it should not.'
+ else:
+ assert reporting.create_report.called, 'IBMCA report has not been created.'
+
+
+def _msg_files(fnames_changed, fnames_untouched):
+ res = []
+ for fname in fnames_changed:
+ res.append(FileInfo(
+ path=fname,
+ exists=True,
+ is_modified=True
+ ))
+
+ for fname in fnames_untouched:
+ res.append(FileInfo(
+ path=fname,
+ exists=True,
+ is_modified=False
+ ))
+
+ return TrackedFilesInfoSource(files=res)
+
+
+# NOTE(pstodulk): Ignoring situation when _SSL_CONF is missing (modified, do not exists).
+# It's not a valid scenario actually, as this file just must exists on the system to
+# consider it in a supported state.
+@pytest.mark.parametrize('msg,openssl_report', (
+ # matrix focused on openssl reports only (positive)
+ (_msg_files([], []), False),
+ (_msg_files([_SSL_CONF], []), True),
+ (_msg_files(['what/ever', _SSL_CONF, 'something'], []), True),
+ (_msg_files(['what/ever'], [_SSL_CONF]), False),
+))
+@pytest.mark.parametrize('src_maj_ver', ('7', '8', '9'))
+def test_check_openssl(monkeypatch, src_maj_ver, msg, openssl_report):
+ monkeypatch.setattr(reporting, "create_report", create_report_mocked())
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(
+ msgs=[msg],
+ src_ver='{}.6'.format(src_maj_ver),
+ dst_ver='{}.0'.format(int(src_maj_ver) + 1)
+ ))
+ checkopensslconf.process()
+
+ if not openssl_report:
+ assert not reporting.create_report.called, 'OpenSSL report created when it should not.'
+ else:
+ assert reporting.create_report.called, 'OpenSSL report has not been created.'
diff --git a/repos/system_upgrade/common/actors/openssl/migrateopensslconf/actor.py b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/actor.py
new file mode 100644
index 00000000..f373b5c4
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/actor.py
@@ -0,0 +1,26 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import migrateopensslconf
+from leapp.tags import ApplicationsPhaseTag, IPUWorkflowTag
+
+
+class MigrateOpenSslConf(Actor):
+ """
+ Enforce the target default configuration file to be used.
+
+ If the /etc/pki/tls/openssl.cnf has been modified and openssl.cnf.rpmnew
+ file is created, backup the original one and replace it by the new default.
+
+ tl;dr: (simplified)
+ if the file is modified; then
+ mv /etc/pki/tls/openssl.cnf{,.leappsave}
+ mv /etc/pki/tls/openssl.cnf{.rpmnew,}
+ fi
+ """
+
+ name = 'migrate_openssl_conf'
+ consumes = ()
+ produces = ()
+ tags = (IPUWorkflowTag, ApplicationsPhaseTag)
+
+ def process(self):
+ migrateopensslconf.process()
diff --git a/repos/system_upgrade/common/actors/openssl/migrateopensslconf/libraries/migrateopensslconf.py b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/libraries/migrateopensslconf.py
new file mode 100644
index 00000000..140c5718
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/libraries/migrateopensslconf.py
@@ -0,0 +1,54 @@
+import os
+
+from leapp.libraries.stdlib import api, CalledProcessError, run
+
+DEFAULT_OPENSSL_CONF = '/etc/pki/tls/openssl.cnf'
+OPENSSL_CONF_RPMNEW = '{}.rpmnew'.format(DEFAULT_OPENSSL_CONF)
+OPENSSL_CONF_BACKUP = '{}.leappsave'.format(DEFAULT_OPENSSL_CONF)
+
+
+def _is_openssl_modified():
+ """
+ Return True if modified in any way
+ """
+ # NOTE(pstodulk): this is different from the approach in scansourcefiles,
+ # where we are interested about modified content. In this case, if the
+ # file is modified in any way, let's do something about that..
+ try:
+ run(['rpm', '-Vf', DEFAULT_OPENSSL_CONF])
+ except CalledProcessError:
+ return True
+ return False
+
+
+def _safe_mv_file(src, dst):
+ """
+ Move the file from src to dst. Return True on success, otherwise False.
+ """
+ try:
+ run(['mv', src, dst])
+ except CalledProcessError:
+ return False
+ return True
+
+
+def process():
+ if not _is_openssl_modified():
+ return
+ if not os.path.exists(OPENSSL_CONF_RPMNEW):
+ api.current_logger().debug('The {} file is modified, but *.rpmsave not found. Cannot do anything.')
+ return
+ if not _safe_mv_file(DEFAULT_OPENSSL_CONF, OPENSSL_CONF_BACKUP):
+ # NOTE(pstodulk): One of reasons could be the file is missing, however
+ # that's not expected to happen at all. If the file is missing before
+ # the upgrade, it will be installed by new openssl* package
+ api.current_logger().error(
+ 'Could not back up the {} file. Skipping other actions.'
+ .format(DEFAULT_OPENSSL_CONF)
+ )
+ return
+ if not _safe_mv_file(OPENSSL_CONF_RPMNEW, DEFAULT_OPENSSL_CONF):
+ # unexpected, it's double seatbelt
+ api.current_logger().error('Cannot apply the new openssl configuration file! Restore it from the backup.')
+ if not _safe_mv_file(OPENSSL_CONF_BACKUP, DEFAULT_OPENSSL_CONF):
+ api.current_logger().error('Cannot restore the openssl configuration file!')
diff --git a/repos/system_upgrade/common/actors/openssl/migrateopensslconf/tests/unit_test_migrateopensslconf.py b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/tests/unit_test_migrateopensslconf.py
new file mode 100644
index 00000000..e9200312
--- /dev/null
+++ b/repos/system_upgrade/common/actors/openssl/migrateopensslconf/tests/unit_test_migrateopensslconf.py
@@ -0,0 +1,145 @@
+import os
+
+import pytest
+
+from leapp.libraries.actor import migrateopensslconf
+from leapp.libraries.common.testutils import CurrentActorMocked, logger_mocked
+from leapp.libraries.stdlib import CalledProcessError
+
+
+class PathExistsMocked(object):
+ def __init__(self, existing_files=None):
+ self.called = 0
+ self._existing_files = existing_files if existing_files else []
+
+ def __call__(self, fpath):
+ self.called += 1
+ return fpath in self._existing_files
+
+
+class IsOpensslModifiedMocked(object):
+ def __init__(self, ret_values):
+ self._ret_values = ret_values
+ # ret_values is list of bools to return on each call. ret_values.pop(0)
+ # if the list becomes empty, returns False
+
+ self.called = 0
+
+ def __call__(self):
+ self.called += 1
+ if not self._ret_values:
+ return False
+ return self._ret_values.pop(0)
+
+
+class SafeMVFileMocked(object):
+ def __init__(self, ret_values):
+ self._ret_values = ret_values
+ # ret_values is list of bools to return on each call. ret_values.pop(0)
+ # if the list becomes empty, returns False
+
+ self.called = 0
+ self.args_list = []
+
+ def __call__(self, src, dst):
+ self.called += 1
+ self.args_list.append((src, dst))
+ if not self._ret_values:
+ return False
+ return self._ret_values.pop(0)
+
+
+def test_migrate_openssl_nothing_to_do(monkeypatch):
+ monkeypatch.setattr(migrateopensslconf.api, 'current_logger', logger_mocked())
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([False]))
+ monkeypatch.setattr(migrateopensslconf, '_safe_mv_file', SafeMVFileMocked([False]))
+ monkeypatch.setattr(os.path, 'exists', PathExistsMocked())
+
+ migrateopensslconf.process()
+ assert not os.path.exists.called
+ assert not migrateopensslconf._safe_mv_file.called
+
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([True]))
+ migrateopensslconf.process()
+ assert os.path.exists.called
+ assert migrateopensslconf.api.current_logger.dbgmsg
+ assert not migrateopensslconf._safe_mv_file.called
+
+
+def test_migrate_openssl_failed_backup(monkeypatch):
+ monkeypatch.setattr(migrateopensslconf.api, 'current_logger', logger_mocked())
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([True]))
+ monkeypatch.setattr(migrateopensslconf, '_safe_mv_file', SafeMVFileMocked([False]))
+ monkeypatch.setattr(os.path, 'exists', PathExistsMocked([migrateopensslconf.OPENSSL_CONF_RPMNEW]))
+
+ migrateopensslconf.process()
+ assert migrateopensslconf._safe_mv_file.called == 1
+ assert migrateopensslconf._safe_mv_file.args_list[0][0] == migrateopensslconf.DEFAULT_OPENSSL_CONF
+ assert migrateopensslconf.api.current_logger.errmsg
+
+
+def test_migrate_openssl_ok(monkeypatch):
+ monkeypatch.setattr(migrateopensslconf.api, 'current_logger', logger_mocked())
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([True]))
+ monkeypatch.setattr(migrateopensslconf, '_safe_mv_file', SafeMVFileMocked([True, True]))
+ monkeypatch.setattr(os.path, 'exists', PathExistsMocked([migrateopensslconf.OPENSSL_CONF_RPMNEW]))
+
+ migrateopensslconf.process()
+ assert migrateopensslconf._safe_mv_file.called == 2
+ assert migrateopensslconf._safe_mv_file.args_list[1][1] == migrateopensslconf.DEFAULT_OPENSSL_CONF
+ assert not migrateopensslconf.api.current_logger.errmsg
+
+
+def test_migrate_openssl_failed_migrate(monkeypatch):
+ monkeypatch.setattr(migrateopensslconf.api, 'current_logger', logger_mocked())
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([True]))
+ monkeypatch.setattr(migrateopensslconf, '_safe_mv_file', SafeMVFileMocked([True, False, True]))
+ monkeypatch.setattr(os.path, 'exists', PathExistsMocked([migrateopensslconf.OPENSSL_CONF_RPMNEW]))
+
+ migrateopensslconf.process()
+ assert migrateopensslconf._safe_mv_file.called == 3
+ assert migrateopensslconf._safe_mv_file.args_list[2][1] == migrateopensslconf.DEFAULT_OPENSSL_CONF
+ assert migrateopensslconf.api.current_logger.errmsg
+
+
+def test_migrate_openssl_failed_restore(monkeypatch):
+ monkeypatch.setattr(migrateopensslconf.api, 'current_logger', logger_mocked())
+ monkeypatch.setattr(migrateopensslconf, '_is_openssl_modified', IsOpensslModifiedMocked([True]))
+ monkeypatch.setattr(migrateopensslconf, '_safe_mv_file', SafeMVFileMocked([True]))
+ monkeypatch.setattr(os.path, 'exists', PathExistsMocked([migrateopensslconf.OPENSSL_CONF_RPMNEW]))
+
+ migrateopensslconf.process()
+ assert migrateopensslconf._safe_mv_file.called == 3
+ assert len(migrateopensslconf.api.current_logger.errmsg) == 2
+
+
+class MockedRun(object):
+ def __init__(self, raise_err):
+ self.called = 0
+ self.args = None
+ self._raise_err = raise_err
+
+ def __call__(self, args):
+ self.called += 1
+ self.args = args
+ if self._raise_err:
+ raise CalledProcessError(
+ message='A Leapp Command Error occurred.',
+ command=args,
+ result={'signal': None, 'exist_code': 1, 'pid': 0, 'stdout': 'fale', 'stderr': 'fake'}
+ )
+ # NOTE(pstodulk) ignore return as the code in the library does not use it
+
+
+@pytest.mark.parametrize('result', (True, False))
+def test_is_openssl_modified(monkeypatch, result):
+ monkeypatch.setattr(migrateopensslconf, 'run', MockedRun(result))
+ assert migrateopensslconf._is_openssl_modified() is result
+ assert migrateopensslconf.run.called == 1
+
+
+@pytest.mark.parametrize('result', (True, False))
+def test_safe_mv_file(monkeypatch, result):
+ monkeypatch.setattr(migrateopensslconf, 'run', MockedRun(not result))
+ assert migrateopensslconf._safe_mv_file('foo', 'bar') is result
+ assert ['mv', 'foo', 'bar'] == migrateopensslconf.run.args
diff --git a/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py b/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
index 33e0275f..16c0e8aa 100644
--- a/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
+++ b/repos/system_upgrade/common/actors/scansourcefiles/libraries/scansourcefiles.py
@@ -9,6 +9,7 @@ from leapp.models import FileInfo, TrackedFilesInfoSource
# '8' (etc..) -> files supposed to be scanned when particular major version of OS is used
TRACKED_FILES = {
'common': [
+ '/etc/pki/tls/openssl.cnf',
],
'8': [
],
--
2.43.0

547
0059-Introduce-custom-modifications-tracking.patch Normal file
View File

@ -0,0 +1,547 @@
From 7c6e0d8ce1ca550309f2e76e1e57bef147f7a86b Mon Sep 17 00:00:00 2001
From: Inessa Vasilevskaya <ivasilev@redhat.com>
Date: Thu, 16 Nov 2023 13:16:01 +0100
Subject: [PATCH 59/60] Introduce custom modifications tracking
This commit introduces two actors:
* the scanner that that scans leapp files and
produces messages with actor name/filepath mapping in case
any unexpected custom files or modified files were discovered.
* the checker that processes CustomModification messages and
produces report entries.
* uses rpms.get_leapp_packages function
* pstodulk: Updated report messages to provide more information to users
The purpose of this change is to help with the investigation
of reported issues as people harm themselves from time to time
and as this is not usually expected, it prolongs the solution
of the problem (people investigating such issues do not check
this possibility as the first thing, which is understandable).
This should help to identify possible root causes faster as
report msg should be always visible.
Jira: RHEL-1774
Co-authored-by: Petr Stodulka <pstodulk@redhat.com>
---
.../actors/checkcustommodifications/actor.py | 19 +++
.../libraries/checkcustommodifications.py | 138 ++++++++++++++++
.../tests/test_checkcustommodifications.py | 35 +++++
.../actors/scancustommodifications/actor.py | 18 +++
.../libraries/scancustommodifications.py | 147 ++++++++++++++++++
.../tests/test_scancustommodifications.py | 89 +++++++++++
.../common/models/custommodifications.py | 13 ++
7 files changed, 459 insertions(+)
create mode 100644 repos/system_upgrade/common/actors/checkcustommodifications/actor.py
create mode 100644 repos/system_upgrade/common/actors/checkcustommodifications/libraries/checkcustommodifications.py
create mode 100644 repos/system_upgrade/common/actors/checkcustommodifications/tests/test_checkcustommodifications.py
create mode 100644 repos/system_upgrade/common/actors/scancustommodifications/actor.py
create mode 100644 repos/system_upgrade/common/actors/scancustommodifications/libraries/scancustommodifications.py
create mode 100644 repos/system_upgrade/common/actors/scancustommodifications/tests/test_scancustommodifications.py
create mode 100644 repos/system_upgrade/common/models/custommodifications.py
diff --git a/repos/system_upgrade/common/actors/checkcustommodifications/actor.py b/repos/system_upgrade/common/actors/checkcustommodifications/actor.py
new file mode 100644
index 00000000..a1a50bad
--- /dev/null
+++ b/repos/system_upgrade/common/actors/checkcustommodifications/actor.py
@@ -0,0 +1,19 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import checkcustommodifications
+from leapp.models import CustomModifications, Report
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+
+
+class CheckCustomModificationsActor(Actor):
+ """
+ Checks CustomModifications messages and produces a report about files in leapp directories that have been
+ modified or newly added.
+ """
+
+ name = 'check_custom_modifications_actor'
+ consumes = (CustomModifications,)
+ produces = (Report,)
+ tags = (IPUWorkflowTag, ChecksPhaseTag)
+
+ def process(self):
+ checkcustommodifications.report_any_modifications()
diff --git a/repos/system_upgrade/common/actors/checkcustommodifications/libraries/checkcustommodifications.py b/repos/system_upgrade/common/actors/checkcustommodifications/libraries/checkcustommodifications.py
new file mode 100644
index 00000000..f1744531
--- /dev/null
+++ b/repos/system_upgrade/common/actors/checkcustommodifications/libraries/checkcustommodifications.py
@@ -0,0 +1,138 @@
+from leapp import reporting
+from leapp.libraries.stdlib import api
+from leapp.models import CustomModifications
+
+FMT_LIST_SEPARATOR = "\n - "
+
+
+def _pretty_files(messages):
+ """
+ Return formatted string of discovered files from obtained CustomModifications messages.
+ """
+ flist = []
+ for msg in messages:
+ actor = ' (Actor: {})'.format(msg.actor_name) if msg.actor_name else ''
+ flist.append(
+ '{sep}{filename}{actor}'.format(
+ sep=FMT_LIST_SEPARATOR,
+ filename=msg.filename,
+ actor=actor
+ )
+ )
+ return ''.join(flist)
+
+
+def _is_modified_config(msg):
+ # NOTE(pstodulk):
+ # We are interested just about modified files for now. Having new created config
+ # files is not so much important for us right now, but in future it could
+ # be changed.
+ if msg.component and msg.component == 'configuration':
+ return msg.type == 'modified'
+ return False
+
+
+def _create_report(title, summary, hint, links=None):
+ report_parts = [
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Groups([reporting.Groups.UPGRADE_PROCESS]),
+ reporting.RemediationHint(hint)
+ ]
+ if links:
+ report_parts += links
+ reporting.create_report(report_parts)
+
+
+def check_configuration_files(msgs):
+ filtered_msgs = [m for m in msgs if _is_modified_config(m)]
+ if not filtered_msgs:
+ return
+ title = 'Detected modified configuration files in leapp configuration directories.'
+ summary = (
+ 'We have detected that some configuration files related to leapp or'
+ ' upgrade process have been modified. Some of these changes could be'
+ ' intended (e.g. modified repomap.json file in case of private cloud'
+ ' regions or customisations done on used Satellite server) so it is'
+ ' not always needed to worry about them. However they can impact'
+ ' the in-place upgrade and it is good to be aware of potential problems'
+ ' or unexpected results if they are not intended.'
+ '\nThe list of modified configuration files:{files}'
+ .format(files=_pretty_files(filtered_msgs))
+ )
+ hint = (
+ 'If some of changes in listed configuration files have not been intended,'
+ ' you can restore original files by following procedure:'
+ '\n1. Remove (or back up) modified files that you want to restore.'
+ '\n2. Reinstall packages which owns these files.'
+ )
+ _create_report(title, summary, hint)
+
+
+def _is_modified_code(msg):
+ if msg.component not in ['framework', 'repository']:
+ return False
+ return msg.type == 'modified'
+
+
+def check_modified_code(msgs):
+ filtered_msgs = [m for m in msgs if _is_modified_code(m)]
+ if not filtered_msgs:
+ return
+ title = 'Detected modified files of the in-place upgrade tooling.'
+ summary = (
+ 'We have detected that some files of the tooling processing the in-place'
+ ' upgrade have been modified. Note that such modifications can be allowed'
+ ' only after consultation with Red Hat - e.g. when support suggests'
+ ' the change to resolve discovered problem.'
+ ' If these changes have not been approved by Red Hat, the in-place upgrade'
+ ' is unsupported.'
+ '\nFollowing files have been modified:{files}'
+ .format(files=_pretty_files(filtered_msgs))
+ )
+ hint = 'To restore original files reinstall related packages.'
+ _create_report(title, summary, hint)
+
+
+def check_custom_actors(msgs):
+ filtered_msgs = [m for m in msgs if m.type == 'custom']
+ if not filtered_msgs:
+ return
+ title = 'Detected custom leapp actors or files.'
+ summary = (
+ 'We have detected installed custom actors or files on the system.'
+ ' These can be provided e.g. by third party vendors, Red Hat consultants,'
+ ' or can be created by users to customize the upgrade (e.g. to migrate'
+ ' custom applications).'
+ ' This is allowed and appreciated. However Red Hat is not responsible'
+ ' for any issues caused by these custom leapp actors.'
+ ' Note that upgrade tooling is under agile development which could'
+ ' require more frequent update of custom actors.'
+ '\nThe list of custom leapp actors and files:{files}'
+ .format(files=_pretty_files(filtered_msgs))
+ )
+ hint = (
+ 'In case of any issues connected to custom or third party actors,'
+ ' contact vendor of such actors. Also we suggest to ensure the installed'
+ ' custom leapp actors are up to date, compatible with the installed'
+ ' packages.'
+ )
+ links = [
+ reporting.ExternalLink(
+ url='https://red.ht/customize-rhel-upgrade',
+ title='Customizing your Red Hat Enterprise Linux in-place upgrade'
+ )
+ ]
+
+ _create_report(title, summary, hint, links)
+
+
+def report_any_modifications():
+ modifications = list(api.consume(CustomModifications))
+ if not modifications:
+ # no modification detected
+ return
+ check_custom_actors(modifications)
+ check_configuration_files(modifications)
+ check_modified_code(modifications)
diff --git a/repos/system_upgrade/common/actors/checkcustommodifications/tests/test_checkcustommodifications.py b/repos/system_upgrade/common/actors/checkcustommodifications/tests/test_checkcustommodifications.py
new file mode 100644
index 00000000..6a538065
--- /dev/null
+++ b/repos/system_upgrade/common/actors/checkcustommodifications/tests/test_checkcustommodifications.py
@@ -0,0 +1,35 @@
+from leapp.libraries.actor import checkcustommodifications
+from leapp.models import CustomModifications, Report
+
+
+def test_report_any_modifications(current_actor_context):
+ discovered_msgs = [CustomModifications(filename='some/changed/leapp/actor/file',
+ type='modified',
+ actor_name='an_actor',
+ component='repository'),
+ CustomModifications(filename='some/new/actor/in/leapp/dir',
+ type='custom',
+ actor_name='a_new_actor',
+ component='repository'),
+ CustomModifications(filename='some/new/actor/in/leapp/dir',
+ type='modified',
+ actor_name='a_new_actor',
+ component='configuration'),
+ CustomModifications(filename='some/changed/file/in/framework',
+ type='modified',
+ actor_name='',
+ component='framework')]
+ for msg in discovered_msgs:
+ current_actor_context.feed(msg)
+ current_actor_context.run()
+ reports = current_actor_context.consume(Report)
+ assert len(reports) == 3
+ assert (reports[0].report['title'] ==
+ 'Detected custom leapp actors or files.')
+ assert 'some/new/actor/in/leapp/dir (Actor: a_new_actor)' in reports[0].report['summary']
+ assert (reports[1].report['title'] ==
+ 'Detected modified configuration files in leapp configuration directories.')
+ assert (reports[2].report['title'] ==
+ 'Detected modified files of the in-place upgrade tooling.')
+ assert 'some/changed/file/in/framework' in reports[2].report['summary']
+ assert 'some/changed/leapp/actor/file (Actor: an_actor)' in reports[2].report['summary']
diff --git a/repos/system_upgrade/common/actors/scancustommodifications/actor.py b/repos/system_upgrade/common/actors/scancustommodifications/actor.py
new file mode 100644
index 00000000..5eae33aa
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scancustommodifications/actor.py
@@ -0,0 +1,18 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import scancustommodifications
+from leapp.models import CustomModifications
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+
+
+class ScanCustomModificationsActor(Actor):
+ """
+ Collects information about files in leapp directories that have been modified or newly added.
+ """
+
+ name = 'scan_custom_modifications_actor'
+ produces = (CustomModifications,)
+ tags = (IPUWorkflowTag, FactsPhaseTag)
+
+ def process(self):
+ for msg in scancustommodifications.scan():
+ self.produce(msg)
diff --git a/repos/system_upgrade/common/actors/scancustommodifications/libraries/scancustommodifications.py b/repos/system_upgrade/common/actors/scancustommodifications/libraries/scancustommodifications.py
new file mode 100644
index 00000000..80137ef4
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scancustommodifications/libraries/scancustommodifications.py
@@ -0,0 +1,147 @@
+import ast
+import os
+
+from leapp.exceptions import StopActorExecution
+from leapp.libraries.common import rpms
+from leapp.libraries.stdlib import api, CalledProcessError, run
+from leapp.models import CustomModifications
+
+LEAPP_REPO_DIRS = ['/usr/share/leapp-repository']
+LEAPP_PACKAGES_TO_IGNORE = ['snactor']
+
+
+def _get_dirs_to_check(component):
+ if component == 'repository':
+ return LEAPP_REPO_DIRS
+ return []
+
+
+def _get_rpms_to_check(component=None):
+ if component == 'repository':
+ return rpms.get_leapp_packages(component=rpms.LeappComponents.REPOSITORY)
+ if component == 'framework':
+ return rpms.get_leapp_packages(component=rpms.LeappComponents.FRAMEWORK)
+ return rpms.get_leapp_packages(components=[rpms.LeappComponents.REPOSITORY, rpms.LeappComponents.FRAMEWORK])
+
+
+def deduce_actor_name(a_file):
+ """
+ A helper to map an actor/library to the actor name
+ If a_file is an actor or an actor library, the name of the actor (name attribute of actor class) will be returned.
+ Empty string is returned if the file could not be associated with any actor.
+ """
+ if not os.path.exists(a_file):
+ return ''
+ # NOTE(ivasilev) Actors reside only in actor.py files, so AST processing any other file can be skipped.
+ # In case this function has been called on a non-actor file, let's go straight to recursive call on the assumed
+ # location of the actor file.
+ if os.path.basename(a_file) == 'actor.py':
+ data = None
+ with open(a_file) as f:
+ try:
+ data = ast.parse(f.read())
+ except TypeError:
+ api.current_logger().warning('An error occurred while parsing %s, can not deduce actor name', a_file)
+ return ''
+ # NOTE(ivasilev) Making proper syntax analysis is not the goal here, so let's get away with the bare minimum.
+ # An actor file will have an Actor ClassDef with a name attribute and a process function defined
+ actor = next((obj for obj in data.body if isinstance(obj, ast.ClassDef) and obj.name and
+ any(isinstance(o, ast.FunctionDef) and o.name == 'process' for o in obj.body)), None)
+ # NOTE(ivasilev) obj.name attribute refers only to Class name, so for fetching name attribute need to go
+ # deeper
+ if actor:
+ try:
+ actor_name = next((expr.value.s for expr in actor.body
+ if isinstance(expr, ast.Assign) and expr.targets[-1].id == 'name'), None)
+ except (AttributeError, IndexError):
+ api.current_logger().warning("Syntax Analysis for %d has failed", a_file)
+ actor_name = None
+ if actor_name:
+ return actor_name
+
+ # Assuming here we are dealing with a library or a file, so let's discover actor filename and deduce actor name
+ # from it. Actor is expected to be found under ../../actor.py
+ def _check_assumed_location(subdir):
+ assumed_actor_file = os.path.join(a_file.split(subdir)[0], 'actor.py')
+ if not os.path.exists(assumed_actor_file):
+ # Nothing more we can do - no actor name mapping, return ''
+ return ''
+ return deduce_actor_name(assumed_actor_file)
+
+ return _check_assumed_location('libraries') or _check_assumed_location('files')
+
+
+def _run_command(cmd, warning_to_log, checked=True):
+ """
+ A helper that executes a command and returns a result or raises StopActorExecution.
+ Upon success results will contain a list with line-by-line output returned by the command.
+ """
+ try:
+ res = run(cmd, checked=checked)
+ output = res['stdout'].strip()
+ if not output:
+ return []
+ return output.split('\n')
+ except CalledProcessError:
+ api.current_logger().warning(warning_to_log)
+ raise StopActorExecution()
+
+
+def _modification_model(filename, change_type, component, rpm_checks_str=''):
+ # XXX FIXME(ivasilev) Actively thinking if different model classes inheriting from CustomModifications
+ # are needed or let's get away with one model for everything (as is implemented now).
+ # The only difference atm is that actor_name makes sense only for repository modifications.
+ return CustomModifications(filename=filename, type=change_type, component=component,
+ actor_name=deduce_actor_name(filename), rpm_checks_str=rpm_checks_str)
+
+
+def check_for_modifications(component):
+ """
+ This will return a list of any untypical files or changes to shipped leapp files discovered on the system.
+ An empty list means that no modifications have been found.
+ """
+ rpms = _get_rpms_to_check(component)
+ dirs = _get_dirs_to_check(component)
+ source_of_truth = []
+ leapp_files = []
+ # Let's collect data about what should have been installed from rpm
+ for rpm in rpms:
+ res = _run_command(['rpm', '-ql', rpm], 'Could not get a list of installed files from rpm {}'.format(rpm))
+ source_of_truth.extend(res)
+ # Let's collect data about what's really on the system
+ for directory in dirs:
+ res = _run_command(['find', directory, '-type', 'f'],
+ 'Could not get a list of leapp files from {}'.format(directory))
+ leapp_files.extend(res)
+ # Let's check for unexpected additions
+ custom_files = sorted(set(leapp_files) - set(source_of_truth))
+ # Now let's check for modifications
+ modified_files = []
+ modified_configs = []
+ for rpm in rpms:
+ res = _run_command(
+ ['rpm', '-V', '--nomtime', rpm], 'Could not check authenticity of the files from {}'.format(rpm),
+ # NOTE(ivasilev) check is False here as in case of any changes found exit code will be 1
+ checked=False)
+ if res:
+ api.current_logger().warning('Modifications to leapp files detected!\n%s', res)
+ for modification_str in res:
+ modification = tuple(modification_str.split())
+ if len(modification) == 3 and modification[1] == 'c':
+ # Dealing with a configuration that will be displayed as ('S.5......', 'c', '/file/path')
+ modified_configs.append(modification)
+ else:
+ # Modification of any other rpm file detected
+ modified_files.append(modification)
+ return ([_modification_model(filename=f[1], component=component, rpm_checks_str=f[0], change_type='modified')
+ # Let's filter out pyc files not to clutter the output as pyc will be present even in case of
+ # a plain open & save-not-changed that we agreed not to react upon.
+ for f in modified_files if not f[1].endswith('.pyc')] +
+ [_modification_model(filename=f, component=component, change_type='custom')
+ for f in custom_files] +
+ [_modification_model(filename=f[2], component='configuration', rpm_checks_str=f[0], change_type='modified')
+ for f in modified_configs])
+
+
+def scan():
+ return check_for_modifications('framework') + check_for_modifications('repository')
diff --git a/repos/system_upgrade/common/actors/scancustommodifications/tests/test_scancustommodifications.py b/repos/system_upgrade/common/actors/scancustommodifications/tests/test_scancustommodifications.py
new file mode 100644
index 00000000..a48869e4
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scancustommodifications/tests/test_scancustommodifications.py
@@ -0,0 +1,89 @@
+import pytest
+
+from leapp.libraries.actor import scancustommodifications
+from leapp.libraries.common.testutils import CurrentActorMocked, produce_mocked
+from leapp.libraries.stdlib import api
+
+FILES_FROM_RPM = """
+repos/system_upgrade/el8toel9/actors/xorgdrvfact/libraries/xorgdriverlib.py
+repos/system_upgrade/el8toel9/actors/anotheractor/actor.py
+repos/system_upgrade/el8toel9/files
+"""
+
+FILES_ON_SYSTEM = """
+repos/system_upgrade/el8toel9/actors/xorgdrvfact/libraries/xorgdriverlib.py
+repos/system_upgrade/el8toel9/actors/anotheractor/actor.py
+repos/system_upgrade/el8toel9/files
+/some/unrelated/to/leapp/file
+repos/system_upgrade/el8toel9/files/file/that/should/not/be/there
+repos/system_upgrade/el8toel9/actors/actor/that/should/not/be/there
+"""
+
+VERIFIED_FILES = """
+.......T. repos/system_upgrade/el8toel9/actors/xorgdrvfact/libraries/xorgdriverlib.py
+S.5....T. repos/system_upgrade/el8toel9/actors/anotheractor/actor.py
+S.5....T. c etc/leapp/files/pes-events.json
+"""
+
+
+@pytest.mark.parametrize('a_file,name', [
+ ('repos/system_upgrade/el8toel9/actors/checkblacklistca/actor.py', 'checkblacklistca'),
+ ('repos/system_upgrade/el7toel8/actors/checkmemcached/actor.py', 'check_memcached'),
+ # actor library
+ ('repos/system_upgrade/el7toel8/actors/checkmemcached/libraries/checkmemcached.py', 'check_memcached'),
+ # actor file
+ ('repos/system_upgrade/common/actors/createresumeservice/files/leapp_resume.service', 'create_systemd_service'),
+ ('repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh',
+ 'common_leapp_dracut_modules'),
+ # not a library and not an actor file
+ ('repos/system_upgrade/el7toel8/models/authselect.py', ''),
+ ('repos/system_upgrade/common/files/rhel_upgrade.py', ''),
+ # common library not tied to any actor
+ ('repos/system_upgrade/common/libraries/mounting.py', ''),
+ ('repos/system_upgrade/common/libraries/config/version.py', ''),
+ ('repos/system_upgrade/common/libraries/multipathutil.py', ''),
+ ('repos/system_upgrade/common/libraries/config/version.py', ''),
+ ('repos/system_upgrade/common/libraries/dnfplugin.py', ''),
+ ('repos/system_upgrade/common/libraries/testutils.py', ''),
+ # the rest of false positives discovered by dkubek
+ ('repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos_repomap.py', 'setuptargetrepos'),
+ ('repos/system_upgrade/el8toel9/actors/sssdfacts/libraries/sssdfacts8to9.py', 'sssd_facts_8to9'),
+ ('repos/system_upgrade/el8toel9/actors/nisscanner/libraries/nisscan.py', 'nis_scanner'),
+ ('repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos_repomap.py', 'setuptargetrepos'),
+ ('repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py', 'repository_mapping'),
+ ('repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner_repomap.py',
+ 'pes_events_scanner')
+])
+def test_deduce_actor_name_from_file(a_file, name):
+ assert scancustommodifications.deduce_actor_name(a_file) == name
+
+
+def mocked__run_command(list_of_args, log_message, checked=True):
+ if list_of_args == ['rpm', '-ql', 'leapp-upgrade-el8toel9']:
+ # get source of truth
+ return FILES_FROM_RPM.strip().split('\n')
+ if list_of_args and list_of_args[0] == 'find':
+ # listing files in directory
+ return FILES_ON_SYSTEM.strip().split('\n')
+ if list_of_args == ['rpm', '-V', '--nomtime', 'leapp-upgrade-el8toel9']:
+ # checking authenticity
+ return VERIFIED_FILES.strip().split('\n')
+ return []
+
+
+def test_check_for_modifications(monkeypatch):
+ monkeypatch.setattr(api, 'current_actor', CurrentActorMocked(arch='x86_64', src_ver='8.9', dst_ver='9.3'))
+ monkeypatch.setattr(scancustommodifications, '_run_command', mocked__run_command)
+ modifications = scancustommodifications.check_for_modifications('repository')
+ modified = [m for m in modifications if m.type == 'modified']
+ custom = [m for m in modifications if m.type == 'custom']
+ configurations = [m for m in modifications if m.component == 'configuration']
+ assert len(modified) == 3
+ assert modified[0].filename == 'repos/system_upgrade/el8toel9/actors/xorgdrvfact/libraries/xorgdriverlib.py'
+ assert modified[0].rpm_checks_str == '.......T.'
+ assert len(custom) == 3
+ assert custom[0].filename == '/some/unrelated/to/leapp/file'
+ assert custom[0].rpm_checks_str == ''
+ assert len(configurations) == 1
+ assert configurations[0].filename == 'etc/leapp/files/pes-events.json'
+ assert configurations[0].rpm_checks_str == 'S.5....T.'
diff --git a/repos/system_upgrade/common/models/custommodifications.py b/repos/system_upgrade/common/models/custommodifications.py
new file mode 100644
index 00000000..51709dde
--- /dev/null
+++ b/repos/system_upgrade/common/models/custommodifications.py
@@ -0,0 +1,13 @@
+from leapp.models import fields, Model
+from leapp.topics import SystemFactsTopic
+
+
+class CustomModifications(Model):
+ """Model to store any custom or modified files that are discovered in leapp directories"""
+ topic = SystemFactsTopic
+
+ filename = fields.String()
+ actor_name = fields.String()
+ type = fields.StringEnum(choices=['custom', 'modified'])
+ rpm_checks_str = fields.String(default='')
+ component = fields.String()
--
2.43.0

1106
0060-Rework-_copy_decouple-to-follow-relative-symlinks-an.patch Normal file
View File

File diff suppressed because it is too large Load Diff

273291
0061-Update-the-data-files-pes-repomap-dddd-CTC2-0.patch Normal file
View File

File diff suppressed because it is too large Load Diff

58
leapp-repository.spec
View File

@ -2,7 +2,7 @@
%global repositorydir %{leapp_datadir}/repositories
%global custom_repositorydir %{leapp_datadir}/custom-repositories
%define leapp_repo_deps 9
%define leapp_repo_deps 10
%if 0%{?rhel} == 7
%define leapp_python_sitelib %{python2_sitelib}
@ -42,13 +42,13 @@ py2_byte_compile "%1" "%2"}
Name: leapp-repository
Version: 0.19.0
Release: 5%{?dist}
Release: 6%{?dist}
Summary: Repositories for leapp
License: ASL 2.0
URL: https://oamg.github.io/leapp/
Source0: https://github.com/oamg/%{name}/archive/v%{version}.tar.gz#/%{name}-%{version}.tar.gz
Source1: deps-pkgs-9.tar.gz
Source1: deps-pkgs-10.tar.gz
# NOTE: Our packages must be noarch. Do no drop this in any way.
BuildArch: noarch
@ -97,6 +97,28 @@ Patch0039: 0039-Fix-several-typos-and-Makefile-help.patch
Patch0040: 0040-Move-code-handling-GPG-keys-to-separate-library.patch
Patch0041: 0041-Check-no-new-unexpected-keys-were-installed-during-t.patch
# CTC2-0
Patch0042: 0042-BZ-2250254-force-removal-of-tomcat-during-the-upgrad.patch
Patch0043: 0043-Add-79to88-and-79to89-aws-upgrade-paths.patch
Patch0044: 0044-Add-7.9to8.10-and-8.10to9.4-upgrade-paths.patch
Patch0045: 0045-Utilize-get_target_major_version-in-no-enabled-targe.patch
Patch0046: 0046-Workaround-tft-issue-with-listing-disabled-plans.patch
Patch0047: 0047-Distribution-agnostick-check-of-signed-packages-1-2.patch
Patch0048: 0048-Distribution-agnostick-check-of-signed-packages-2-2.patch
Patch0049: 0049-Pylint-fix-superfluous-parens-in-the-code.patch
Patch0050: 0050-distributionsignedrpmscanner-refactoring-gpg-pubkey-.patch
Patch0051: 0051-Introduce-two-functions-for-listing-which-packages-a.patch
Patch0052: 0052-Switch-test-repo-branch-to-main.patch
Patch0053: 0053-Update-dependencies-require-xfsprogs-and-e2fsprogs.patch
Patch0054: 0054-Several-enhancements-to-the-Makefile.patch
Patch0055: 0055-pes_events_scanner-Ignore-Leapp-related-PES-events.patch
Patch0056: 0056-Use-library-functions-for-getting-leapp-packages.patch
Patch0057: 0057-Introduce-TrackedFilesInfoSource-message-and-new-act.patch
Patch0058: 0058-Add-actors-for-OpenSSL-conf-and-IBMCA.patch
Patch0059: 0059-Introduce-custom-modifications-tracking.patch
Patch0060: 0060-Rework-_copy_decouple-to-follow-relative-symlinks-an.patch
Patch0061: 0061-Update-the-data-files-pes-repomap-dddd-CTC2-0.patch
%description
%{summary}
@ -279,6 +301,26 @@ Requires: python3-gobject-base
%patch0039 -p1
%patch0040 -p1
%patch0041 -p1
%patch0042 -p1
%patch0043 -p1
%patch0044 -p1
%patch0045 -p1
%patch0046 -p1
%patch0047 -p1
%patch0048 -p1
%patch0049 -p1
%patch0050 -p1
%patch0051 -p1
%patch0052 -p1
%patch0053 -p1
%patch0054 -p1
%patch0055 -p1
%patch0056 -p1
%patch0057 -p1
%patch0058 -p1
%patch0059 -p1
%patch0060 -p1
%patch0061 -p1
%build
@ -356,6 +398,16 @@ done;
# no files here
%changelog
* Fri Jan 12 2024 Petr Stodulka <pstodulk@redhat.com> - 0.19.0-6
- Add detection of possible usage of OpenSSL IBMCA engine on IBM Z machines
- Add detection of modified /etc/pki/tls/openssl.cnf file
- Update the leapp upgrade data files
- Fix handling of symlinks under /etc/pki with relative paths specified
- Report custom actors and modifications of the upgrade tooling
- Requires xfsprogs and e2fsprogs to ensure that Ext4 and XFS tools are installed
- Bump leapp-repository-dependencies to 10
- Resolves: RHEL-1774, RHEL-16729
* Thu Nov 16 2023 Petr Stodulka <pstodulk@redhat.com> - 0.19.0-5
- Enable new upgrade path for RHEL 8.10 -> RHEL 9.4 (including RHEL with SAP HANA)
- Introduce generic transition of systemd services states during the IPU

2
sources
View File

@ -1,2 +1,2 @@
SHA512 (deps-pkgs-9.tar.gz) = 55593b4384b1fbfbf150585d2a124d9b0fdb8cb1e7449a372be5b2889b04fe72fc5cb567f9ee2b2a573d630b50e16f13467ad94596306fafad9658b9964fb84e
SHA512 (leapp-repository-0.19.0.tar.gz) = e7e913cd635c8101dc5dcd65929d19a21ce72fd9291b84ea60a20e6dbdf4a65553c890770bf16000145f601242ed7f047cae1e283966c8b385ea9bf61e04ef65
SHA512 (deps-pkgs-10.tar.gz) = e63f77e439456e0a8b0fc338b370ee7e2d7824b1d62c75f2209b283905c8c0641d504bfe910021317884fa1662429d952fd4c9b9ee457c48b34182e6f975aa0e