ykohut/leapp-repository
1
0
Fork 0
forked from rpms/leapp-repository
Code Pull Requests Activity
9a1e34c143
leapp-repository/SOURCES/leapp-repository-0.16.0-elevate.patch

3338 lines
144 KiB
Diff
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

diff --git a/README.md b/README.md
index 4de458b..9975f06 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,61 @@
-**Before doing anything, please read
-[Leapp framework documentation](https://leapp.readthedocs.io/).**
+# Leapp ELevate Repository
----
+**Before doing anything, please read [Leapp framework documentation](https://leapp.readthedocs.io/).**
+
+## Running
+Make sure your system is fully updated before starting the upgrade process.
+
+```bash
+sudo yum update -y
+```
+
+Install `elevate-release` package with the project repo and GPG key.
+
+`sudo yum install -y http://repo.almalinux.org/elevate/elevate-release-latest-el7.noarch.rpm`
+
+Install leapp packages and migration data for the OS you want to upgrade. Possible options are:
+ - leapp-data-almalinux
+ - leapp-data-centos
+ - leapp-data-eurolinux
+ - leapp-data-oraclelinux
+ - leapp-data-rocky
+
+`sudo yum install -y leapp-upgrade leapp-data-almalinux`
+
+Start a preupgrade check. In the meantime, the Leapp utility creates a special /var/log/leapp/leapp-report.txt file that contains possible problems and recommended solutions. No rpm packages will be installed at this phase.
+
+`sudo leapp preupgrade`
+
+The preupgrade process may stall with the following message:
+> Inhibitor: Newest installed kernel not in use
+
+Make sure your system is running the latest kernel before proceeding with the upgrade. If you updated the system recently, a reboot may be sufficient to do so. Otherwise, edit your Grub configuration accordingly.
+
+> NOTE: In certain configurations, Leapp generates `/var/log/leapp/answerfile` with true/false questions. Leapp utility requires answers to all these questions in order to proceed with the upgrade.
+
+Once the preupgrade process completes, the results will be contained in `/var/log/leapp/leapp-report.txt` file.
+It's advised to review the report and consider how the changes will affect your system.
+
+Start an upgrade. You’ll be offered to reboot the system after this process is completed.
+
+```bash
+sudo leapp upgrade
+sudo reboot
+```
+
+> NOTE: The upgrade process after the reboot may take a long time, up to 40-50 minutes, depending on the machine resources. If the machine remains unresponsive for more than 2 hours, assume the upgrade process failed during the post-reboot phase.
+> If it's still possible to access the machine in some way, for example, through remote VNC access, the logs containing the information on what went wrong are located in this folder: `/var/log/leapp`
+
+A new entry in GRUB called ELevate-Upgrade-Initramfs will appear. The system will be automatically booted into it. Observe the update process in the console.
+
+After the reboot, login into the system and check the migration report. Verify that the current OS is the one you need.
+
+```bash
+cat /etc/redhat-release
+cat /etc/os-release
+```
+
+Check the leapp logs for .rpmnew configuration files that may have been created during the upgrade process. In some cases os-release or yum package files may not be replaced automatically, requiring the user to rename the .rpmnew files manually.
## Troubleshooting
@@ -11,6 +65,15 @@
- Leapp framework: [https://github.com/oamg/leapp/issues/new/choose](https://github.com/oamg/leapp/issues/new/choose)
- Leapp actors: [https://github.com/oamg/leapp-repository/issues/new/choose](https://github.com/oamg/leapp-repository/issues/new/choose)
+### Where can I report an issue or RFE related to the AlmaLinux actor or data modifications?
+- GitHub issues are preferred:
+ - Leapp actors: [https://github.com/AlmaLinux/leapp-repository/issues/new/choose](https://github.com/AlmaLinux/leapp-repository/issues/new/choose)
+ - Leapp data: [https://github.com/AlmaLinux/leapp-data/issues/new/choose](https://github.com/AlmaLinux/leapp-data/issues/new/choose)
+
+### What data should be provided when making a report?
+
+Before gathering data, if possible, run the *leapp* command that encountered an issue with the `--debug` flag, e.g.: `leapp upgrade --debug`.
+
- When filing an issue, include:
- Steps to reproduce the issue
- *All files in /var/log/leapp*
@@ -25,7 +88,613 @@
Then you may attach only the `leapp-logs.tgz` file.
### Where can I seek help?
-We’ll gladly answer your questions and lead you to through any troubles with the
-actor development.
+We’ll gladly answer your questions and lead you to through any troubles with the actor development.
+
+You can reach the primary Leapp development team at IRC: `#leapp` on freenode.
+
+## Third-party integration
+
+If you would like to add your **signed** 3rd party packages into the upgrade process, you can use the third-party integration mechanism.
+
+There are four components for adding your information to the elevation process:
+- <vendor_name>_map.json: repository mapping file
+- <vendor_name>.repo: package repository information
+- <vendor_name>.sigs: list of package signatures of vendor repositories
+- <vendor_name>_pes.json: package migration event list
+
+All these files **must** have the same <vendor_name> part.
+
+### Repository mapping file
+
+This JSON file provides information on mappings between source system repositories (repositories present on the system being upgraded) and target system repositories (package repositories to be used during the upgrade).
+
+The file contains two sections, `mapping` and `repositories`.
+
+`repositories` descripes the source and target repositories themselves. Each entry should have a unique string ID specific to mapping/PES files - `pesid`, and a list of attributes:
+- major_version: major system version that this repository targets
+- repo_type: repository type, see below
+- repoid: repository ID, same as in *.repo files. Doesn't have to exactly match `pesid`
+- arch: system architecture for which this repository is relevant
+- channel: repository channel, see below
+
+
+**Repository types**:
+- rpm: normal RPM packages
+- srpm: source packages
+- debuginfo: packages with debug information
+
+**Repository channels**:
+- ga: general availability repositories
+ - AKA stable repositories.
+- beta: beta-testing repositories
+- eus, e4s, aus, tus: Extended Update Support, Update Services for SAP Solutions, Advanced Update Support, Telco Extended Update Support
+ - Red Hat update channel classification. Most of the time you won't need to use these.
+
+`mapping` establishes connections between described repositories.
+Each entry in the list defines a mapping between major system versions, and contains the following elements:
+- source_major_version: major system version from which the system would be upgraded
+- target_major_version: major system version to which the system would be elevated
+- entries: the list of repository mappings
+ - source: source repository, one that would be found on a pre-upgrade system
+ - target: a list of target upgrade repositores that would contain new package versions. Each source repository can map to one or multiple target repositories
+
+
+> **Important**: The repository mapping file also defines whether a vendor's packages will be included into the upgrade process at all.
+> If at least one source repository listed in the file is present on the system, the vendor is considered active, and package repositories/PES events are enabled - otherwise, they **will not** affect the upgrade process.
+
+### Package repository information
+
+This file defines the vendor's package repositories to be used during the upgrade.
+
+The file has the same format normal YUM/DNF package repository files do.
+
+> NOTE: The repositories listed in this file are only used *during* the upgrade. Package repositories on the post-upgrade system should be provided through updated packages or custom repository deployment.
+
+### Package signature list
+
+This file should contain the list of public signature headers that the packages are signed with, one entry per line.
+
+You can find signature headers for your packages by running the following command:
+
+`rpm -qa --queryformat "%{NAME} || %|DSAHEADER?{%{DSAHEADER:pgpsig}}:{%|RSAHEADER?{%{RSAHEADER:pgpsig}}:{(none)}|}|\n" <PACKAGE_NAME>`
+
+rpm will return an entry like the following:
+`package-name || DSA/SHA1, Mon Aug 23 08:17:13 2021, Key ID 8c55a6628608cb71`
+
+The value after "Key ID", in this case, `8c55a6628608cb71`, is what you should put into the signature list file.
+
+### Package migration event list
+
+The Leapp upgrade process uses information from the AlmaLinux PES (Package Evolution System) to keep track of how packages change between the OS versions. This data is located in `leapp-data/vendors.d/<vendor_name>_pes.json` in the GitHub repository and in `/etc/leapp/files/vendors.d/<vendor_name>_pes.json` on a system being upgraded.
+
+> **Warning**: leapp doesn't force packages from out_packageset to be installed from the specific repository; instead, it enables repo from out_packageset and then DNF installs the latest package version from all enabled repos.
+
+#### Creating event lists through PES
+
+The recommended way to create new event lists is to use the PES mechanism.
+
+The web interface can create, manage and export groups of events to JSON files.
+
+This video demonstration walks through the steps of adding an action event group and exporting it as a JSON file to make use of it in the elevation process.
+
+> https://drive.google.com/file/d/1VqnQkUsxzLijIqySMBGu5lDrA72BVd5A/view?usp=sharing
+
+Please refer to the [PES contribution guide](https://wiki.almalinux.org/elevate/Contribution-guide.html) for additional information on entry fields.
+
+#### Manual editing
+
+To add new rules to the list, add a new entry to the `packageinfo` array.
+
+**Important**: actions from PES JSON files will be in effect only for those packages that are signed **and** have their signatures in one of the active <vendor_name>.sigs files. Unsigned packages will be updated only if some signed package requires a new version, otherwise they will by left as they are.
+
+Required fields:
+
+- action: what action to perform on the listed package
+- 0 - present
+- 1 - removed
+- 2 - deprecated
+- 3 - replaced
+- 4 - split
+- 5 - merged
+- 6 - moved to new repository
+- 7 - renamed
+- arches: what system architectures the listed entry relates to
+- id: entry ID, must be unique
+- in_packageset: set of packages on the old system
+- out_packageset: set of packages to switch to, empty if removed or deprecated
+- initial_release: source OS release
+- release: target OS release
+
+`in_packageset` and `out_packageset` have the following format:
+
+```json
+ "in_packageset": {
+ "package": [
+ {
+ "module_stream": null,
+ "name": "PackageKit",
+ "repository": "base"
+ },
+ {
+ "module_stream": null,
+ "name": "PackageKit-yum",
+ "repository": "base"
+ }
+ ],
+ "set_id": 1592
+ },
+```
+
+For `in_packageset`, `repository` field defines the package repository the package was installed from on the source system.
+For `out_packageset`, `repository` field for packages should be the same as the "Target system repo name in PES" field in the associated vendor repository mapping file.
+
+### Providing the data
+
+Once you've prepared the vendor data for migration, you can make a pull request to https://github.com/AlmaLinux/leapp-data/ to make it available publicly.
+Files should be placed into the `vendors.d` subfolder if the data should be available for all elevation target OS variants, or into the `files/<target_OS>/vendors.d/` if intended for a specific one.
+
+Alternatively, you can deploy the vendor files on a system prior to starting the upgrade. In this case, place the files into the folder `/etc/leapp/files/vendors.d/`.
+
+## Adding complex changes (custom actors for migration)
+To perform any changes of arbitrary complexity during the migration process, add a component to the existing Leapp pipeline.
+
+To begin, clone the code repository: https://github.com/AlmaLinux/leapp-repository
+For instructions on how to deploy a development enviroment, refer to [Leapp framework documentation](https://leapp.readthedocs.io/en/latest/devenv-install.html).
+
+Create an actor inside the main system_upgrade leapp repository:
+
+```bash
+cd ./leapp-repository/repos/system_upgrade/common
+snactor new-actor testactor
+```
+
+Alternatively, you can [create your own repository](https://leapp.readthedocs.io/en/latest/create-repository.html) in the system_upgrade folder, if you wish to keep your actors separate from others.
+Keep in mind that you’ll need to link all other repositories whose functions you will use.
+The created subfolder will contain the main Python file of your new actor.
+
+The actor’s main class has three fields of interest:
+- consumes
+- produces
+- tags
+
+consumes and produces defines the [data that the actor may receive or provide to other actors](https://leapp.readthedocs.io/en/latest/messaging.html).
+
+Tags define the phase of the upgrade process during which the actor runs.
+All actors also must be assigned the `IPUWorkflowTag` to mark them as a part of the in-place upgrade process.
+The file `leapp-repository/repos/system_upgrade/common/workflows/inplace_upgrade.py` lists all phases of the elevation process.
+
+### Submitting changes
+Changes you want to submit upstream should be sent through pull requests to repositories https://github.com/AlmaLinux/leapp-repository and https://github.com/AlmaLinux/leapp-data.
+The standard GitHub contribution process applies - fork the repository, make your changes inside of it, then submit the pull request to be reviewed.
+
+### Custom actor example
+
+"Actors" in Leapp terminology are Python scripts that run during the upgrade process.
+Actors are a core concept of the framework, and the entire process is built from them.
+
+Custom actors are the actors that are added by third-party developers, and are not present in the upstream Leapp repository.
+
+Actors can gather data, communicate with each other and modify the system during the upgrade.
+
+Let's examine how an upgrade problem might be resolved with a custom actor.
+
+#### Problem
+
+If you ever ran `leapp preupgrade` on unprepared systems before, you likely have seen the following message:
+
+```
+Upgrade has been inhibited due to the following problems:
+ 1. Inhibitor: Possible problems with remote login using root account
+```
+
+It's caused by the change in default behaviour for permitting root logins between RHEL 7 and 8.
+In RHEL 8 logging in as root via password authentication is no longer allowed by default, which means that some machines can become inaccessible after the upgrade.
+
+Some configurations require an administrator's intervention to resolve this issue, but SSHD configurations where no `PermitRootLogin` options were explicitly set can be modified to preserve the RHEL 7 default behaviour and not require manual modification.
+
+Let's create a custom actor to handle such cases for us.
+
+#### Creating an actor
+
+Actors are contained in ["repositories"](https://leapp.readthedocs.io/en/latest/leapp-repositories.html) - subfolders containing compartmentalized code and resources that the Leapp framework will use during the upgrade.
+
+> Do not confuse Leapp repositories with Git repositories - these are two different concepts, independent of one another.
+
+Inside the `leapp-repository` GitHub repo, Leapp repositories are contained inside the `repos` subfolder.
+
+Everything related to system upgrade proper is inside the `system_upgrade` folder.
+`el7toel8` contains resources used when upgrading from RHEL 7 to RHEL 8, `el8toel9` - RHEL 8 to 9, `common` - shared resources.
+
+Since the change in system behaviour we're looking to mitigate occurs between RHEL 7 and 8, the appopriate repository to place the actor in is `el7toel8`.
+
+You can [create new actors](https://leapp.readthedocs.io/en/latest/first-actor.html) by using the `snactor` tool provided by Leapp, or manually.
+
+`snactor new-actor ACTOR_NAME`
+
+The bare-bones actor code consists of a file named `actor.py` contained inside the `actors/<actor_name>` subfolder of a Leapp repository.
+
+In this case, then, it should be located in a directory like `leapp-repository/repos/system_upgrade/el7toel8/actors/opensshmodifypermitroot`
+
+If you used snactor to create it, you'll see contents like the following:
+
+```python
+from leapp.actors import Actor
+
+
+class OpenSSHModifyPermitRoot(Actor):
+ """
+ No documentation has been provided for the open_ssh_actor_example actor.
+ """
+
+ name = 'openssh_modify_permit_root'
+ consumes = ()
+ produces = ()
+ tags = ()
+
+ def process(self):
+ pass
+```
+
+#### Configuring the actor
+
+Actors' `consumes` and `produces` attributes define types of [*messages*](https://leapp.readthedocs.io/en/latest/messaging.html) these actors receive or send.
+
+For instance, during the initial upgrade stages several standard actors gather system information and *produce* messages with gathered data to other actors.
+
+> Messages are defined by *message models*, which are contained inside Leapp repository's `models` subfolder, just like all actors are contained in `actors`.
+
+Actors' `tags` attributes define the [phase of the upgrade](https://leapp.readthedocs.io/en/latest/working-with-workflows.html) during which that actor gets executed.
+
+> The list of all phases can be found in file `leapp-repository/repos/system_upgrade/common/workflows/inplace_upgrade.py`.
+
+##### Receiving messages
+
+Leapp already provides information about the OpenSSH configuration through the `OpenSshConfigScanner` actor. This actor provides a message with a message model `OpenSshConfig`.
+
+Instead of opening and reading the configuration file in our own actor, we can simply read the provided message to see if we can safely alter the configuration automatically.
+
+To begin with, import the message model from `leapp.models`:
+
+```python
+from leapp.models import OpenSshConfig
+```
+
+> It doesn't matter in which Leapp repository the model is located. Leapp will gather all availabile data inside its submodules.
+
+Add the message model to the list of messages to be received:
+
+```python
+consumes = (OpenSshConfig, )
+```
+
+The actor now will be able to read messages of this format provided by other actors that were executed prior to its own execution.
+
+##### Sending messages
+
+To ensure that the user knows about the automatic configuration change that will occur, we can send a *report*.
+
+> Reports are a built-in type of Leapp messages that are added to the `/var/log/leapp/leapp-report.txt` file at the end of the upgrade process.
+
+To start off with, add a Report message model to the `produces` attribute of the actor.
+
+```python
+produces = (Report, )
+```
+
+Don't forget to import the model type from `leapp.models`.
+
+All done - now we're ready to make use of the models inside the actor's code.
+
+
+##### Running phase
+
+Both workflow and phase tags are imported from leapp.tags:
+
+```python
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+```
+
+All actors to be run during the upgrade must contain the upgrade workflow tag. It looks as follows:
+
+```python
+tags = (IPUWorkflowTag, )
+```
+
+To define the upgrade phase during which an actor will run, set the appropriate tag in the `tags` attribute.
+
+Standard actor `OpenSshPermitRootLoginCheck` that blocks the upgrade if it detects potential problems in SSH configuration, runs during the *checks* phase, and has the `ChecksPhaseTag` inside its `tags`.
+
+Therefore, we want to run our new actor before it. We can select an earlier phase from the list of phases - or we can mark our actor to run *before other actors* in the phase with a modifier as follows:
+
+```python
+tags = (ChecksPhaseTag.Before, IPUWorkflowTag, )
+```
+
+All phases have built-in `.Before` and `.After` stages that can be used this way. Now our actor is guaranteed to be run before the `OpenSshPermitRootLoginCheck` actor.
+
+
+#### Actor code
+
+With configuration done, it's time to write the actual code of the actor that will be executed during the upgrade.
+
+The entry point for it is the actor's `process` function.
+
+First, let's start by reading the SSH config message we've set the actor to receive.
+
+```python
+# Importing from Leapp built-ins.
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.stdlib import api
+
+def process(self):
+ # Retreive the OpenSshConfig message.
+
+ # Actors have `consume` and `produce` methods that work with messages.
+ # `consume` expects a message type that is listed inside the `consumes` attribute.
+ openssh_messages = self.consume(OpenSshConfig)
+
+ # The return value of self.consume is a generator of messages of the provided type.
+ config = next(openssh_messages, None)
+ # We expect to get only one message of this type. If there's more than one, something's wrong.
+ if list(openssh_messages):
+ # api.current_logger lets you pass messages into Leapp's log. By default, they will
+ # be displayed in `/var/log/leapp/leapp-preupgrade.log`
+ # or `/var/log/leapp/leapp-upgrade.log`, depending on which command you ran.
+ api.current_logger().warning('Unexpectedly received more than one OpenSshConfig message.')
+ # If the config message is not present, the standard actor failed to read it.
+ # Stop here.
+ if not config:
+ # StopActorExecutionError is a Leapp built-in exception type that halts the actor execution.
+ # By default this will also halt the upgrade phase and the upgrade process in general.
+ raise StopActorExecutionError(
+ 'Could not check openssh configuration', details={'details': 'No OpenSshConfig facts found.'}
+ )
+```
+
+Next, let's read the received message and see if we can modify the configuration.
+
+```python
+import errno
+
+CONFIG = '/etc/ssh/sshd_config'
+CONFIG_BACKUP = '/etc/ssh/sshd_config.leapp_backup'
+
+ # The OpenSshConfig model has a permit_root_login attribute that contains
+ # all instances of PermitRootLogin option present in the config.
+ # See leapp-repository/repos/system_upgrade/el7toel8/models/opensshconfig.py
+
+ # We can only safely modify the config to preserve the default behaviour if no
+ # explicit PermitRootLogin option was set anywhere in the config.
+ if not config.permit_root_login:
+ try:
+ # Read the config into memory to prepare for its modification.
+ with open(CONFIG, 'r') as fd:
+ sshd_config = fd.readlines()
+
+ # These are the lines we want to add to the configuration file.
+ permit_autoconf = [
+ "# Automatically added by Leapp to preserve RHEL7 default\n",
+ "# behaviour after migration.\n",
+ "# Placed on top of the file to avoid being included into Match blocks.\n",
+ "PermitRootLogin yes\n"
+ "\n",
+ ]
+ permit_autoconf.extend(sshd_config)
+ # Write the changed config into the file.
+ with open(CONFIG, 'w') as fd:
+ fd.writelines(permit_autoconf)
+ # Write the backup file with the old configuration.
+ with open(CONFIG_BACKUP, 'w') as fd:
+ fd.writelines(sshd_config)
+
+ # Handle errors.
+ except IOError as err:
+ if err.errno != errno.ENOENT:
+ error = 'Failed to open sshd_config: {}'.format(str(err))
+ api.current_logger().error(error)
+ return
+```
+
+The functional part of the actor itself is done. Now, let's add a report to let the user know
+the machine's SSH configuration has changed.
+
+```python
+# These Leapp imports are required to create reports.
+from leapp import reporting
+from leapp.models import Report
+from leapp.reporting import create_report
+
+# Tags signify the categories the report and the associated issue are related to.
+COMMON_REPORT_TAGS = [
+ reporting.Tags.AUTHENTICATION,
+ reporting.Tags.SECURITY,
+ reporting.Tags.NETWORK,
+ reporting.Tags.SERVICES
+]
+
+ # Related resources are listed in the report to help resolving the issue.
+ resources = [
+ reporting.RelatedResource('package', 'openssh-server'),
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config')
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config.leapp_backup')
+ ]
+ # This function creates and submits the actual report message.
+ # Normally you'd need to call self.produce() to send messages,
+ # but reports are a special case that gets handled automatically.
+ create_report([
+ # Report title and summary.
+ reporting.Title('SSH configuration automatically modified to permit root login'),
+ reporting.Summary(
+ 'Your OpenSSH configuration file does not explicitly state '
+ 'the option PermitRootLogin in sshd_config file. '
+ 'Its default is "yes" in RHEL7, but will change in '
+ 'RHEL8 to "prohibit-password", which may affect your ability '
+ 'to log onto this machine after the upgrade. '
+ 'To prevent this from occuring, the PermitRootLogin option '
+ 'has been explicity set to "yes" to preserve the default behaivour '
+ 'after migration.'
+ 'The original configuration file has been backed up to'
+ '/etc/ssh/sshd_config.leapp_backup'
+ ),
+ # Reports are ordered by severity in the list.
+ reporting.Severity(reporting.Severity.MEDIUM),
+ reporting.Tags(COMMON_REPORT_TAGS),
+ # Remediation section contains hints on how to resolve the reported (potential) problem.
+ reporting.Remediation(
+ hint='If you would prefer to configure the root login policy yourself, '
+ 'consider setting the PermitRootLogin option '
+ 'in sshd_config explicitly.'
+ )
+ ] + resources) # Resources are added to the list of data for the report.
+```
+
+The actor code is now complete. The final version with less verbose comments will look something like this:
+
+```python
+from leapp import reporting
+from leapp.actors import Actor
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.stdlib import api
+from leapp.models import OpenSshConfig, Report
+from leapp.reporting import create_report
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+
+import errno
+
+CONFIG = '/etc/ssh/sshd_config'
+CONFIG_BACKUP = '/etc/ssh/sshd_config.leapp_backup'
+
+COMMON_REPORT_TAGS = [
+ reporting.Tags.AUTHENTICATION,
+ reporting.Tags.SECURITY,
+ reporting.Tags.NETWORK,
+ reporting.Tags.SERVICES
+]
+
+
+class OpenSSHModifyPermitRoot(Actor):
+ """
+ OpenSSH doesn't allow root logins with password by default on RHEL8.
+
+ Check the values of PermitRootLogin in OpenSSH server configuration file
+ and see if it was set explicitly.
+ If not, adding an explicit "PermitRootLogin yes" will preserve the current
+ default behaviour.
+ """
+
+ name = 'openssh_modify_permit_root'
+ consumes = (OpenSshConfig, )
+ produces = (Report, )
+ tags = (ChecksPhaseTag.Before, IPUWorkflowTag, )
+
+ def process(self):
+ # Retreive the OpenSshConfig message.
+ openssh_messages = self.consume(OpenSshConfig)
+ config = next(openssh_messages, None)
+ if list(openssh_messages):
+ api.current_logger().warning('Unexpectedly received more than one OpenSshConfig message.')
+ if not config:
+ raise StopActorExecutionError(
+ 'Could not check openssh configuration', details={'details': 'No OpenSshConfig facts found.'}
+ )
+
+ # Read and modify the config.
+ # Only act if there's no explicit PermitRootLogin option set anywhere in the config.
+ if not config.permit_root_login:
+ try:
+ with open(CONFIG, 'r') as fd:
+ sshd_config = fd.readlines()
+
+ permit_autoconf = [
+ "# Automatically added by Leapp to preserve RHEL7 default\n",
+ "# behaviour after migration.\n",
+ "# Placed on top of the file to avoid being included into Match blocks.\n",
+ "PermitRootLogin yes\n"
+ "\n",
+ ]
+ permit_autoconf.extend(sshd_config)
+ with open(CONFIG, 'w') as fd:
+ fd.writelines(permit_autoconf)
+ with open(CONFIG_BACKUP, 'w') as fd:
+ fd.writelines(sshd_config)
+
+ except IOError as err:
+ if err.errno != errno.ENOENT:
+ error = 'Failed to open sshd_config: {}'.format(str(err))
+ api.current_logger().error(error)
+ return
+
+ # Create a report letting the user know what happened.
+ resources = [
+ reporting.RelatedResource('package', 'openssh-server'),
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config'),
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config.leapp_backup')
+ ]
+ create_report([
+ reporting.Title('SSH configuration automatically modified to permit root login'),
+ reporting.Summary(
+ 'Your OpenSSH configuration file does not explicitly state '
+ 'the option PermitRootLogin in sshd_config file. '
+ 'Its default is "yes" in RHEL7, but will change in '
+ 'RHEL8 to "prohibit-password", which may affect your ability '
+ 'to log onto this machine after the upgrade. '
+ 'To prevent this from occuring, the PermitRootLogin option '
+ 'has been explicity set to "yes" to preserve the default behaivour '
+ 'after migration.'
+ 'The original configuration file has been backed up to'
+ '/etc/ssh/sshd_config.leapp_backup'
+ ),
+ reporting.Severity(reporting.Severity.MEDIUM),
+ reporting.Tags(COMMON_REPORT_TAGS),
+ reporting.Remediation(
+ hint='If you would prefer to configure the root login policy yourself, '
+ 'consider setting the PermitRootLogin option '
+ 'in sshd_config explicitly.'
+ )
+ ] + resources)
+```
+
+Due to this actor's small size, the entire code can be fit inside the `process` function.
+If it grows beyond manageable size, or you want to run unit tests on its components, it's advised to move out all of the functional parts from the `process` function into the *actor library*.
+
+#### Libraries
+
+Larger actors can import code from [common libraries](https://leapp.readthedocs.io/en/latest/best-practices.html#move-generic-functionality-to-libraries) or define their own "libraries" and run code from them inside the `process` function.
+
+In such cases, the directory layout looks like this:
+```
+actors
++ example_actor_name
+| + libraries
+| + example_actor_name.py
+| + actor.py
+...
+```
+
+and importing code from them looks like this:
+
+`from leapp.libraries.actor.example_actor_name import example_lib_function`
+
+This is also the main way of [writing unit-testable code](https://leapp.readthedocs.io/en/latest/best-practices.html#write-unit-testable-code), since the code contained inside the `process` function cannot be unit-tested normally.
+
+In this actor format, you would move all of the actual actor code into the associated library, leaving only preparation and function calls inside the `process` function.
+
+#### Debugging
+
+The Leapp utility `snactor` can also be used for unit-testing the created actors.
+
+It is capable of saving the output of actors as locally stored messages, so that they can be consumed by other actors that are being developed.
+
+For example, to test our new actor, we need the OpenSshConfig message, which is produced by the OpenSshConfigScanner standard actor. To make the data consumable, run the actor producing the data with the –save-output option:
+
+`snactor run --save-output OpenSshConfigScanner`
+
+The output of the actor is stored in the local repository data file, and it can be used by other actors. To flush all saved messages from the repository database, run `snactor messages clear`.
+
+With the input messages available and stored, the actor being developed can be tested.
+
+`snactor run --print-output OpenSshModifyPermitRoot`
+
+#### Additional information
-You can reach us at IRC: `#leapp` on freenode.
+For more information about Leapp and additional tutorials, visit the [official Leapp documentation](https://leapp.readthedocs.io/en/latest/tutorials.html).
diff --git a/commands/command_utils.py b/commands/command_utils.py
index da62c50..a8e7d76 100644
--- a/commands/command_utils.py
+++ b/commands/command_utils.py
@@ -12,7 +12,7 @@ LEAPP_UPGRADE_FLAVOUR_DEFAULT = 'default'
LEAPP_UPGRADE_FLAVOUR_SAP_HANA = 'saphana'
LEAPP_UPGRADE_PATHS = 'upgrade_paths.json'
-VERSION_REGEX = re.compile(r"^([1-9]\d*)\.(\d+)$")
+VERSION_REGEX = re.compile(r"^([1-9]\d*)(\.(\d+))?$")
def check_version(version):
diff --git a/commands/upgrade/__init__.py b/commands/upgrade/__init__.py
index c9c2741..7646459 100644
--- a/commands/upgrade/__init__.py
+++ b/commands/upgrade/__init__.py
@@ -86,7 +86,7 @@ def upgrade(args, breadcrumbs):
workflow = repositories.lookup_workflow('IPUWorkflow')(auto_reboot=args.reboot)
util.process_whitelist_experimental(repositories, workflow, configuration, logger)
util.warn_if_unsupported(configuration)
- with beautify_actor_exception():
+ with util.format_actor_exceptions(logger):
logger.info("Using answerfile at %s", answerfile_path)
workflow.load_answers(answerfile_path, userchoices_path)
@@ -99,13 +99,16 @@ def upgrade(args, breadcrumbs):
logger.info("Answerfile will be created at %s", answerfile_path)
workflow.save_answers(answerfile_path, userchoices_path)
report_errors(workflow.errors)
+ util.log_errors(workflow.errors, logger)
report_inhibitors(context)
+ util.log_inhibitors(context, logger)
util.generate_report_files(context, report_schema)
report_files = util.get_cfg_files('report', cfg)
log_files = util.get_cfg_files('logs', cfg)
report_info(report_files, log_files, answerfile_path, fail=workflow.failure)
if workflow.failure:
+ logger.error("Upgrade workflow failed, check log for details")
sys.exit(1)
diff --git a/commands/upgrade/util.py b/commands/upgrade/util.py
index 75ffa6a..379d480 100644
--- a/commands/upgrade/util.py
+++ b/commands/upgrade/util.py
@@ -2,18 +2,23 @@ import functools
import itertools
import json
import os
+import sys
import shutil
import tarfile
from datetime import datetime
+from contextlib import contextmanager
from leapp.cli.commands import command_utils
from leapp.cli.commands.config import get_config
-from leapp.exceptions import CommandError
+from leapp.exceptions import CommandError, LeappRuntimeError
from leapp.repository.scan import find_and_scan_repositories
from leapp.utils import audit
from leapp.utils.audit import get_checkpoints, get_connection, get_messages
-from leapp.utils.output import report_unsupported
+from leapp.utils.output import report_unsupported, pretty_block_text
from leapp.utils.report import fetch_upgrade_report_messages, generate_report_file
+from leapp.models import ErrorModel
+
+
def disable_database_sync():
@@ -236,3 +241,61 @@ def process_report_schema(args, configuration):
raise CommandError('--report-schema version can not be greater that the '
'actual {} one.'.format(default_report_schema))
return args.report_schema or default_report_schema
+
+
+# TODO: This and the following functions should eventually be placed into the
+# leapp.utils.output module.
+def pretty_block_log(string, logger_level, width=60):
+ log_str = "\n{separator}\n{text}\n{separator}\n".format(
+ separator="=" * width,
+ text=string.center(width))
+ logger_level(log_str)
+
+
+@contextmanager
+def format_actor_exceptions(logger):
+ try:
+ try:
+ yield
+ except LeappRuntimeError as e:
+ # TODO: This only reports the actor that raised an exception
+ # and the return code.
+ # The traceback gets eaten on the framework level, and is only
+ # seen in stderr. Changing that will require modifying the framework
+ # code itself.
+ msg = '{} - Please check the above details'.format(e.message)
+ sys.stderr.write("\n")
+ sys.stderr.write(pretty_block_text(msg, color="", width=len(msg)))
+ logger.error(e.message)
+ finally:
+ pass
+
+
+def log_errors(errors, logger):
+ if errors:
+ pretty_block_log("ERRORS", logger.info)
+
+ for error in errors:
+ model = ErrorModel.create(json.loads(error['message']['data']))
+ logger.error("{time} [{severity}] Actor: {actor}\nMessage: {message}\n".format(
+ severity=model.severity.upper(),
+ message=model.message, time=model.time, actor=model.actor))
+ if model.details:
+ print('Summary:')
+ details = json.loads(model.details)
+ for detail in details:
+ print(' {k}: {v}'.format(
+ k=detail.capitalize(),
+ v=details[detail].rstrip().replace('\n', '\n' + ' ' * (6 + len(detail)))))
+
+
+def log_inhibitors(context_id, logger):
+ from leapp.reporting import Flags # pylint: disable=import-outside-toplevel
+ reports = fetch_upgrade_report_messages(context_id)
+ inhibitors = [report for report in reports if Flags.INHIBITOR in report.get('flags', [])]
+ if inhibitors:
+ pretty_block_log("UPGRADE INHIBITED", logger.error)
+ logger.error('Upgrade has been inhibited due to the following problems:')
+ for position, report in enumerate(inhibitors, start=1):
+ logger.error('{idx:5}. Inhibitor: {title}'.format(idx=position, title=report['title']))
+ logger.info('Consult the pre-upgrade report for details and possible remediation.')
diff --git a/packaging/leapp-repository.spec b/packaging/leapp-repository.spec
index af4b31d..0d8f6c8 100644
--- a/packaging/leapp-repository.spec
+++ b/packaging/leapp-repository.spec
@@ -196,6 +196,8 @@ rm -rf %{buildroot}%{leapp_python_sitelib}/leapp/cli/commands/tests
rm -rf %{buildroot}%{repositorydir}/system_upgrade/el8toel9
%else
rm -rf %{buildroot}%{repositorydir}/system_upgrade/el7toel8
+# CloudLinux migration only supports el7 to el8
+rm -rf %{buildroot}%{repositorydir}/system_upgrade/cloudlinux
%endif
# remove component/unit tests, Makefiles, ... stuff that related to testing only
diff --git a/repos/system_upgrade/cloudlinux/.leapp/info b/repos/system_upgrade/cloudlinux/.leapp/info
new file mode 100644
index 0000000..1f16b9f
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/.leapp/info
@@ -0,0 +1 @@
+{"name": "cloudlinux", "id": "427ddd90-9b5e-4400-b21e-73d77791f175", "repos": ["644900a5-c347-43a3-bfab-f448f46d9647", "c47fbc3d-ae38-416e-9176-7163d67d94f6", "efcf9016-f2d1-4609-9329-a298e6587b3c"]}
\ No newline at end of file
diff --git a/repos/system_upgrade/cloudlinux/.leapp/leapp.conf b/repos/system_upgrade/cloudlinux/.leapp/leapp.conf
new file mode 100644
index 0000000..b459134
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/.leapp/leapp.conf
@@ -0,0 +1,6 @@
+
+[repositories]
+repo_path=${repository:root_dir}
+
+[database]
+path=${repository:state_dir}/leapp.db
diff --git a/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/actor.py b/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/actor.py
new file mode 100644
index 0000000..783e347
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/actor.py
@@ -0,0 +1,21 @@
+from leapp.actors import Actor
+from leapp.tags import FirstBootPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+from leapp.libraries.actor.addcustomrepositories import add_custom
+
+
+class AddCustomRepositories(Actor):
+ """
+ Move the files inside the custom-repos folder of this leapp repository into the /etc/yum.repos.d repository.
+ """
+
+ name = 'add_custom_repositories'
+ consumes = ()
+ produces = ()
+ tags = (IPUWorkflowTag, FirstBootPhaseTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ # We only want to run this actor on CloudLinux systems.
+ # current_version returns a tuple (release_name, version_value).
+ add_custom(self.log)
diff --git a/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/libraries/addcustomrepositories.py b/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/libraries/addcustomrepositories.py
new file mode 100644
index 0000000..74ba425
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/addcustomrepositories/libraries/addcustomrepositories.py
@@ -0,0 +1,26 @@
+import os
+import os.path
+import shutil
+import logging
+
+from leapp.libraries.stdlib import api
+
+CUSTOM_REPOS_FOLDER = 'custom-repos'
+REPO_ROOT_PATH = "/etc/yum.repos.d"
+
+
+def add_custom(log):
+ # type: (logging.Logger) -> None
+ custom_repo_dir = api.get_common_folder_path(CUSTOM_REPOS_FOLDER)
+ repofiles = os.listdir(custom_repo_dir)
+
+ # If any components are missing, halt.
+ if not repofiles or not custom_repo_dir:
+ return
+
+ for repofile in repofiles:
+ full_repo_path = os.path.join(custom_repo_dir, repofile)
+
+ log.debug("Copying repo file {} to {}".format(repofile, REPO_ROOT_PATH))
+
+ shutil.copy(full_repo_path, REPO_ROOT_PATH)
diff --git a/repos/system_upgrade/cloudlinux/actors/checkcllicense/actor.py b/repos/system_upgrade/cloudlinux/actors/checkcllicense/actor.py
new file mode 100644
index 0000000..2c935d7
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/checkcllicense/actor.py
@@ -0,0 +1,41 @@
+from leapp.actors import Actor
+from leapp import reporting
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+from leapp.libraries.stdlib import CalledProcessError, run
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+import os
+
+
+class CheckClLicense(Actor):
+ """
+ Check if the server has a CL license
+ """
+
+ name = 'check_cl_license'
+ consumes = ()
+ produces = ()
+ tags = (ChecksPhaseTag, IPUWorkflowTag)
+
+ system_id_path = '/etc/sysconfig/rhn/systemid'
+ rhn_check_bin = '/usr/sbin/rhn_check'
+
+ @run_on_cloudlinux
+ def process(self):
+ res = None
+ if os.path.exists(self.system_id_path):
+ res = run([self.rhn_check_bin])
+ self.log.debug('rhn_check result: %s', res)
+ if not res or res['exit_code'] != 0 or res['stderr']:
+ title = 'Server does not have an active CloudLinux license'
+ summary = 'Server does not have an active CloudLinux license. This renders key CloudLinux packages ' \
+ 'inaccessible, inhibiting the upgrade process.'
+ remediation = 'Activate a CloudLinux license on this machine before running Leapp again.'
+ reporting.create_report([
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Tags([reporting.Tags.OS_FACTS]),
+ reporting.Flags([reporting.Flags.INHIBITOR]),
+ reporting.Remediation(hint=remediation),
+ ])
diff --git a/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/actor.py b/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/actor.py
new file mode 100644
index 0000000..a1c1cee
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/actor.py
@@ -0,0 +1,58 @@
+from leapp.actors import Actor
+from leapp import reporting
+from leapp.reporting import Report
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+from leapp.libraries.actor.version import (
+ Version, VersionParsingError,
+)
+
+import subprocess
+
+
+class CheckRhnClientToolsVersion(Actor):
+ """
+ Check the rhn-client-tools package version
+ """
+
+ name = 'check_rhn_client_tools_version'
+ consumes = ()
+ produces = (Report,)
+ tags = (ChecksPhaseTag, IPUWorkflowTag)
+
+ minimal_version = Version('2.0.2')
+ minimal_release_int = 43
+ minimal_release = '%s.el7.cloudlinux' % minimal_release_int
+
+ @run_on_cloudlinux
+ def process(self):
+ title, summary, remediation = None, None, None
+ # ex:
+ # Version : 2.0.2
+ # Release : 43.el7.cloudlinux
+ # res is: b'2.0.2\n43.el7.cloudlinux\n'
+ cmd = "yum info installed rhn-client-tools | grep '^Version' -A 1 | awk '{print $3}'"
+ res = subprocess.check_output(cmd, shell=True)
+ rhn_version, rhn_release = res.decode().split()
+ self.log.info('Current rhn-client-tools version: "%s"', rhn_version)
+ try:
+ current_version = Version(rhn_version)
+ except VersionParsingError:
+ title = 'rhn-client-tools: package is not installed'
+ summary = 'rhn-client-tools package is required to perform elevation.'
+ remediation = 'Install rhn-client-tools "%s" version before running Leapp again.' % self.minimal_version
+ else:
+ if current_version < self.minimal_version or int(rhn_release.split('.')[0]) < self.minimal_release_int:
+ title = 'rhn-client-tools: package version is too low'
+ summary = 'Current version of the rhn-client-tools package has no capability to perform elevation.'
+ remediation = 'Update rhn-client-tools to "%s %s" version before running Leapp again.' % (self.minimal_version, self.minimal_release)
+ if title:
+ reporting.create_report([
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Tags([reporting.Tags.OS_FACTS]),
+ reporting.Flags([reporting.Flags.INHIBITOR]),
+ reporting.Remediation(hint=remediation),
+ ])
diff --git a/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/libraries/version.py b/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/libraries/version.py
new file mode 100644
index 0000000..149bce2
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/checkrhnclienttools/libraries/version.py
@@ -0,0 +1,46 @@
+from six import reraise as raise_
+import sys
+
+
+class VersionException(Exception):
+ pass
+
+
+class VersionParsingError(VersionException):
+ pass
+
+
+class Version(object):
+ def __init__(self, version):
+ self._raw = version
+ try:
+ self.value = tuple(
+ map(lambda x: int(x), version.split('.'))
+ )
+ except Exception:
+ tb = sys.exc_info()[2]
+ raise_(VersionParsingError, 'failed to parse version: "%s"' % self._raw, tb)
+
+ def __eq__(self, other):
+ return self.value == other.value
+
+ def __gt__(self, other):
+ return any(
+ [v[0] > v[1] for v in zip(self.value, other.value)]
+ )
+
+ def __ge__(self, other):
+ return all(
+ [v[0] >= v[1] for v in zip(self.value, other.value)]
+ )
+
+ def __lt__(self, other):
+ return any(
+ [v[0] < v[1] for v in zip(self.value, other.value)]
+ )
+
+ def __le__(self, other):
+ return all(
+ [v[0] <= v[1] for v in zip(self.value, other.value)]
+ )
+
diff --git a/repos/system_upgrade/cloudlinux/actors/checkrhnversionoverride/actor.py b/repos/system_upgrade/cloudlinux/actors/checkrhnversionoverride/actor.py
new file mode 100644
index 0000000..d832d34
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/checkrhnversionoverride/actor.py
@@ -0,0 +1,38 @@
+from leapp.actors import Actor
+from leapp import reporting
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+
+class CheckRhnVersionOverride(Actor):
+ """
+ Check if the up2date versionOverride option has not been set.
+ """
+
+ name = 'check_rhn_version_override'
+ consumes = ()
+ produces = ()
+ tags = (ChecksPhaseTag, IPUWorkflowTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ up2date_config = '/etc/sysconfig/rhn/up2date'
+ with open(up2date_config, 'r') as f:
+ config_data = f.readlines()
+ for line in config_data:
+ if line.startswith('versionOverride=') and line != 'versionOverride=':
+ title = 'RHN up2date: versionOverride not empty'
+ summary = ('The RHN config file up2date has a set value of the versionOverride option.'
+ ' This value will get overwritten by the upgrade process, and non-supported values'
+ ' carry a risk of causing issues during the upgrade.')
+ remediation = ('Remove the versionOverride value from the up2date config file'
+ ' before running Leapp again.')
+ reporting.create_report([
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Tags([reporting.Tags.OS_FACTS]),
+ reporting.Flags([reporting.Flags.INHIBITOR]),
+ reporting.Remediation(hint=remediation),
+ reporting.RelatedResource('file', '/etc/sysconfig/rhn/up2date')
+ ])
diff --git a/repos/system_upgrade/cloudlinux/actors/checkup2dateconfig/actor.py b/repos/system_upgrade/cloudlinux/actors/checkup2dateconfig/actor.py
new file mode 100644
index 0000000..bfc0642
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/checkup2dateconfig/actor.py
@@ -0,0 +1,48 @@
+from leapp.actors import Actor
+from leapp.tags import FirstBootPhaseTag, IPUWorkflowTag
+from leapp import reporting
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+import os
+
+
+class CheckUp2dateConfig(Actor):
+ """
+ Move up2date.rpmnew config to the old one's place
+ """
+
+ name = 'check_up2date_config'
+ consumes = ()
+ produces = ()
+ tags = (FirstBootPhaseTag, IPUWorkflowTag)
+
+ original = '/etc/sysconfig/rhn/up2date'
+ new = original + '.rpmnew'
+
+ @run_on_cloudlinux
+ def process(self):
+ """
+ For some reason we get new .rpmnew file instead of the modified `original`
+ This actor tries to save the old `serverURL` parameter to new config and move new instead of old one
+ """
+ replace, old_lines, new_lines = None, None, None
+ if os.path.exists(self.new):
+ self.log.warning('"%s" config found, trying to replace the old one', self.new)
+ with open(self.original) as o, open(self.new) as n:
+ old_lines = o.readlines()
+ new_lines = n.readlines()
+ for l in old_lines:
+ if l.startswith('serverURL=') and l not in new_lines:
+ replace = l
+ break
+ if replace:
+ for i, line in enumerate(new_lines):
+ if line.startswith('serverURL='):
+ new_lines[i] = replace
+ self.log.warning('"serverURL" parameter will be saved as "%s"', line.strip())
+ break
+ with open(self.original, 'w') as f:
+ f.writelines(new_lines)
+ self.log.info('"%s" config is overwritten by the contents of "%s"', self.original, self.new)
+ os.unlink(self.new)
+ self.log.info('"%s" config deleted', self.new)
diff --git a/repos/system_upgrade/cloudlinux/actors/clearpackageconflicts/actor.py b/repos/system_upgrade/cloudlinux/actors/clearpackageconflicts/actor.py
new file mode 100644
index 0000000..cd6801b
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/clearpackageconflicts/actor.py
@@ -0,0 +1,36 @@
+import os
+import errno
+import shutil
+
+from leapp.actors import Actor
+from leapp.libraries.common.rpms import has_package
+from leapp.models import InstalledRPM
+from leapp.tags import DownloadPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+class ClearPackageConflicts(Actor):
+ """
+ Remove several python package files manually to resolve conflicts between versions of packages to be upgraded.
+ """
+
+ name = 'clear_package_conflicts'
+ consumes = (InstalledRPM,)
+ produces = ()
+ tags = (DownloadPhaseTag.Before, IPUWorkflowTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ problem_packages = ["alt-python37-six", "alt-python37-pytz"]
+ problem_packages_installed = any([has_package(InstalledRPM, pkg) for pkg in problem_packages])
+
+ if problem_packages_installed:
+ problem_dirs = [
+ "/opt/alt/python37/lib/python3.7/site-packages/six-1.15.0-py3.7.egg-info",
+ "/opt/alt/python37/lib/python3.7/site-packages/pytz-2017.2-py3.7.egg-info"]
+ for p_dir in problem_dirs:
+ try:
+ if os.path.isdir(p_dir):
+ shutil.rmtree(p_dir)
+ except OSError as e:
+ if e.errno != errno.ENOENT:
+ raise
diff --git a/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/actor.py b/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/actor.py
new file mode 100644
index 0000000..7b947f9
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/actor.py
@@ -0,0 +1,44 @@
+from leapp import reporting
+from leapp.actors import Actor
+from leapp.reporting import Report
+from leapp.tags import ChecksPhaseTag, IPUWorkflowTag
+
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+from leapp.libraries.actor.detectcontrolpanel import detect_panel, UNKNOWN_NAME
+
+
+class DetectControlPanel(Actor):
+ """
+ Check for a presence of a control panel, and inhibit the upgrade if one is found.
+ """
+
+ name = 'detect_control_panel'
+ consumes = ()
+ produces = (Report,)
+ tags = (ChecksPhaseTag, IPUWorkflowTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ panel = detect_panel()
+
+ if panel:
+ summary_info = "Detected panel: {}".format(panel)
+ if panel == UNKNOWN_NAME:
+ summary_info = "Legacy custom panel script detected in CloudLinux configuration"
+
+ # Block the upgrade on any systems with a panel detected.
+ reporting.create_report([
+ reporting.Title("The upgrade process should not be run on systems with a control panel present."),
+ reporting.Summary(
+ "Systems with a control panel present are not supported at the moment."
+ " No control panels are currently included in the Leapp database, which"
+ " makes loss of functionality after the upgrade extremely likely."
+ " {}.".format(summary_info)),
+ reporting.Severity(reporting.Severity.HIGH),
+ reporting.Tags([
+ reporting.Tags.OS_FACTS
+ ]),
+ reporting.Flags([
+ reporting.Flags.INHIBITOR
+ ])
+ ])
diff --git a/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/libraries/detectcontrolpanel.py b/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/libraries/detectcontrolpanel.py
new file mode 100644
index 0000000..15b797a
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/detectcontrolpanel/libraries/detectcontrolpanel.py
@@ -0,0 +1,68 @@
+import os
+import os.path
+
+from leapp.libraries.stdlib import api
+
+
+CPANEL_NAME = 'cPanel'
+DIRECTADMIN_NAME = 'DirectAdmin'
+PLESK_NAME = 'Plesk'
+ISPMANAGER_NAME = 'ISPManager'
+INTERWORX_NAME = 'InterWorx'
+UNKNOWN_NAME = 'Unknown'
+INTEGRATED_NAME = 'Integrated'
+
+CLSYSCONFIG = '/etc/sysconfig/cloudlinux'
+
+
+def lvectl_custompanel_script():
+ """
+ Retrives custom panel script for lvectl from CL config file
+ :return: Script path or None if script filename wasn't found in config
+ """
+ config_param_name = 'CUSTOM_GETPACKAGE_SCRIPT'
+ try:
+ # Try to determine the custom script name
+ if os.path.exists(CLSYSCONFIG):
+ with open(CLSYSCONFIG, 'r') as f:
+ file_lines = f.readlines()
+ for line in file_lines:
+ line = line.strip()
+ if line.startswith(config_param_name):
+ line_parts = line.split('=')
+ if len(line_parts) == 2 and line_parts[0].strip() == config_param_name:
+ script_name = line_parts[1].strip()
+ if os.path.exists(script_name):
+ return script_name
+ except (OSError, IOError, IndexError):
+ # Ignore errors - what's important is that the script wasn't found
+ pass
+ return None
+
+
+def detect_panel():
+ """
+ This function will try to detect control panels supported by CloudLinux
+ :return: Detected control panel name or None
+ """
+ panel_name = None
+ if os.path.isfile('/opt/cpvendor/etc/integration.ini'):
+ panel_name = INTEGRATED_NAME
+ elif os.path.isfile('/usr/local/cpanel/cpanel'):
+ panel_name = CPANEL_NAME
+ elif os.path.isfile('/usr/local/directadmin/directadmin') or\
+ os.path.isfile('/usr/local/directadmin/custombuild/build'):
+ panel_name = DIRECTADMIN_NAME
+ elif os.path.isfile('/usr/local/psa/version'):
+ panel_name = PLESK_NAME
+ # ispmanager must have:
+ # v5: /usr/local/mgr5/ directory,
+ # v4: /usr/local/ispmgr/bin/ispmgr file
+ elif os.path.isfile('/usr/local/ispmgr/bin/ispmgr') or os.path.isdir('/usr/local/mgr5'):
+ panel_name = ISPMANAGER_NAME
+ elif os.path.isdir('/usr/local/interworx'):
+ panel_name = INTERWORX_NAME
+ # Check if the CL config has a legacy custom script for a control panel
+ elif lvectl_custompanel_script():
+ panel_name = UNKNOWN_NAME
+ return panel_name
diff --git a/repos/system_upgrade/cloudlinux/actors/enableyumspacewalkplugin/actor.py b/repos/system_upgrade/cloudlinux/actors/enableyumspacewalkplugin/actor.py
new file mode 100644
index 0000000..1e353b1
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/enableyumspacewalkplugin/actor.py
@@ -0,0 +1,55 @@
+from leapp.actors import Actor
+from leapp.tags import FirstBootPhaseTag, IPUWorkflowTag
+from leapp import reporting
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+try:
+ # py2
+ import ConfigParser as configparser
+ ParserClass = configparser.SafeConfigParser
+except Exception:
+ # py3
+ import configparser
+ ParserClass = configparser.ConfigParser
+
+
+class EnableYumSpacewalkPlugin(Actor):
+ """
+ Enable yum spacewalk plugin if it's disabled
+ Required for the CLN channel functionality to work properly
+ """
+
+ name = 'enable_yum_spacewalk_plugin'
+ consumes = ()
+ produces = ()
+ tags = (FirstBootPhaseTag, IPUWorkflowTag)
+
+ config = '/etc/yum/pluginconf.d/spacewalk.conf'
+
+ @run_on_cloudlinux
+ def process(self):
+ summary = 'yum spacewalk plugin must be enabled for the CLN channels to work properly. ' \
+ 'Please make sure it is enabled. Default config path is "%s"' % self.config
+ title = None
+
+ parser = ParserClass(allow_no_value=True)
+ try:
+ red = parser.read(self.config)
+ if not red:
+ title = 'yum spacewalk plugin config not found'
+ if parser.get('main', 'enabled') != '1':
+ parser.set('main', 'enabled', '1')
+ with open(self.config, 'w') as f:
+ parser.write(f)
+ self.log.info('yum spacewalk plugin enabled')
+ return
+ except Exception as e:
+ title = 'yum spacewalk plugin config error: %s' % e
+
+ if title:
+ reporting.create_report([
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.MEDIUM),
+ reporting.Tags([reporting.Tags.SANITY])
+ ])
diff --git a/repos/system_upgrade/cloudlinux/actors/resetrhnversionoverride/actor.py b/repos/system_upgrade/cloudlinux/actors/resetrhnversionoverride/actor.py
new file mode 100644
index 0000000..21b2164
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/resetrhnversionoverride/actor.py
@@ -0,0 +1,25 @@
+from leapp.actors import Actor
+from leapp.tags import FinalizationPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+
+class ResetRhnVersionOverride(Actor):
+ """
+ Reset the versionOverride value in the RHN up2date config to empty.
+ """
+
+ name = 'reset_rhn_version_override'
+ consumes = ()
+ produces = ()
+ tags = (FinalizationPhaseTag, IPUWorkflowTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ up2date_config = '/etc/sysconfig/rhn/up2date'
+ with open(up2date_config, 'r') as f:
+ config_data = f.readlines()
+ for line in config_data:
+ if line.startswith('versionOverride='):
+ line = 'versionOverride='
+ with open(up2date_config, 'w') as f:
+ f.writelines(config_data)
diff --git a/repos/system_upgrade/cloudlinux/actors/switchclnchannel/actor.py b/repos/system_upgrade/cloudlinux/actors/switchclnchannel/actor.py
new file mode 100644
index 0000000..ae01af1
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/switchclnchannel/actor.py
@@ -0,0 +1,33 @@
+from leapp.actors import Actor
+from leapp.libraries.stdlib import api
+from leapp.tags import DownloadPhaseTag, IPUWorkflowTag
+from leapp.libraries.stdlib import CalledProcessError, run
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+
+class SwitchClnChannel(Actor):
+ """
+ Switch CLN channel from 7 to 8 to be able to download upgrade packages
+ """
+
+ name = 'switch_cln_channel'
+ consumes = ()
+ produces = ()
+ tags = (IPUWorkflowTag, DownloadPhaseTag.Before)
+
+ switch_bin = '/usr/sbin/cln-switch-channel'
+
+ @run_on_cloudlinux
+ def process(self):
+ switch_cmd = [self.switch_bin, '-t', '8', '-o', '-f']
+ yum_clean_cmd = ['yum', 'clean', 'all']
+ update_release_cmd = ['yum', 'update', '-y', 'cloudlinux-release']
+ try:
+ res = run(switch_cmd)
+ self.log.debug('Command "%s" result: %s', switch_cmd, res)
+ res = run(yum_clean_cmd) # required to update the repolist
+ self.log.debug('Command "%s" result: %s', yum_clean_cmd, res)
+ res = run(update_release_cmd)
+ self.log.debug('Command "%s" result: %s', update_release_cmd, res)
+ except OSError as e:
+ api.current_logger().error('Could not call RHN command: Message: %s', str(e), exc_info=True)
diff --git a/repos/system_upgrade/cloudlinux/actors/updatecagefs/actor.py b/repos/system_upgrade/cloudlinux/actors/updatecagefs/actor.py
new file mode 100644
index 0000000..71e3c66
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/actors/updatecagefs/actor.py
@@ -0,0 +1,36 @@
+import os
+
+from leapp.actors import Actor
+from leapp.libraries.stdlib import run, CalledProcessError
+from leapp.reporting import Report, create_report
+from leapp import reporting
+from leapp.tags import FirstBootPhaseTag, IPUWorkflowTag
+from leapp.libraries.common.cllaunch import run_on_cloudlinux
+
+
+class UpdateCagefs(Actor):
+ """
+ Force update of cagefs.
+
+ cagefs should reflect massive changes in system made in previous phases
+ """
+
+ name = 'update_cagefs'
+ consumes = ()
+ produces = ()
+ tags = (FirstBootPhaseTag, IPUWorkflowTag)
+
+ @run_on_cloudlinux
+ def process(self):
+ if os.path.exists('/usr/sbin/cagefsctl'):
+ try:
+ run(['/usr/sbin/cagefsctl', '--force-update'], checked=True)
+ self.log.info('cagefs update was successful')
+ except CalledProcessError as e:
+ # cagefsctl prints errors in stdout
+ self.log.error(e.stdout)
+ self.log.error('Command "cagefsctl --force-update" finished with exit code {}, '
+ 'the filesystem inside cagefs may be out-of-date.\n'
+ 'Check cagefsctl output above and in /var/log/cagefs-update.log, '
+ 'rerun "cagefsctl --force-update" after fixing the issues.'.format(e.exit_code)
+ )
diff --git a/repos/system_upgrade/cloudlinux/libraries/cllaunch.py b/repos/system_upgrade/cloudlinux/libraries/cllaunch.py
new file mode 100644
index 0000000..6cbab5d
--- /dev/null
+++ b/repos/system_upgrade/cloudlinux/libraries/cllaunch.py
@@ -0,0 +1,11 @@
+import functools
+from leapp.libraries.common.config import version
+
+
+def run_on_cloudlinux(func):
+ @functools.wraps(func)
+ def wrapper(*args, **kwargs):
+ if (version.current_version()[0] != "cloudlinux"):
+ return
+ return func(*args, **kwargs)
+ return wrapper
diff --git a/repos/system_upgrade/common/actors/addupgradebootentry/libraries/addupgradebootentry.py b/repos/system_upgrade/common/actors/addupgradebootentry/libraries/addupgradebootentry.py
index a2cede0..5ff1c76 100644
--- a/repos/system_upgrade/common/actors/addupgradebootentry/libraries/addupgradebootentry.py
+++ b/repos/system_upgrade/common/actors/addupgradebootentry/libraries/addupgradebootentry.py
@@ -17,7 +17,7 @@ def add_boot_entry(configs=None):
'/usr/sbin/grubby',
'--add-kernel', '{0}'.format(kernel_dst_path),
'--initrd', '{0}'.format(initram_dst_path),
- '--title', 'RHEL-Upgrade-Initramfs',
+ '--title', 'ELevate-Upgrade-Initramfs',
'--copy-default',
'--make-default',
'--args', '{DEBUG} enforcing=0 rd.plymouth=0 plymouth.enable=0'.format(DEBUG=debug)
diff --git a/repos/system_upgrade/common/actors/addupgradebootentry/tests/unit_test_addupgradebootentry.py b/repos/system_upgrade/common/actors/addupgradebootentry/tests/unit_test_addupgradebootentry.py
index bb89c9f..2b8e7c8 100644
--- a/repos/system_upgrade/common/actors/addupgradebootentry/tests/unit_test_addupgradebootentry.py
+++ b/repos/system_upgrade/common/actors/addupgradebootentry/tests/unit_test_addupgradebootentry.py
@@ -42,7 +42,7 @@ run_args_add = [
'/usr/sbin/grubby',
'--add-kernel', '/abc',
'--initrd', '/def',
- '--title', 'RHEL-Upgrade-Initramfs',
+ '--title', 'ELevate-Upgrade-Initramfs',
'--copy-default',
'--make-default',
'--args',
diff --git a/repos/system_upgrade/common/actors/checkenabledvendorrepos/actor.py b/repos/system_upgrade/common/actors/checkenabledvendorrepos/actor.py
new file mode 100644
index 0000000..5284aec
--- /dev/null
+++ b/repos/system_upgrade/common/actors/checkenabledvendorrepos/actor.py
@@ -0,0 +1,53 @@
+from leapp.actors import Actor
+from leapp.libraries.stdlib import api
+from leapp.models import (
+ RepositoriesFacts,
+ VendorSourceRepos,
+ ActiveVendorList,
+)
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+
+
+class CheckEnabledVendorRepos(Actor):
+ """
+ Create a list of vendors whose repositories are present on the system.
+ Only those vendors' configurations (new repositories, PES actions, etc.)
+ will be included in the upgrade process.
+ """
+
+ name = "check_enabled_vendor_repos"
+ consumes = (RepositoriesFacts, VendorSourceRepos)
+ produces = (ActiveVendorList)
+ tags = (IPUWorkflowTag, FactsPhaseTag.Before)
+
+ def process(self):
+ vendor_mapping_data = {}
+ active_vendors = set()
+
+ # Make a dict for easy lookup of repoid -> vendor name.
+ for vendor_src_repodata in api.consume(VendorSourceRepos):
+ for vendor_src_repo in vendor_src_repodata.source_repoids:
+ vendor_mapping_data[vendor_src_repo] = vendor_src_repodata.vendor
+
+ # Is the repo listed in the vendor map as from_repoid present on the system?
+ for repos in api.consume(RepositoriesFacts):
+ for repo_file in repos.repositories:
+ for repo in repo_file.data:
+ self.log.debug(
+ "Looking for repository {} in vendor maps".format(repo.repoid)
+ )
+ if repo.repoid in vendor_mapping_data:
+ # If the vendor's repository is present in the system, count the vendor as active.
+ new_vendor = vendor_mapping_data[repo.repoid]
+ self.log.debug(
+ "Repository {} found, enabling vendor {}".format(
+ repo.repoid, new_vendor
+ )
+ )
+ active_vendors.add(new_vendor)
+
+ if active_vendors:
+ self.log.debug("Active vendor list: {}".format(active_vendors))
+ api.produce(ActiveVendorList(data=list(active_vendors)))
+ else:
+ self.log.info("No active vendors found, vendor list not generated")
diff --git a/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh b/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh
index acdb93b..da1e814 100755
--- a/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh
+++ b/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/85sys-upgrade-redhat/do-upgrade.sh
@@ -8,7 +8,7 @@ fi
type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
get_rhel_major_release() {
- local os_version=$(cat /etc/initrd-release | grep -o '^VERSION="[0-9][0-9]*\.' | grep -o '[0-9]*')
+ local os_version=$(cat /etc/initrd-release | grep -o '^VERSION="[0-9][0-9]*' | grep -o '[0-9]*')
[ -z "$os_version" ] && {
# This should not happen as /etc/initrd-release is supposed to have API
# stability, but check is better than broken system.
@@ -326,4 +326,3 @@ getarg 'rd.break=leapp-logs' && emergency_shell -n upgrade "Break after LEAPP sa
sync
mount -o "remount,$old_opts" $NEWROOT
exit $result
-
diff --git a/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/90sys-upgrade/initrd-system-upgrade-generator b/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/90sys-upgrade/initrd-system-upgrade-generator
index 14bd6e3..f6adacf 100755
--- a/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/90sys-upgrade/initrd-system-upgrade-generator
+++ b/repos/system_upgrade/common/actors/commonleappdracutmodules/files/dracut/90sys-upgrade/initrd-system-upgrade-generator
@@ -1,7 +1,7 @@
#!/bin/sh
get_rhel_major_release() {
- local os_version=$(cat /etc/initrd-release | grep -o '^VERSION="[0-9][0-9]*\.' | grep -o '[0-9]*')
+ local os_version=$(cat /etc/initrd-release | grep -o '^VERSION="[0-9][0-9]*' | grep -o '[0-9]*')
[ -z "$os_version" ] && {
# This should not happen as /etc/initrd-release is supposed to have API
# stability, but check is better than broken system.
diff --git a/repos/system_upgrade/common/actors/efibootorderfix/finalization/actor.py b/repos/system_upgrade/common/actors/efibootorderfix/finalization/actor.py
index f42909f..171fdd2 100644
--- a/repos/system_upgrade/common/actors/efibootorderfix/finalization/actor.py
+++ b/repos/system_upgrade/common/actors/efibootorderfix/finalization/actor.py
@@ -1,17 +1,101 @@
+import os
+import re
+
+from leapp.libraries.stdlib import run, api
from leapp.actors import Actor
-from leapp.libraries.common import efi_reboot_fix
+from leapp.models import InstalledTargetKernelVersion, KernelCmdlineArg, FirmwareFacts, MountEntry
from leapp.tags import FinalizationPhaseTag, IPUWorkflowTag
+from leapp.exceptions import StopActorExecutionError
class EfiFinalizationFix(Actor):
"""
- Adjust EFI boot entry for final reboot
+ Ensure that EFI boot order is updated, which is particularly necessary
+ when upgrading to a different OS distro. Also rebuilds grub config
+ if necessary.
"""
name = 'efi_finalization_fix'
- consumes = ()
+ consumes = (KernelCmdlineArg, InstalledTargetKernelVersion, FirmwareFacts, MountEntry)
produces = ()
tags = (FinalizationPhaseTag, IPUWorkflowTag)
def process(self):
- efi_reboot_fix.maybe_emit_updated_boot_entry()
+ is_system_efi = False
+ ff = next(self.consume(FirmwareFacts), None)
+
+ dirname = {
+ 'AlmaLinux': 'almalinux',
+ 'CentOS Linux': 'centos',
+ 'CentOS Stream': 'centos',
+ 'Oracle Linux Server': 'redhat',
+ 'Red Hat Enterprise Linux': 'redhat',
+ 'Rocky Linux': 'rocky',
+ 'CloudLinux': 'centos',
+ }
+
+ efi_shimname_dict = {
+ 'x86_64': 'shimx64.efi',
+ 'aarch64': 'shimaa64.efi'
+ }
+
+ def devparts(dev):
+ part = next(re.finditer(r'\d+$', dev)).group(0)
+ dev = dev[:-len(part)]
+ return [dev, part];
+
+ with open('/etc/system-release', 'r') as sr:
+ release_line = next(line for line in sr if 'release' in line)
+ distro = release_line.split(' release ', 1)[0]
+
+ efi_bootentry_label = distro
+ distro_dir = dirname.get(distro, 'default')
+ shim_filename = efi_shimname_dict.get(api.current_actor().configuration.architecture, 'shimx64.efi')
+
+ shim_path = '/boot/efi/EFI/' + distro_dir + '/' + shim_filename
+ grub_cfg_path = '/boot/efi/EFI/' + distro_dir + '/grub.cfg'
+ bootmgr_path = '\\EFI\\' + distro_dir + '\\' + shim_filename
+
+ has_efibootmgr = os.path.exists('/sbin/efibootmgr')
+ has_shim = os.path.exists(shim_path)
+ has_grub_cfg = os.path.exists(grub_cfg_path)
+
+ if not ff:
+ raise StopActorExecutionError(
+ 'Could not identify system firmware',
+ details={'details': 'Actor did not receive FirmwareFacts message.'}
+ )
+
+ if not has_efibootmgr:
+ return
+
+ for fact in self.consume(FirmwareFacts):
+ if fact.firmware == 'efi':
+ is_system_efi = True
+ break
+
+ if is_system_efi and has_shim:
+ efidevlist = []
+ with open('/proc/mounts', 'r') as fp:
+ for line in fp:
+ if '/boot/efi' in line:
+ efidevpath = line.split(' ', 1)[0]
+ efidevpart = efidevpath.split('/')[-1]
+ if os.path.exists('/proc/mdstat'):
+ with open('/proc/mdstat', 'r') as mds:
+ for line in mds:
+ if line.startswith(efidevpart):
+ mddev = line.split(' ')
+ for md in mddev:
+ if '[' in md:
+ efimd = md.split('[', 1)[0]
+ efidp = efidevpath.replace(efidevpart, efimd)
+ efidevlist.append(efidp)
+ if len(efidevlist) == 0:
+ efidevlist.append(efidevpath)
+ for devpath in efidevlist:
+ efidev, efipart = devparts(devpath)
+ run(['/sbin/efibootmgr', '-c', '-d', efidev, '-p', efipart, '-l', bootmgr_path, '-L', efi_bootentry_label])
+
+ if not has_grub_cfg:
+ run(['/sbin/grub2-mkconfig', '-o', grub_cfg_path])
diff --git a/repos/system_upgrade/common/actors/ipuworkflowconfig/libraries/ipuworkflowconfig.py b/repos/system_upgrade/common/actors/ipuworkflowconfig/libraries/ipuworkflowconfig.py
index edf978f..7fea4ec 100644
--- a/repos/system_upgrade/common/actors/ipuworkflowconfig/libraries/ipuworkflowconfig.py
+++ b/repos/system_upgrade/common/actors/ipuworkflowconfig/libraries/ipuworkflowconfig.py
@@ -47,6 +47,7 @@ def get_os_release(path):
:return: `OSRelease` model if the file can be parsed
:raises: `IOError`
"""
+ os_version = '.'.join(platform.dist()[1].split('.')[:2])
try:
with open(path) as f:
data = dict(l.strip().split('=', 1) for l in f.readlines() if '=' in l)
@@ -55,7 +56,7 @@ def get_os_release(path):
name=data.get('NAME', '').strip('"'),
pretty_name=data.get('PRETTY_NAME', '').strip('"'),
version=data.get('VERSION', '').strip('"'),
- version_id=data.get('VERSION_ID', '').strip('"'),
+ version_id=os_version,
variant=data.get('VARIANT', '').strip('"') or None,
variant_id=data.get('VARIANT_ID', '').strip('"') or None
)
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/actor.py b/repos/system_upgrade/common/actors/peseventsscanner/actor.py
index fadf76b..7ef2664 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/actor.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/actor.py
@@ -1,3 +1,6 @@
+import os
+import os.path
+
from leapp.actors import Actor
from leapp.libraries.actor.peseventsscanner import pes_events_scanner
from leapp.models import (
@@ -9,11 +12,15 @@ from leapp.models import (
RepositoriesMapping,
RepositoriesSetupTasks,
RHUIInfo,
- RpmTransactionTasks
+ RpmTransactionTasks,
+ ActiveVendorList,
)
from leapp.reporting import Report
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+LEAPP_FILES_DIR = "/etc/leapp/files"
+VENDORS_DIR = "/etc/leapp/files/vendors.d"
+
class PesEventsScanner(Actor):
"""
@@ -32,9 +39,22 @@ class PesEventsScanner(Actor):
RepositoriesMapping,
RHUIInfo,
RpmTransactionTasks,
+ ActiveVendorList,
)
produces = (PESRpmTransactionTasks, RepositoriesSetupTasks, Report)
tags = (IPUWorkflowTag, FactsPhaseTag)
def process(self):
- pes_events_scanner('/etc/leapp/files', 'pes-events.json')
+ pes_events_scanner(LEAPP_FILES_DIR, "pes-events.json")
+
+ active_vendors = []
+ for vendor_list in self.consume(ActiveVendorList):
+ active_vendors.extend(vendor_list.data)
+
+ pes_json_suffix = "_pes.json"
+ if os.path.isdir(VENDORS_DIR):
+ vendor_pesfiles = list(filter(lambda vfile: pes_json_suffix in vfile, os.listdir(VENDORS_DIR)))
+
+ for pesfile in vendor_pesfiles:
+ if pesfile[:-len(pes_json_suffix)] in active_vendors:
+ pes_events_scanner(VENDORS_DIR, pesfile)
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner.py b/repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner.py
index 1be2caa..072de17 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/libraries/peseventsscanner.py
@@ -138,19 +138,26 @@ def _get_repositories_mapping(target_pesids):
:return: Dictionary with all repositories mapped.
"""
- repositories_map_msgs = api.consume(RepositoriesMapping)
- repositories_map_msg = next(repositories_map_msgs, None)
- if list(repositories_map_msgs):
- api.current_logger().warning('Unexpectedly received more than one RepositoriesMapping message.')
- if not repositories_map_msg:
- raise StopActorExecutionError(
- 'Cannot parse RepositoriesMapping data properly',
- details={'Problem': 'Did not receive a message with mapped repositories'}
- )
+ composite_mapping = []
+ composite_repos = []
+
+ for repomap_msg in api.consume(RepositoriesMapping):
+ if not repomap_msg:
+ raise StopActorExecutionError(
+ 'Cannot parse RepositoriesMapping data properly',
+ details={'Problem': 'Received a blank message with mapped repositories'}
+ )
+ composite_mapping.extend(repomap_msg.mapping)
+ composite_repos.extend(repomap_msg.repositories)
+
+ composite_map_msg = RepositoriesMapping(
+ mapping=composite_mapping,
+ repositories=composite_repos
+ )
rhui_info = next(api.consume(RHUIInfo), RHUIInfo(provider=''))
- repomap = peseventsscanner_repomap.RepoMapDataHandler(repositories_map_msg, cloud_provider=rhui_info.provider)
+ repomap = peseventsscanner_repomap.RepoMapDataHandler(composite_map_msg, cloud_provider=rhui_info.provider)
# NOTE: We have to calculate expected target repositories
# like in the setuptargetrepos actor. It's planned to handle this in different
# way in future...
@@ -324,7 +331,7 @@ def parse_pes_events(json_data):
:return: List of Event tuples, where each event contains event type and input/output pkgs
"""
data = json.loads(json_data)
- if not isinstance(data, dict) or not data.get('packageinfo'):
+ if not isinstance(data, dict) or data.get('packageinfo') is None:
raise ValueError('Found PES data with invalid structure')
return list(chain(*[parse_entry(entry) for entry in data['packageinfo']]))
diff --git a/repos/system_upgrade/common/actors/peseventsscanner/tests/unit_test_peseventsscanner.py b/repos/system_upgrade/common/actors/peseventsscanner/tests/unit_test_peseventsscanner.py
index f4b02e9..c22165e 100644
--- a/repos/system_upgrade/common/actors/peseventsscanner/tests/unit_test_peseventsscanner.py
+++ b/repos/system_upgrade/common/actors/peseventsscanner/tests/unit_test_peseventsscanner.py
@@ -492,6 +492,10 @@ def test_get_events(monkeypatch):
assert reporting.create_report.called == 1
assert 'inhibitor' in reporting.create_report.report_fields['flags']
+ with open(os.path.join(CUR_DIR, 'files/sample04.json')) as f:
+ events = parse_pes_events(f.read())
+ assert len(events) == 0
+
def test_pes_data_not_found(monkeypatch):
def read_or_fetch_mocked(filename, directory="/etc/leapp/files", service=None, allow_empty=False):
diff --git a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py b/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
index 01f6df3..0bb0726 100644
--- a/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
+++ b/repos/system_upgrade/common/actors/redhatsignedrpmscanner/actor.py
@@ -1,27 +1,65 @@
from leapp.actors import Actor
from leapp.libraries.common import rhui
-from leapp.models import InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM
+from leapp.models import InstalledRedHatSignedRPM, InstalledRPM, InstalledUnsignedRPM, VendorSignatures
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
-class RedHatSignedRpmScanner(Actor):
+VENDOR_SIGS = {
+ 'rhel': ['199e2f91fd431d51',
+ '5326810137017186',
+ '938a80caf21541eb',
+ 'fd372689897da07a',
+ '45689c882fa658e0'],
+ 'centos': ['24c6a8a7f4a80eb5',
+ '05b555b38483c65d',
+ '4eb84e71f2ee9d55'],
+ 'cloudlinux': ['8c55a6628608cb71'],
+ 'almalinux': ['51d6647ec21ad6ea',
+ 'd36cb86cb86b3716'],
+ 'rocky': ['15af5dac6d745a60',
+ '702d426d350d275d'],
+ 'ol': ['72f97b74ec551f03',
+ '82562ea9ad986da3',
+ 'bc4d06a08d8b756f'],
+ 'eurolinux': ['75c333f418cd4a9e',
+ 'b413acad6275f250',
+ 'f7ad3e5a1c9fd080']
+}
+
+VENDOR_PACKAGERS = {
+ "rhel": "Red Hat, Inc.",
+ "centos": "CentOS",
+ "cloudlinux": "CloudLinux Packaging Team",
+ "almalinux": "AlmaLinux Packaging Team",
+ "rocky": "infrastructure@rockylinux.org",
+ "eurolinux": "EuroLinux",
+}
+
+
+class VendorSignedRpmScanner(Actor):
"""Provide data about installed RPM Packages signed by Red Hat.
After filtering the list of installed RPM packages by signature, a message
with relevant data will be produced.
"""
- name = 'red_hat_signed_rpm_scanner'
- consumes = (InstalledRPM,)
- produces = (InstalledRedHatSignedRPM, InstalledUnsignedRPM,)
+ name = "vendor_signed_rpm_scanner"
+ consumes = (InstalledRPM, VendorSignatures)
+ produces = (
+ InstalledRedHatSignedRPM,
+ InstalledUnsignedRPM,
+ )
tags = (IPUWorkflowTag, FactsPhaseTag)
def process(self):
- RH_SIGS = ['199e2f91fd431d51',
- '5326810137017186',
- '938a80caf21541eb',
- 'fd372689897da07a',
- '45689c882fa658e0']
+ vendor = self.configuration.os_release.release_id
+ vendor_keys = sum(VENDOR_SIGS.values(), [])
+ vendor_packager = VENDOR_PACKAGERS.get(vendor, "not-available")
+
+ for siglist in self.consume(VendorSignatures):
+ vendor_keys.extend(siglist.sigs)
+
+ self.log.debug("Signature list: {}".format(vendor_keys))
signed_pkgs = InstalledRedHatSignedRPM()
unsigned_pkgs = InstalledUnsignedRPM()
@@ -32,11 +70,11 @@ class RedHatSignedRpmScanner(Actor):
all_signed = [
env
for env in env_vars
- if env.name == 'LEAPP_DEVEL_RPMS_ALL_SIGNED' and env.value == '1'
+ if env.name == "LEAPP_DEVEL_RPMS_ALL_SIGNED" and env.value == "1"
]
- def has_rhsig(pkg):
- return any(key in pkg.pgpsig for key in RH_SIGS)
+ def has_vendorsig(pkg):
+ return any(key in pkg.pgpsig for key in vendor_keys)
def is_gpg_pubkey(pkg):
"""Check if gpg-pubkey pkg exists or LEAPP_DEVEL_RPMS_ALL_SIGNED=1
@@ -44,15 +82,15 @@ class RedHatSignedRpmScanner(Actor):
gpg-pubkey is not signed as it would require another package
to verify its signature
"""
- return ( # pylint: disable-msg=consider-using-ternary
- pkg.name == 'gpg-pubkey'
- and pkg.packager.startswith('Red Hat, Inc.')
- or all_signed
+ return ( # pylint: disable-msg=consider-using-ternary
+ pkg.name == "gpg-pubkey"
+ and (pkg.packager.startswith(vendor_packager))
+ or all_signed
)
def has_katello_prefix(pkg):
"""Whitelist the katello package."""
- return pkg.name.startswith('katello-ca-consumer')
+ return pkg.name.startswith("katello-ca-consumer")
def is_azure_pkg(pkg):
"""Whitelist Azure config package."""
@@ -68,16 +106,24 @@ class RedHatSignedRpmScanner(Actor):
for pkg in rpm_pkgs.items:
if any(
[
- has_rhsig(pkg),
+ has_vendorsig(pkg),
is_gpg_pubkey(pkg),
has_katello_prefix(pkg),
is_azure_pkg(pkg),
]
):
signed_pkgs.items.append(pkg)
+ self.log.debug(
+ "Package {} is signed, packager: {}, signature: {}".format(
+ pkg.name, pkg.packager, pkg.pgpsig
+ )
+ )
continue
unsigned_pkgs.items.append(pkg)
+ self.log.debug(
+ "Package {} is unsigned, packager: {}, signature: {}".format(pkg.name, pkg.packager, pkg.pgpsig)
+ )
self.produce(signed_pkgs)
self.produce(unsigned_pkgs)
diff --git a/repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py b/repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py
index b2d00f3..e9458c5 100644
--- a/repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py
+++ b/repos/system_upgrade/common/actors/repositoriesmapping/libraries/repositoriesmapping.py
@@ -1,12 +1,9 @@
-from collections import defaultdict
-import json
import os
-from leapp.exceptions import StopActorExecutionError
from leapp.libraries.common.config.version import get_target_major_version, get_source_major_version
-from leapp.libraries.common.fetch import read_or_fetch
+from leapp.libraries.common.repomaputils import RepoMapData, read_repofile, inhibit_upgrade
from leapp.libraries.stdlib import api
-from leapp.models import RepositoriesMapping, PESIDRepositoryEntry, RepoMapEntry
+from leapp.models import RepositoriesMapping
from leapp.models.fields import ModelViolationError
OLD_REPOMAP_FILE = 'repomap.csv'
@@ -16,144 +13,9 @@ REPOMAP_FILE = 'repomap.json'
"""The name of the new repository mapping file."""
-class RepoMapData(object):
- VERSION_FORMAT = '1.0.0'
-
- def __init__(self):
- self.repositories = []
- self.mapping = {}
-
- def add_repository(self, data, pesid):
- """
- Add new PESIDRepositoryEntry with given pesid from the provided dictionary.
-
- :param data: A dict containing the data of the added repository. The dictionary structure corresponds
- to the repositories entries in the repository mapping JSON schema.
- :type data: Dict[str, str]
- :param pesid: PES id of the repository family that the newly added repository belongs to.
- :type pesid: str
- """
- self.repositories.append(PESIDRepositoryEntry(
- repoid=data['repoid'],
- channel=data['channel'],
- rhui=data.get('rhui', ''),
- repo_type=data['repo_type'],
- arch=data['arch'],
- major_version=data['major_version'],
- pesid=pesid
- ))
-
- def get_repositories(self, valid_major_versions):
- """
- Return the list of PESIDRepositoryEntry object matching the specified major versions.
- """
- return [repo for repo in self.repositories if repo.major_version in valid_major_versions]
-
- def add_mapping(self, source_major_version, target_major_version, source_pesid, target_pesid):
- """
- Add a new mapping entry that is mapping the source pesid to the destination pesid(s),
- relevant in an IPU from the supplied source major version to the supplied target
- major version.
-
- :param str source_major_version: Specifies the major version of the source system
- for which the added mapping applies.
- :param str target_major_version: Specifies the major version of the target system
- for which the added mapping applies.
- :param str source_pesid: PESID of the source repository.
- :param Union[str|List[str]] target_pesid: A single target PESID or a list of target
- PESIDs of the added mapping.
- """
- # NOTE: it could be more simple, but I prefer to be sure the input data
- # contains just one map per source PESID.
- key = '{}:{}'.format(source_major_version, target_major_version)
- rmap = self.mapping.get(key, defaultdict(set))
- self.mapping[key] = rmap
- if isinstance(target_pesid, list):
- rmap[source_pesid].update(target_pesid)
- else:
- rmap[source_pesid].add(target_pesid)
-
- def get_mappings(self, src_major_version, dst_major_version):
- """
- Return the list of RepoMapEntry objects for the specified upgrade path.
-
- IOW, the whole mapping for specified IPU.
- """
- key = '{}:{}'.format(src_major_version, dst_major_version)
- rmap = self.mapping.get(key, None)
- if not rmap:
- return None
- map_list = []
- for src_pesid in sorted(rmap.keys()):
- map_list.append(RepoMapEntry(source=src_pesid, target=sorted(rmap[src_pesid])))
- return map_list
-
- @staticmethod
- def load_from_dict(data):
- if data['version_format'] != RepoMapData.VERSION_FORMAT:
- raise ValueError(
- 'The obtained repomap data has unsupported version of format.'
- ' Get {} required {}'
- .format(data['version_format'], RepoMapData.VERSION_FORMAT)
- )
-
- repomap = RepoMapData()
-
- # Load reposiories
- existing_pesids = set()
- for repo_family in data['repositories']:
- existing_pesids.add(repo_family['pesid'])
- for repo in repo_family['entries']:
- repomap.add_repository(repo, repo_family['pesid'])
-
- # Load mappings
- for mapping in data['mapping']:
- for entry in mapping['entries']:
- if not isinstance(entry['target'], list):
- raise ValueError(
- 'The target field of a mapping entry is not a list: {}'
- .format(entry)
- )
-
- for pesid in [entry['source']] + entry['target']:
- if pesid not in existing_pesids:
- raise ValueError(
- 'The {} pesid is not related to any repository.'
- .format(pesid)
- )
- repomap.add_mapping(
- source_major_version=mapping['source_major_version'],
- target_major_version=mapping['target_major_version'],
- source_pesid=entry['source'],
- target_pesid=entry['target'],
- )
- return repomap
-
-
-def _inhibit_upgrade(msg):
- raise StopActorExecutionError(
- msg,
- details={'hint': ('Read documentation at the following link for more'
- ' information about how to retrieve the valid file:'
- ' https://access.redhat.com/articles/3664871')})
-
-
-def _read_repofile(repofile):
- # NOTE: what about catch StopActorExecution error when the file cannot be
- # obtained -> then check whether old_repomap file exists and in such a case
- # inform user they have to provde the new repomap.json file (we have the
- # warning now only which could be potentially overlooked)
- try:
- return json.loads(read_or_fetch(repofile))
- except ValueError:
- # The data does not contain a valid json
- _inhibit_upgrade('The repository mapping file is invalid: file does not contain a valid JSON object.')
- return None # Avoids inconsistent-return-statements warning
-
-
-def scan_repositories(read_repofile_func=_read_repofile):
+def scan_repositories(read_repofile_func=read_repofile):
"""
- Scan the repository mapping file and produce RepositoriesMap msg.
+ Scan the repository mapping file and produce RepositoriesMapping msg.
See the description of the actor for more details.
"""
@@ -185,10 +47,10 @@ def scan_repositories(read_repofile_func=_read_repofile):
'the JSON does not match required schema (wrong field type/value): {}'
.format(err)
)
- _inhibit_upgrade(err_message)
+ inhibit_upgrade(err_message)
except KeyError as err:
- _inhibit_upgrade(
+ inhibit_upgrade(
'The repository mapping file is invalid: the JSON is missing a required field: {}'.format(err))
except ValueError as err:
# The error should contain enough information, so we do not need to clarify it further
- _inhibit_upgrade('The repository mapping file is invalid: {}'.format(err))
+ inhibit_upgrade('The repository mapping file is invalid: {}'.format(err))
diff --git a/repos/system_upgrade/common/actors/repositoriesmapping/tests/unit_test_repositoriesmapping.py b/repos/system_upgrade/common/actors/repositoriesmapping/tests/unit_test_repositoriesmapping.py
index 3c0b04b..3480432 100644
--- a/repos/system_upgrade/common/actors/repositoriesmapping/tests/unit_test_repositoriesmapping.py
+++ b/repos/system_upgrade/common/actors/repositoriesmapping/tests/unit_test_repositoriesmapping.py
@@ -15,7 +15,6 @@ from leapp.models import PESIDRepositoryEntry
CUR_DIR = os.path.dirname(os.path.abspath(__file__))
-
@pytest.fixture
def adjust_cwd():
previous_cwd = os.getcwd()
diff --git a/repos/system_upgrade/common/actors/scancustomrepofile/actor.py b/repos/system_upgrade/common/actors/scancustomrepofile/actor.py
index d46018f..bb49b4e 100644
--- a/repos/system_upgrade/common/actors/scancustomrepofile/actor.py
+++ b/repos/system_upgrade/common/actors/scancustomrepofile/actor.py
@@ -1,6 +1,9 @@
from leapp.actors import Actor
from leapp.libraries.actor import scancustomrepofile
-from leapp.models import CustomTargetRepository, CustomTargetRepositoryFile
+from leapp.models import (
+ CustomTargetRepository,
+ CustomTargetRepositoryFile,
+)
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
@@ -18,7 +21,7 @@ class ScanCustomRepofile(Actor):
If the file doesn't exist, nothing happens.
"""
- name = 'scan_custom_repofile'
+ name = "scan_custom_repofile"
consumes = ()
produces = (CustomTargetRepository, CustomTargetRepositoryFile)
tags = (FactsPhaseTag, IPUWorkflowTag)
diff --git a/repos/system_upgrade/common/actors/scancustomrepofile/libraries/scancustomrepofile.py b/repos/system_upgrade/common/actors/scancustomrepofile/libraries/scancustomrepofile.py
index c294193..c0820eb 100644
--- a/repos/system_upgrade/common/actors/scancustomrepofile/libraries/scancustomrepofile.py
+++ b/repos/system_upgrade/common/actors/scancustomrepofile/libraries/scancustomrepofile.py
@@ -18,18 +18,27 @@ def process():
"""
if not os.path.isfile(CUSTOM_REPO_PATH):
api.current_logger().debug(
- "The {} file doesn't exist. Nothing to do."
- .format(CUSTOM_REPO_PATH))
+ "The {} file doesn't exist. Nothing to do.".format(CUSTOM_REPO_PATH)
+ )
return
- api.current_logger().info("The {} file exists.".format(CUSTOM_REPO_PATH))
+
repofile = repofileutils.parse_repofile(CUSTOM_REPO_PATH)
if not repofile.data:
+ api.current_logger().info(
+ "The {} file exists, but is empty. Nothing to do.".format(CUSTOM_REPO_PATH)
+ )
return
api.produce(CustomTargetRepositoryFile(file=CUSTOM_REPO_PATH))
+
for repo in repofile.data:
- api.produce(CustomTargetRepository(
- repoid=repo.repoid,
- name=repo.name,
- baseurl=repo.baseurl,
- enabled=repo.enabled,
- ))
+ api.produce(
+ CustomTargetRepository(
+ repoid=repo.repoid,
+ name=repo.name,
+ baseurl=repo.baseurl,
+ enabled=repo.enabled,
+ )
+ )
+ api.current_logger().info(
+ "The {} file exists, custom repositories loaded.".format(CUSTOM_REPO_PATH)
+ )
diff --git a/repos/system_upgrade/common/actors/scancustomrepofile/tests/test_scancustomrepofile.py b/repos/system_upgrade/common/actors/scancustomrepofile/tests/test_scancustomrepofile.py
index 4ea33a2..aaec273 100644
--- a/repos/system_upgrade/common/actors/scancustomrepofile/tests/test_scancustomrepofile.py
+++ b/repos/system_upgrade/common/actors/scancustomrepofile/tests/test_scancustomrepofile.py
@@ -4,8 +4,13 @@ from leapp.libraries.actor import scancustomrepofile
from leapp.libraries.common import repofileutils
from leapp.libraries.common.testutils import produce_mocked
from leapp.libraries.stdlib import api
-from leapp.models import CustomTargetRepository, CustomTargetRepositoryFile, RepositoryData, RepositoryFile
+from leapp.models import (
+ CustomTargetRepository,
+ CustomTargetRepositoryFile,
+ RepositoryData,
+ RepositoryFile,
+)
_REPODATA = [
RepositoryData(repoid="repo1", name="repo1name", baseurl="repo1url", enabled=True),
@@ -57,7 +62,7 @@ def test_valid_repofile_exists(monkeypatch):
monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
monkeypatch.setattr(api, 'current_logger', LoggerMocked())
scancustomrepofile.process()
- msg = "The {} file exists.".format(scancustomrepofile.CUSTOM_REPO_PATH)
+ msg = "The {} file exists, custom repositories loaded.".format(scancustomrepofile.CUSTOM_REPO_PATH)
assert api.current_logger.infomsg == msg
assert api.produce.called == len(_CUSTOM_REPOS) + 1
assert _CUSTOM_REPO_FILE_MSG in api.produce.model_instances
@@ -73,6 +78,6 @@ def test_empty_repofile_exists(monkeypatch):
monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
monkeypatch.setattr(api, 'current_logger', LoggerMocked())
scancustomrepofile.process()
- msg = "The {} file exists.".format(scancustomrepofile.CUSTOM_REPO_PATH)
+ msg = "The {} file exists, but is empty. Nothing to do.".format(scancustomrepofile.CUSTOM_REPO_PATH)
assert api.current_logger.infomsg == msg
assert not api.produce.called
diff --git a/repos/system_upgrade/common/actors/scanvendorrepofiles/actor.py b/repos/system_upgrade/common/actors/scanvendorrepofiles/actor.py
new file mode 100644
index 0000000..6c7f3a3
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scanvendorrepofiles/actor.py
@@ -0,0 +1,20 @@
+from leapp.actors import Actor
+from leapp.libraries.actor import scanvendorrepofiles
+from leapp.models import CustomTargetRepository, CustomTargetRepositoryFile, ActiveVendorList
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+from leapp.libraries.stdlib import api
+
+
+class ScanVendorRepofiles(Actor):
+ """
+ Load and produce custom repository data from vendor-provided files.
+ Only those vendors whose source system repoids were found on the system will be included.
+ """
+
+ name = "scan_vendor_repofiles"
+ consumes = (ActiveVendorList)
+ produces = (CustomTargetRepository, CustomTargetRepositoryFile)
+ tags = (FactsPhaseTag, IPUWorkflowTag)
+
+ def process(self):
+ scanvendorrepofiles.process()
diff --git a/repos/system_upgrade/common/actors/scanvendorrepofiles/libraries/scanvendorrepofiles.py b/repos/system_upgrade/common/actors/scanvendorrepofiles/libraries/scanvendorrepofiles.py
new file mode 100644
index 0000000..5a65f5a
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scanvendorrepofiles/libraries/scanvendorrepofiles.py
@@ -0,0 +1,68 @@
+import os
+
+from leapp.libraries.common import repofileutils
+from leapp.libraries.stdlib import api
+from leapp.models import CustomTargetRepository, CustomTargetRepositoryFile, ActiveVendorList
+
+
+VENDORS_DIR = "/etc/leapp/files/vendors.d/"
+REPOFILE_SUFFIX = ".repo"
+
+
+def process():
+ """
+ Produce CustomTargetRepository msgs for the vendor repo files inside the
+ <CUSTOM_REPO_DIR>.
+
+ The CustomTargetRepository messages are produced only if a "from" vendor repository
+ listed indide its map matched one of the repositories active on the system.
+ """
+ if not os.path.isdir(VENDORS_DIR):
+ api.current_logger().debug(
+ "The {} directory doesn't exist. Nothing to do.".format(VENDORS_DIR)
+ )
+ return
+
+ for reponame in os.listdir(VENDORS_DIR):
+ if not reponame.endswith(REPOFILE_SUFFIX):
+ continue
+ # Cut the .repo part to get only the name.
+ vendor_name = reponame[:-5]
+
+ active_vendors = []
+ for vendor_list in api.consume(ActiveVendorList):
+ active_vendors.extend(vendor_list.data)
+
+ api.current_logger().debug(
+ "Active vendor list: {}".format(active_vendors)
+ )
+
+ if vendor_name not in active_vendors:
+ api.current_logger().debug(
+ "Vendor {} not in active list, skipping".format(vendor_name)
+ )
+ continue
+
+ api.current_logger().debug(
+ "Vendor {} found in active list, processing file".format(vendor_name)
+ )
+ full_repo_path = os.path.join(VENDORS_DIR, reponame)
+ repofile = repofileutils.parse_repofile(full_repo_path)
+
+ api.produce(CustomTargetRepositoryFile(file=full_repo_path))
+ for repo in repofile.data:
+ api.current_logger().debug(
+ "Loaded repository {} from file {}".format(repo.repoid, reponame)
+ )
+ api.produce(
+ CustomTargetRepository(
+ repoid=repo.repoid,
+ name=repo.name,
+ baseurl=repo.baseurl,
+ enabled=repo.enabled,
+ )
+ )
+
+ api.current_logger().info(
+ "The {} directory exists, vendor repositories loaded.".format(VENDORS_DIR)
+ )
diff --git a/repos/system_upgrade/common/actors/scanvendorrepofiles/tests/test_scanvendorrepofiles.py b/repos/system_upgrade/common/actors/scanvendorrepofiles/tests/test_scanvendorrepofiles.py
new file mode 100644
index 0000000..cb5c7ab
--- /dev/null
+++ b/repos/system_upgrade/common/actors/scanvendorrepofiles/tests/test_scanvendorrepofiles.py
@@ -0,0 +1,131 @@
+import os
+
+from leapp.libraries.actor import scancustomrepofile
+from leapp.libraries.common import repofileutils
+from leapp.libraries.common.testutils import produce_mocked
+from leapp.libraries.stdlib import api
+
+from leapp.models import (CustomTargetRepository, CustomTargetRepositoryFile,
+ RepositoryData, RepositoryFile)
+
+
+_REPODATA = [
+ RepositoryData(repoid="repo1", name="repo1name", baseurl="repo1url", enabled=True),
+ RepositoryData(repoid="repo2", name="repo2name", baseurl="repo2url", enabled=False),
+ RepositoryData(repoid="repo3", name="repo3name", enabled=True),
+ RepositoryData(repoid="repo4", name="repo4name", mirrorlist="mirror4list", enabled=True),
+]
+
+_CUSTOM_REPOS = [
+ CustomTargetRepository(repoid="repo1", name="repo1name", baseurl="repo1url", enabled=True),
+ CustomTargetRepository(repoid="repo2", name="repo2name", baseurl="repo2url", enabled=False),
+ CustomTargetRepository(repoid="repo3", name="repo3name", baseurl=None, enabled=True),
+ CustomTargetRepository(repoid="repo4", name="repo4name", baseurl=None, enabled=True),
+]
+
+_CUSTOM_REPO_FILE_MSG = CustomTargetRepositoryFile(file=scancustomrepofile.CUSTOM_REPO_PATH)
+
+
+_TESTING_REPODATA = [
+ RepositoryData(repoid="repo1-stable", name="repo1name", baseurl="repo1url", enabled=True),
+ RepositoryData(repoid="repo2-testing", name="repo2name", baseurl="repo2url", enabled=False),
+ RepositoryData(repoid="repo3-stable", name="repo3name", enabled=False),
+ RepositoryData(repoid="repo4-testing", name="repo4name", mirrorlist="mirror4list", enabled=True),
+]
+
+_TESTING_CUSTOM_REPOS_STABLE_TARGET = [
+ CustomTargetRepository(repoid="repo1-stable", name="repo1name", baseurl="repo1url", enabled=True),
+ CustomTargetRepository(repoid="repo2-testing", name="repo2name", baseurl="repo2url", enabled=False),
+ CustomTargetRepository(repoid="repo3-stable", name="repo3name", baseurl=None, enabled=False),
+ CustomTargetRepository(repoid="repo4-testing", name="repo4name", baseurl=None, enabled=True),
+]
+
+_TESTING_CUSTOM_REPOS_BETA_TARGET = [
+ CustomTargetRepository(repoid="repo1-stable", name="repo1name", baseurl="repo1url", enabled=True),
+ CustomTargetRepository(repoid="repo2-testing", name="repo2name", baseurl="repo2url", enabled=True),
+ CustomTargetRepository(repoid="repo3-stable", name="repo3name", baseurl=None, enabled=False),
+ CustomTargetRepository(repoid="repo4-testing", name="repo4name", baseurl=None, enabled=True),
+]
+
+_PROCESS_STABLE_TARGET = "stable"
+_PROCESS_BETA_TARGET = "beta"
+
+
+class LoggerMocked(object):
+ def __init__(self):
+ self.infomsg = None
+ self.debugmsg = None
+
+ def info(self, msg):
+ self.infomsg = msg
+
+ def debug(self, msg):
+ self.debugmsg = msg
+
+ def __call__(self):
+ return self
+
+
+def test_no_repofile(monkeypatch):
+ monkeypatch.setattr(os.path, 'isfile', lambda dummy: False)
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(api, 'current_logger', LoggerMocked())
+ scancustomrepofile.process()
+ msg = "The {} file doesn't exist. Nothing to do.".format(scancustomrepofile.CUSTOM_REPO_PATH)
+ assert api.current_logger.debugmsg == msg
+ assert not api.produce.called
+
+
+def test_valid_repofile_exists(monkeypatch):
+ def _mocked_parse_repofile(fpath):
+ return RepositoryFile(file=fpath, data=_REPODATA)
+ monkeypatch.setattr(os.path, 'isfile', lambda dummy: True)
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
+ monkeypatch.setattr(api, 'current_logger', LoggerMocked())
+ scancustomrepofile.process()
+ msg = "The {} file exists, custom repositories loaded.".format(scancustomrepofile.CUSTOM_REPO_PATH)
+ assert api.current_logger.infomsg == msg
+ assert api.produce.called == len(_CUSTOM_REPOS) + 1
+ assert _CUSTOM_REPO_FILE_MSG in api.produce.model_instances
+ for crepo in _CUSTOM_REPOS:
+ assert crepo in api.produce.model_instances
+
+
+def test_target_stable_repos(monkeypatch):
+ def _mocked_parse_repofile(fpath):
+ return RepositoryFile(file=fpath, data=_TESTING_REPODATA)
+ monkeypatch.setattr(os.path, 'isfile', lambda dummy: True)
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
+
+ scancustomrepofile.process(_PROCESS_STABLE_TARGET)
+ assert api.produce.called == len(_TESTING_CUSTOM_REPOS_STABLE_TARGET) + 1
+ for crepo in _TESTING_CUSTOM_REPOS_STABLE_TARGET:
+ assert crepo in api.produce.model_instances
+
+
+def test_target_beta_repos(monkeypatch):
+ def _mocked_parse_repofile(fpath):
+ return RepositoryFile(file=fpath, data=_TESTING_REPODATA)
+ monkeypatch.setattr(os.path, 'isfile', lambda dummy: True)
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
+
+ scancustomrepofile.process(_PROCESS_BETA_TARGET)
+ assert api.produce.called == len(_TESTING_CUSTOM_REPOS_BETA_TARGET) + 1
+ for crepo in _TESTING_CUSTOM_REPOS_BETA_TARGET:
+ assert crepo in api.produce.model_instances
+
+
+def test_empty_repofile_exists(monkeypatch):
+ def _mocked_parse_repofile(fpath):
+ return RepositoryFile(file=fpath, data=[])
+ monkeypatch.setattr(os.path, 'isfile', lambda dummy: True)
+ monkeypatch.setattr(api, 'produce', produce_mocked())
+ monkeypatch.setattr(repofileutils, 'parse_repofile', _mocked_parse_repofile)
+ monkeypatch.setattr(api, 'current_logger', LoggerMocked())
+ scancustomrepofile.process()
+ msg = "The {} file exists, but is empty. Nothing to do.".format(scancustomrepofile.CUSTOM_REPO_PATH)
+ assert api.current_logger.infomsg == msg
+ assert not api.produce.called
diff --git a/repos/system_upgrade/common/actors/setuptargetrepos/actor.py b/repos/system_upgrade/common/actors/setuptargetrepos/actor.py
index 00de073..fb86639 100644
--- a/repos/system_upgrade/common/actors/setuptargetrepos/actor.py
+++ b/repos/system_upgrade/common/actors/setuptargetrepos/actor.py
@@ -12,6 +12,7 @@ from leapp.models import (
UsedRepositories
)
from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+from leapp.libraries.stdlib import api
class SetupTargetRepos(Actor):
diff --git a/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py b/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
index 3f34aed..2992037 100644
--- a/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
+++ b/repos/system_upgrade/common/actors/setuptargetrepos/libraries/setuptargetrepos.py
@@ -59,9 +59,20 @@ def _get_used_repo_dict():
def _setup_repomap_handler(src_repoids):
- repo_mappig_msg = next(api.consume(RepositoriesMapping), RepositoriesMapping())
+ combined_mapping = []
+ combined_repositories = []
+ # Depending on whether there are any vendors present, we might get more than one message.
+ for msg in api.consume(RepositoriesMapping):
+ combined_mapping.extend(msg.mapping)
+ combined_repositories.extend(msg.repositories)
+
+ combined_repomapping = RepositoriesMapping(
+ mapping=combined_mapping,
+ repositories=combined_repositories
+ )
+
rhui_info = next(api.consume(RHUIInfo), RHUIInfo(provider=''))
- repomap = setuptargetrepos_repomap.RepoMapDataHandler(repo_mappig_msg, cloud_provider=rhui_info.provider)
+ repomap = setuptargetrepos_repomap.RepoMapDataHandler(combined_repomapping, cloud_provider=rhui_info.provider)
# TODO(pstodulk): what about skip this completely and keep the default 'ga'..?
default_channels = setuptargetrepos_repomap.get_default_repository_channels(repomap, src_repoids)
repomap.set_default_channels(default_channels)
diff --git a/repos/system_upgrade/common/actors/systemfacts/actor.py b/repos/system_upgrade/common/actors/systemfacts/actor.py
index 59b12c8..85d4a09 100644
--- a/repos/system_upgrade/common/actors/systemfacts/actor.py
+++ b/repos/system_upgrade/common/actors/systemfacts/actor.py
@@ -47,7 +47,7 @@ class SystemFactsActor(Actor):
GrubCfgBios,
Report
)
- tags = (IPUWorkflowTag, FactsPhaseTag,)
+ tags = (IPUWorkflowTag, FactsPhaseTag.Before,)
def process(self):
self.produce(systemfacts.get_sysctls_status())
diff --git a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
index 7a8bd99..f59c909 100644
--- a/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
+++ b/repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py
@@ -592,6 +592,7 @@ def _install_custom_repofiles(context, custom_repofiles):
"""
for rfile in custom_repofiles:
_dst_path = os.path.join('/etc/yum.repos.d', os.path.basename(rfile.file))
+ api.current_logger().debug("Copying {} to {}".format(rfile.file, _dst_path))
context.copy_to(rfile.file, _dst_path)
diff --git a/repos/system_upgrade/common/actors/vendorreposignaturescanner/actor.py b/repos/system_upgrade/common/actors/vendorreposignaturescanner/actor.py
new file mode 100644
index 0000000..f74de27
--- /dev/null
+++ b/repos/system_upgrade/common/actors/vendorreposignaturescanner/actor.py
@@ -0,0 +1,70 @@
+import os
+
+from leapp.actors import Actor
+from leapp.models import VendorSignatures, ActiveVendorList
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+
+
+VENDORS_DIR = "/etc/leapp/files/vendors.d/"
+SIGFILE_SUFFIX = ".sigs"
+
+
+class VendorRepoSignatureScanner(Actor):
+ """
+ Produce VendorSignatures msgs for the vendor signature files inside the
+ <VENDORS_DIR>.
+
+ The messages are produced only if a "from" vendor repository
+ listed indide its map matched one of the repositories active on the system.
+ """
+
+ name = 'vendor_repo_signature_scanner'
+ consumes = (ActiveVendorList)
+ produces = (VendorSignatures)
+ tags = (IPUWorkflowTag, FactsPhaseTag.Before)
+
+ def process(self):
+ if not os.path.isdir(VENDORS_DIR):
+ self.log.debug(
+ "The {} directory doesn't exist. Nothing to do.".format(VENDORS_DIR)
+ )
+ return
+
+ for sigfile_name in os.listdir(VENDORS_DIR):
+ if not sigfile_name.endswith(SIGFILE_SUFFIX):
+ continue
+ # Cut the suffix part to get only the name.
+ vendor_name = sigfile_name[:-5]
+
+ active_vendors = []
+ for vendor_list in self.consume(ActiveVendorList):
+ active_vendors.extend(vendor_list.data)
+
+ self.log.debug(
+ "Active vendor list: {}".format(active_vendors)
+ )
+
+ if vendor_name not in active_vendors:
+ self.log.debug(
+ "Vendor {} not in active list, skipping".format(vendor_name)
+ )
+ continue
+
+ self.log.debug(
+ "Vendor {} found in active list, processing file".format(vendor_name)
+ )
+
+ full_sigfile_path = os.path.join(VENDORS_DIR, sigfile_name)
+ with open(full_sigfile_path) as f:
+ signatures = [line for line in f.read().splitlines() if line]
+
+ self.produce(
+ VendorSignatures(
+ vendor=vendor_name,
+ sigs=signatures,
+ )
+ )
+
+ self.log.info(
+ "The {} directory exists, vendor signatures loaded.".format(VENDORS_DIR)
+ )
diff --git a/repos/system_upgrade/common/actors/vendorrepositoriesmapping/actor.py b/repos/system_upgrade/common/actors/vendorrepositoriesmapping/actor.py
new file mode 100644
index 0000000..1325647
--- /dev/null
+++ b/repos/system_upgrade/common/actors/vendorrepositoriesmapping/actor.py
@@ -0,0 +1,19 @@
+from leapp.actors import Actor
+# from leapp.libraries.common.repomaputils import scan_vendor_repomaps, VENDOR_REPOMAP_DIR
+from leapp.libraries.actor.vendorrepositoriesmapping import scan_vendor_repomaps
+from leapp.models import VendorSourceRepos, RepositoriesMapping
+from leapp.tags import FactsPhaseTag, IPUWorkflowTag
+
+
+class VendorRepositoriesMapping(Actor):
+ """
+ Scan the vendor repository mapping files and provide the data to other actors.
+ """
+
+ name = "vendor_repositories_mapping"
+ consumes = ()
+ produces = (RepositoriesMapping, VendorSourceRepos,)
+ tags = (IPUWorkflowTag, FactsPhaseTag.Before)
+
+ def process(self):
+ scan_vendor_repomaps()
diff --git a/repos/system_upgrade/common/actors/vendorrepositoriesmapping/libraries/vendorrepositoriesmapping.py b/repos/system_upgrade/common/actors/vendorrepositoriesmapping/libraries/vendorrepositoriesmapping.py
new file mode 100644
index 0000000..204d0dc
--- /dev/null
+++ b/repos/system_upgrade/common/actors/vendorrepositoriesmapping/libraries/vendorrepositoriesmapping.py
@@ -0,0 +1,66 @@
+import os
+
+from leapp.libraries.common.config.version import get_target_major_version, get_source_major_version
+from leapp.libraries.common.repomaputils import RepoMapData, read_repofile, inhibit_upgrade
+from leapp.libraries.stdlib import api
+from leapp.models import VendorSourceRepos, RepositoriesMapping
+from leapp.models.fields import ModelViolationError
+
+
+VENDORS_DIR = "/etc/leapp/files/vendors.d"
+"""The folder containing the vendor repository mapping files."""
+
+
+def read_repomap_file(repomap_file, read_repofile_func, vendor_name):
+ json_data = read_repofile_func(repomap_file, VENDORS_DIR)
+ try:
+ repomap_data = RepoMapData.load_from_dict(json_data)
+
+ api.produce(VendorSourceRepos(
+ vendor=vendor_name,
+ source_repoids=repomap_data.get_version_repoids(get_source_major_version())
+ ))
+
+ mapping = repomap_data.get_mappings(get_source_major_version(), get_target_major_version())
+ valid_major_versions = [get_source_major_version(), get_target_major_version()]
+ api.produce(RepositoriesMapping(
+ mapping=mapping,
+ repositories=repomap_data.get_repositories(valid_major_versions)
+ ))
+ except ModelViolationError as err:
+ err_message = (
+ 'The repository mapping file is invalid: '
+ 'the JSON does not match required schema (wrong field type/value): {}'
+ .format(err)
+ )
+ inhibit_upgrade(err_message)
+ except KeyError as err:
+ inhibit_upgrade(
+ 'The repository mapping file is invalid: the JSON is missing a required field: {}'.format(err))
+ except ValueError as err:
+ # The error should contain enough information, so we do not need to clarify it further
+ inhibit_upgrade('The repository mapping file is invalid: {}'.format(err))
+
+
+def scan_vendor_repomaps(read_repofile_func=read_repofile):
+ """
+ Scan the repository mapping file and produce RepositoriesMapping msg.
+
+ See the description of the actor for more details.
+ """
+
+ map_json_suffix = "_map.json"
+ if os.path.isdir(VENDORS_DIR):
+ vendor_mapfiles = list(filter(lambda vfile: map_json_suffix in vfile, os.listdir(VENDORS_DIR)))
+
+ for mapfile in vendor_mapfiles:
+ read_repomap_file(mapfile, read_repofile_func, mapfile[:-len(map_json_suffix)])
+ else:
+ api.current_logger().debug(
+ "The {} directory doesn't exist. Nothing to do.".format(VENDORS_DIR)
+ )
+ # vendor_repomap_collection = scan_vendor_repomaps(VENDOR_REPOMAP_DIR)
+ # if vendor_repomap_collection:
+ # self.produce(vendor_repomap_collection)
+ # for repomap in vendor_repomap_collection.maps:
+ # self.produce(repomap)
diff --git a/repos/system_upgrade/common/libraries/config/version.py b/repos/system_upgrade/common/libraries/config/version.py
index 03f3cd4..efc25d7 100644
--- a/repos/system_upgrade/common/libraries/config/version.py
+++ b/repos/system_upgrade/common/libraries/config/version.py
@@ -13,8 +13,8 @@ OP_MAP = {
_SUPPORTED_VERSIONS = {
# Note: 'rhel-alt' is detected when on 'rhel' with kernel 4.x
- '7': {'rhel': ['7.9'], 'rhel-alt': ['7.6'], 'rhel-saphana': ['7.9']},
- '8': {'rhel': ['8.5', '8.6']},
+ '7': {'rhel': ['7.9'], 'rhel-alt': ['7.6'], 'rhel-saphana': ['7.9'], 'centos': ['7.9'], 'eurolinux': ['7.9'], 'ol': ['7.9'], 'cloudlinux': ['7.9']},
+ '8': {'rhel': ['8.5', '8.6'], 'centos': ['8.5'], 'almalinux': ['8.6'], 'eurolinux': ['8.6'], 'ol': ['8.6'], 'rocky': ['8.6']},
}
diff --git a/repos/system_upgrade/common/libraries/dnfplugin.py b/repos/system_upgrade/common/libraries/dnfplugin.py
index 4010e9f..fdb8e66 100644
--- a/repos/system_upgrade/common/libraries/dnfplugin.py
+++ b/repos/system_upgrade/common/libraries/dnfplugin.py
@@ -4,6 +4,8 @@ import json
import os
import shutil
+import six
+
from leapp.exceptions import StopActorExecutionError
from leapp.libraries.common import dnfconfig, guards, mounting, overlaygen, rhsm, utils
from leapp.libraries.common.config.version import get_target_major_version, get_target_version
@@ -213,10 +215,17 @@ def _transaction(context, stage, target_repoids, tasks, plugin_info, test=False,
message='Failed to execute dnf. Reason: {}'.format(str(e))
)
except CalledProcessError as e:
+ err_stdout = e.stdout
+ err_stderr = e.stderr
+ if six.PY2:
+ err_stdout = e.stdout.encode('utf-8', 'xmlcharrefreplace')
+ err_stderr = e.stderr.encode('utf-8', 'xmlcharrefreplace')
+
api.current_logger().error('DNF execution failed: ')
raise StopActorExecutionError(
message='DNF execution failed with non zero exit code.\nSTDOUT:\n{stdout}\nSTDERR:\n{stderr}'.format(
- stdout=e.stdout, stderr=e.stderr)
+ stdout=err_stdout, stderr=err_stderr
+ )
)
finally:
if stage == 'check':
diff --git a/repos/system_upgrade/common/libraries/fetch.py b/repos/system_upgrade/common/libraries/fetch.py
index 1c58148..37313b6 100644
--- a/repos/system_upgrade/common/libraries/fetch.py
+++ b/repos/system_upgrade/common/libraries/fetch.py
@@ -73,7 +73,7 @@ def read_or_fetch(filename, directory="/etc/leapp/files", service=None, allow_em
data = f.read()
if not allow_empty and not data:
_raise_error(local_path, "File {lp} exists but is empty".format(lp=local_path))
- logger.warning("File {lp} successfully read ({l} bytes)".format(lp=local_path, l=len(data)))
+ logger.debug("File {lp} successfully read ({l} bytes)".format(lp=local_path, l=len(data)))
return data
except EnvironmentError:
_raise_error(local_path, "File {lp} exists but couldn't be read".format(lp=local_path))
diff --git a/repos/system_upgrade/common/libraries/repomaputils.py b/repos/system_upgrade/common/libraries/repomaputils.py
new file mode 100644
index 0000000..5c41620
--- /dev/null
+++ b/repos/system_upgrade/common/libraries/repomaputils.py
@@ -0,0 +1,147 @@
+import json
+from collections import defaultdict
+
+from leapp.exceptions import StopActorExecutionError
+from leapp.libraries.common.fetch import read_or_fetch
+from leapp.models import PESIDRepositoryEntry, RepoMapEntry
+
+
+def inhibit_upgrade(msg):
+ raise StopActorExecutionError(
+ msg,
+ details={'hint': ('Read documentation at the following link for more'
+ ' information about how to retrieve the valid file:'
+ ' https://access.redhat.com/articles/3664871')})
+
+
+def read_repofile(repofile, directory="/etc/leapp/files"):
+ # NOTE: what about catch StopActorExecution error when the file cannot be
+ # obtained -> then check whether old_repomap file exists and in such a case
+ # inform user they have to provde the new repomap.json file (we have the
+ # warning now only which could be potentially overlooked)
+ try:
+ return json.loads(read_or_fetch(repofile, directory))
+ except ValueError:
+ # The data does not contain a valid json
+ inhibit_upgrade('The repository mapping file is invalid: file does not contain a valid JSON object.')
+ return None # Avoids inconsistent-return-statements warning
+
+
+class RepoMapData(object):
+ VERSION_FORMAT = '1.0.0'
+
+ def __init__(self):
+ self.repositories = []
+ self.mapping = {}
+
+ def add_repository(self, data, pesid):
+ """
+ Add new PESIDRepositoryEntry with given pesid from the provided dictionary.
+
+ :param data: A dict containing the data of the added repository. The dictionary structure corresponds
+ to the repositories entries in the repository mapping JSON schema.
+ :type data: Dict[str, str]
+ :param pesid: PES id of the repository family that the newly added repository belongs to.
+ :type pesid: str
+ """
+ self.repositories.append(PESIDRepositoryEntry(
+ repoid=data['repoid'],
+ channel=data['channel'],
+ rhui=data.get('rhui', ''),
+ repo_type=data['repo_type'],
+ arch=data['arch'],
+ major_version=data['major_version'],
+ pesid=pesid
+ ))
+
+ def get_repositories(self, valid_major_versions):
+ """
+ Return the list of PESIDRepositoryEntry object matching the specified major versions.
+ """
+ return [repo for repo in self.repositories if repo.major_version in valid_major_versions]
+
+ def get_version_repoids(self, major_version):
+ """
+ Return the list of repository ID strings for repositories matching the specified major version.
+ """
+ return [repo.repoid for repo in self.repositories if repo.major_version == major_version]
+
+ def add_mapping(self, source_major_version, target_major_version, source_pesid, target_pesid):
+ """
+ Add a new mapping entry that is mapping the source pesid to the destination pesid(s),
+ relevant in an IPU from the supplied source major version to the supplied target
+ major version.
+
+ :param str source_major_version: Specifies the major version of the source system
+ for which the added mapping applies.
+ :param str target_major_version: Specifies the major version of the target system
+ for which the added mapping applies.
+ :param str source_pesid: PESID of the source repository.
+ :param Union[str|List[str]] target_pesid: A single target PESID or a list of target
+ PESIDs of the added mapping.
+ """
+ # NOTE: it could be more simple, but I prefer to be sure the input data
+ # contains just one map per source PESID.
+ key = '{}:{}'.format(source_major_version, target_major_version)
+ rmap = self.mapping.get(key, defaultdict(set))
+ self.mapping[key] = rmap
+ if isinstance(target_pesid, list):
+ rmap[source_pesid].update(target_pesid)
+ else:
+ rmap[source_pesid].add(target_pesid)
+
+ def get_mappings(self, src_major_version, dst_major_version):
+ """
+ Return the list of RepoMapEntry objects for the specified upgrade path.
+
+ IOW, the whole mapping for specified IPU.
+ """
+ key = '{}:{}'.format(src_major_version, dst_major_version)
+ rmap = self.mapping.get(key, None)
+ if not rmap:
+ return None
+ map_list = []
+ for src_pesid in sorted(rmap.keys()):
+ map_list.append(RepoMapEntry(source=src_pesid, target=sorted(rmap[src_pesid])))
+ return map_list
+
+ @staticmethod
+ def load_from_dict(data):
+ if data['version_format'] != RepoMapData.VERSION_FORMAT:
+ raise ValueError(
+ 'The obtained repomap data has unsupported version of format.'
+ ' Get {} required {}'
+ .format(data['version_format'], RepoMapData.VERSION_FORMAT)
+ )
+
+ repomap = RepoMapData()
+
+ # Load reposiories
+ existing_pesids = set()
+ for repo_family in data['repositories']:
+ existing_pesids.add(repo_family['pesid'])
+ for repo in repo_family['entries']:
+ repomap.add_repository(repo, repo_family['pesid'])
+
+ # Load mappings
+ for mapping in data['mapping']:
+ for entry in mapping['entries']:
+ if not isinstance(entry['target'], list):
+ raise ValueError(
+ 'The target field of a mapping entry is not a list: {}'
+ .format(entry)
+ )
+
+ for pesid in [entry['source']] + entry['target']:
+ if pesid not in existing_pesids:
+ raise ValueError(
+ 'The {} pesid is not related to any repository.'
+ .format(pesid)
+ )
+ repomap.add_mapping(
+ source_major_version=mapping['source_major_version'],
+ target_major_version=mapping['target_major_version'],
+ source_pesid=entry['source'],
+ target_pesid=entry['target'],
+ )
+ return repomap
diff --git a/repos/system_upgrade/common/libraries/rhsm.py b/repos/system_upgrade/common/libraries/rhsm.py
index b7e4b21..dc038bf 100644
--- a/repos/system_upgrade/common/libraries/rhsm.py
+++ b/repos/system_upgrade/common/libraries/rhsm.py
@@ -92,7 +92,7 @@ def _handle_rhsm_exceptions(hint=None):
def skip_rhsm():
"""Check whether we should skip RHSM related code."""
- return get_env('LEAPP_NO_RHSM', '0') == '1'
+ return True
def with_rhsm(f):
diff --git a/repos/system_upgrade/common/models/activevendorlist.py b/repos/system_upgrade/common/models/activevendorlist.py
new file mode 100644
index 0000000..de4056f
--- /dev/null
+++ b/repos/system_upgrade/common/models/activevendorlist.py
@@ -0,0 +1,7 @@
+from leapp.models import Model, fields
+from leapp.topics import VendorTopic
+
+
+class ActiveVendorList(Model):
+ topic = VendorTopic
+ data = fields.List(fields.String())
diff --git a/repos/system_upgrade/common/models/repositoriesmap.py b/repos/system_upgrade/common/models/repositoriesmap.py
index c187333..a068a70 100644
--- a/repos/system_upgrade/common/models/repositoriesmap.py
+++ b/repos/system_upgrade/common/models/repositoriesmap.py
@@ -92,3 +92,4 @@ class RepositoriesMapping(Model):
mapping = fields.List(fields.Model(RepoMapEntry), default=[])
repositories = fields.List(fields.Model(PESIDRepositoryEntry), default=[])
+
diff --git a/repos/system_upgrade/common/models/vendorsignatures.py b/repos/system_upgrade/common/models/vendorsignatures.py
new file mode 100644
index 0000000..f456aec
--- /dev/null
+++ b/repos/system_upgrade/common/models/vendorsignatures.py
@@ -0,0 +1,8 @@
+from leapp.models import Model, fields
+from leapp.topics import VendorTopic
+
+
+class VendorSignatures(Model):
+ topic = VendorTopic
+ vendor = fields.String()
+ sigs = fields.List(fields.String())
diff --git a/repos/system_upgrade/common/models/vendorsourcerepos.py b/repos/system_upgrade/common/models/vendorsourcerepos.py
new file mode 100644
index 0000000..b7a219b
--- /dev/null
+++ b/repos/system_upgrade/common/models/vendorsourcerepos.py
@@ -0,0 +1,12 @@
+from leapp.models import Model, fields
+from leapp.topics import VendorTopic
+
+
+class VendorSourceRepos(Model):
+ """
+ This model contains the data on all source repositories associated with a specific vendor.
+ Its data is used to determine whether the vendor should be included into the upgrade process.
+ """
+ topic = VendorTopic
+ vendor = fields.String()
+ source_repoids = fields.List(fields.String())
diff --git a/repos/system_upgrade/common/topics/vendortopic.py b/repos/system_upgrade/common/topics/vendortopic.py
new file mode 100644
index 0000000..014b7af
--- /dev/null
+++ b/repos/system_upgrade/common/topics/vendortopic.py
@@ -0,0 +1,5 @@
+from leapp.topics import Topic
+
+
+class VendorTopic(Topic):
+ name = 'vendor_topic'
diff --git a/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/actor.py b/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/actor.py
index 69ca0f0..c488eb0 100644
--- a/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/actor.py
+++ b/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/actor.py
@@ -2,6 +2,7 @@ from leapp.actors import Actor
from leapp.libraries.stdlib import CalledProcessError, run
from leapp.models import NetworkManagerConfig
from leapp.tags import FirstBootPhaseTag, IPUWorkflowTag
+from leapp import reporting
class NetworkManagerUpdateConnections(Actor):
@@ -26,8 +27,22 @@ class NetworkManagerUpdateConnections(Actor):
return
try:
- r = run(['/usr/bin/python3', 'tools/nm-update-client-ids.py'])['stdout']
- self.log.info('Updated client-ids: {}'.format(r))
+ r = run(['/usr/bin/python3', 'tools/nm-update-client-ids.py'])
+ if r['exit_code'] == 79:
+ title = 'NetworkManager connection update failed - PyGObject bindings for NetworkManager not found.'
+ summary = 'When using dhcp=dhclient on Red Hat Enterprise Linux 7, a non-hexadecimal ' \
+ 'client-id (a string) is sent on the wire as is. On Red Hat Enterprise Linux 8, a zero ' \
+ 'byte is prepended to string-only client-ids. If you wish to preserve the RHEL 7 behaviour, ' \
+ 'you may want to convert your client-ids to hexadecimal form manually.'
+ reporting.create_report([
+ reporting.Title(title),
+ reporting.Summary(summary),
+ reporting.Severity(reporting.Severity.MEDIUM),
+ reporting.Tags([reporting.Tags.NETWORK])
+ ])
+ continue
+
+ self.log.info('Updated client-ids: {}'.format(r['stdout']))
except (OSError, CalledProcessError) as e:
self.log.warning('Error calling nm-update-client-ids script: {}'.format(e))
diff --git a/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/tools/nm-update-client-ids.py b/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/tools/nm-update-client-ids.py
index 923bf80..9972204 100755
--- a/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/tools/nm-update-client-ids.py
+++ b/repos/system_upgrade/el7toel8/actors/networkmanagerupdateconnections/tools/nm-update-client-ids.py
@@ -3,12 +3,26 @@ from __future__ import print_function
import sys
import gi
-gi.require_version('NM', '1.0')
+
+try:
+ gi.require_version("NM", "1.0")
+except ValueError:
+ # If we're missing NetworkManager-libnm, the script won't function.
+ print(
+ "PyGObject bindings for NetworkManager not found - do you have NetworkManager-libnm installed?"
+ )
+ print(
+ "If you have dhcp=dhclient, you may need to convert your string-formatted client IDs to hexadecimal"
+ "to preserve the format they're sent on the wire with. Otherwise, they will now have a zero byte"
+ "prepended while being sent."
+ )
+ sys.exit(79)
+
from gi.repository import NM # noqa: E402; pylint: disable=wrong-import-position
def is_hexstring(s):
- arr = s.split(':')
+ arr = s.split(":")
for a in arr:
if len(a) != 1 and len(a) != 2:
return False
@@ -21,8 +35,8 @@ def is_hexstring(s):
client = NM.Client.new(None)
if not client:
- print('Cannot create NM client instance')
- sys.exit(0)
+ print("Cannot create NM client instance")
+ sys.exit(79)
processed = 0
changed = 0
@@ -35,15 +49,20 @@ for c in client.get_connections():
client_id = s_ip4.get_dhcp_client_id()
if client_id is not None:
if not is_hexstring(client_id):
- new_client_id = ':'.join(hex(ord(x))[2:] for x in client_id)
+ new_client_id = ":".join(hex(ord(x))[2:] for x in client_id)
s_ip4.set_property(NM.SETTING_IP4_CONFIG_DHCP_CLIENT_ID, new_client_id)
success = c.commit_changes(True, None)
if success:
changed += 1
else:
errors += 1
- print('Connection {}: \'{}\' -> \'{}\' ({})'.format(c.get_uuid(),
- client_id, new_client_id,
- 'OK' if success else 'FAIL'))
+ print(
+ "Connection {}: '{}' -> '{}' ({})".format(
+ c.get_uuid(),
+ client_id,
+ new_client_id,
+ "OK" if success else "FAIL",
+ )
+ )
print("{} processed, {} changed, {} errors".format(processed, changed, errors))
diff --git a/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/actor.py b/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/actor.py
index f13a767..0ca198f 100644
--- a/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/actor.py
+++ b/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/actor.py
@@ -1,7 +1,7 @@
from leapp import reporting
from leapp.actors import Actor
from leapp.exceptions import StopActorExecutionError
-from leapp.libraries.actor.opensshpermitrootlogincheck import semantics_changes
+from leapp.libraries.actor.opensshpermitrootlogincheck import semantics_changes, add_permitrootlogin_conf
from leapp.libraries.stdlib import api
from leapp.models import OpenSshConfig, Report
from leapp.reporting import create_report
@@ -39,28 +39,32 @@ class OpenSshPermitRootLoginCheck(Actor):
resources = [
reporting.RelatedResource('package', 'openssh-server'),
- reporting.RelatedResource('file', '/etc/ssh/sshd_config')
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config'),
+ reporting.RelatedResource('file', '/etc/ssh/sshd_config.leapp_backup')
]
- # When the configuration does not contain the PermitRootLogin directive and
- # the configuration file was locally modified, it will not get updated by
- # RPM and the user might be locked away from the server. Warn the user here.
- if not config.permit_root_login and config.modified:
+ if not config.permit_root_login:
+ add_permitrootlogin_conf()
create_report([
- reporting.Title('Possible problems with remote login using root account'),
+ reporting.Title('SSH configuration automatically modified to permit root login'),
reporting.Summary(
- 'OpenSSH configuration file does not explicitly state '
- 'the option PermitRootLogin in sshd_config file, '
- 'which will default in RHEL8 to "prohibit-password".'
+ 'Your OpenSSH configuration file does not explicitly state '
+ 'the option PermitRootLogin in sshd_config file. '
+ 'Its default is "yes" in RHEL7, but will change in '
+ 'RHEL8 to "prohibit-password", which may affect your ability '
+ 'to log onto this machine after the upgrade. '
+ 'To prevent this from occuring, the PermitRootLogin option '
+ 'has been explicity set to "yes" to preserve the default behaivour '
+ 'after migration.'
+ 'The original configuration file has been backed up to'
+ '/etc/ssh/sshd_config.leapp_backup'
),
- reporting.Severity(reporting.Severity.HIGH),
+ reporting.Severity(reporting.Severity.MEDIUM),
reporting.Tags(COMMON_REPORT_TAGS),
reporting.Remediation(
- hint='If you depend on remote root logins using '
- 'passwords, consider setting up a different '
- 'user for remote administration or adding '
- '"PermitRootLogin yes" to sshd_config.'
- ),
- reporting.Flags([reporting.Flags.INHIBITOR])
+ hint='If you would prefer to configure the root login policy yourself, '
+ 'consider setting the PermitRootLogin option '
+ 'in sshd_config explicitly.'
+ )
] + resources)
# Check if there is at least one PermitRootLogin other than "no"
@@ -68,7 +72,7 @@ class OpenSshPermitRootLoginCheck(Actor):
# This usually means some more complicated setup depending on the
# default value being globally "yes" and being overwritten by this
# match block
- if semantics_changes(config):
+ elif semantics_changes(config):
create_report([
reporting.Title('OpenSSH configured to allow root login'),
reporting.Summary(
@@ -76,7 +80,7 @@ class OpenSshPermitRootLoginCheck(Actor):
'blocks, but not explicitly enabled in global or '
'"Match all" context. This update changes the '
'default to disable root logins using paswords '
- 'so your server migth get inaccessible.'
+ 'so your server might become inaccessible.'
),
reporting.Severity(reporting.Severity.HIGH),
reporting.Tags(COMMON_REPORT_TAGS),
diff --git a/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/libraries/opensshpermitrootlogincheck.py b/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/libraries/opensshpermitrootlogincheck.py
index 0cb9081..7a962b7 100644
--- a/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/libraries/opensshpermitrootlogincheck.py
+++ b/repos/system_upgrade/el7toel8/actors/opensshpermitrootlogincheck/libraries/opensshpermitrootlogincheck.py
@@ -1,3 +1,5 @@
+import errno
+from leapp.libraries.stdlib import api
def semantics_changes(config):
@@ -13,3 +15,30 @@ def semantics_changes(config):
globally_enabled = True
return not globally_enabled and in_match_disabled
+
+
+def add_permitrootlogin_conf():
+ CONFIG = '/etc/ssh/sshd_config'
+ CONFIG_BACKUP = '/etc/ssh/sshd_config.leapp_backup'
+ try:
+ with open(CONFIG, 'r') as fd:
+ sshd_config = fd.readlines()
+
+ permit_autoconf = [
+ "# Automatically added by Leapp to preserve RHEL7 default\n",
+ "# behaviour after migration.\n",
+ "# Placed on top of the file to avoid being included into Match blocks.\n",
+ "PermitRootLogin yes\n"
+ "\n",
+ ]
+ permit_autoconf.extend(sshd_config)
+ with open(CONFIG, 'w') as fd:
+ fd.writelines(permit_autoconf)
+ with open(CONFIG_BACKUP, 'w') as fd:
+ fd.writelines(sshd_config)
+
+ except IOError as err:
+ if err.errno != errno.ENOENT:
+ error = 'Failed to open sshd_config: {}'.format(str(err))
+ api.current_logger().error(error)
+ return
View Git Blame Copy Permalink