Compare commits

...

No commits in common. 'c8' and 'c9' have entirely different histories.
c8 ... c9

@ -1,33 +1,32 @@
From 5e1e568d7085fd4443b4e3ccc492f5e31747e270 Mon Sep 17 00:00:00 2001
From: Amy Chen <xiachen@redhat.com>
Date: Wed, 20 Apr 2022 10:59:48 +0800
From 7daf47d1ee60a132244f2a9ec4e89ad496c32c3e Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Mon, 28 Mar 2022 14:08:05 +0200
Subject: Add initial redhat setup
Merged patches (22.1)
- d18029bf Add netifaces package as a Requires in cloud-init.spec.template
- 31adf961 Add gdisk and openssl as deps to fix UEFI / Azure initialization
- f4a2905d Add dhcp-client as a dependency
- 290e14cc cloud-init.spec.template: update %systemd_postun parameter
- 9be4ae9b (tag: cloud-init-21.1-1.el8) Update to cloud-init-21.1-1.el8
Merged patches (22.1):
- 70f80f82 Use _systemdgeneratordir macro for cloud-init-generator handling
- 0f74e7b8 Add dhcp-client as a dependency
- 791eb2b6 Report full specific version with "cloud-init --version"
- 865805d6 source-git: Use dynamic SRPM_NAME
- 45ddc666 add the drop-in also in the %files section of cloud-init.spec
- 08d939b2 Add gdisk and openssl as deps to fix UEFI / Azure initialization
- 0531cd68 Add netifaces package as a Requires in cloud-init.spec.template
Conflicts:
cloudinit/config/cc_chef.py Using double quotes instead of single quotes
- .gitignore
- cloudinit/config/cc_chef.py = use double quotes instead of single quotes
- cloudinit/settings.py = use rhel settings
- cloudinit/sources/DataSourceAzure.py = remove temporary_hostname and helper
- requirements.txt = just add netifaces and not jsonschema dep
- setup.py = use double quotes instead of single quotes
cloudinit/settings.py
- Using rhel settings
- Using double quotes instead of single quotes
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
setup.py
- Following the changes of 21.1 rebase
- Using double quotes instead of single quotes
redhat/cloud-init.spec.template
- Add the drop-in to the right cloud-init.spec used by our package builder, which is downstream-only part of the bz 2002492 fix.
redhat/Makefile.common
- Backport the build handling fixes from patch "Update to cloud-init-21.1-1.el8"
Signed-off-by: Amy Chen <xiachen@redhat.com>
Merged patches (RHEL-9/21.1):
- 5688a1d0 Removing python-nose and python-tox as dependency
- 237d57f9 Removing mock dependency
- d1c2f496 Removing python-jsonschema dependency
- 0d1cd14c Don't override default network configuration
Merged patches (21.1):
- 915d30ad Change gating file to correct rhel version
@ -60,61 +59,72 @@ Merged patches (18.5):
Conflicts:
cloudinit/config/cc_chef.py:
- Updated header documentation text
- Replacing double quotes by simple quotes
- Updated header documentation text
- Replacing double quotes by simple quotes
setup.py:
- Adding missing cmdclass info
- Adding missing cmdclass info
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Changes:
- move redhat to .distro to use new build script structure
- Fixing changelog for RHEL 9
Merged patches (21.1):
- 69bd7f71 DataSourceAzure.py: use hostnamectl to set hostname
- 0407867e Remove race condition between cloud-init and NetworkManager
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
---
.distro/.gitignore | 1 +
.distro/Makefile | 74 ++++
.distro/Makefile.common | 20 +
.distro/cloud-init-tmpfiles.conf | 1 +
.distro/cloud-init.spec.template | 505 ++++++++++++++++++++++++++
.distro/gating.yaml | 8 +
.distro/rpmbuild/BUILD/.gitignore | 3 +
.distro/rpmbuild/RPMS/.gitignore | 3 +
.distro/rpmbuild/SOURCES/.gitignore | 3 +
.distro/rpmbuild/SPECS/.gitignore | 3 +
.distro/rpmbuild/SRPMS/.gitignore | 3 +
.distro/scripts/frh.py | 27 ++
.distro/scripts/git-backport-diff | 327 +++++++++++++++++
.distro/scripts/git-compile-check | 215 +++++++++++
.distro/scripts/process-patches.sh | 88 +++++
.distro/scripts/tarball_checksum.sh | 3 +
.gitignore | 1 +
cloudinit/config/cc_chef.py | 65 ++-
cloudinit/settings.py | 7 +-
redhat/.gitignore | 1 +
redhat/Makefile | 71 +++
redhat/Makefile.common | 37 ++
redhat/cloud-init-tmpfiles.conf | 1 +
redhat/cloud-init.spec.template | 696 ++++++++++++++++++++++++++
redhat/gating.yaml | 8 +
redhat/rpmbuild/BUILD/.gitignore | 3 +
redhat/rpmbuild/RPMS/.gitignore | 3 +
redhat/rpmbuild/SOURCES/.gitignore | 3 +
redhat/rpmbuild/SPECS/.gitignore | 3 +
redhat/rpmbuild/SRPMS/.gitignore | 3 +
redhat/scripts/frh.py | 25 +
redhat/scripts/git-backport-diff | 327 ++++++++++++
redhat/scripts/git-compile-check | 215 ++++++++
redhat/scripts/process-patches.sh | 92 ++++
redhat/scripts/tarball_checksum.sh | 3 +
cloudinit/config/cc_chef.py | 65 +++-
cloudinit/settings.py | 21 +-
requirements.txt | 3 -
rhel/README.rhel | 5 +
rhel/cloud-init-tmpfiles.conf | 1 +
rhel/cloud.cfg | 69 +++
rhel/cloud.cfg | 69 ++++
rhel/systemd/cloud-config.service | 18 +
rhel/systemd/cloud-config.target | 11 +
rhel/systemd/cloud-final.service | 24 +
rhel/systemd/cloud-final.service | 24 ++
rhel/systemd/cloud-init-local.service | 31 ++
rhel/systemd/cloud-init.service | 25 +
rhel/systemd/cloud-init.service | 26 ++
rhel/systemd/cloud-init.target | 7 +
setup.py | 28 +-
tools/read-version | 28 +-
30 files changed, 1756 insertions(+), 55 deletions(-)
create mode 100644 redhat/.gitignore
create mode 100644 redhat/Makefile
create mode 100644 redhat/Makefile.common
create mode 100644 redhat/cloud-init-tmpfiles.conf
create mode 100644 redhat/cloud-init.spec.template
create mode 100644 redhat/gating.yaml
create mode 100644 redhat/rpmbuild/BUILD/.gitignore
create mode 100644 redhat/rpmbuild/RPMS/.gitignore
create mode 100644 redhat/rpmbuild/SOURCES/.gitignore
create mode 100644 redhat/rpmbuild/SPECS/.gitignore
create mode 100644 redhat/rpmbuild/SRPMS/.gitignore
create mode 100755 redhat/scripts/frh.py
create mode 100755 redhat/scripts/git-backport-diff
create mode 100755 redhat/scripts/git-compile-check
create mode 100755 redhat/scripts/process-patches.sh
create mode 100755 redhat/scripts/tarball_checksum.sh
31 files changed, 1557 insertions(+), 65 deletions(-)
create mode 100644 .distro/.gitignore
create mode 100644 .distro/Makefile
create mode 100644 .distro/Makefile.common
create mode 100644 .distro/cloud-init-tmpfiles.conf
create mode 100644 .distro/cloud-init.spec.template
create mode 100644 .distro/gating.yaml
create mode 100644 .distro/rpmbuild/BUILD/.gitignore
create mode 100644 .distro/rpmbuild/RPMS/.gitignore
create mode 100644 .distro/rpmbuild/SOURCES/.gitignore
create mode 100644 .distro/rpmbuild/SPECS/.gitignore
create mode 100644 .distro/rpmbuild/SRPMS/.gitignore
create mode 100755 .distro/scripts/frh.py
create mode 100755 .distro/scripts/git-backport-diff
create mode 100755 .distro/scripts/git-compile-check
create mode 100755 .distro/scripts/process-patches.sh
create mode 100755 .distro/scripts/tarball_checksum.sh
create mode 100644 rhel/README.rhel
create mode 100644 rhel/cloud-init-tmpfiles.conf
create mode 100644 rhel/cloud.cfg
@ -202,28 +212,52 @@ index fdb3a6e3..d028c548 100644
import itertools
import json
diff --git a/cloudinit/settings.py b/cloudinit/settings.py
index ecc1403b..39650a5b 100644
index ecc1403b..a780e21e 100644
--- a/cloudinit/settings.py
+++ b/cloudinit/settings.py
@@ -50,13 +50,16 @@ CFG_BUILTIN = {
@@ -48,16 +48,19 @@ CFG_BUILTIN = {
# At the end to act as a 'catch' when none of the above work...
"None",
],
"def_log_file": "/var/log/cloud-init.log",
"log_cfgs": [],
- "def_log_file": "/var/log/cloud-init.log",
- "log_cfgs": [],
- "syslog_fix_perms": ["syslog:adm", "root:adm", "root:wheel", "root:root"],
+ "mount_default_fields": [None, None, "auto", "defaults,nofail", "0", "2"],
+ "ssh_deletekeys": False,
+ "ssh_genkeytypes": [],
+ "syslog_fix_perms": [],
"system_info": {
"paths": {
"cloud_dir": "/var/lib/cloud",
"templates_dir": "/etc/cloud/templates/",
- "system_info": {
- "paths": {
- "cloud_dir": "/var/lib/cloud",
- "templates_dir": "/etc/cloud/templates/",
+ 'def_log_file': '/var/log/cloud-init.log',
+ 'log_cfgs': [],
+ 'mount_default_fields': [None, None, 'auto', 'defaults,nofail', '0', '2'],
+ 'ssh_deletekeys': False,
+ 'ssh_genkeytypes': [],
+ 'syslog_fix_perms': [],
+ 'system_info': {
+ 'paths': {
+ 'cloud_dir': '/var/lib/cloud',
+ 'templates_dir': '/etc/cloud/templates/',
},
- "distro": "ubuntu",
+ "distro": "rhel",
"network": {"renderers": None},
- "network": {"renderers": None},
+ 'distro': 'rhel',
+ 'network': {'renderers': None},
},
"vendor_data": {"enabled": True, "prefix": []},
"vendor_data2": {"enabled": True, "prefix": []},
diff --git a/requirements.txt b/requirements.txt
index c4adc455..f77f1d73 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -30,9 +30,6 @@ requests
# For patching pieces of cloud-config together
jsonpatch
-# For validating cloud-config sections per schema definitions
-jsonschema
-
# Used by DataSourceVMware to inspect the host's network configuration during
# the "setup()" function.
#
diff --git a/rhel/README.rhel b/rhel/README.rhel
new file mode 100644
index 00000000..aa29630d
@ -244,7 +278,7 @@ index 00000000..0c6d2a3b
+d /run/cloud-init 0700 root root - -
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
new file mode 100644
index 00000000..82e8bf62
index 00000000..9ecba215
--- /dev/null
+++ b/rhel/cloud.cfg
@@ -0,0 +1,69 @@
@ -256,7 +290,7 @@ index 00000000..82e8bf62
+
+mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
+resize_rootfs_tmp: /dev
+ssh_deletekeys: 0
+ssh_deletekeys: 1
+ssh_genkeytypes: ~
+syslog_fix_perms: ~
+disable_vmware_customization: false
@ -427,10 +461,10 @@ index 00000000..8f9f6c9f
+WantedBy=cloud-init.target
diff --git a/rhel/systemd/cloud-init.service b/rhel/systemd/cloud-init.service
new file mode 100644
index 00000000..d0023a05
index 00000000..0b3d796d
--- /dev/null
+++ b/rhel/systemd/cloud-init.service
@@ -0,0 +1,25 @@
@@ -0,0 +1,26 @@
+[Unit]
+Description=Initial cloud-init job (metadata service crawler)
+Wants=cloud-init-local.service
@ -438,6 +472,7 @@ index 00000000..d0023a05
+Wants=sshd.service
+After=cloud-init-local.service
+After=NetworkManager.service network.service
+After=NetworkManager-wait-online.service
+Before=network-online.target
+Before=sshd-keygen.service
+Before=sshd.service

@ -1,4 +1,4 @@
From e0dc628ac553072891fa6607dc91b652efd99be2 Mon Sep 17 00:00:00 2001
From c521ec2ce5b1d9a7322ce152011b8792f121bf5c Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 7 May 2021 13:36:06 +0200
Subject: Do not write NM_CONTROLLED=no in generated interface config files
@ -8,16 +8,19 @@ Conflicts 20.3:
mechanism to identify if cloud-init is running on RHEL, having the
correct settings for NM_CONTROLLED.
Merged patches (21.1):
- ecbace48 sysconfig: Don't write BOOTPROTO=dhcp for ipv6 dhcp
- a1a00383 include 'NOZEROCONF=yes' in /etc/sysconfig/network
X-downstream-only: true
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Ryan McCabe <rmccabe@redhat.com>
---
cloudinit/net/sysconfig.py | 1 -
cloudinit/net/sysconfig.py | 12 ++++++++++--
tests/unittests/test_net.py | 28 ----------------------------
2 files changed, 29 deletions(-)
2 files changed, 10 insertions(+), 30 deletions(-)
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index ba85c4f6..e06ddee7 100644
index ba85c4f6..d8c53312 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -336,7 +336,6 @@ class Renderer(renderer.Renderer):
@ -28,6 +31,24 @@ index ba85c4f6..e06ddee7 100644
"BOOTPROTO": "none",
},
"suse": {"BOOTPROTO": "static", "STARTMODE": "auto"},
@@ -1039,7 +1038,16 @@ class Renderer(renderer.Renderer):
# Distros configuring /etc/sysconfig/network as a file e.g. Centos
if sysconfig_path.endswith("network"):
util.ensure_dir(os.path.dirname(sysconfig_path))
- netcfg = [_make_header(), "NETWORKING=yes"]
+ netcfg = []
+ for line in util.load_file(sysconfig_path, quiet=True).split('\n'):
+ if 'cloud-init' in line:
+ break
+ if not line.startswith(('NETWORKING=',
+ 'IPV6_AUTOCONF=',
+ 'NETWORKING_IPV6=')):
+ netcfg.append(line)
+ # Now generate the cloud-init portion of sysconfig/network
+ netcfg.extend([_make_header(), 'NETWORKING=yes'])
if network_state.use_ipv6:
netcfg.append("NETWORKING_IPV6=yes")
netcfg.append("IPV6_AUTOCONF=no")
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index 47e4ba00..591241b3 100644
--- a/tests/unittests/test_net.py

@ -0,0 +1,42 @@
From b952fa472be3f417e0d857c8647a1b930624c247 Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 25 Feb 2022 05:05:17 -0500
Subject: Adding _netdev to the default mount configuration
RH-Author: Eduardo Otubo <otubo@redhat.com>
RH-MergeRequest: 21: Adding _netdev to the default mount configuration
RH-Commit: [1/1] 250860a24db396a5088d207d6526a0028ac73eb3 (otubo/cloud-init-src)
RH-Bugzilla: 1998445
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
RH-Acked-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Adding _netdev option also to the default configuration for RHEL.
rhbz: 1998445
x-downstream-only: yes
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
patch_name: ci-Adding-_netdev-to-the-default-mount-configuration.patch
present_in_specfile: true
location_in_specfile: 29
---
rhel/cloud.cfg | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
index 9ecba215..1ec1a6c6 100644
--- a/rhel/cloud.cfg
+++ b/rhel/cloud.cfg
@@ -4,7 +4,7 @@ users:
disable_root: 1
ssh_pwauth: 0
-mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
+mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service,_netdev', '0', '2']
resize_rootfs_tmp: /dev
ssh_deletekeys: 1
ssh_genkeytypes: ~
--
2.31.1

@ -0,0 +1,49 @@
From a14df44ffdc880ae16c691901e2671458ab234ff Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Thu, 17 Feb 2022 15:32:35 +0100
Subject: Setting highest autoconnect priority for network-scripts
RH-Author: Eduardo Otubo <otubo@redhat.com>
RH-MergeRequest: 22: Setting highest autoconnect priority for network-scripts
RH-Commit: [1/1] 34f1d62f8934a983a124df95b861a1e448681d3b (otubo/cloud-init-src)
RH-Bugzilla: 2036060
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
RH-Acked-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Set the highest autoconnect priority for network-scripts which is
loaded by NetworkManager ifcfg-rh plugin. Note that keyfile is the only
and default existing plugin on RHEL9, by setting the highest autoconnect
priority for network-scripts, NetworkManager will activate
network-scripts but keyfile. Network-scripts path:
Since this is a blocking issue, we decided to have this one-liner
downstream-only patch so we can move forward and have a better
NetworkManager support later on the release.
rhbz: 2036060
x-downstream-only: yes
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
patch_name: ci-Setting-highest-autoconnect-priority-for-network-scr.patch
present_in_specfile: true
location_in_specfile: 30
---
cloudinit/net/sysconfig.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index d8c53312..b50035b5 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -337,6 +337,7 @@ class Renderer(renderer.Renderer):
"ONBOOT": True,
"USERCTL": False,
"BOOTPROTO": "none",
+ "AUTOCONNECT_PRIORITY": 999
},
"suse": {"BOOTPROTO": "static", "STARTMODE": "auto"},
}
--
2.31.1

@ -1,52 +0,0 @@
From ffa647e83efd4293bd027e9e390274aad8a12d94 Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 7 May 2021 13:36:13 +0200
Subject: include 'NOZEROCONF=yes' in /etc/sysconfig/network
RH-Author: Eduardo Otubo <otubo@redhat.com>
Message-id: <20190320114559.23708-1-otubo@redhat.com>
Patchwork-id: 84937
O-Subject: [RHEL-7.7 cloud-init PATCH] include 'NOZEROCONF=yes' in /etc/sysconfig/network
Bugzilla: 1653131
RH-Acked-by: Cathy Avery <cavery@redhat.com>
RH-Acked-by: Mohammed Gamal <mgamal@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
The option NOZEROCONF=yes is not included by default in
/etc/sysconfig/network, which is required by Overcloud instances. The
patch also includes tests for the modifications.
X-downstream-only: yes
Resolves: rhbz#1653131
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
---
cloudinit/net/sysconfig.py | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index e06ddee7..362e8d19 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -1038,7 +1038,16 @@ class Renderer(renderer.Renderer):
# Distros configuring /etc/sysconfig/network as a file e.g. Centos
if sysconfig_path.endswith("network"):
util.ensure_dir(os.path.dirname(sysconfig_path))
- netcfg = [_make_header(), "NETWORKING=yes"]
+ netcfg = []
+ for line in util.load_file(sysconfig_path, quiet=True).split("\n"):
+ if "cloud-init" in line:
+ break
+ if not line.startswith(("NETWORKING=",
+ "IPV6_AUTOCONF=",
+ "NETWORKING_IPV6=")):
+ netcfg.append(line)
+ # Now generate the cloud-init portion of sysconfig/network
+ netcfg.extend([_make_header(), "NETWORKING=yes"])
if network_state.use_ipv6:
netcfg.append("NETWORKING_IPV6=yes")
netcfg.append("IPV6_AUTOCONF=no")
--
2.31.1

@ -1,148 +0,0 @@
From 386f0a82bfdfd62e506bf4251c17263260d3250a Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 7 May 2021 13:36:14 +0200
Subject: Remove race condition between cloud-init and NetworkManager
Message-id: <20200302104635.11648-1-otubo@redhat.com>
Patchwork-id: 94098
O-Subject: [RHEL-7.9/RHEL-8.2.0 cloud-init PATCH] Remove race condition between cloud-init and NetworkManager
Bugzilla: 1807797
RH-Acked-by: Cathy Avery <cavery@redhat.com>
RH-Acked-by: Mohammed Gamal <mgamal@redhat.com>
BZ: 1748015
BRANCH: rhel7/master-18.5
BREW: 26924611
BZ: 1807797
BRANCH: rhel820/master-18.5
BREW: 26924957
cloud-init service is set to start before NetworkManager service starts,
but this does not avoid a race condition between them. NetworkManager
starts before cloud-init can write `dns=none' to the file:
/etc/NetworkManager/conf.d/99-cloud-init.conf. This way NetworkManager
doesn't read the configuration and erases all resolv.conf values upon
shutdown. On the next reboot neither cloud-init or NetworkManager will
write anything to resolv.conf, leaving it blank.
This patch introduces a NM reload (try-restart) at the end of cloud-init
start up so it won't erase resolv.conf upon first shutdown.
x-downstream-only: yes
resolves: rhbz#1748015, rhbz#1807797 and rhbz#1804780
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
This commit is a squash and also includes the folloowing commits:
commit 316a17b7c02a87fa9b2981535be0b20d165adc46
Author: Eduardo Otubo <otubo@redhat.com>
Date: Mon Jun 1 11:58:06 2020 +0200
Make cloud-init.service execute after network is up
RH-Author: Eduardo Otubo <otubo@redhat.com>
Message-id: <20200526090804.2047-1-otubo@redhat.com>
Patchwork-id: 96809
O-Subject: [RHEL-8.2.1 cloud-init PATCH] Make cloud-init.service execute after network is up
Bugzilla: 1803928
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
cloud-init.service needs to wait until network is fully up before
continuing executing and configuring its service.
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
x-downstream-only: yes
Resolves: rhbz#1831646
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
commit 0422ba0e773d1a8257a3f2bf3db05f3bc7917eb7
Author: Eduardo Otubo <otubo@redhat.com>
Date: Thu May 28 08:44:08 2020 +0200
Remove race condition between cloud-init and NetworkManager
RH-Author: Eduardo Otubo <otubo@redhat.com>
Message-id: <20200327121911.17699-1-otubo@redhat.com>
Patchwork-id: 94453
O-Subject: [RHEL-7.9/RHEL-8.2.0 cloud-init PATCHv2] Remove race condition between cloud-init and NetworkManager
Bugzilla: 1840648
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
RH-Acked-by: Cathy Avery <cavery@redhat.com>
cloud-init service is set to start before NetworkManager service starts,
but this does not avoid a race condition between them. NetworkManager
starts before cloud-init can write `dns=none' to the file:
/etc/NetworkManager/conf.d/99-cloud-init.conf. This way NetworkManager
doesn't read the configuration and erases all resolv.conf values upon
shutdown. On the next reboot neither cloud-init or NetworkManager will
write anything to resolv.conf, leaving it blank.
This patch introduces a NM reload (try-reload-or-restart) at the end of cloud-init
start up so it won't erase resolv.conf upon first shutdown.
x-downstream-only: yes
Signed-off-by: Eduardo Otubo otubo@redhat.com
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
commit e0b48a936433faea7f56dbc29dda35acf7d375f7
Author: Eduardo Otubo <otubo@redhat.com>
Date: Thu May 28 08:44:06 2020 +0200
Enable ssh_deletekeys by default
RH-Author: Eduardo Otubo <otubo@redhat.com>
Message-id: <20200317091705.15715-1-otubo@redhat.com>
Patchwork-id: 94365
O-Subject: [RHEL-7.9/RHEL-8.2.0 cloud-init PATCH] Enable ssh_deletekeys by default
Bugzilla: 1814152
RH-Acked-by: Mohammed Gamal <mgamal@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
The configuration option ssh_deletekeys will trigger the generation
of new ssh keys for every new instance deployed.
x-downstream-only: yes
resolves: rhbz#1814152
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
---
rhel/cloud.cfg | 2 +-
rhel/systemd/cloud-init.service | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
index 82e8bf62..9ecba215 100644
--- a/rhel/cloud.cfg
+++ b/rhel/cloud.cfg
@@ -6,7 +6,7 @@ ssh_pwauth: 0
mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
resize_rootfs_tmp: /dev
-ssh_deletekeys: 0
+ssh_deletekeys: 1
ssh_genkeytypes: ~
syslog_fix_perms: ~
disable_vmware_customization: false
diff --git a/rhel/systemd/cloud-init.service b/rhel/systemd/cloud-init.service
index d0023a05..0b3d796d 100644
--- a/rhel/systemd/cloud-init.service
+++ b/rhel/systemd/cloud-init.service
@@ -5,6 +5,7 @@ Wants=sshd-keygen.service
Wants=sshd.service
After=cloud-init-local.service
After=NetworkManager.service network.service
+After=NetworkManager-wait-online.service
Before=network-online.target
Before=sshd-keygen.service
Before=sshd.service
--
2.31.1

@ -1,4 +1,4 @@
From cb7b35ca10c82c9725c3527e3ec5fb8cb7c61bc0 Mon Sep 17 00:00:00 2001
From 40ad855b883050069393b9c00db2a6d222d949db Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 7 May 2021 13:36:08 +0200
Subject: limit permissions on def_log_file
@ -22,17 +22,17 @@ Signed-off-by: Eduardo Otubo <otubo@redhat.com>
3 files changed, 6 insertions(+)
diff --git a/cloudinit/settings.py b/cloudinit/settings.py
index 39650a5b..3c2145e9 100644
index a780e21e..aa2d6b95 100644
--- a/cloudinit/settings.py
+++ b/cloudinit/settings.py
@@ -49,6 +49,7 @@ CFG_BUILTIN = {
"None",
],
"def_log_file": "/var/log/cloud-init.log",
+ "def_log_file_mode": 0o600,
"log_cfgs": [],
"mount_default_fields": [None, None, "auto", "defaults,nofail", "0", "2"],
"ssh_deletekeys": False,
'def_log_file': '/var/log/cloud-init.log',
+ 'def_log_file_mode': 0o600,
'log_cfgs': [],
'mount_default_fields': [None, None, 'auto', 'defaults,nofail', '0', '2'],
'ssh_deletekeys': False,
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index 3f17294b..61db1dbd 100644
--- a/cloudinit/stages.py

@ -1,4 +1,4 @@
From b545a0cbabe8924d048b7172b30e7aad59ed32d5 Mon Sep 17 00:00:00 2001
From 00f1f910d8d166ebe2913c12549f212c2d666c11 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Thu, 20 May 2021 08:53:55 +0200
Subject: rhel/cloud.cfg: remove ssh_genkeytypes in settings.py and set in
@ -35,24 +35,24 @@ Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
2 files changed, 1 insertion(+), 3 deletions(-)
diff --git a/cloudinit/settings.py b/cloudinit/settings.py
index 3c2145e9..71672e10 100644
index aa2d6b95..38a90b70 100644
--- a/cloudinit/settings.py
+++ b/cloudinit/settings.py
@@ -52,8 +52,6 @@ CFG_BUILTIN = {
"def_log_file_mode": 0o600,
"log_cfgs": [],
"mount_default_fields": [None, None, "auto", "defaults,nofail", "0", "2"],
- "ssh_deletekeys": False,
- "ssh_genkeytypes": [],
"syslog_fix_perms": [],
"system_info": {
"paths": {
'def_log_file_mode': 0o600,
'log_cfgs': [],
'mount_default_fields': [None, None, 'auto', 'defaults,nofail', '0', '2'],
- 'ssh_deletekeys': False,
- 'ssh_genkeytypes': [],
'syslog_fix_perms': [],
'system_info': {
'paths': {
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
index 9ecba215..cbee197a 100644
index 1ec1a6c6..75d5c84b 100644
--- a/rhel/cloud.cfg
+++ b/rhel/cloud.cfg
@@ -7,7 +7,7 @@ ssh_pwauth: 0
mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service,_netdev', '0', '2']
resize_rootfs_tmp: /dev
ssh_deletekeys: 1
-ssh_genkeytypes: ~

@ -1,15 +1,14 @@
From 0d93e53fd05c44b62e3456b7580c9de8135e6b5a Mon Sep 17 00:00:00 2001
From f0ae77cbf4a5e269da54fc2783a2a836023bbd86 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Mon, 2 May 2022 14:21:24 +0200
Subject: [PATCH 1/4] Add native NetworkManager support (#1224)
Date: Mon, 2 May 2022 14:42:52 +0200
Subject: [PATCH 1/5] Add native NetworkManager support (#1224)
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 57: Add native NetworkManager support (#1224)
RH-Commit: [1/2] 56b9ed40840a4930c421c2749e8aa385097bef93
RH-Bugzilla: 2059872
RH-MergeRequest: 24: Add native NetworkManager support (#1224)
RH-Commit: [1/3] 65231ba68460c505646807faf186c704d67678b5 (eesposit/cloud-init-centos-)
RH-Bugzilla: 2056964
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Jon Maloy <jmaloy@redhat.com>
RH-Acked-by: Eduardo Otubo <otubo@redhat.com>
commit feda344e6cf9d37b09bc13cf333a717d1654c26c
Author: Lubomir Rintel <lkundrak@v3.sk>
@ -570,7 +569,7 @@ index c755f04c..7edc34b5 100644
"netbsd",
"openbsd",
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index 362e8d19..c3b0c795 100644
index b50035b5..2a45a4fa 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -5,8 +5,6 @@ import io
@ -607,7 +606,7 @@ index 362e8d19..c3b0c795 100644
class ConfigMap(object):
"""Sysconfig like dictionary object."""
@@ -1031,8 +1011,6 @@ class Renderer(renderer.Renderer):
@@ -1032,8 +1012,6 @@ class Renderer(renderer.Renderer):
netrules_content = self._render_persistent_net(network_state)
netrules_path = subp.target_path(target, self.netrules_path)
util.write_file(netrules_path, netrules_content, file_mode)
@ -616,7 +615,7 @@ index 362e8d19..c3b0c795 100644
sysconfig_path = subp.target_path(target, templates.get("control"))
# Distros configuring /etc/sysconfig/network as a file e.g. Centos
@@ -1071,14 +1049,9 @@ def _supported_vlan_names(rdev, vid):
@@ -1072,14 +1050,9 @@ def _supported_vlan_names(rdev, vid):
def available(target=None):
@ -633,7 +632,7 @@ index 362e8d19..c3b0c795 100644
expected = ["ifup", "ifdown"]
search = ["/sbin", "/usr/sbin"]
for p in expected:
@@ -1095,10 +1068,4 @@ def available_sysconfig(target=None):
@@ -1096,10 +1069,4 @@ def available_sysconfig(target=None):
return False
@ -2296,5 +2295,5 @@ index 3c29e2f7..4525c49c 100644
NETWORKD_BRING_DOWN_CALL_LIST = [
--
2.35.3
2.31.1

@ -1,15 +1,14 @@
From 5c99ba05086b1ec83ce7e0c64edb4add4b47d923 Mon Sep 17 00:00:00 2001
From 8e599c618ba33f7ed572f752fc9201ca44e41868 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Thu, 19 May 2022 11:14:39 +0200
Subject: [PATCH 3/4] Align rhel custom files with upstream (#1431)
Date: Wed, 18 May 2022 16:21:45 +0200
Subject: [PATCH 4/5] Align rhel custom files with upstream (#1431)
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 65: Align rhel custom files with upstream (#1431)
RH-Commit: [1/2] 5d9067175688b1006472a477b0916b81c73d5e07
RH-Bugzilla: 2082071
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
RH-Acked-by: Eduardo Otubo <otubo@redhat.com>
RH-MergeRequest: 26: Align rhel custom files with upstream (#1431)
RH-Commit: [1/2] edac80c4fa3a11d093ee0e7260796566a7eb141e (eesposit/cloud-init-centos-)
RH-Bugzilla: 2088448
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
commit 9624758f91b61f4711e8d7b5c83075b5d23e0c43
Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
@ -253,5 +252,5 @@ index 30fbd1a4..9f95d448 100644
}
default_user = system_cfg["system_info"]["default_user"]["name"]
--
2.35.3
2.31.1

@ -0,0 +1,516 @@
From 1176a788c23697099093b4d8a9a21f10f71ebb12 Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 1 Feb 2023 10:47:07 +0100
Subject: [PATCH] Allow growpart to resize encrypted partitions (#1316)
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2166245
commit d95a331d1035d52443c470e0c00765a2c2b271cc
Author: James Falcon <james.falcon@canonical.com>
Date: Tue Apr 26 19:03:13 2022 -0500
Allow growpart to resize encrypted partitions (#1316)
Adds the ability for growpart to resize a LUKS formatted partition.
This involves resizing the underlying partition as well as the
filesystem. 'cryptsetup' is used for resizing.
This relies on a file present at /cc_growpart_keydata containing
json formatted 'key' and 'slot' keys, with the key being
base64 encoded. After resize, cloud-init will destroy
the luks slot used for resizing and remove the key file.
Conflicts:
cloudinit/config/cc_growpart.py (includes only)
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
cloudinit/config/cc_growpart.py | 171 +++++++++++++++-
test-requirements.txt | 1 +
tests/unittests/config/test_cc_growpart.py | 228 +++++++++++++++++++++
tox.ini | 1 +
4 files changed, 400 insertions(+), 1 deletion(-)
diff --git a/cloudinit/config/cc_growpart.py b/cloudinit/config/cc_growpart.py
index 43334caa..bdf17aba 100644
--- a/cloudinit/config/cc_growpart.py
+++ b/cloudinit/config/cc_growpart.py
@@ -64,10 +64,16 @@ growpart is::
ignore_growroot_disabled: <true/false>
"""
+import base64
+import copy
+import json
import os
import os.path
import re
import stat
+from contextlib import suppress
+from pathlib import Path
+from typing import Tuple
from cloudinit import log as logging
from cloudinit import subp, temp_utils, util
@@ -81,6 +87,8 @@ DEFAULT_CONFIG = {
"ignore_growroot_disabled": False,
}
+KEYDATA_PATH = Path("/cc_growpart_keydata")
+
class RESIZE(object):
SKIPPED = "SKIPPED"
@@ -289,10 +297,128 @@ def devent2dev(devent):
return dev
+def get_mapped_device(blockdev):
+ """Returns underlying block device for a mapped device.
+
+ If it is mapped, blockdev will usually take the form of
+ /dev/mapper/some_name
+
+ If blockdev is a symlink pointing to a /dev/dm-* device, return
+ the device pointed to. Otherwise, return None.
+ """
+ realpath = os.path.realpath(blockdev)
+ if realpath.startswith("/dev/dm-"):
+ LOG.debug("%s is a mapped device pointing to %s", blockdev, realpath)
+ return realpath
+ return None
+
+
+def is_encrypted(blockdev, partition) -> bool:
+ """
+ Check if a device is an encrypted device. blockdev should have
+ a /dev/dm-* path whereas partition is something like /dev/sda1.
+ """
+ if not subp.which("cryptsetup"):
+ LOG.debug("cryptsetup not found. Assuming no encrypted partitions")
+ return False
+ try:
+ subp.subp(["cryptsetup", "status", blockdev])
+ except subp.ProcessExecutionError as e:
+ if e.exit_code == 4:
+ LOG.debug("Determined that %s is not encrypted", blockdev)
+ else:
+ LOG.warning(
+ "Received unexpected exit code %s from "
+ "cryptsetup status. Assuming no encrypted partitions.",
+ e.exit_code,
+ )
+ return False
+ with suppress(subp.ProcessExecutionError):
+ subp.subp(["cryptsetup", "isLuks", partition])
+ LOG.debug("Determined that %s is encrypted", blockdev)
+ return True
+ return False
+
+
+def get_underlying_partition(blockdev):
+ command = ["dmsetup", "deps", "--options=devname", blockdev]
+ dep: str = subp.subp(command)[0] # type: ignore
+ # Returned result should look something like:
+ # 1 dependencies : (vdb1)
+ if not dep.startswith("1 depend"):
+ raise RuntimeError(
+ f"Expecting '1 dependencies' from 'dmsetup'. Received: {dep}"
+ )
+ try:
+ return f'/dev/{dep.split(": (")[1].split(")")[0]}'
+ except IndexError as e:
+ raise RuntimeError(
+ f"Ran `{command}`, but received unexpected stdout: `{dep}`"
+ ) from e
+
+
+def resize_encrypted(blockdev, partition) -> Tuple[str, str]:
+ """Use 'cryptsetup resize' to resize LUKS volume.
+
+ The loaded keyfile is json formatted with 'key' and 'slot' keys.
+ key is base64 encoded. Example:
+ {"key":"XFmCwX2FHIQp0LBWaLEMiHIyfxt1SGm16VvUAVledlY=","slot":5}
+ """
+ if not KEYDATA_PATH.exists():
+ return (RESIZE.SKIPPED, "No encryption keyfile found")
+ try:
+ with KEYDATA_PATH.open() as f:
+ keydata = json.load(f)
+ key = keydata["key"]
+ decoded_key = base64.b64decode(key)
+ slot = keydata["slot"]
+ except Exception as e:
+ raise RuntimeError(
+ "Could not load encryption key. This is expected if "
+ "the volume has been previously resized."
+ ) from e
+
+ try:
+ subp.subp(
+ ["cryptsetup", "--key-file", "-", "resize", blockdev],
+ data=decoded_key,
+ )
+ finally:
+ try:
+ subp.subp(
+ [
+ "cryptsetup",
+ "luksKillSlot",
+ "--batch-mode",
+ partition,
+ str(slot),
+ ]
+ )
+ except subp.ProcessExecutionError as e:
+ LOG.warning(
+ "Failed to kill luks slot after resizing encrypted volume: %s",
+ e,
+ )
+ try:
+ KEYDATA_PATH.unlink()
+ except Exception:
+ util.logexc(
+ LOG, "Failed to remove keyfile after resizing encrypted volume"
+ )
+
+ return (
+ RESIZE.CHANGED,
+ f"Successfully resized encrypted volume '{blockdev}'",
+ )
+
+
def resize_devices(resizer, devices):
# returns a tuple of tuples containing (entry-in-devices, action, message)
+ devices = copy.copy(devices)
info = []
- for devent in devices:
+
+ while devices:
+ devent = devices.pop(0)
try:
blockdev = devent2dev(devent)
except ValueError as e:
@@ -329,6 +455,49 @@ def resize_devices(resizer, devices):
)
continue
+ underlying_blockdev = get_mapped_device(blockdev)
+ if underlying_blockdev:
+ try:
+ # We need to resize the underlying partition first
+ partition = get_underlying_partition(blockdev)
+ if is_encrypted(underlying_blockdev, partition):
+ if partition not in [x[0] for x in info]:
+ # We shouldn't attempt to resize this mapped partition
+ # until the underlying partition is resized, so re-add
+ # our device to the beginning of the list we're
+ # iterating over, then add our underlying partition
+ # so it can get processed first
+ devices.insert(0, devent)
+ devices.insert(0, partition)
+ continue
+ status, message = resize_encrypted(blockdev, partition)
+ info.append(
+ (
+ devent,
+ status,
+ message,
+ )
+ )
+ else:
+ info.append(
+ (
+ devent,
+ RESIZE.SKIPPED,
+ f"Resizing mapped device ({blockdev}) skipped "
+ "as it is not encrypted.",
+ )
+ )
+ except Exception as e:
+ info.append(
+ (
+ devent,
+ RESIZE.FAILED,
+ f"Resizing encrypted device ({blockdev}) failed: {e}",
+ )
+ )
+ # At this point, we WON'T resize a non-encrypted mapped device
+ # though we should probably grow the ability to
+ continue
try:
(disk, ptnum) = device_part_info(blockdev)
except (TypeError, ValueError) as e:
diff --git a/test-requirements.txt b/test-requirements.txt
index 06dfbbec..7160416a 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -2,6 +2,7 @@
httpretty>=0.7.1
pytest
pytest-cov
+pytest-mock
# Only really needed on older versions of python
setuptools
diff --git a/tests/unittests/config/test_cc_growpart.py b/tests/unittests/config/test_cc_growpart.py
index ba66f136..7d4e2629 100644
--- a/tests/unittests/config/test_cc_growpart.py
+++ b/tests/unittests/config/test_cc_growpart.py
@@ -8,6 +8,7 @@ import shutil
import stat
import unittest
from contextlib import ExitStack
+from itertools import chain
from unittest import mock
from cloudinit import cloud, subp, temp_utils
@@ -342,6 +343,233 @@ class TestResize(unittest.TestCase):
os.stat = real_stat
+class TestEncrypted:
+ """Attempt end-to-end scenarios using encrypted devices.
+
+ Things are mocked such that:
+ - "/fake_encrypted" is mounted onto "/dev/mapper/fake"
+ - "/dev/mapper/fake" is a LUKS device and symlinked to /dev/dm-1
+ - The partition backing "/dev/mapper/fake" is "/dev/vdx1"
+ - "/" is not encrypted and mounted onto "/dev/vdz1"
+
+ Note that we don't (yet) support non-encrypted mapped drives, such
+ as LVM volumes. If our mount point is /dev/mapper/*, then we will
+ not resize it if it is not encrypted.
+ """
+
+ def _subp_side_effect(self, value, good=True, **kwargs):
+ if value[0] == "dmsetup":
+ return ("1 dependencies : (vdx1)",)
+ return mock.Mock()
+
+ def _device_part_info_side_effect(self, value):
+ if value.startswith("/dev/mapper/"):
+ raise TypeError(f"{value} not a partition")
+ return (1024, 1024)
+
+ def _devent2dev_side_effect(self, value):
+ if value == "/fake_encrypted":
+ return "/dev/mapper/fake"
+ elif value == "/":
+ return "/dev/vdz"
+ elif value.startswith("/dev"):
+ return value
+ raise Exception(f"unexpected value {value}")
+
+ def _realpath_side_effect(self, value):
+ return "/dev/dm-1" if value.startswith("/dev/mapper") else value
+
+ def assert_resize_and_cleanup(self):
+ all_subp_args = list(
+ chain(*[args[0][0] for args in self.m_subp.call_args_list])
+ )
+ assert "resize" in all_subp_args
+ assert "luksKillSlot" in all_subp_args
+ self.m_unlink.assert_called_once()
+
+ def assert_no_resize_or_cleanup(self):
+ all_subp_args = list(
+ chain(*[args[0][0] for args in self.m_subp.call_args_list])
+ )
+ assert "resize" not in all_subp_args
+ assert "luksKillSlot" not in all_subp_args
+ self.m_unlink.assert_not_called()
+
+ @pytest.fixture
+ def common_mocks(self, mocker):
+ # These are all "happy path" mocks which will get overridden
+ # when needed
+ mocker.patch(
+ "cloudinit.config.cc_growpart.device_part_info",
+ side_effect=self._device_part_info_side_effect,
+ )
+ mocker.patch("os.stat")
+ mocker.patch("stat.S_ISBLK")
+ mocker.patch("stat.S_ISCHR")
+ mocker.patch(
+ "cloudinit.config.cc_growpart.devent2dev",
+ side_effect=self._devent2dev_side_effect,
+ )
+ mocker.patch(
+ "os.path.realpath", side_effect=self._realpath_side_effect
+ )
+ # Only place subp.which is used in cc_growpart is for cryptsetup
+ mocker.patch(
+ "cloudinit.config.cc_growpart.subp.which",
+ return_value="/usr/sbin/cryptsetup",
+ )
+ self.m_subp = mocker.patch(
+ "cloudinit.config.cc_growpart.subp.subp",
+ side_effect=self._subp_side_effect,
+ )
+ mocker.patch(
+ "pathlib.Path.open",
+ new_callable=mock.mock_open,
+ read_data=(
+ '{"key":"XFmCwX2FHIQp0LBWaLEMiHIyfxt1SGm16VvUAVledlY=",'
+ '"slot":5}'
+ ),
+ )
+ mocker.patch("pathlib.Path.exists", return_value=True)
+ self.m_unlink = mocker.patch("pathlib.Path.unlink", autospec=True)
+
+ self.resizer = mock.Mock()
+ self.resizer.resize = mock.Mock(return_value=(1024, 1024))
+
+ def test_resize_when_encrypted(self, common_mocks, caplog):
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 2
+ assert info[0][0] == "/dev/vdx1"
+ assert info[0][2].startswith("no change necessary")
+ assert info[1][0] == "/fake_encrypted"
+ assert (
+ info[1][2]
+ == "Successfully resized encrypted volume '/dev/mapper/fake'"
+ )
+ assert (
+ "/dev/mapper/fake is a mapped device pointing to /dev/dm-1"
+ in caplog.text
+ )
+ assert "Determined that /dev/dm-1 is encrypted" in caplog.text
+
+ self.assert_resize_and_cleanup()
+
+ def test_resize_when_unencrypted(self, common_mocks):
+ info = cc_growpart.resize_devices(self.resizer, ["/"])
+ assert len(info) == 1
+ assert info[0][0] == "/"
+ assert "encrypted" not in info[0][2]
+ self.assert_no_resize_or_cleanup()
+
+ def test_encrypted_but_cryptsetup_not_found(
+ self, common_mocks, mocker, caplog
+ ):
+ mocker.patch(
+ "cloudinit.config.cc_growpart.subp.which",
+ return_value=None,
+ )
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+
+ assert len(info) == 1
+ assert "skipped as it is not encrypted" in info[0][2]
+ assert "cryptsetup not found" in caplog.text
+ self.assert_no_resize_or_cleanup()
+
+ def test_dmsetup_not_found(self, common_mocks, mocker, caplog):
+ def _subp_side_effect(value, **kwargs):
+ if value[0] == "dmsetup":
+ raise subp.ProcessExecutionError()
+
+ mocker.patch(
+ "cloudinit.config.cc_growpart.subp.subp",
+ side_effect=_subp_side_effect,
+ )
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 1
+ assert info[0][0] == "/fake_encrypted"
+ assert info[0][1] == "FAILED"
+ assert (
+ "Resizing encrypted device (/dev/mapper/fake) failed" in info[0][2]
+ )
+ self.assert_no_resize_or_cleanup()
+
+ def test_unparsable_dmsetup(self, common_mocks, mocker, caplog):
+ def _subp_side_effect(value, **kwargs):
+ if value[0] == "dmsetup":
+ return ("2 dependencies",)
+ return mock.Mock()
+
+ mocker.patch(
+ "cloudinit.config.cc_growpart.subp.subp",
+ side_effect=_subp_side_effect,
+ )
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 1
+ assert info[0][0] == "/fake_encrypted"
+ assert info[0][1] == "FAILED"
+ assert (
+ "Resizing encrypted device (/dev/mapper/fake) failed" in info[0][2]
+ )
+ self.assert_no_resize_or_cleanup()
+
+ def test_missing_keydata(self, common_mocks, mocker, caplog):
+ # Note that this will be standard behavior after first boot
+ # on a system with an encrypted root partition
+ mocker.patch("pathlib.Path.open", side_effect=FileNotFoundError())
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 2
+ assert info[0][0] == "/dev/vdx1"
+ assert info[0][2].startswith("no change necessary")
+ assert info[1][0] == "/fake_encrypted"
+ assert info[1][1] == "FAILED"
+ assert (
+ info[1][2]
+ == "Resizing encrypted device (/dev/mapper/fake) failed: Could "
+ "not load encryption key. This is expected if the volume has "
+ "been previously resized."
+ )
+ self.assert_no_resize_or_cleanup()
+
+ def test_resize_failed(self, common_mocks, mocker, caplog):
+ def _subp_side_effect(value, **kwargs):
+ if value[0] == "dmsetup":
+ return ("1 dependencies : (vdx1)",)
+ elif value[0] == "cryptsetup" and "resize" in value:
+ raise subp.ProcessExecutionError()
+ return mock.Mock()
+
+ self.m_subp = mocker.patch(
+ "cloudinit.config.cc_growpart.subp.subp",
+ side_effect=_subp_side_effect,
+ )
+
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 2
+ assert info[0][0] == "/dev/vdx1"
+ assert info[0][2].startswith("no change necessary")
+ assert info[1][0] == "/fake_encrypted"
+ assert info[1][1] == "FAILED"
+ assert (
+ "Resizing encrypted device (/dev/mapper/fake) failed" in info[1][2]
+ )
+ # Assert we still cleanup
+ all_subp_args = list(
+ chain(*[args[0][0] for args in self.m_subp.call_args_list])
+ )
+ assert "luksKillSlot" in all_subp_args
+ self.m_unlink.assert_called_once()
+
+ def test_resize_skipped(self, common_mocks, mocker, caplog):
+ mocker.patch("pathlib.Path.exists", return_value=False)
+ info = cc_growpart.resize_devices(self.resizer, ["/fake_encrypted"])
+ assert len(info) == 2
+ assert info[1] == (
+ "/fake_encrypted",
+ "SKIPPED",
+ "No encryption keyfile found",
+ )
+
+
def simple_device_part_info(devpath):
# simple stupid return (/dev/vda, 1) for /dev/vda
ret = re.search("([^0-9]*)([0-9]*)$", devpath)
diff --git a/tox.ini b/tox.ini
index c494cb94..04a206f2 100644
--- a/tox.ini
+++ b/tox.ini
@@ -108,6 +108,7 @@ deps =
# test-requirements
pytest==3.3.2
pytest-cov==2.5.1
+ pytest-mock==1.7.1
# Needed by pytest and default causes failures
attrs==17.4.0
--
2.39.1

@ -1,17 +1,10 @@
From df1c0f391537071c34652ee6df9bff87e5aea230 Mon Sep 17 00:00:00 2001
From 14d1952c17637b80923d1bfaf3b6b5f8cf032147 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Wed, 14 Dec 2022 09:20:47 +0100
Date: Wed, 14 Dec 2022 09:31:51 +0100
Subject: [PATCH] Ensure network ready before cloud-init service runs on RHEL
(#1893)
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 87: Ensure network ready before cloud-init service runs on RHEL (#1893)
RH-Bugzilla: 2151861
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
RH-Acked-by: Camilla Conte <cconte@redhat.com>
RH-Commit: [1/1] 5bb5f6f94a205854633fb1606ccc68e838c2030d
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2151861
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2152100
commit 6e725f36647407d201af0603d7db11fc96a93d4d
Author: James Falcon <james.falcon@canonical.com>

@ -0,0 +1,135 @@
From 53e3f8ab9008fec8400f96918c2129f7defe6a70 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Fri, 10 Jun 2022 20:51:55 +0200
Subject: [PATCH 1/3] Honor system locale for RHEL (#1355)
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 29: Honor system locale for RHEL (#1355)
RH-Commit: [1/1] d571126fe6add8dc34a22c869d4e1a07a7373d8d (eesposit/cloud-init-centos-)
RH-Bugzilla: 2061604
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
commit 58da7d856274e9ca2b507128d6f186e0e6abfe06
Author: Wei Shi <wshi@redhat.com>
Date: Wed Mar 30 23:55:30 2022 +0800
Honor system locale for RHEL (#1355)
Make sure to use system locale as default on RHEL if locale is not
set in cloud-config.
RHEL has a pre-installed cloud image using C.UTF-8 for system locale
just like ubuntu-minimal cloud image, without this patch, locale
module will set it to en_US.UTF-8 from ds default value during config
stage.
Authored-by: Wei Shi <shi2wei3@hotmail.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
cloudinit/distros/rhel.py | 32 +++++++++++++++++++++++++
tests/unittests/distros/test_generic.py | 10 ++++----
tools/.github-cla-signers | 1 +
3 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/cloudinit/distros/rhel.py b/cloudinit/distros/rhel.py
index 84744ece..320f4ba1 100644
--- a/cloudinit/distros/rhel.py
+++ b/cloudinit/distros/rhel.py
@@ -7,6 +7,7 @@
# Author: Joshua Harlow <harlowja@yahoo-inc.com>
#
# This file is part of cloud-init. See LICENSE file for license information.
+import os
from cloudinit import distros, helpers
from cloudinit import log as logging
@@ -57,11 +58,25 @@ class Distro(distros.Distro):
# should only happen say once per instance...)
self._runner = helpers.Runners(paths)
self.osfamily = "redhat"
+ self.default_locale = "en_US.UTF-8"
+ self.system_locale = None
cfg["ssh_svcname"] = "sshd"
def install_packages(self, pkglist):
self.package_command("install", pkgs=pkglist)
+ def get_locale(self):
+ """Return the default locale if set, else use system locale"""
+
+ # read system locale value
+ if not self.system_locale:
+ self.system_locale = self._read_system_locale()
+
+ # Return system_locale setting if valid, else use default locale
+ return (
+ self.system_locale if self.system_locale else self.default_locale
+ )
+
def apply_locale(self, locale, out_fn=None):
if self.uses_systemd():
if not out_fn:
@@ -75,6 +90,23 @@ class Distro(distros.Distro):
}
rhel_util.update_sysconfig_file(out_fn, locale_cfg)
+ def _read_system_locale(self, keyname="LANG"):
+ """Read system default locale setting, if present"""
+ if self.uses_systemd():
+ locale_fn = self.systemd_locale_conf_fn
+ else:
+ locale_fn = self.locale_conf_fn
+
+ if not locale_fn:
+ raise ValueError("Invalid path: %s" % locale_fn)
+
+ if os.path.exists(locale_fn):
+ (_exists, contents) = rhel_util.read_sysconfig_file(locale_fn)
+ if keyname in contents:
+ return contents[keyname]
+ else:
+ return None
+
def _write_hostname(self, hostname, filename):
# systemd will never update previous-hostname for us, so
# we need to do it ourselves
diff --git a/tests/unittests/distros/test_generic.py b/tests/unittests/distros/test_generic.py
index 93c5395c..fedc7300 100644
--- a/tests/unittests/distros/test_generic.py
+++ b/tests/unittests/distros/test_generic.py
@@ -187,12 +187,14 @@ class TestGenericDistro(helpers.FilesystemMockingTestCase):
locale = d.get_locale()
self.assertEqual("C.UTF-8", locale)
- def test_get_locale_rhel(self):
- """Test rhel distro returns NotImplementedError exception"""
+ @mock.patch("cloudinit.distros.rhel.Distro._read_system_locale")
+ def test_get_locale_rhel(self, m_locale):
+ """Test rhel distro returns locale set to C.UTF-8"""
+ m_locale.return_value = "C.UTF-8"
cls = distros.fetch("rhel")
d = cls("rhel", {}, None)
- with self.assertRaises(NotImplementedError):
- d.get_locale()
+ locale = d.get_locale()
+ self.assertEqual("C.UTF-8", locale)
def test_expire_passwd_uses_chpasswd(self):
"""Test ubuntu.expire_passwd uses the passwd command."""
diff --git a/tools/.github-cla-signers b/tools/.github-cla-signers
index 9f71ea0c..9eb2ae38 100644
--- a/tools/.github-cla-signers
+++ b/tools/.github-cla-signers
@@ -70,6 +70,7 @@ renanrodrigo
rhansen
riedel
sarahwzadara
+shi2wei3
slingamn
slyon
smoser
--
2.35.1

@ -1,15 +1,14 @@
From d43f0d93386f123892451d923c2b3c6fe7130c39 Mon Sep 17 00:00:00 2001
From 0ae221fe8f29555370520d05f6558eb75fe3cd42 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Thu, 19 May 2022 11:38:22 +0200
Subject: [PATCH 4/4] Remove rhel specific files
Date: Thu, 5 May 2022 11:31:33 +0200
Subject: [PATCH 5/5] Remove rhel specific files
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 65: Align rhel custom files with upstream (#1431)
RH-Commit: [2/2] 5e31f0bcb500682e7746ccbd2e628c2ef339d6c6
RH-Bugzilla: 2082071
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
RH-Acked-by: Eduardo Otubo <otubo@redhat.com>
RH-MergeRequest: 26: Align rhel custom files with upstream (#1431)
RH-Commit: [2/2] 0b9687ebab4f4039d5bbe05c00806ec7923e898d (eesposit/cloud-init-centos-)
RH-Bugzilla: 2088448
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
Remove all files in rhel/ directory and related commands that copy
and replace them with the generated ones.
@ -26,7 +25,6 @@ X-downstream-only: yes
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
redhat/cloud-init.spec.template | 21 ++------
rhel/README.rhel | 5 --
rhel/cloud-init-tmpfiles.conf | 1 -
rhel/cloud.cfg | 69 ---------------------------
@ -48,7 +46,6 @@ Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
delete mode 100644 rhel/systemd/cloud-init.service
delete mode 100644 rhel/systemd/cloud-init.target
diff --git a/rhel/README.rhel b/rhel/README.rhel
deleted file mode 100644
index aa29630d..00000000
@ -69,7 +66,7 @@ index 0c6d2a3b..00000000
-d /run/cloud-init 0700 root root - -
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
deleted file mode 100644
index cbee197a..00000000
index 75d5c84b..00000000
--- a/rhel/cloud.cfg
+++ /dev/null
@@ -1,69 +0,0 @@
@ -79,7 +76,7 @@ index cbee197a..00000000
-disable_root: 1
-ssh_pwauth: 0
-
-mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
-mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service,_netdev', '0', '2']
-resize_rootfs_tmp: /dev
-ssh_deletekeys: 1
-ssh_genkeytypes: ['rsa', 'ecdsa', 'ed25519']
@ -369,5 +366,5 @@ index 3c377eaa..a9132d2c 100755
"console_scripts": [
"cloud-init = cloudinit.cmd.main:main",
--
2.35.3
2.31.1

@ -1,18 +1,19 @@
From f1836e78d20ef34b05b6aba002fc10a97eceb454 Mon Sep 17 00:00:00 2001
From 35bd50e66f636a3f3923b6980bdee3ba33f7457d Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Mon, 8 Aug 2022 10:08:50 +0200
Subject: [PATCH 1/2] Revert "Add native NetworkManager support (#1224)"
Date: Mon, 8 Aug 2022 10:01:16 +0200
Subject: [PATCH 1/3] Revert "Add native NetworkManager support (#1224)"
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 81: Revert "Use Network-Manager and Netplan as default renderers for RHEL and Fedora (#1465)"
RH-Commit: [1/2] 5b3e51502a89c2dcfbc97dc08a86b792454fedd3
RH-Bugzilla: 2107464 2110066 2117526 2104393 2098624
RH-MergeRequest: 31: Revert "Revert "Setting highest autoconnect priority for network-scripts""
RH-Commit: [1/3] 38dcbc9ec19412601e96305fcac09642c89d73b8 (eesposit/cloud-init-centos-)
RH-Bugzilla: 2107463 2104389 2117532 2098501
RH-Acked-by: Eduardo Otubo <otubo@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
NM is not yet stable, so we don't want to support it for now.
This reverts commit 0d93e53fd05c44b62e3456b7580c9de8135e6b5a.
NM is still not stable, revert it for now.
This reverts commit 588deeb5b3f87ffe40d9ecaf6da3639176f806c4.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
@ -540,7 +541,7 @@ index 7edc34b5..c755f04c 100644
"netbsd",
"openbsd",
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index c3b0c795..362e8d19 100644
index dc401d78..d8c53312 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -5,6 +5,8 @@ import io
@ -2262,5 +2263,5 @@ index 4525c49c..3c29e2f7 100644
NETWORKD_BRING_DOWN_CALL_LIST = [
--
2.27.0
2.31.1

@ -0,0 +1,37 @@
From c843f5cf7152846da3422185a6ad344dd47604e6 Mon Sep 17 00:00:00 2001
From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Date: Mon, 8 Aug 2022 10:02:48 +0200
Subject: [PATCH 3/3] Revert "Revert "Setting highest autoconnect priority for
network-scripts""
RH-Author: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RH-MergeRequest: 31: Revert "Revert "Setting highest autoconnect priority for network-scripts""
RH-Commit: [3/3] 09b83fb559675a2fcd4fc5394b251f034b92c1bc (eesposit/cloud-init-centos-)
RH-Bugzilla: 2107463 2104389 2117532 2098501
RH-Acked-by: Eduardo Otubo <otubo@redhat.com>
RH-Acked-by: Vitaly Kuznetsov <vkuznets@redhat.com>
RH-Acked-by: Mohamed Gamal Morsy <mmorsy@redhat.com>
Reverting NM means we also need to restore this patch
This reverts commit 0eba5c6194017ef493a735cb24757c57d8af7b59.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
cloudinit/net/sysconfig.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index d8c53312..b50035b5 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -337,6 +337,7 @@ class Renderer(renderer.Renderer):