cloud-init/SOURCES/0001-Add-initial-redhat-setup.patch

From 4b84d29211b7b2121afe9045c71ded5381536d8b Mon Sep 17 00:00:00 2001
From: Eduardo Otubo <otubo@redhat.com>
Date: Fri, 7 May 2021 13:36:03 +0200
Subject: Add initial redhat setup

Merged patches (RHEL-9/21.1):
- 5688a1d0 Removing python-nose and python-tox as dependency
- 237d57f9 Removing mock dependency
- d1c2f496 Removing python-jsonschema dependency
- 0d1cd14c Don't override default network configuration

Merged patches (21.1):
- 915d30ad Change gating file to correct rhel version
- 311f318d Removing net-tools dependency
- 74731806 Adding man pages to Red Hat spec file
- 758d333d Removing blocking test from yaml configuration file
- c7e7c59c Changing permission of cloud-init-generator to 755
- 8b85abbb Installing man pages in the correct place with correct permissions
- c6808d8d Fix unit failure of cloud-final.service if NetworkManager was not present.
- 11866ef6 Report full specific version with "cloud-init --version"

Rebase notes (18.5):
- added bash_completition file
- added cloud-id file

Merged patches (20.3):
- 01900d0 changing ds-identify patch from /usr/lib to /usr/libexec
- 7f47ca3 Render the generator from template instead of cp

Merged patches (19.4):
- 4ab5a61 Fix for network configuration not persisting after reboot
- 84cf125 Removing cloud-user from wheel
- 31290ab Adding gating tests for Azure, ESXi and AWS

Merged patches (18.5):
- 2d6b469 add power-state-change module to cloud_final_modules
- 764159f Adding systemd mount options to wait for cloud-init
- da4d99e Adding disk_setup to rhel/cloud.cfg
- f5c6832 Enable cloud-init by default on vmware

Conflicts:
cloudinit/config/cc_chef.py:
 - Updated header documentation text
 - Replacing double quotes by simple quotes

setup.py:
 - Adding missing cmdclass info

Signed-off-by: Eduardo Otubo <otubo@redhat.com>

Changes:
- move redhat to .distro to use new build script structure
- Fixing changelog for RHEL 9

Merged patches (21.1):
- 69bd7f71 DataSourceAzure.py: use hostnamectl to set hostname
- 0407867e Remove race condition between cloud-init and NetworkManager

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
---
 .distro/.gitignore                    |   1 +
 .distro/Makefile                      |  74 +++++
 .distro/Makefile.common               |  30 ++
 .distro/cloud-init-tmpfiles.conf      |   1 +
 .distro/cloud-init.spec.template      | 383 ++++++++++++++++++++++++++
 .distro/gating.yaml                   |   8 +
 .distro/rpmbuild/BUILD/.gitignore     |   3 +
 .distro/rpmbuild/RPMS/.gitignore      |   3 +
 .distro/rpmbuild/SOURCES/.gitignore   |   3 +
 .distro/rpmbuild/SPECS/.gitignore     |   3 +
 .distro/rpmbuild/SRPMS/.gitignore     |   3 +
 .distro/scripts/frh.py                |  27 ++
 .distro/scripts/git-backport-diff     | 327 ++++++++++++++++++++++
 .distro/scripts/git-compile-check     | 215 +++++++++++++++
 .distro/scripts/process-patches.sh    |  88 ++++++
 .distro/scripts/tarball_checksum.sh   |   3 +
 .gitignore                            |   1 +
 cloudinit/config/cc_chef.py           |  67 ++++-
 cloudinit/settings.py                 |   7 +-
 cloudinit/sources/DataSourceAzure.py  |   2 +-
 requirements.txt                      |   3 -
 rhel/README.rhel                      |   5 +
 rhel/cloud-init-tmpfiles.conf         |   1 +
 rhel/cloud.cfg                        |  69 +++++
 rhel/systemd/cloud-config.service     |  18 ++
 rhel/systemd/cloud-config.target      |  11 +
 rhel/systemd/cloud-final.service      |  24 ++
 rhel/systemd/cloud-init-local.service |  31 +++
 rhel/systemd/cloud-init.service       |  26 ++
 rhel/systemd/cloud-init.target        |   7 +
 setup.py                              |  23 +-
 tools/read-version                    |  28 +-
 32 files changed, 1441 insertions(+), 54 deletions(-)
 create mode 100644 .distro/.gitignore
 create mode 100644 .distro/Makefile
 create mode 100644 .distro/Makefile.common
 create mode 100644 .distro/cloud-init-tmpfiles.conf
 create mode 100644 .distro/cloud-init.spec.template
 create mode 100644 .distro/gating.yaml
 create mode 100644 .distro/rpmbuild/BUILD/.gitignore
 create mode 100644 .distro/rpmbuild/RPMS/.gitignore
 create mode 100644 .distro/rpmbuild/SOURCES/.gitignore
 create mode 100644 .distro/rpmbuild/SPECS/.gitignore
 create mode 100644 .distro/rpmbuild/SRPMS/.gitignore
 create mode 100755 .distro/scripts/frh.py
 create mode 100755 .distro/scripts/git-backport-diff
 create mode 100755 .distro/scripts/git-compile-check
 create mode 100755 .distro/scripts/process-patches.sh
 create mode 100755 .distro/scripts/tarball_checksum.sh
 create mode 100644 rhel/README.rhel
 create mode 100644 rhel/cloud-init-tmpfiles.conf
 create mode 100644 rhel/cloud.cfg
 create mode 100644 rhel/systemd/cloud-config.service
 create mode 100644 rhel/systemd/cloud-config.target
 create mode 100644 rhel/systemd/cloud-final.service
 create mode 100644 rhel/systemd/cloud-init-local.service
 create mode 100644 rhel/systemd/cloud-init.service
 create mode 100644 rhel/systemd/cloud-init.target

diff --git a/cloudinit/config/cc_chef.py b/cloudinit/config/cc_chef.py
index aaf71366..97ef649a 100644
--- a/cloudinit/config/cc_chef.py
+++ b/cloudinit/config/cc_chef.py
@@ -6,7 +6,70 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
-"""Chef: module that configures, starts and installs chef."""
+"""
+Chef
+----
+**Summary:** module that configures, starts and installs chef.
+
+This module enables chef to be installed (from packages or
+from gems, or from omnibus). Before this occurs chef configurations are
+written to disk (validation.pem, client.pem, firstboot.json, client.rb),
+and needed chef folders/directories are created (/etc/chef and /var/log/chef
+and so-on). Then once installing proceeds correctly if configured chef will
+be started (in daemon mode or in non-daemon mode) and then once that has
+finished (if ran in non-daemon mode this will be when chef finishes
+converging, if ran in daemon mode then no further actions are possible since
+chef will have forked into its own process) then a post run function can
+run that can do finishing activities (such as removing the validation pem
+file).
+
+**Internal name:** ``cc_chef``
+
+**Module frequency:** per always
+
+**Supported distros:** all
+
+**Config keys**::
+
+    chef:
+       directories: (defaulting to /etc/chef, /var/log/chef, /var/lib/chef,
+                     /var/cache/chef, /var/backups/chef, /run/chef)
+       validation_cert: (optional string to be written to file validation_key)
+                        special value 'system' means set use existing file
+       validation_key: (optional the path for validation_cert. default
+                        /etc/chef/validation.pem)
+       firstboot_path: (path to write run_list and initial_attributes keys that
+                        should also be present in this configuration, defaults
+                        to /etc/chef/firstboot.json)
+       exec: boolean to run or not run chef (defaults to false, unless
+                                             a gem installed is requested
+                                             where this will then default
+                                             to true)
+
+    chef.rb template keys (if falsey, then will be skipped and not
+                           written to /etc/chef/client.rb)
+
+    chef:
+      client_key:
+      encrypted_data_bag_secret:
+      environment:
+      file_backup_path:
+      file_cache_path:
+      json_attribs:
+      log_level:
+      log_location:
+      node_name:
+      omnibus_url:
+      omnibus_url_retries:
+      omnibus_version:
+      pid_file:
+      server_url:
+      show_time:
+      ssl_verify_mode:
+      validation_cert:
+      validation_key:
+      validation_name:
+"""
 
 import itertools
 import json
@@ -31,7 +94,7 @@ CHEF_DIRS = tuple([
     '/var/lib/chef',
     '/var/cache/chef',
     '/var/backups/chef',
-    '/var/run/chef',
+    '/run/chef',
 ])
 REQUIRED_CHEF_DIRS = tuple([
     '/etc/chef',
diff --git a/cloudinit/settings.py b/cloudinit/settings.py
index 91e1bfe7..e690c0fd 100644
--- a/cloudinit/settings.py
+++ b/cloudinit/settings.py
@@ -47,13 +47,16 @@ CFG_BUILTIN = {
     ],
     'def_log_file': '/var/log/cloud-init.log',
     'log_cfgs': [],
-    'syslog_fix_perms': ['syslog:adm', 'root:adm', 'root:wheel', 'root:root'],
+    'mount_default_fields': [None, None, 'auto', 'defaults,nofail', '0', '2'],
+    'ssh_deletekeys': False,
+    'ssh_genkeytypes': [],
+    'syslog_fix_perms': [],
     'system_info': {
         'paths': {
             'cloud_dir': '/var/lib/cloud',
             'templates_dir': '/etc/cloud/templates/',
         },
-        'distro': 'ubuntu',
+        'distro': 'rhel',
         'network': {'renderers': None},
     },
     'vendor_data': {'enabled': True, 'prefix': []},
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index cee630f7..553b5a7e 100755
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -296,7 +296,7 @@ def get_hostname(hostname_command='hostname'):
 
 
 def set_hostname(hostname, hostname_command='hostname'):
-    subp.subp([hostname_command, hostname])
+    util.subp(['hostnamectl', 'set-hostname', str(hostname)])
 
 
 @azure_ds_telemetry_reporter
diff --git a/requirements.txt b/requirements.txt
index 5817da3b..5b8becd7 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -29,6 +29,3 @@ requests
 
 # For patching pieces of cloud-config together
 jsonpatch
-
-# For validating cloud-config sections per schema definitions
-jsonschema
diff --git a/rhel/README.rhel b/rhel/README.rhel
new file mode 100644
index 00000000..aa29630d
--- /dev/null
+++ b/rhel/README.rhel
@@ -0,0 +1,5 @@
+The following cloud-init modules are currently unsupported on this OS:
+ - apt_update_upgrade ('apt_update', 'apt_upgrade', 'apt_mirror', 'apt_preserve_sources_list', 'apt_old_mirror', 'apt_sources', 'debconf_selections', 'packages' options)
+ - byobu ('byobu_by_default' option)
+ - chef
+ - grub_dpkg
diff --git a/rhel/cloud-init-tmpfiles.conf b/rhel/cloud-init-tmpfiles.conf
new file mode 100644
index 00000000..0c6d2a3b
--- /dev/null
+++ b/rhel/cloud-init-tmpfiles.conf
@@ -0,0 +1 @@
+d /run/cloud-init 0700 root root - -
diff --git a/rhel/cloud.cfg b/rhel/cloud.cfg
new file mode 100644
index 00000000..9ecba215
--- /dev/null
+++ b/rhel/cloud.cfg
@@ -0,0 +1,69 @@
+users:
+ - default
+
+disable_root: 1
+ssh_pwauth:   0
+
+mount_default_fields: [~, ~, 'auto', 'defaults,nofail,x-systemd.requires=cloud-init.service', '0', '2']
+resize_rootfs_tmp: /dev
+ssh_deletekeys:   1
+ssh_genkeytypes:  ~
+syslog_fix_perms: ~
+disable_vmware_customization: false
+
+cloud_init_modules:
+ - disk_setup
+ - migrator
+ - bootcmd
+ - write-files
+ - growpart
+ - resizefs
+ - set_hostname
+ - update_hostname
+ - update_etc_hosts
+ - rsyslog
+ - users-groups
+ - ssh
+
+cloud_config_modules:
+ - mounts
+ - locale
+ - set-passwords
+ - rh_subscription
+ - yum-add-repo
+ - package-update-upgrade-install
+ - timezone
+ - puppet
+ - chef
+ - salt-minion
+ - mcollective
+ - disable-ec2-metadata
+ - runcmd
+
+cloud_final_modules:
+ - rightscale_userdata
+ - scripts-per-once
+ - scripts-per-boot
+ - scripts-per-instance
+ - scripts-user
+ - ssh-authkey-fingerprints
+ - keys-to-console
+ - phone-home
+ - final-message
+ - power-state-change
+
+system_info:
+  default_user:
+    name: cloud-user
+    lock_passwd: true
+    gecos: Cloud User
+    groups: [adm, systemd-journal]
+    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
+    shell: /bin/bash
+  distro: rhel
+  paths:
+    cloud_dir: /var/lib/cloud
+    templates_dir: /etc/cloud/templates
+  ssh_svcname: sshd
+
+# vim:syntax=yaml
diff --git a/rhel/systemd/cloud-config.service b/rhel/systemd/cloud-config.service
new file mode 100644
index 00000000..f3dcd4be
--- /dev/null
+++ b/rhel/systemd/cloud-config.service
@@ -0,0 +1,18 @@
+[Unit]
+Description=Apply the settings specified in cloud-config
+After=network-online.target cloud-config.target
+Wants=network-online.target cloud-config.target
+ConditionPathExists=!/etc/cloud/cloud-init.disabled
+ConditionKernelCommandLine=!cloud-init=disabled
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/cloud-init modules --mode=config
+RemainAfterExit=yes
+TimeoutSec=0
+
+# Output needs to appear in instance console output
+StandardOutput=journal+console
+
+[Install]
+WantedBy=cloud-init.target
diff --git a/rhel/systemd/cloud-config.target b/rhel/systemd/cloud-config.target
new file mode 100644
index 00000000..ae9b7d02
--- /dev/null
+++ b/rhel/systemd/cloud-config.target
@@ -0,0 +1,11 @@
+# cloud-init normally emits a "cloud-config" upstart event to inform third
+# parties that cloud-config is available, which does us no good when we're
+# using systemd.  cloud-config.target serves as this synchronization point
+# instead.  Services that would "start on cloud-config" with upstart can
+# instead use "After=cloud-config.target" and "Wants=cloud-config.target"
+# as appropriate.
+
+[Unit]
+Description=Cloud-config availability
+Wants=cloud-init-local.service cloud-init.service
+After=cloud-init-local.service cloud-init.service
diff --git a/rhel/systemd/cloud-final.service b/rhel/systemd/cloud-final.service
new file mode 100644
index 00000000..e281c0cf
--- /dev/null
+++ b/rhel/systemd/cloud-final.service
@@ -0,0 +1,24 @@
+[Unit]
+Description=Execute cloud user/final scripts
+After=network-online.target cloud-config.service rc-local.service
+Wants=network-online.target cloud-config.service
+ConditionPathExists=!/etc/cloud/cloud-init.disabled
+ConditionKernelCommandLine=!cloud-init=disabled
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/cloud-init modules --mode=final
+RemainAfterExit=yes
+TimeoutSec=0
+KillMode=process
+# Restart NetworkManager if it is present and running.
+ExecStartPost=/bin/sh -c 'u=NetworkManager.service; \
+ out=$(systemctl show --property=SubState $u) || exit; \
+ [ "$out" = "SubState=running" ] || exit 0; \
+ systemctl reload-or-try-restart $u'
+
+# Output needs to appear in instance console output
+StandardOutput=journal+console
+
+[Install]
+WantedBy=cloud-init.target
diff --git a/rhel/systemd/cloud-init-local.service b/rhel/systemd/cloud-init-local.service
new file mode 100644
index 00000000..8f9f6c9f
--- /dev/null
+++ b/rhel/systemd/cloud-init-local.service
@@ -0,0 +1,31 @@
+[Unit]
+Description=Initial cloud-init job (pre-networking)
+DefaultDependencies=no
+Wants=network-pre.target
+After=systemd-remount-fs.service
+Requires=dbus.socket
+After=dbus.socket
+Before=NetworkManager.service network.service
+Before=network-pre.target
+Before=shutdown.target
+Before=firewalld.target
+Conflicts=shutdown.target
+RequiresMountsFor=/var/lib/cloud
+ConditionPathExists=!/etc/cloud/cloud-init.disabled
+ConditionKernelCommandLine=!cloud-init=disabled
+
+[Service]
+Type=oneshot
+ExecStartPre=/bin/mkdir -p /run/cloud-init
+ExecStartPre=/sbin/restorecon /run/cloud-init
+ExecStartPre=/usr/bin/touch /run/cloud-init/enabled
+ExecStart=/usr/bin/cloud-init init --local
+ExecStart=/bin/touch /run/cloud-init/network-config-ready
+RemainAfterExit=yes
+TimeoutSec=0
+
+# Output needs to appear in instance console output
+StandardOutput=journal+console
+
+[Install]
+WantedBy=cloud-init.target
diff --git a/rhel/systemd/cloud-init.service b/rhel/systemd/cloud-init.service
new file mode 100644
index 00000000..0b3d796d
--- /dev/null
+++ b/rhel/systemd/cloud-init.service
@@ -0,0 +1,26 @@
+[Unit]
+Description=Initial cloud-init job (metadata service crawler)
+Wants=cloud-init-local.service
+Wants=sshd-keygen.service
+Wants=sshd.service
+After=cloud-init-local.service
+After=NetworkManager.service network.service
+After=NetworkManager-wait-online.service
+Before=network-online.target
+Before=sshd-keygen.service
+Before=sshd.service
+Before=systemd-user-sessions.service
+ConditionPathExists=!/etc/cloud/cloud-init.disabled
+ConditionKernelCommandLine=!cloud-init=disabled
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/cloud-init init
+RemainAfterExit=yes
+TimeoutSec=0
+
+# Output needs to appear in instance console output
+StandardOutput=journal+console
+
+[Install]
+WantedBy=cloud-init.target
diff --git a/rhel/systemd/cloud-init.target b/rhel/systemd/cloud-init.target
new file mode 100644
index 00000000..083c3b6f
--- /dev/null
+++ b/rhel/systemd/cloud-init.target
@@ -0,0 +1,7 @@
+# cloud-init target is enabled by cloud-init-generator
+# To disable it you can either:
+#  a.) boot with kernel cmdline of 'cloud-init=disabled'
+#  b.) touch a file /etc/cloud/cloud-init.disabled
+[Unit]
+Description=Cloud-init target
+After=multi-user.target
diff --git a/setup.py b/setup.py
index cbacf48e..d5cd01a4 100755
--- a/setup.py
+++ b/setup.py
@@ -125,14 +125,6 @@ INITSYS_FILES = {
     'sysvinit_deb': [f for f in glob('sysvinit/debian/*') if is_f(f)],
     'sysvinit_openrc': [f for f in glob('sysvinit/gentoo/*') if is_f(f)],
     'sysvinit_suse': [f for f in glob('sysvinit/suse/*') if is_f(f)],
-    'systemd': [render_tmpl(f)
-                for f in (glob('systemd/*.tmpl') +
-                          glob('systemd/*.service') +
-                          glob('systemd/*.target'))
-                if (is_f(f) and not is_generator(f))],
-    'systemd.generators': [
-        render_tmpl(f, mode=0o755)
-        for f in glob('systemd/*') if is_f(f) and is_generator(f)],
     'upstart': [f for f in glob('upstart/*') if is_f(f)],
 }
 INITSYS_ROOTS = {
@@ -142,9 +134,6 @@ INITSYS_ROOTS = {
     'sysvinit_deb': 'etc/init.d',
     'sysvinit_openrc': 'etc/init.d',
     'sysvinit_suse': 'etc/init.d',
-    'systemd': pkg_config_read('systemd', 'systemdsystemunitdir'),
-    'systemd.generators': pkg_config_read('systemd',
-                                          'systemdsystemgeneratordir'),
     'upstart': 'etc/init/',
 }
 INITSYS_TYPES = sorted([f.partition(".")[0] for f in INITSYS_ROOTS.keys()])
@@ -245,14 +234,11 @@ if not in_virtualenv():
         INITSYS_ROOTS[k] = "/" + INITSYS_ROOTS[k]
 
 data_files = [
-    (ETC + '/cloud', [render_tmpl("config/cloud.cfg.tmpl")]),
+    (ETC + '/bash_completion.d', ['bash_completion/cloud-init']),
     (ETC + '/cloud/cloud.cfg.d', glob('config/cloud.cfg.d/*')),
     (ETC + '/cloud/templates', glob('templates/*')),
-    (USR_LIB_EXEC + '/cloud-init', ['tools/ds-identify',
-                                    'tools/uncloud-init',
+    (USR_LIB_EXEC + '/cloud-init', ['tools/uncloud-init',
                                     'tools/write-ssh-key-fingerprints']),
-    (USR + '/share/bash-completion/completions',
-     ['bash_completion/cloud-init']),
     (USR + '/share/doc/cloud-init', [f for f in glob('doc/*') if is_f(f)]),
     (USR + '/share/doc/cloud-init/examples',
         [f for f in glob('doc/examples/*') if is_f(f)]),
@@ -263,8 +249,7 @@ if not platform.system().endswith('BSD'):
     data_files.extend([
         (ETC + '/NetworkManager/dispatcher.d/',
          ['tools/hook-network-manager']),
-        (ETC + '/dhcp/dhclient-exit-hooks.d/', ['tools/hook-dhclient']),
-        (LIB + '/udev/rules.d', [f for f in glob('udev/*.rules')])
+        ('/usr/lib/udev/rules.d', [f for f in glob('udev/*.rules')])
     ])
 # Use a subclass for install that handles
 # adding on the right init system configuration files
@@ -286,8 +271,6 @@ setuptools.setup(
     scripts=['tools/cloud-init-per'],
     license='Dual-licensed under GPLv3 or Apache 2.0',
     data_files=data_files,
-    install_requires=requirements,
-    cmdclass=cmdclass,
     entry_points={
         'console_scripts': [
             'cloud-init = cloudinit.cmd.main:main',
diff --git a/tools/read-version b/tools/read-version
index 02c90643..79755f78 100755
--- a/tools/read-version
+++ b/tools/read-version
@@ -71,32 +71,8 @@ version_long = None
 is_release_branch_ci = (
     os.environ.get("TRAVIS_PULL_REQUEST_BRANCH", "").startswith("upstream/")
 )
-if is_gitdir(_tdir) and which("git") and not is_release_branch_ci:
-    flags = []
-    if use_tags:
-        flags = ['--tags']
-    cmd = ['git', 'describe', '--abbrev=8', '--match=[0-9]*'] + flags
-
-    try:
-        version = tiny_p(cmd).strip()
-    except RuntimeError:
-        version = None
-
-    if version is None or not version.startswith(src_version):
-        sys.stderr.write("git describe version (%s) differs from "
-                         "cloudinit.version (%s)\n" % (version, src_version))
-        sys.stderr.write(
-            "Please get the latest upstream tags.\n"
-            "As an example, this can be done with the following:\n"
-            "$ git remote add upstream https://git.launchpad.net/cloud-init\n"
-            "$ git fetch upstream --tags\n"
-        )
-        sys.exit(1)
-
-    version_long = tiny_p(cmd + ["--long"]).strip()
-else:
-    version = src_version
-    version_long = None
+version = src_version
+version_long = None
 
 # version is X.Y.Z[+xxx.gHASH]
 # version_long is None or X.Y.Z-xxx-gHASH
-- 
2.27.0