Compare commits

...

No commits in common. "c8" and "imports/c8-beta/oscap-anaconda-addon-1.1.1-7.el8" have entirely different histories.

18 changed files with 5886 additions and 2042 deletions

2
.gitignore vendored
View File

@ -1 +1 @@
SOURCES/oscap-anaconda-addon-1.2.1.tar.gz
SOURCES/oscap-anaconda-addon-1.1.1.tar.gz

View File

@ -0,0 +1 @@
789fbf8b6c8618619fb484a635c29d638202d015 SOURCES/oscap-anaconda-addon-1.1.1.tar.gz

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,25 @@
From 316d1fdd46e962c2eca32dde8b6d391286bd3d28 Mon Sep 17 00:00:00 2001
From: Masahiro Matsuya <mmatsuya@redhat.com>
Date: Fri, 7 Aug 2020 15:50:05 +0900
Subject: [PATCH] avoiding UnicodeDecodeError in decode(). Output more errors
which starts with E: oscap
---
org_fedora_oscap/common.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/org_fedora_oscap/common.py b/org_fedora_oscap/common.py
index 9b4b7bf..45ce727 100644
--- a/org_fedora_oscap/common.py
+++ b/org_fedora_oscap/common.py
@@ -136,8 +136,9 @@ def execute(self, ** kwargs):
(stdout, stderr) = proc.communicate()
self.stdout = stdout.decode()
- self.stderr = stderr.decode()
+ self.stderr = stderr.decode(errors="replace")
self.messages = re.findall(r'OpenSCAP Error:.*', self.stderr)
+ self.messages = self.messages + re.findall(r'E: oscap:.*', self.stderr)
self.returncode = proc.returncode

View File

@ -0,0 +1,101 @@
From 2646bb88746aa7b1abc715b40fa0a07a865d9576 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Wed, 17 Jun 2020 14:30:52 +0200
Subject: [PATCH 1/3] Improve handling of languages.
- fix the Makefile so translations are copied to the right directory in the image
- allow generation of update image with or without translations
- regenerate the create_update_image script with latest argbash
---
Makefile | 6 ++-
create_update_image.sh | 86 ++++++++++++++++++++++++++++--------------
2 files changed, 63 insertions(+), 29 deletions(-)
diff --git a/Makefile b/Makefile
index 2c2059c..dc8e9a0 100644
--- a/Makefile
+++ b/Makefile
@@ -6,6 +6,8 @@ ADDON = org_fedora_oscap
TESTS = tests \
testing_files
+DEFAULT_INSTALL_OF_PO_FILES ?= yes
+
OSVERSION := $(shell grep -o " [0-9]\{1,\}" /etc/redhat-release | sed "s/ //g")
ifeq ($(OSVERSION),7)
PYVERSION = ""
@@ -37,7 +39,9 @@ NUM_PROCS = $$(getconf _NPROCESSORS_ONLN)
install:
mkdir -p $(DESTDIR)$(ADDONDIR)
cp -rv $(ADDON) $(DESTDIR)$(ADDONDIR)
+ifeq ($(DEFAULT_INSTALL_OF_PO_FILES),yes)
$(MAKE) install-po-files
+endif
uninstall:
rm -rfv $(DESTDIR)$(ADDONDIR)
@@ -78,7 +82,7 @@ push-pot: potfile
zanata push $(ZANATA_PUSH_ARGS)
install-po-files:
- $(MAKE) -C po install
+ $(MAKE) -C po install RPM_BUILD_ROOT=$(DESTDIR)
test:
@echo "***Running pylint$(PYVERSION) checks***"
From 7434500bcd94c2c53d61c84a7364493372f7e053 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Wed, 17 Jun 2020 14:32:25 +0200
Subject: [PATCH 2/3] Dont use capital letters for the spoke title.
---
org_fedora_oscap/gui/spokes/oscap.glade | 2 +-
po/oscap-anaconda-addon.pot | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/org_fedora_oscap/gui/spokes/oscap.glade b/org_fedora_oscap/gui/spokes/oscap.glade
index e9e2fc0..e8516c0 100644
--- a/org_fedora_oscap/gui/spokes/oscap.glade
+++ b/org_fedora_oscap/gui/spokes/oscap.glade
@@ -37,7 +37,7 @@
<property name="can_focus">False</property>
<property name="hexpand">True</property>
<property name="vexpand">True</property>
- <property name="window_name" translatable="yes">SECURITY POLICY</property>
+ <property name="window_name" translatable="yes">Security Policy</property>
<signal name="button-clicked" handler="on_back_clicked" swapped="no"/>
<child internal-child="main_box">
<object class="GtkBox" id="AnacondaSpokeWindow-main_box1">
From 00c3dd557881def88c2898fa6ced17dddfef5213 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Thu, 18 Jun 2020 15:12:40 +0200
Subject: [PATCH 3/3] Changed URI back to URL in order not to mess with
translations.
The PR #112 marked this string as translatable, and changed it in the process.
However, the string is already present elsewhere in the source code, so this move
messed things up for translators.
This change reverts the URL->URI change, and leaves the GUI string "translatable",
as there is no way how to record the reason for this decision, so the situation doesn't repeat again.
As a result, it is now the job of the translation framework
to deal with the duplication gracefully.
---
org_fedora_oscap/gui/spokes/oscap.glade | 2 +-
po/oscap-anaconda-addon.pot | 9 ++-------
2 files changed, 3 insertions(+), 8 deletions(-)
diff --git a/org_fedora_oscap/gui/spokes/oscap.glade b/org_fedora_oscap/gui/spokes/oscap.glade
index e8516c0..1adcfcf 100644
--- a/org_fedora_oscap/gui/spokes/oscap.glade
+++ b/org_fedora_oscap/gui/spokes/oscap.glade
@@ -446,7 +446,7 @@
<object class="GtkLabel" id="noContentLabel">
<property name="visible">True</property>
<property name="can_focus">False</property>
- <property name="label" translatable="yes">No content found. Please enter data stream content or archive URI below:</property>
+ <property name="label" translatable="yes">No content found. Please enter data stream content or archive URL below:</property>
<property name="wrap">True</property>
<property name="xalign">0</property>
</object>

View File

@ -0,0 +1,41 @@
From 943b2570c3196aec12a46d5b0e261f026ace0e86 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Thu, 9 Jul 2020 17:15:11 +0200
Subject: [PATCH] Changed the spoke label back to uppercase.
The spoke title which is defined in the Python code and visible in the hub
should have capitalization that respects the local language convention.
However, the much less visible spoke label that is defined in the glade file
should be uppercase, and so should be translations.
---
org_fedora_oscap/gui/spokes/oscap.glade | 2 +-
org_fedora_oscap/gui/spokes/oscap.py | 3 +++
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/org_fedora_oscap/gui/spokes/oscap.glade b/org_fedora_oscap/gui/spokes/oscap.glade
index 1adcfcf..1fd2664 100644
--- a/org_fedora_oscap/gui/spokes/oscap.glade
+++ b/org_fedora_oscap/gui/spokes/oscap.glade
@@ -37,7 +37,7 @@
<property name="can_focus">False</property>
<property name="hexpand">True</property>
<property name="vexpand">True</property>
- <property name="window_name" translatable="yes">Security Policy</property>
+ <property name="window_name" translatable="yes">SECURITY POLICY</property>
<signal name="button-clicked" handler="on_back_clicked" swapped="no"/>
<child internal-child="main_box">
<object class="GtkBox" id="AnacondaSpokeWindow-main_box1">
diff --git a/org_fedora_oscap/gui/spokes/oscap.py b/org_fedora_oscap/gui/spokes/oscap.py
index f21f7d3..b9fcd31 100644
--- a/org_fedora_oscap/gui/spokes/oscap.py
+++ b/org_fedora_oscap/gui/spokes/oscap.py
@@ -196,6 +196,9 @@ class OSCAPSpoke(NormalSpoke):
# title of the spoke (will be displayed on the hub)
title = N_("_Security Policy")
+ # The string "SECURITY POLICY" in oscap.glade is meant to be uppercase,
+ # as it is displayed inside the spoke as the spoke label,
+ # and spoke labels are all uppercase by a convention.
# methods defined by API and helper methods #
def __init__(self, data, storage, payload):

View File

@ -0,0 +1,25 @@
From 9e514460c584666308967d611b26ea2927ea0535 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Thu, 18 Jun 2020 11:28:15 +0200
Subject: [PATCH] Added nfs-utils as an essential package for GUI envs.
---
org_fedora_oscap/rule_handling.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/org_fedora_oscap/rule_handling.py b/org_fedora_oscap/rule_handling.py
index 3728f89..6a3a04e 100644
--- a/org_fedora_oscap/rule_handling.py
+++ b/org_fedora_oscap/rule_handling.py
@@ -43,7 +43,10 @@
ESSENTIAL_PACKAGES = {
"xorg-x11-server-common": {
"env": ["graphical-server-environment", "workstation-product-environment"],
- }
+ },
+ "nfs-utils": {
+ "env": ["graphical-server-environment", "workstation-product-environment"],
+ },
}
log = logging.getLogger("anaconda")

View File

@ -0,0 +1,82 @@
From 6c285154723f618675c3a216ce84b480d770c10d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Tue, 9 Jun 2020 16:56:32 +0200
Subject: [PATCH 1/2] Address incompatible profiles and software selections.
This change introduces a mechanism that allows to vet packages marked for removal.
Such package can now have a record in the ESSENTIAL_PACKAGES dict,
that define whether the package is essential => cant be removed
based on the environment and groups selected in the Software Selection Anaconda spoke.
In case when one first selects the profile and then changes the Software Selection
to an incompatible setting, the Selection spoke will raise an error, as it already
tries to apply the blacklist with its environment/groups.
---
org_fedora_oscap/rule_handling.py | 38 +++++++++++++++++++++++++++----
1 file changed, 33 insertions(+), 5 deletions(-)
diff --git a/org_fedora_oscap/rule_handling.py b/org_fedora_oscap/rule_handling.py
index cd67822..3728f89 100644
--- a/org_fedora_oscap/rule_handling.py
+++ b/org_fedora_oscap/rule_handling.py
@@ -40,6 +40,12 @@
__all__ = ["RuleData"]
+ESSENTIAL_PACKAGES = {
+ "xorg-x11-server-common": {
+ "env": ["graphical-server-environment", "workstation-product-environment"],
+ }
+}
+
log = logging.getLogger("anaconda")
_ = common._
@@ -627,6 +633,20 @@ def __str__(self):
return ret
+ def _package_is_essential(self, package_name, ksdata_packages):
+ if package_name not in ESSENTIAL_PACKAGES:
+ return False
+ if package_name in ksdata_packages.packageList:
+ return True
+ selected_install_env = ksdata_packages.environment
+ if selected_install_env in ESSENTIAL_PACKAGES[package_name].get("env"):
+ return True
+ selected_install_groups_names = {g.name for g in ksdata_packages.groupList}
+ for g in ESSENTIAL_PACKAGES[package_name].get("groups", []):
+ if g in selected_install_groups_names:
+ return True
+ return False
+
def eval_rules(self, ksdata, storage, report_only=False):
""":see: RuleHandler.eval_rules"""
@@ -655,13 +675,21 @@ def eval_rules(self, ksdata, storage, report_only=False):
common.MESSAGE_TYPE_INFO, msg))
# now do the same for the packages that should be excluded
-
# add messages for the already excluded packages
for pkg in self._removed_pkgs:
- msg = _("package '%s' has been added to the list of excluded "
- "packages" % pkg)
- messages.append(RuleMessage(self.__class__,
- common.MESSAGE_TYPE_INFO, msg))
+ if self._package_is_essential(pkg, ksdata.packages):
+ msg = _(
+ "package '{package}' has been added to the list "
+ "of excluded packages, but it can't be removed "
+ "from the current software selection without breaking the installation."
+ .format(package=pkg))
+ messages.append(RuleMessage(self.__class__,
+ common.MESSAGE_TYPE_FATAL, msg))
+ else:
+ msg = _("package '%s' has been added to the list of excluded "
+ "packages" % pkg)
+ messages.append(RuleMessage(self.__class__,
+ common.MESSAGE_TYPE_INFO, msg))
# packages, that should be added
packages_to_remove = (pkg for pkg in self._remove_pkgs

View File

@ -0,0 +1,50 @@
From 1de759e8bcf1caddddfdda59496473ed9b525365 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Wed, 4 Nov 2020 17:48:35 +0100
Subject: [PATCH] Expanded group data to detect more package collisions.
This should cover RHEL8 OSPP and CIS profiles.
---
org_fedora_oscap/rule_handling.py | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/org_fedora_oscap/rule_handling.py b/org_fedora_oscap/rule_handling.py
index 6a3a04e..80d86c7 100644
--- a/org_fedora_oscap/rule_handling.py
+++ b/org_fedora_oscap/rule_handling.py
@@ -40,12 +40,26 @@
__all__ = ["RuleData"]
+# Mapping of packages to package environments and/or groups that depends on them
+# See also https://access.redhat.com/solutions/1201413 how to get group IDs.
+# on RHEL8, use e.g. grep -R "<id>" /var/cache/dnf/*
ESSENTIAL_PACKAGES = {
"xorg-x11-server-common": {
"env": ["graphical-server-environment", "workstation-product-environment"],
+ "groups": ["workstation-product-environment"],
},
"nfs-utils": {
"env": ["graphical-server-environment", "workstation-product-environment"],
+ "groups": ["workstation-product-environment"],
+ },
+ "tftp": {
+ "groups": ["network-server"],
+ },
+ "abrt": {
+ "groups": ["debugging"],
+ },
+ "gssproxy": {
+ "groups": ["file-server"],
},
}
@@ -642,7 +656,7 @@ def _package_is_essential(self, package_name, ksdata_packages):
if package_name in ksdata_packages.packageList:
return True
selected_install_env = ksdata_packages.environment
- if selected_install_env in ESSENTIAL_PACKAGES[package_name].get("env"):
+ if selected_install_env in ESSENTIAL_PACKAGES[package_name].get("env", []):
return True
selected_install_groups_names = {g.name for g in ksdata_packages.groupList}
for g in ESSENTIAL_PACKAGES[package_name].get("groups", []):

View File

@ -1,206 +0,0 @@
From 8eacfad08b3c27aa9510f2c3337356581bd9bebd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Mon, 3 Jan 2022 17:31:49 +0100
Subject: [PATCH 1/3] Add oscap sanity check before attempting remediation
If something is obviously wrong with the scanner, then don't attempt to remediate
and try to show relevant information in a dialog window.
---
org_fedora_oscap/common.py | 39 ++++++++++++++++++++++++++++--------
org_fedora_oscap/ks/oscap.py | 11 ++++++++++
tests/test_common.py | 8 ++++++++
3 files changed, 50 insertions(+), 8 deletions(-)
diff --git a/org_fedora_oscap/common.py b/org_fedora_oscap/common.py
index 884bbc8..05829ce 100644
--- a/org_fedora_oscap/common.py
+++ b/org_fedora_oscap/common.py
@@ -139,7 +139,8 @@ def execute(self, ** kwargs):
proc = subprocess.Popen(self.args, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, ** kwargs)
except OSError as oserr:
- msg = "Failed to run the oscap tool: %s" % oserr
+ msg = ("Failed to execute command '{command_string}': {oserr}"
+ .format(command_string=command_string, oserr=oserr))
raise OSCAPaddonError(msg)
(stdout, stderr) = proc.communicate()
@@ -215,6 +216,34 @@ def _run_oscap_gen_fix(profile, fpath, template, ds_id="", xccdf_id="",
return proc.stdout
+def do_chroot(chroot):
+ """Helper function doing the chroot if requested."""
+ if chroot and chroot != "/":
+ os.chroot(chroot)
+ os.chdir("/")
+
+
+def assert_scanner_works(chroot, executable="oscap"):
+ args = [executable, "--version"]
+ command = " ".join(args)
+
+ try:
+ proc = subprocess.Popen(
+ args, preexec_fn=lambda: do_chroot(chroot),
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ (stdout, stderr) = proc.communicate()
+ stderr = stderr.decode(errors="replace")
+ except OSError as exc:
+ msg = _(f"Basic invocation '{command}' fails: {str(exc)}")
+ raise OSCAPaddonError(msg)
+ if proc.returncode != 0:
+ msg = _(
+ f"Basic scanner invocation '{command}' exited "
+ "with non-zero error code {proc.returncode}: {stderr}")
+ raise OSCAPaddonError(msg)
+ return True
+
+
def run_oscap_remediate(profile, fpath, ds_id="", xccdf_id="", tailoring="",
chroot=""):
"""
@@ -244,12 +273,6 @@ def run_oscap_remediate(profile, fpath, ds_id="", xccdf_id="", tailoring="",
if not profile:
return ""
- def do_chroot():
- """Helper function doing the chroot if requested."""
- if chroot and chroot != "/":
- os.chroot(chroot)
- os.chdir("/")
-
# make sure the directory for the results exists
results_dir = os.path.dirname(RESULTS_PATH)
if chroot:
@@ -274,7 +297,7 @@ def do_chroot():
args.append(fpath)
proc = SubprocessLauncher(args)
- proc.execute(preexec_fn=do_chroot)
+ proc.execute(preexec_fn=lambda: do_chroot(chroot))
proc.log_messages()
if proc.returncode not in (0, 2):
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index 65d74cf..da1600f 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -488,6 +488,17 @@ def execute(self, storage, ksdata, users, payload):
# selected
return
+ try:
+ common.assert_scanner_works(
+ chroot=conf.target.system_root, executable="oscap")
+ except Exception as exc:
+ msg_lines = [_(
+ "The 'oscap' scanner doesn't work in the installed system: {error}"
+ .format(error=str(exc)))]
+ msg_lines.append(_("As a result, the installed system can't be hardened."))
+ self._terminate("\n".join(msg_lines))
+ return
+
target_content_dir = utils.join_paths(conf.target.system_root,
common.TARGET_CONTENT_DIR)
utils.ensure_dir_exists(target_content_dir)
diff --git a/tests/test_common.py b/tests/test_common.py
index 9f7a16a..4f25379 100644
--- a/tests/test_common.py
+++ b/tests/test_common.py
@@ -77,6 +77,14 @@ def _run_oscap(mock_subprocess, additional_args):
return expected_args, kwargs
+def test_oscap_works():
+ assert common.assert_scanner_works(chroot="/")
+ with pytest.raises(common.OSCAPaddonError, match="No such file"):
+ common.assert_scanner_works(chroot="/", executable="i_dont_exist")
+ with pytest.raises(common.OSCAPaddonError, match="non-zero"):
+ common.assert_scanner_works(chroot="/", executable="false")
+
+
def test_run_oscap_remediate_profile_only(mock_subprocess, monkeypatch):
return run_oscap_remediate_profile(
mock_subprocess, monkeypatch,
From b54cf2bddba56e5b776fb60514a5e29d47c74cac Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Mon, 3 Jan 2022 17:42:31 +0100
Subject: [PATCH 2/3] Don't raise exceptions in execute()
Those result in tracebacks during the installation,
while a dialog window presents a more useful form of user interaction.
---
org_fedora_oscap/ks/oscap.py | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index da1600f..d3f0dbe 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -513,8 +513,9 @@ def execute(self, storage, ksdata, users, payload):
ret = util.execInSysroot("yum", ["-y", "--nogpg", "install",
self.raw_postinst_content_path])
if ret != 0:
- raise common.ExtractionError("Failed to install content "
- "RPM to the target system")
+ msg = _(f"Failed to install content RPM to the target system.")
+ self._terminate(msg)
+ return
elif self.content_type == "scap-security-guide":
# nothing needed
pass
@@ -525,10 +526,15 @@ def execute(self, storage, ksdata, users, payload):
if os.path.exists(self.preinst_tailoring_path):
shutil.copy2(self.preinst_tailoring_path, target_content_dir)
- common.run_oscap_remediate(self.profile_id, self.postinst_content_path,
- self.datastream_id, self.xccdf_id,
- self.postinst_tailoring_path,
- chroot=conf.target.system_root)
+ try:
+ common.run_oscap_remediate(self.profile_id, self.postinst_content_path,
+ self.datastream_id, self.xccdf_id,
+ self.postinst_tailoring_path,
+ chroot=conf.target.system_root)
+ except Exception as exc:
+ msg = _(f"Something went wrong during the final hardening: {str(exc)}.")
+ self._terminate(msg)
+ return
def clear_all(self):
"""Clear all the stored values."""
From 00d770d1b7f8e1f0734e93da227f1c3e445033c8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Mon, 3 Jan 2022 17:44:12 +0100
Subject: [PATCH 3/3] Change the error feedback based on the installation mode
The original approach was confusing, because non-interactive installs run without any user input,
and the message assumed that the user is able to answer installer's questions.
---
org_fedora_oscap/ks/oscap.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index d3f0dbe..ef34448 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -372,13 +372,14 @@ def postinst_tailoring_path(self):
self.tailoring_path)
def _terminate(self, message):
- message += "\n" + _("The installation should be aborted.")
- message += " " + _("Do you wish to continue anyway?")
if flags.flags.automatedInstall and not flags.flags.ksprompt:
# cannot have ask in a non-interactive kickstart
# installation
+ message += "\n" + _("Aborting the installation.")
raise errors.CmdlineError(message)
+ message += "\n" + _("The installation should be aborted.")
+ message += " " + _("Do you wish to continue anyway?")
answ = errors.errorHandler.ui.showYesNoQuestion(message)
if answ == errors.ERROR_CONTINUE:
# prevent any futher actions here by switching to the dry

View File

@ -1,39 +0,0 @@
From 1abc4e96638e819d3fbee74396b36a6ccaf0ab29 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Tue, 3 Aug 2021 11:01:59 +0200
Subject: [PATCH] Refactor content identification
Don't use the multiprocessing pool - it sometimes creates probems during
its initialization:
https://bugzilla.redhat.com/show_bug.cgi?id=1989441
---
org_fedora_oscap/content_handling.py | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/org_fedora_oscap/content_handling.py b/org_fedora_oscap/content_handling.py
index f2af22f..65d5a28 100644
--- a/org_fedora_oscap/content_handling.py
+++ b/org_fedora_oscap/content_handling.py
@@ -111,9 +111,8 @@ def parse_HTML_from_content(content):
def identify_files(fpaths):
- with multiprocessing.Pool(os.cpu_count()) as p:
- labels = p.map(get_doc_type, fpaths)
- return {path: label for (path, label) in zip(fpaths, labels)}
+ result = {path: get_doc_type(path) for path in fpaths}
+ return result
def get_doc_type(file_path):
@@ -131,7 +130,9 @@ def get_doc_type(file_path):
except UnicodeDecodeError:
# 'oscap info' supplied weird output, which happens when it tries
# to explain why it can't examine e.g. a JPG.
- return None
+ pass
+ except Exception as e:
+ log.warning(f"OSCAP addon: Unexpected error when looking at {file_path}: {str(e)}")
log.info("OSCAP addon: Identified {file_path} as {content_type}"
.format(file_path=file_path, content_type=content_type))
return content_type

View File

@ -1,51 +0,0 @@
From 3377a914f4668af3d72216468ae192bc300890f9 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Mon, 9 Aug 2021 15:45:58 +0200
Subject: [PATCH 1/2] Fix archive handling in GUI installs
GUI downloads an archive, so the ensuing installation doesn't have to.
However, the installation has to be able to discover files recovered
from the archive.
The fix makes sure that files are discovered also in subdirectories.
---
org_fedora_oscap/content_discovery.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index f6b4d27..5fc7343 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -196,7 +196,8 @@ def _gather_available_files(self, actually_fetched_content, dest_filename):
if not dest_filename: # using scap-security-guide
fpaths = [self.DEFAULT_SSG_DATA_STREAM_PATH]
else: # Using downloaded XCCDF/OVAL/DS/tailoring
- fpaths = glob(str(self.CONTENT_DOWNLOAD_LOCATION / "*.xml"))
+ fpaths = pathlib.Path(self.CONTENT_DOWNLOAD_LOCATION).rglob("*")
+ fpaths = [str(p) for p in fpaths if p.is_file()]
else:
dest_filename = pathlib.Path(dest_filename)
# RPM is an archive at this phase
From 191df327e3e51f486fb655e97acac30222c264fa Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Mon, 9 Aug 2021 15:48:50 +0200
Subject: [PATCH 2/2] Improve logging
Logs written to log files can contain specific details.
---
org_fedora_oscap/ks/oscap.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index d1b8c9e..65d74cf 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -393,7 +393,7 @@ def _terminate(self, message):
time.sleep(100000)
def _handle_error(self, exception):
- log.error("Failed to fetch and initialize SCAP content!")
+ log.error(f"Failed to fetch and initialize SCAP content: {str(exception)}")
if isinstance(exception, ContentCheckError):
msg = _("The integrity check of the security content failed.")

View File

@ -1,32 +0,0 @@
From 6ac75d5052fff5a7d4b7e249ef198ccecd1f86a4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mat=C4=9Bj=20T=C3=BD=C4=8D?= <matyc@redhat.com>
Date: Mon, 17 Jul 2023 17:08:54 +0200
Subject: [PATCH] Make tar extraction safer
See also https://bugzilla.redhat.com/show_bug.cgi?id=2218875
---
org_fedora_oscap/common.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/org_fedora_oscap/common.py b/org_fedora_oscap/common.py
index 05829ce..b27276e 100644
--- a/org_fedora_oscap/common.py
+++ b/org_fedora_oscap/common.py
@@ -360,7 +360,7 @@ def extract_data(archive, out_dir, ensure_has_files=None):
raise ExtractionError(msg)
utils.ensure_dir_exists(out_dir)
- zfile.extractall(path=out_dir)
+ zfile.extractall(path=out_dir, filter="data")
result = [utils.join_paths(out_dir, info.filename) for info in zfile.filelist]
zfile.close()
elif archive.endswith(".tar"):
@@ -418,7 +418,7 @@ def _extract_tarball(archive, out_dir, ensure_has_files, alg):
raise ExtractionError(msg)
utils.ensure_dir_exists(out_dir)
- tfile.extractall(path=out_dir)
+ tfile.extractall(path=out_dir, filter="data")
result = [utils.join_paths(out_dir, member.path) for member in tfile.getmembers()]
tfile.close()

View File

@ -1,372 +0,0 @@
From e8e303aa3ca9db564ea52258de15a81851c3b265 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 12 Oct 2022 11:37:04 +0200
Subject: [PATCH 1/5] Add capability to preselect content from archives
Users can specify content path and tailoring path in kickstarts,
and the addon should be able to assure that those files are available,
and that they have precedence over other files.
---
org_fedora_oscap/content_discovery.py | 35 +++++++++++++++++++
tests/test_content_discovery.py | 48 +++++++++++++++++++++++++++
2 files changed, 83 insertions(+)
create mode 100644 tests/test_content_discovery.py
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index 5fc7343..f654449 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -11,6 +11,7 @@
from org_fedora_oscap import data_fetch, utils
from org_fedora_oscap import common
from org_fedora_oscap import content_handling
+from org_fedora_oscap.content_handling import CONTENT_TYPES
from org_fedora_oscap.common import _
@@ -167,6 +168,38 @@ def _verify_fingerprint(self, dest_filename, fingerprint=""):
msg = _(f"Integrity check of the content failed - {hash_obj.name} hash didn't match")
raise content_handling.ContentCheckError(msg)
+ def filter_discovered_content(self, labelled_files):
+ expected_path = self._addon_data.content_path
+ categories = (CONTENT_TYPES["DATASTREAM"], CONTENT_TYPES["XCCDF_CHECKLIST"])
+ if expected_path:
+ labelled_files = self.reduce_files(labelled_files, expected_path, categories)
+
+ expected_path = self._addon_data.tailoring_path
+ categories = (CONTENT_TYPES["TAILORING"], )
+ if expected_path:
+ labelled_files = self.reduce_files(labelled_files, expected_path, categories)
+
+ expected_path = self._addon_data.cpe_path
+ categories = (CONTENT_TYPES["CPE_DICT"], )
+ if expected_path:
+ labelled_files = self.reduce_files(labelled_files, expected_path, categories)
+
+ return labelled_files
+
+ def reduce_files(self, labelled_files, expected_path, categories):
+ reduced_files = dict()
+ if expected_path not in labelled_files:
+ msg = (
+ f"Expected a file {expected_path} to be part of the supplied content, "
+ f"but it was not the case, got only {list(labelled_files.keys())}"
+ )
+ raise RuntimeError(msg)
+ for path, label in labelled_files.items():
+ if label in categories and path != expected_path:
+ continue
+ reduced_files[path] = label
+ return reduced_files
+
def _finish_actual_fetch(self, wait_for, fingerprint, report_callback, dest_filename):
threadMgr.wait(wait_for)
actually_fetched_content = wait_for is not None
@@ -182,6 +215,8 @@ def _finish_actual_fetch(self, wait_for, fingerprint, report_callback, dest_file
structured_content.add_content_archive(dest_filename)
labelled_files = content_handling.identify_files(fpaths)
+ labelled_files = self.filter_discovered_content(labelled_files)
+
for fname, label in labelled_files.items():
structured_content.add_file(fname, label)
diff --git a/tests/test_content_discovery.py b/tests/test_content_discovery.py
new file mode 100644
index 0000000..5463c9a
--- /dev/null
+++ b/tests/test_content_discovery.py
@@ -0,0 +1,48 @@
+import pytest
+
+import org_fedora_oscap.content_discovery as tested_module
+
+
+@pytest.fixture
+def labelled_files():
+ return {
+ "dir/datastream": "D",
+ "dir/datastream2": "D",
+ "dir/dir/datastream3": "D",
+ "dir/dir/datastream3": "D",
+ "dir/XCCDF": "X",
+ "XCCDF2": "X",
+ "cpe": "C",
+ "t1": "T",
+ "dir3/t2": "T",
+ }
+
+
+def test_reduce(labelled_files):
+ bringer = tested_module.ContentBringer(None)
+
+ d_count = 0
+ x_count = 0
+ for l in labelled_files.values():
+ if l == "D":
+ d_count += 1
+ elif l == "X":
+ x_count += 1
+
+ reduced = bringer.reduce_files(labelled_files, "dir/datastream", ["D"])
+ assert len(reduced) == len(labelled_files) - d_count + 1
+ assert "dir/datastream" in reduced
+
+ reduced = bringer.reduce_files(labelled_files, "dir/datastream", ["D", "X"])
+ assert len(reduced) == len(labelled_files) - d_count - x_count + 1
+ assert "dir/datastream" in reduced
+
+ reduced = bringer.reduce_files(labelled_files, "dir/XCCDF", ["D", "X"])
+ assert len(reduced) == len(labelled_files) - d_count - x_count + 1
+ assert "dir/XCCDF" in reduced
+
+ with pytest.raises(RuntimeError, match="dir/datastream4"):
+ bringer.reduce_files(labelled_files, "dir/datastream4", ["D"])
+
+ reduced = bringer.reduce_files(labelled_files, "cpe", ["C"])
+ assert reduced == labelled_files
From 82c1950903fcce079cd71f021c1fde25f75f9521 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 12 Oct 2022 11:40:11 +0200
Subject: [PATCH 2/5] Handle changes in content identification
The code is able to handle changes in the way how oscap identifies
content much more gracefully.
---
org_fedora_oscap/content_discovery.py | 13 +++++++++----
org_fedora_oscap/content_handling.py | 5 +++++
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index f654449..b20f3a6 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -2,6 +2,7 @@
import logging
import pathlib
import shutil
+import os
from glob import glob
from pyanaconda.core import constants
@@ -214,11 +215,15 @@ def _finish_actual_fetch(self, wait_for, fingerprint, report_callback, dest_file
if content_type in ("archive", "rpm"):
structured_content.add_content_archive(dest_filename)
- labelled_files = content_handling.identify_files(fpaths)
- labelled_files = self.filter_discovered_content(labelled_files)
+ labelled_filenames = content_handling.identify_files(fpaths)
+ labelled_relative_filenames = {
+ os.path.relpath(path, self.CONTENT_DOWNLOAD_LOCATION): label
+ for path, label in labelled_filenames.items()}
+ labelled_relative_filenames = self.filter_discovered_content(labelled_relative_filenames)
- for fname, label in labelled_files.items():
- structured_content.add_file(fname, label)
+ for rel_fname, label in labelled_relative_filenames.items():
+ fname = self.CONTENT_DOWNLOAD_LOCATION / rel_fname
+ structured_content.add_file(str(fname), label)
if fingerprint and dest_filename:
structured_content.record_verification(dest_filename)
diff --git a/org_fedora_oscap/content_handling.py b/org_fedora_oscap/content_handling.py
index 65d5a28..3e2ecae 100644
--- a/org_fedora_oscap/content_handling.py
+++ b/org_fedora_oscap/content_handling.py
@@ -122,6 +122,11 @@ def get_doc_type(file_path):
if line.startswith("Document type:"):
_prefix, _sep, type_info = line.partition(":")
content_type = type_info.strip()
+ if content_type not in CONTENT_TYPES.values():
+ log.info(
+ f"File {file_path} labelled by oscap as {content_type}, "
+ "which is an unexpected type.")
+ content_type = f"unknown - {content_type}"
break
except OSError:
# 'oscap info' exitted with a non-zero exit code -> unknown doc
From b6bf5a6c96f5dbbd78043455802ebc0033cf1a6a Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 12 Oct 2022 11:38:51 +0200
Subject: [PATCH 3/5] Remove unused code
The function is not referenced anywhere in the project
---
org_fedora_oscap/content_handling.py | 40 ----------------------------
1 file changed, 40 deletions(-)
diff --git a/org_fedora_oscap/content_handling.py b/org_fedora_oscap/content_handling.py
index 3e2ecae..5096bab 100644
--- a/org_fedora_oscap/content_handling.py
+++ b/org_fedora_oscap/content_handling.py
@@ -141,43 +141,3 @@ def get_doc_type(file_path):
log.info("OSCAP addon: Identified {file_path} as {content_type}"
.format(file_path=file_path, content_type=content_type))
return content_type
-
-
-def explore_content_files(fpaths):
- """
- Function for finding content files in a list of file paths. SIMPLY PICKS
- THE FIRST USABLE CONTENT FILE OF A PARTICULAR TYPE AND JUST PREFERS DATA
- STREAMS OVER STANDALONE BENCHMARKS.
-
- :param fpaths: a list of file paths to search for content files in
- :type fpaths: [str]
- :return: ContentFiles instance containing the file names of the XCCDF file,
- CPE dictionary and tailoring file or "" in place of those items
- if not found
- :rtype: ContentFiles
-
- """
- xccdf_file = ""
- cpe_file = ""
- tailoring_file = ""
- found_ds = False
-
- for fpath in fpaths:
- doc_type = get_doc_type(fpath)
- if not doc_type:
- continue
-
- # prefer DS over standalone XCCDF
- if doc_type == "Source Data Stream" and (not xccdf_file or not found_ds):
- xccdf_file = fpath
- found_ds = True
- elif doc_type == "XCCDF Checklist" and not xccdf_file:
- xccdf_file = fpath
- elif doc_type == "CPE Dictionary" and not cpe_file:
- cpe_file = fpath
- elif doc_type == "XCCDF Tailoring" and not tailoring_file:
- tailoring_file = fpath
-
- # TODO: raise exception if no xccdf_file is found?
- files = ContentFiles(xccdf_file, cpe_file, tailoring_file)
- return files
From a990568ccddb2864c8daeae91fdc1f6588b3c6f3 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Thu, 13 Oct 2022 14:11:25 +0200
Subject: [PATCH 4/5] Dont use tailoring if it is not expected
Take tailorings into account only if it is specified in the kickstart.
Compulsive usage of tailoring may be unwanted.
---
org_fedora_oscap/content_discovery.py | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index b20f3a6..e9cf34a 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -169,16 +169,25 @@ def _verify_fingerprint(self, dest_filename, fingerprint=""):
msg = _(f"Integrity check of the content failed - {hash_obj.name} hash didn't match")
raise content_handling.ContentCheckError(msg)
+ def allow_one_expected_tailoring_or_no_tailoring(self, labelled_files):
+ expected_tailoring = self._addon_data.tailoring_path
+ tailoring_label = CONTENT_TYPES["TAILORING"]
+ if expected_tailoring:
+ labelled_files = self.reduce_files(labelled_files, expected_tailoring, [tailoring_label])
+ else:
+ labelled_files = {
+ path: label for path, label in labelled_files.items()
+ if label != tailoring_label
+ }
+ return labelled_files
+
def filter_discovered_content(self, labelled_files):
expected_path = self._addon_data.content_path
categories = (CONTENT_TYPES["DATASTREAM"], CONTENT_TYPES["XCCDF_CHECKLIST"])
if expected_path:
labelled_files = self.reduce_files(labelled_files, expected_path, categories)
- expected_path = self._addon_data.tailoring_path
- categories = (CONTENT_TYPES["TAILORING"], )
- if expected_path:
- labelled_files = self.reduce_files(labelled_files, expected_path, categories)
+ labelled_files = self.allow_one_expected_tailoring_or_no_tailoring(labelled_files)
expected_path = self._addon_data.cpe_path
categories = (CONTENT_TYPES["CPE_DICT"], )
From c4cb296ca3838a0967c8258b9ed5221691884a36 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Tue, 8 Nov 2022 10:46:59 +0100
Subject: [PATCH 5/5] Make the content RPM installation robust
If a package manager fails to install the package,
use the rpm command directly and skip deps.
---
org_fedora_oscap/ks/oscap.py | 41 ++++++++++++++++++++++++++++--------
1 file changed, 32 insertions(+), 9 deletions(-)
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index e47d6ba..dac273d 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -23,6 +23,7 @@
import shutil
import re
import os
+import io
import time
import logging
import pathlib
@@ -473,6 +474,33 @@ def setup(self, storage, ksdata, payload):
if pkg not in ksdata.packages.packageList:
ksdata.packages.packageList.append(pkg)
+ def _attempt_rpm_installation(self):
+ log.info("OSCAP addon: Installing the security content RPM to the installed system.")
+ stdout = io.StringIO()
+ ret = util.execWithRedirect(
+ "yum", ["-y", "--nogpg", "install", self.raw_postinst_content_path],
+ stdout=stdout, root=conf.target.system_root)
+ stdout.seek(0)
+ if ret != 0:
+ log.error(
+ "OSCAP addon: Error installing security content RPM using yum: {0}",
+ stdout.read())
+
+ stdout = io.StringIO()
+ ret = util.execWithRedirect(
+ "rpm", ["--install", "--nodeps", self.raw_postinst_content_path],
+ stdout=stdout, root=conf.target.system_root)
+ if ret != 0:
+ log.error(
+ "OSCAP addon: Error installing security content RPM using rpm: {0}",
+ stdout.read())
+ msg = _(f"Failed to install content RPM to the target system.")
+ raise RuntimeError(msg)
+
+ def _copy_rpm_to_target_and_install(self, target_content_dir):
+ shutil.copy2(self.raw_preinst_content_path, target_content_dir)
+ self._attempt_rpm_installation()
+
def execute(self, storage, ksdata, users, payload):
"""
The execute method that should make changes to the installed system. It
@@ -507,15 +535,10 @@ def execute(self, storage, ksdata, users, payload):
if self.content_type == "datastream":
shutil.copy2(self.preinst_content_path, target_content_dir)
elif self.content_type == "rpm":
- # copy the RPM to the target system
- shutil.copy2(self.raw_preinst_content_path, target_content_dir)
-
- # and install it with yum
- ret = util.execInSysroot("yum", ["-y", "--nogpg", "install",
- self.raw_postinst_content_path])
- if ret != 0:
- msg = _(f"Failed to install content RPM to the target system.")
- self._terminate(msg)
+ try:
+ self._copy_rpm_to_target_and_install(target_content_dir)
+ except Exception as exc:
+ self._terminate(str(exc))
return
elif self.content_type == "scap-security-guide":
# nothing needed

View File

@ -1,74 +0,0 @@
From 55cc3b685dd5a9ca6059459f41876dd9f19f900d Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Tue, 11 Oct 2022 17:07:28 +0200
Subject: [PATCH 1/2] Remove redundant message
The send_ready already performs what the removed call
could aim to accomplish.
---
org_fedora_oscap/gui/spokes/oscap.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/org_fedora_oscap/gui/spokes/oscap.py b/org_fedora_oscap/gui/spokes/oscap.py
index c57b1cd..4f8702a 100644
--- a/org_fedora_oscap/gui/spokes/oscap.py
+++ b/org_fedora_oscap/gui/spokes/oscap.py
@@ -150,7 +150,6 @@ def decorated(self, *args, **kwargs):
self._ready = True
# pylint: disable-msg=E1101
hubQ.send_ready(self.__class__.__name__, True)
- hubQ.send_message(self.__class__.__name__, self.status)
return ret
From 3f7c560947a17d1696899857e70ebcc8cba44019 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Thu, 13 Oct 2022 17:19:17 +0200
Subject: [PATCH 2/2] Increase robustness of fetching state detection
It is not completely practical to rely on locks alone,
and we can elliminate some corner cases by looking
whether well-known UI threads exist.
---
org_fedora_oscap/gui/spokes/oscap.py | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/org_fedora_oscap/gui/spokes/oscap.py b/org_fedora_oscap/gui/spokes/oscap.py
index 4f8702a..d8e6ce2 100644
--- a/org_fedora_oscap/gui/spokes/oscap.py
+++ b/org_fedora_oscap/gui/spokes/oscap.py
@@ -363,11 +363,14 @@ def _render_selected(self, column, renderer, model, itr, user_data=None):
else:
renderer.set_property("stock-id", None)
+ def _still_fetching(self):
+ return self._fetching or threadMgr.get('OSCAPguiWaitForDataFetchThread')
+
def _fetch_data_and_initialize(self):
"""Fetch data from a specified URL and initialize everything."""
with self._fetch_flag_lock:
- if self._fetching:
+ if self._still_fetching():
# prevent multiple fetches running simultaneously
return
self._fetching = True
@@ -894,7 +897,7 @@ def refresh(self):
# hide the progress box, no progress now
with self._fetch_flag_lock:
- if not self._fetching:
+ if not self._still_fetching():
really_hide(self._progress_box)
self._content_url_entry.set_sensitive(True)
@@ -1117,7 +1120,7 @@ def on_fetch_button_clicked(self, *args):
"""Handler for the Fetch button"""
with self._fetch_flag_lock:
- if self._fetching:
+ if self._still_fetching():
# some other fetching/pre-processing running, give up
log.warn("Clicked the fetch button, although the GUI is in the fetching mode.")
return

View File

@ -1,202 +0,0 @@
From 08d3da5640e5c16cda4e79cc13ac7921f1ebd964 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Tue, 15 Nov 2022 15:37:28 +0100
Subject: [PATCH 1/2] Fix handling of content paths
Archives and ready-to-use content use paths differently.
Archives get unpacked into a directory, where they need to be unpacked,
analyzed, and cross-checked with e.g. the supplied content path,
whereas ready-to-use content can be used directly.
As the current codebase doesn't untangle all possible ways how to obtain
existing content in a way of decomposing those into layers, this change
just makes the current code working at the expense of making it worse to
maintain.
---
org_fedora_oscap/content_discovery.py | 34 ++++++++++++++++++---------
org_fedora_oscap/ks/oscap.py | 6 ++++-
tests/test_content_discovery.py | 21 +++++++++++++++++
3 files changed, 49 insertions(+), 12 deletions(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index e9cf34a..2b71b1f 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -25,6 +25,14 @@ def is_network(scheme):
for net_prefix in data_fetch.NET_URL_PREFIXES)
+def path_is_present_among_paths(path, paths):
+ absolute_path = os.path.abspath(path)
+ for second_path in paths:
+ if absolute_path == os.path.abspath(second_path):
+ return True
+ return False
+
+
class ContentBringer:
CONTENT_DOWNLOAD_LOCATION = pathlib.Path(common.INSTALLATION_CONTENT_DIR)
DEFAULT_SSG_DATA_STREAM_PATH = f"{common.SSG_DIR}/{common.SSG_CONTENT}"
@@ -170,7 +178,7 @@ def _verify_fingerprint(self, dest_filename, fingerprint=""):
raise content_handling.ContentCheckError(msg)
def allow_one_expected_tailoring_or_no_tailoring(self, labelled_files):
- expected_tailoring = self._addon_data.tailoring_path
+ expected_tailoring = self._addon_data.preinst_tailoring_path
tailoring_label = CONTENT_TYPES["TAILORING"]
if expected_tailoring:
labelled_files = self.reduce_files(labelled_files, expected_tailoring, [tailoring_label])
@@ -182,7 +190,7 @@ def allow_one_expected_tailoring_or_no_tailoring(self, labelled_files):
return labelled_files
def filter_discovered_content(self, labelled_files):
- expected_path = self._addon_data.content_path
+ expected_path = self._addon_data.preinst_content_path
categories = (CONTENT_TYPES["DATASTREAM"], CONTENT_TYPES["XCCDF_CHECKLIST"])
if expected_path:
labelled_files = self.reduce_files(labelled_files, expected_path, categories)
@@ -198,7 +206,7 @@ def filter_discovered_content(self, labelled_files):
def reduce_files(self, labelled_files, expected_path, categories):
reduced_files = dict()
- if expected_path not in labelled_files:
+ if not path_is_present_among_paths(expected_path, labelled_files.keys()):
msg = (
f"Expected a file {expected_path} to be part of the supplied content, "
f"but it was not the case, got only {list(labelled_files.keys())}"
@@ -225,13 +233,9 @@ def _finish_actual_fetch(self, wait_for, fingerprint, report_callback, dest_file
structured_content.add_content_archive(dest_filename)
labelled_filenames = content_handling.identify_files(fpaths)
- labelled_relative_filenames = {
- os.path.relpath(path, self.CONTENT_DOWNLOAD_LOCATION): label
- for path, label in labelled_filenames.items()}
- labelled_relative_filenames = self.filter_discovered_content(labelled_relative_filenames)
+ labelled_filenames = self.filter_discovered_content(labelled_filenames)
- for rel_fname, label in labelled_relative_filenames.items():
- fname = self.CONTENT_DOWNLOAD_LOCATION / rel_fname
+ for fname, label in labelled_filenames.items():
structured_content.add_file(str(fname), label)
if fingerprint and dest_filename:
@@ -274,11 +278,18 @@ def use_downloaded_content(self, content):
# We know that we have ended up with a datastream-like content,
# but if we can't convert an archive to a datastream.
# self._addon_data.content_type = "datastream"
- self._addon_data.content_path = str(preferred_content.relative_to(content.root))
+ content_type = self._addon_data.content_type
+ if content_type in ("archive", "rpm"):
+ self._addon_data.content_path = str(preferred_content.relative_to(content.root))
+ else:
+ self._addon_data.content_path = str(preferred_content)
preferred_tailoring = self.get_preferred_tailoring(content)
if content.tailoring:
- self._addon_data.tailoring_path = str(preferred_tailoring.relative_to(content.root))
+ if content_type in ("archive", "rpm"):
+ self._addon_data.tailoring_path = str(preferred_tailoring.relative_to(content.root))
+ else:
+ self._addon_data.tailoring_path = str(preferred_tailoring)
def use_system_content(self, content=None):
self._addon_data.clear_all()
@@ -372,6 +383,7 @@ def _xccdf_content(self):
def find_expected_usable_content(self, relative_expected_content_path):
content_path = self.root / relative_expected_content_path
+ content_path = content_path.resolve()
eligible_main_content = (self._datastream_content(), self._xccdf_content())
if content_path in eligible_main_content:
diff --git a/org_fedora_oscap/ks/oscap.py b/org_fedora_oscap/ks/oscap.py
index dac273d..7d4a131 100644
--- a/org_fedora_oscap/ks/oscap.py
+++ b/org_fedora_oscap/ks/oscap.py
@@ -179,7 +179,11 @@ def _parse_profile_id(self, value):
self.profile_id = value
def _parse_content_path(self, value):
- # need to be checked?
+ if self.content_type in ("archive", "rpm") and os.path.isabs(self.content_path):
+ msg = (
+ "When using archives-like content input, the corresponding content path "
+ "has to be relative, but got '{self.content_path}'.")
+ raise KickstartValueError(msg)
self.content_path = value
def _parse_cpe_path(self, value):
diff --git a/tests/test_content_discovery.py b/tests/test_content_discovery.py
index 5463c9a..d6e14d9 100644
--- a/tests/test_content_discovery.py
+++ b/tests/test_content_discovery.py
@@ -1,3 +1,5 @@
+import os
+
import pytest
import org_fedora_oscap.content_discovery as tested_module
@@ -46,3 +48,22 @@ def test_reduce(labelled_files):
reduced = bringer.reduce_files(labelled_files, "cpe", ["C"])
assert reduced == labelled_files
+
+
+def test_path_presence_detection():
+ list_of_paths = ["file1", os.path.abspath("file2"), os.path.abspath("dir///file3")]
+
+ list_of_paths_in_list = [
+ "file1", os.path.abspath("file1"), "./file1",
+ "file2", "dir/..//file2",
+ "dir/../dir/file3", "dir/file3",
+ ]
+ list_of_paths_not_in_list = [
+ "../file1", "file3"
+ ]
+
+ for path in list_of_paths_in_list:
+ assert tested_module.path_is_present_among_paths(path, list_of_paths)
+
+ for path in list_of_paths_not_in_list:
+ assert not tested_module.path_is_present_among_paths(path, list_of_paths)
From 786ec5d90d12a1321fbff86f5d8d4a534059ad22 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 16 Nov 2022 15:35:09 +0100
Subject: [PATCH 2/2] Compare paths according to their equivalence
not according their arbitrary string form
---
org_fedora_oscap/content_discovery.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index 2b71b1f..42c61e0 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -25,10 +25,14 @@ def is_network(scheme):
for net_prefix in data_fetch.NET_URL_PREFIXES)
+def paths_are_equivalent(p1, p2):
+ return os.path.abspath(p1) == os.path.abspath(p2)
+
+
def path_is_present_among_paths(path, paths):
absolute_path = os.path.abspath(path)
for second_path in paths:
- if absolute_path == os.path.abspath(second_path):
+ if paths_are_equivalent(path, second_path):
return True
return False
@@ -213,7 +217,7 @@ def reduce_files(self, labelled_files, expected_path, categories):
)
raise RuntimeError(msg)
for path, label in labelled_files.items():
- if label in categories and path != expected_path:
+ if label in categories and not paths_are_equivalent(path, expected_path):
continue
reduced_files[path] = label
return reduced_files

View File

@ -1,66 +0,0 @@
From 58d4847dc4b55b9d4982be9505127679beca87c6 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 18 Jan 2023 16:36:36 +0100
Subject: [PATCH 1/2] Handle the URL with missing ://
---
org_fedora_oscap/content_discovery.py | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/org_fedora_oscap/content_discovery.py b/org_fedora_oscap/content_discovery.py
index 42c61e0..23fdafd 100644
--- a/org_fedora_oscap/content_discovery.py
+++ b/org_fedora_oscap/content_discovery.py
@@ -67,9 +67,14 @@ def content_uri(self):
@content_uri.setter
def content_uri(self, uri):
- scheme, path = uri.split("://", 1)
- self.content_uri_path = path
- self.content_uri_scheme = scheme
+ scheme_and_maybe_path = uri.split("://")
+ if len(scheme_and_maybe_path) == 1:
+ msg = (
+ f"Invalid supplied content URL '{uri}', "
+ "use the 'scheme://path' form.")
+ raise KickstartValueError(msg)
+ self.content_uri_path = scheme_and_maybe_path[1]
+ self.content_uri_scheme = scheme_and_maybe_path[0]
def fetch_content(self, what_if_fail, ca_certs_path=""):
"""
@@ -80,7 +85,10 @@ def fetch_content(self, what_if_fail, ca_certs_path=""):
should handle them in the calling layer.
ca_certs_path: Path to the HTTPS certificate file
"""
- self.content_uri = self._addon_data.content_url
+ try:
+ self.content_uri = self._addon_data.content_url
+ except Exception as exc:
+ what_if_fail(exc)
shutil.rmtree(self.CONTENT_DOWNLOAD_LOCATION, ignore_errors=True)
self.CONTENT_DOWNLOAD_LOCATION.mkdir(parents=True, exist_ok=True)
fetching_thread_name = self._fetch_files(
From cbfdae4f43ade3ef982a967f3e2844e66db3f9a0 Mon Sep 17 00:00:00 2001
From: Matej Tyc <matyc@redhat.com>
Date: Wed, 18 Jan 2023 16:36:53 +0100
Subject: [PATCH 2/2] Stop fetching when there is an invalid profile
---
org_fedora_oscap/gui/spokes/oscap.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/org_fedora_oscap/gui/spokes/oscap.py b/org_fedora_oscap/gui/spokes/oscap.py
index d8e6ce2..54eae1e 100644
--- a/org_fedora_oscap/gui/spokes/oscap.py
+++ b/org_fedora_oscap/gui/spokes/oscap.py
@@ -469,6 +469,8 @@ def update_progress_label(msg):
if self._addon_data.profile_id and not selected:
# profile ID given, but it was impossible to select it -> invalid
# profile ID given
+ with self._fetch_flag_lock:
+ self._fetching = False
self._invalid_profile_id()
return

View File

@ -2,8 +2,8 @@
%global _default_patch_flags --no-backup-if-mismatch
Name: oscap-anaconda-addon
Version: 1.2.1
Release: 14%{?dist}
Version: 1.1.1
Release: 7%{?dist}
Summary: Anaconda addon integrating OpenSCAP to the installation process
License: GPLv2+
@ -19,14 +19,12 @@ Source0: %{name}-%{version}.tar.gz
# Let the Patch1 be reserved for translations patches
Patch1: lang.patch
Patch2: oscap-anaconda-addon-1.2.2-content_ident-PR_167.patch
Patch3: oscap-anaconda-addon-1.2.2-deep_archives-PR_168.patch
Patch4: oscap-anaconda-addon-1.2.2-absent_appstream-PR_184.patch
Patch5: oscap-anaconda-addon-1.3.0-better_archive_handling-PR_220.patch
Patch6: oscap-anaconda-addon-1.3.0-clicking_nocrash-PR_221.patch
Patch7: oscap-anaconda-addon-1.3.0-fix_content_paths-PR_225.patch
Patch8: oscap-anaconda-addon-null-http_content_url-PR_232.patch
Patch9: oscap-anaconda-addon-1.2.2-tar-extraction-PR_249.patch
Patch2: oaa-1.2_warn-xorg.patch
Patch3: oaa-1.2_warn-nfs-utils.patch
Patch4: oaa-1.2_lang-streamline.patch
Patch5: oaa-1.2_spoke-window_PR122.patch
Patch6: oaa-1.2-unicode_issues_PR124.patch
Patch7: oscap-anaconda-addon-1.2-new_package_groups-PR_128.patch
BuildArch: noarch
BuildRequires: make
@ -50,17 +48,16 @@ content.
%prep
%setup -q -n %{name}-%{version}
# As patches may translates the strings that are updated by later patches,
# Patch1 needs to be aplied last.
%patch1 -p1
%patch2 -p1
%patch3 -p1
%patch4 -p1
%patch5 -p1
%patch6 -p1
%patch7 -p1
%patch8 -p1
%patch9 -p1
# As patches may translates the strings that are updated by later patches,
# Patch1 needs to be aplied last.
%patch1 -p1
# NOTE CONCERNING TRANSLATION PATCHES
# When preparing translation patches, don't consider that some languages are unsupported -
# we aim to include all applicable translation texts to the appropriate patch.
@ -82,74 +79,6 @@ make install DESTDIR=%{buildroot}
%doc COPYING ChangeLog README.md
%changelog
* Wed Aug 02 2023 Jan Černý <jcerny@redhat.com> - 1.2.1-14
- Rebuild after tests update
* Wed Jul 19 2023 Jan Černý <jcerny@redhat.com> - 1.2.1-13
- Fix tar file extraction (rhbz#2219408)
- Update translations (rhbz#2189572)
* Wed Feb 08 2023 Matej Tyc <matyc@redhat.com> - 1.2.1-12
- Update translations
Resolves: rhbz#2139743
* Mon Jan 23 2023 Matej Tyc <matyc@redhat.com> - 1.2.1-11
- Fix a reaction to invalid content URI
Resolves: rhbz#2148509
* Wed Nov 23 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-10
- Fix regression introduced when fixing content archive input
Resolves: rhbz#2129008
* Thu Nov 10 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-9
- Fix problems with handling multi-datastream archives
Resolves: rhbz#2129008
- Fix a crash when compulsively clicking in the GUI
Resolves: rhbz#2000998
* Wed Jul 20 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-8
- Update translations
Resolves: rhbz#2062707
* Fri Jun 10 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-7
- Remove the firstboot remediation feature completely.
We can't have it, while maintaining the standard UX.
Resolves: rhbz#2063179
* Mon Mar 21 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-6
- Introduce the firstboot remediation
Resolves: rhbz#1834716
- Add better error handling of installation using unsupported installation sources
Resolves: rhbz#2007981
* Fri Jan 21 2022 Matej Tyc <matyc@redhat.com> - 1.2.1-5
- Updated translations
Resolves: rhbz#2017356
* Fri Aug 20 2021 Matej Tyc <matyc@redhat.com> - 1.2.1-4
- Updated translations
Resolves: rhbz#1962007
* Mon Aug 09 2021 Matej Tyc <matyc@redhat.com> - 1.2.1-3
- Fix handling of archives with directories in GUI installs
- Resolves: rhbz#1691305
* Tue Aug 03 2021 Matej Tyc <matyc@redhat.com> - 1.2.1-2
- Refactor content identification
- Resolves: rhbz#1989441
* Fri Jul 30 2021 Matej Tyc <matyc@redhat.com> - 1.2.1-1
- Rebase to the new upstream version.
- Resolves: rhbz#1691305
* Fri Jul 16 2021 Matej Tyc <matyc@redhat.com> - 1.2.0-2
- Updated translations
- Resolves: rhbz#1938623
* Fri Jun 25 2021 Matej Tyc <matyc@redhat.com> - 1.2.0-1
- Rebase to the new upstream version.
- Resolves: rhbz#1691305
* Mon Feb 15 2021 Matej Tyc <matyc@redhat.com> - 1.1.1-7
- Updated translations.