metalefty/sos
1
0
Fork 0
forked from rpms/sos
Code Pull Requests Activity

Merge branch 'c8' into a8

Browse Source
This commit is contained in:
eabdullin 2022-05-25 07:44:29 +00:00 committed by Stepan Oksanichenko
commit 9bb48f561c
3 changed files with 2388 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

560
SOURCES/sos-bz2036697-ocp-backports.patch
View File

@ -3392,6 +3392,50 @@ index 8a9dbd7a..ab7f23cc 100644
--
2.31.1
From a7ffacd855fcf2e9a136c6946744cbc99bc91272 Mon Sep 17 00:00:00 2001
From: Jake Hunsaker <jhunsake@redhat.com>
Date: Tue, 19 Oct 2021 14:31:37 -0400
Subject: [PATCH] [Plugin] Wrap add_service_status to add_cmd_output
The changes to allow writing command output to a file highlighted a
short coming in add_service_status - by wrapping to `_add_cmd_output()`
instead of `add_cmd_output()`, we are not applying the default values
for kwarg parameters and thus potentially causing undesired behavior
during the actual collection.
Fix this by wrapping to `add_cmd_output()`, which will then in turn call
`_add_cmd_output()`.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
---
sos/report/plugins/__init__.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sos/report/plugins/__init__.py b/sos/report/plugins/__init__.py
index 38529a9d..98f163ab 100644
--- a/sos/report/plugins/__init__.py
+++ b/sos/report/plugins/__init__.py
@@ -2493,7 +2493,7 @@ class Plugin():
:param services: Service name(s) to collect statuses for
:type services: ``str`` or a ``list`` of strings
- :param kwargs: Optional arguments to pass to _add_cmd_output
+ :param kwargs: Optional arguments to pass to add_cmd_output
(timeout, predicate, suggest_filename,..)
"""
@@ -2508,7 +2508,7 @@ class Plugin():
return
for service in services:
- self._add_cmd_output(cmd="%s %s" % (query, service), **kwargs)
+ self.add_cmd_output("%s %s" % (query, service), **kwargs)
def add_journal(self, units=None, boot=None, since=None, until=None,
lines=None, allfields=False, output=None,
--
2.27.0
From 38a0533de3dd2613eefcc4865a2916e225e3ceed Mon Sep 17 00:00:00 2001
From: Pavel Moravec <pmoravec@redhat.com>
Date: Tue, 9 Nov 2021 19:34:25 +0100
@ -3526,6 +3570,32 @@ index e84b52da..1bfa741f 100644
--
2.31.1
From 83bade79da931225f12a1f40e576884bfe7173dd Mon Sep 17 00:00:00 2001
From: Michael Cambria <mcambria@redhat.com>
Date: Fri, 12 Nov 2021 09:07:24 -0500
Subject: [PATCH] Collect "ip route cache" data
Signed-off-by: Michael Cambria <mcambria@redhat.com>
---
sos/report/plugins/networking.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/sos/report/plugins/networking.py b/sos/report/plugins/networking.py
index f2bdca06..cc93c9e9 100644
--- a/sos/report/plugins/networking.py
+++ b/sos/report/plugins/networking.py
@@ -95,6 +95,8 @@ class Networking(Plugin):
"networkctl status -a",
"ip route show table all",
"ip -6 route show table all",
+ "ip -d route show cache",
+ "ip -d -6 route show cache",
"ip -4 rule",
"ip -6 rule",
"ip -s -d link",
--
2.27.0
From 8bf602108f75db10e449eff5e2266c6466504086 Mon Sep 17 00:00:00 2001
From: Nadia Pinaeva <npinaeva@redhat.com>
Date: Thu, 2 Dec 2021 16:30:44 +0100
@ -5024,6 +5094,496 @@ index cb20772fd..b59eade9a 100644
def test_ip_parser_valid_ipv4_line(self):
line = 'foobar foo 10.0.0.1/24 barfoo bar'
From 134451fe8fc80c67b8714713ab49c55245fd7727 Mon Sep 17 00:00:00 2001
From: Jake Hunsaker <jhunsake@redhat.com>
Date: Thu, 20 Jan 2022 10:21:32 -0500
Subject: [PATCH 1/2] [Plugin] Add support for containers to `add_copy_spec()`
This commit adds the ability for `add_copy_spec()` to copy files from
containers running on the host via a supported container runtime.
`add_copy_spec()` now has a `container` parameter which, if set, will
generate the command needed to copy a file from a container to our temp
directory. This will be collected after host files but before command
collection.
Runtimes have been updated with a `get_copy_command()` method that will
return the command string needed to copy a given file from a given
container to a given path on the host. Note that the `crio` runtime does
not currently provide a copy mechanism like `docker` or `podman`, so
file collections from containers will not be succesful on hosts using
that as their default runtime.
Finally, the manifest entries for plugins have been updated with a new
`containers` dict field whose entries are container names that have been
collected from. Those fields are also dicts having the same `files` and
`commands` keys/content as those in the plugin entry directly.
Closes: #2439
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
---
sos/policies/runtimes/__init__.py | 32 ++++++
sos/policies/runtimes/crio.py | 3 +
sos/policies/runtimes/docker.py | 3 +
sos/report/plugins/__init__.py | 180 +++++++++++++++++++++++++-----
4 files changed, 189 insertions(+), 29 deletions(-)
diff --git a/sos/policies/runtimes/__init__.py b/sos/policies/runtimes/__init__.py
index 4e9a45c1..5ac67354 100644
--- a/sos/policies/runtimes/__init__.py
+++ b/sos/policies/runtimes/__init__.py
@@ -69,6 +69,12 @@ class ContainerRuntime():
return True
return False
+ def check_can_copy(self):
+ """Check if the runtime supports copying files out of containers and
+ onto the host filesystem
+ """
+ return True
+
def get_containers(self, get_all=False):
"""Get a list of containers present on the system.
@@ -199,5 +205,31 @@ class ContainerRuntime():
"""
return "%s logs -t %s" % (self.binary, container)
+ def get_copy_command(self, container, path, dest, sizelimit=None):
+ """Generate the command string used to copy a file out of a container
+ by way of the runtime.
+
+ :param container: The name or ID of the container
+ :type container: ``str``
+
+ :param path: The path to copy from the container. Note that at
+ this time, no supported runtime supports globbing
+ :type path: ``str``
+
+ :param dest: The destination on the *host* filesystem to write
+ the file to
+ :type dest: ``str``
+
+ :param sizelimit: Limit the collection to the last X bytes of the
+ file at PATH
+ :type sizelimit: ``int``
+
+ :returns: Formatted runtime command to copy a file from a container
+ :rtype: ``str``
+ """
+ if sizelimit:
+ return "%s %s tail -c %s %s" % (self.run_cmd, container, sizelimit,
+ path)
+ return "%s cp %s:%s %s" % (self.binary, container, path, dest)
# vim: set et ts=4 sw=4 :
diff --git a/sos/policies/runtimes/crio.py b/sos/policies/runtimes/crio.py
index 980c3ea1..55082d07 100644
--- a/sos/policies/runtimes/crio.py
+++ b/sos/policies/runtimes/crio.py
@@ -19,6 +19,9 @@ class CrioContainerRuntime(ContainerRuntime):
name = 'crio'
binary = 'crictl'
+ def check_can_copy(self):
+ return False
+
def get_containers(self, get_all=False):
"""Get a list of containers present on the system.
diff --git a/sos/policies/runtimes/docker.py b/sos/policies/runtimes/docker.py
index e81f580e..c0cc156c 100644
--- a/sos/policies/runtimes/docker.py
+++ b/sos/policies/runtimes/docker.py
@@ -27,4 +27,7 @@ class DockerContainerRuntime(ContainerRuntime):
return True
return False
+ def check_can_copy(self):
+ return self.check_is_active(sysroot=self.policy.sysroot)
+
# vim: set et ts=4 sw=4 :
diff --git a/sos/report/plugins/__init__.py b/sos/report/plugins/__init__.py
index ca58c22c..0bdc1632 100644
--- a/sos/report/plugins/__init__.py
+++ b/sos/report/plugins/__init__.py
@@ -487,6 +487,7 @@
self.commons = commons
self.forbidden_paths = []
self.copy_paths = set()
+ self.container_copy_paths = []
self.copy_strings = []
self.collect_cmds = []
self.sysroot = commons['sysroot']
@@ -538,6 +539,7 @@
self.manifest.add_field('command_timeout', self.cmdtimeout)
self.manifest.add_list('commands', [])
self.manifest.add_list('files', [])
+ self.manifest.add_field('containers', {})
def timeout_from_options(self, optname, plugoptname, default_timeout):
"""Returns either the default [plugin|cmd] timeout value, the value as
@@ -1558,7 +1560,7 @@ class Plugin():
self.manifest.files.append(manifest_data)
def add_copy_spec(self, copyspecs, sizelimit=None, maxage=None,
- tailit=True, pred=None, tags=[]):
+ tailit=True, pred=None, tags=[], container=None):
"""Add a file, directory, or regex matching filepaths to the archive
:param copyspecs: A file, directory, or regex matching filepaths
@@ -1583,10 +1585,17 @@ class Plugin():
for this collection
:type tags: ``str`` or a ``list`` of strings
+ :param container: Container(s) from which this file should be copied
+ :type container: ``str`` or a ``list`` of strings
+
`copyspecs` will be expanded and/or globbed as appropriate. Specifying
a directory here will cause the plugin to attempt to collect the entire
directory, recursively.
+ If `container` is specified, `copyspecs` may only be explicit paths,
+ not globs as currently container runtimes do not support glob expansion
+ as part of the copy operation.
+
Note that `sizelimit` is applied to each `copyspec`, not each file
individually. For example, a copyspec of
``['/etc/foo', '/etc/bar.conf']`` and a `sizelimit` of 25 means that
@@ -1623,28 +1632,79 @@ class Plugin():
if isinstance(tags, str):
tags = [tags]
+ def get_filename_tag(fname):
+ """Generate a tag to add for a single file copyspec
+
+ This tag will be set to the filename, minus any extensions
+ except '.conf' which will be converted to '_conf'
+ """
+ fname = fname.replace('-', '_')
+ if fname.endswith('.conf'):
+ return fname.replace('.', '_')
+ return fname.split('.')[0]
+
for copyspec in copyspecs:
if not (copyspec and len(copyspec)):
return False
- if self.use_sysroot():
- copyspec = self.path_join(copyspec)
-
- files = self._expand_copy_spec(copyspec)
+ if not container:
+ if self.use_sysroot():
+ copyspec = self.path_join(copyspec)
+ files = self._expand_copy_spec(copyspec)
+ if len(files) == 0:
+ continue
+ else:
+ files = [copyspec]
- if len(files) == 0:
- continue
+ _spec_tags = []
+ if len(files) == 1:
+ _spec_tags = [get_filename_tag(files[0].split('/')[-1])]
- def get_filename_tag(fname):
- """Generate a tag to add for a single file copyspec
+ _spec_tags.extend(tags)
+ _spec_tags = list(set(_spec_tags))
- This tag will be set to the filename, minus any extensions
- except '.conf' which will be converted to '_conf'
- """
- fname = fname.replace('-', '_')
- if fname.endswith('.conf'):
- return fname.replace('.', '_')
- return fname.split('.')[0]
+ if container:
+ if isinstance(container, str):
+ container = [container]
+ for con in container:
+ if not self.container_exists(con):
+ continue
+ _tail = False
+ if sizelimit:
+ # to get just the size, stat requires a literal '%s'
+ # which conflicts with python string formatting
+ cmd = "stat -c %s " + copyspec
+ ret = self.exec_cmd(cmd, container=con)
+ if ret['status'] == 0:
+ try:
+ consize = int(ret['output'])
+ if consize > sizelimit:
+ _tail = True
+ except ValueError:
+ self._log_info(
+ "unable to determine size of '%s' in "
+ "container '%s'. Skipping collection."
+ % (copyspec, con)
+ )
+ continue
+ else:
+ self._log_debug(
+ "stat of '%s' in container '%s' failed, "
+ "skipping collection: %s"
+ % (copyspec, con, ret['output'])
+ )
+ continue
+ self.container_copy_paths.append(
+ (con, copyspec, sizelimit, _tail, _spec_tags)
+ )
+ self._log_info(
+ "added collection of '%s' from container '%s'"
+ % (copyspec, con)
+ )
+ # break out of the normal flow here as container file
+ # copies are done via command execution, not raw cp/mv
+ # operations
+ continue
# Files hould be sorted in most-recently-modified order, so that
# we collect the newest data first before reaching the limit.
@@ -1668,12 +1728,6 @@ class Plugin():
return False
return True
- _spec_tags = []
- if len(files) == 1:
- _spec_tags = [get_filename_tag(files[0].split('/')[-1])]
-
- _spec_tags.extend(tags)
-
if since or maxage:
files = list(filter(lambda f: time_filter(f), files))
@@ -1742,13 +1796,14 @@ class Plugin():
# should collect the whole file and stop
limit_reached = (sizelimit and current_size == sizelimit)
- _spec_tags = list(set(_spec_tags))
- if self.manifest:
- self.manifest.files.append({
- 'specification': copyspec,
- 'files_copied': _manifest_files,
- 'tags': _spec_tags
- })
+ if not container:
+ # container collection manifest additions are handled later
+ if self.manifest:
+ self.manifest.files.append({
+ 'specification': copyspec,
+ 'files_copied': _manifest_files,
+ 'tags': _spec_tags
+ })
def add_blockdev_cmd(self, cmds, devices='block', timeout=None,
sizelimit=None, chroot=True, runat=None, env=None,
@@ -2460,6 +2515,30 @@ class Plugin():
chdir=runat, binary=binary, env=env,
foreground=foreground, stderr=stderr)
+ def _add_container_file_to_manifest(self, container, path, arcpath, tags):
+ """Adds a file collection to the manifest for a particular container
+ and file path.
+
+ :param container: The name of the container
+ :type container: ``str``
+
+ :param path: The filename from the container filesystem
+ :type path: ``str``
+
+ :param arcpath: Where in the archive the file is written to
+ :type arcpath: ``str``
+
+ :param tags: Metadata tags for this collection
+ :type tags: ``str`` or ``list`` of strings
+ """
+ if container not in self.manifest.containers:
+ self.manifest.containers[container] = {'files': [], 'commands': []}
+ self.manifest.containers[container]['files'].append({
+ 'specification': path,
+ 'files_copied': arcpath,
+ 'tags': tags
+ })
+
def _get_container_runtime(self, runtime=None):
"""Based on policy and request by the plugin, return a usable
ContainerRuntime if one exists
@@ -2842,6 +2921,48 @@ class Plugin():
self._do_copy_path(path)
self.generate_copyspec_tags()
+ def _collect_container_copy_specs(self):
+ """Copy any requested files from containers here. This is done
+ separately from normal file collection as this requires the use of
+ a container runtime.
+
+ This method will iterate over self.container_copy_paths which is a set
+ of 5-tuples as (container, path, sizelimit, stdout, tags).
+ """
+ if not self.container_copy_paths:
+ return
+ rt = self._get_container_runtime()
+ if not rt:
+ self._log_info("Cannot collect container based files - no runtime "
+ "is present/active.")
+ return
+ if not rt.check_can_copy():
+ self._log_info("Loaded runtime '%s' does not support copying "
+ "files from containers. Skipping collections.")
+ return
+ for contup in self.container_copy_paths:
+ con, path, sizelimit, tailit, tags = contup
+ self._log_info("collecting '%s' from container '%s'" % (path, con))
+
+ arcdest = "sos_containers/%s/%s" % (con, path.lstrip('/'))
+ self.archive.check_path(arcdest, P_FILE)
+ dest = self.archive.dest_path(arcdest)
+
+ cpcmd = rt.get_copy_command(
+ con, path, dest, sizelimit=sizelimit if tailit else None
+ )
+ cpret = self.exec_cmd(cpcmd, timeout=10)
+
+ if cpret['status'] == 0:
+ if tailit:
+ # current runtimes convert files sent to stdout to tar
+ # archives, with no way to control that
+ self.archive.add_string(cpret['output'], arcdest)
+ self._add_container_file_to_manifest(con, path, arcdest, tags)
+ else:
+ self._log_info("error copying '%s' from container '%s': %s"
+ % (path, con, cpret['output']))
+
def _collect_cmds(self):
self.collect_cmds.sort(key=lambda x: x.priority)
for soscmd in self.collect_cmds:
@@ -2875,6 +2996,7 @@ class Plugin():
"""Collect the data for a plugin."""
start = time()
self._collect_copy_specs()
+ self._collect_container_copy_specs()
self._collect_cmds()
self._collect_strings()
fields = (self.name(), time() - start)
--
2.27.0
From 5114e164e38f6aa09d1efdd30ab5d2e9266272cc Mon Sep 17 00:00:00 2001
From: Jake Hunsaker <jhunsake@redhat.com>
Date: Thu, 20 Jan 2022 11:30:30 -0500
Subject: [PATCH 2/2] [Plugin] Add container-based command collections in
manifest
Following the previous commit of adding a `containers` key to a Plugin's
manifest entry, we will now make an entry in the relevant `containers`
entry for commands executed in containers.
Additionally, we will make a symlink from the relevant `sos_containers/`
path to the collection under `sos_commands/$plugin/`. This should allow
for easier spot analysis of a container by providing a kind of "micro
sos report" for each container collections happen from under the
`sos_containers/` top level directory.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
---
sos/report/plugins/__init__.py | 41 ++++++++++++++++++++++++++++++++--
1 file changed, 39 insertions(+), 2 deletions(-)
diff --git a/sos/report/plugins/__init__.py b/sos/report/plugins/__init__.py
index 0bdc1632..2988be08 100644
--- a/sos/report/plugins/__init__.py
+++ b/sos/report/plugins/__init__.py
@@ -2019,8 +2019,10 @@ class Plugin():
if pred is None:
pred = self.get_predicate(cmd=True)
for cmd in cmds:
+ container_cmd = None
if container:
ocmd = cmd
+ container_cmd = (ocmd, container)
cmd = self.fmt_container_cmd(container, cmd)
if not cmd:
self._log_debug("Skipping command '%s' as the requested "
@@ -1805,7 +1807,8 @@
env=env, binary=binary, sizelimit=sizelimit,
pred=pred, subdir=subdir, tags=tags,
changes=changes, foreground=foreground,
- priority=priority, cmd_as_tag=cmd_as_tag)
+ priority=priority, cmd_as_tag=cmd_as_tag,
+ container_cmd=container_cmd)
def add_cmd_tags(self, tagdict):
"""Retroactively add tags to any commands that have been run by this
@@ -2006,7 +2006,8 @@
stderr=True, chroot=True, runat=None, env=None,
binary=False, sizelimit=None, subdir=None,
changes=False, foreground=False, tags=[],
- priority=10, cmd_as_tag=False):
+ priority=10, cmd_as_tag=False,
+ container_cmd=False):
"""Execute a command and save the output to a file for inclusion in the
report.
@@ -2362,10 +2365,14 @@ class Plugin():
os.path.join(self.archive.get_archive_path(), outfn) if outfn else
''
)
+
if self.manifest:
manifest_cmd['filepath'] = outfn
manifest_cmd['run_time'] = run_time
self.manifest.commands.append(manifest_cmd)
+ if container_cmd:
+ self._add_container_cmd_to_manifest(manifest_cmd.copy(),
+ container_cmd)
return result
def collect_cmd_output(self, cmd, suggest_filename=None,
@@ -2539,6 +2546,36 @@ class Plugin():
'tags': tags
})
+ def _add_container_cmd_to_manifest(self, manifest, contup):
+ """Adds a command collection to the manifest for a particular container
+ and creates a symlink to that collection from the relevant
+ sos_containers/ location
+
+ :param manifest: The manifest entry for the command
+ :type manifest: ``dict``
+
+ :param contup: A tuple of (original_cmd, container_name)
+ :type contup: ``tuple``
+ """
+
+ cmd, container = contup
+ if container not in self.manifest.containers:
+ self.manifest.containers[container] = {'files': [], 'commands': []}
+ manifest['exec'] = cmd
+ manifest['command'] = cmd.split(' ')[0]
+ manifest['parameters'] = cmd.split(' ')[1:]
+
+ _cdir = "sos_containers/%s/sos_commands/%s" % (container, self.name())
+ _outloc = "../../../../%s" % manifest['filepath']
+ cmdfn = self._mangle_command(cmd)
+ conlnk = "%s/%s" % (_cdir, cmdfn)
+
+ self.archive.check_path(conlnk, P_FILE)
+ os.symlink(_outloc, self.archive.dest_path(conlnk))
+
+ manifest['filepath'] = conlnk
+ self.manifest.containers[container]['commands'].append(manifest)
+
def _get_container_runtime(self, runtime=None):
"""Based on policy and request by the plugin, return a usable
ContainerRuntime if one exists
--
2.27.0
From 2ae16e0245e1b01b8547e507abb69c11871a8467 Mon Sep 17 00:00:00 2001
From: Jake Hunsaker <jhunsake@redhat.com>
Date: Mon, 21 Feb 2022 14:37:09 -0500

1797
SOURCES/sos-bz2071825-merged-8.6.z.patch Normal file
View File

File diff suppressed because it is too large Load Diff

33
SPECS/sos.spec
View File

@ -5,7 +5,7 @@
Summary: A set of tools to gather troubleshooting information from a system
Name: sos
Version: 4.2
Release: 15%{?dist}.alma
Release: 19%{?dist}.alma
Group: Applications/System
Source0: https://github.com/sosreport/sos/archive/%{version}/sos-%{version}.tar.gz
Source1: sos-audit-%{auditversion}.tgz
@ -44,6 +44,7 @@ Patch20: sos-bz2041488-virsh-in-foreground.patch
Patch21: sos-bz2042966-ovn-proper-package-enablement.patch
Patch22: sos-bz2054882-plugopt-logging-effective-opts.patch
Patch23: sos-bz2055547-honour-plugins-timeout-hardcoded.patch
Patch24: sos-bz2071825-merged-8.6.z.patch
# almalinux
Patch1000: sos-almalinux-branding.patch
@ -80,6 +81,7 @@ support technicians and developers.
%patch21 -p1
%patch22 -p1
%patch23 -p1
%patch24 -p1
# almalinux
%patch1000 -p1
@ -149,11 +151,38 @@ of the system. Currently storage and filesystem commands are audited.
%ghost /etc/audit/rules.d/40-sos-storage.rules
%changelog
* Tue May 10 2022 Andrew Lukoshko <alukoshko@almalinux.org> - 4.2-15.alma
* Wed May 25 2022 Andrew Lukoshko <alukoshko@almalinux.org> - 4.2-19.alma
- Debrand for AlmaLinux
* Tue Nov 30 2021 Pavel Moravec <pmoravec@redhat.com> = 4.1-9
* Mon May 09 2022 Jan Jansky <jjansky@redhat.com> = 4.2-19
- OCP backport
Resolves: bz2071824
- [pacemaker] Update collect cluster profile for pacemaker
Resolves: bz2071695
- [Plugin] oom excessive memory usage
Resolves: bz2071825
* Fri Apr 22 2022 Jan Jansky <jjansky@redhat.com> = 4.2-18
- OCP backport
Resolves: bz2071824
- [pacemaker] Update collect cluster profile for pacemaker
Resolves: bz2071695
- [Plugin] oom excessive memory usage
Resolves: bz2071825
* Wed Apr 20 2022 Jan Jansky <jjansky@redhat.com> = 4.2-17
- increased release version
* Wed Apr 13 2022 Jan Jansky <jjansky@redhat.com> = 4.2-16
- OCP backport
Resolves: bz2071824
- [pacemaker] Update collect cluster profile for pacemaker
Resolves: bz2071695
- [Plugin] oom excessive memory usage
Resolves: bz2071825
* Wed Feb 23 2022 Pavel Moravec <pmoravec@redhat.com> = 4.2-15
- [sosnode] Handle downstream versioning for runtime option
Resolves: bz2036697