rpms/rteval
7
0
Fork 0
Code Issues Pull Requests Releases Activity

Rebase to upstream rteval-3.9

Browse Source 
Resolves: RHEL-65483
Signed-off-by: John Kacur <jkacur@redhat.com>
This commit is contained in:
John Kacur 2024-11-15 13:57:57 -05:00
parent 00cdc543d3
commit 1632376582
26 changed files with 8 additions and 3389 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -8,3 +8,4 @@
/0002-rteval-Remove-upstream-spec-file.patch /0002-rteval-Remove-upstream-spec-file.patch
/rteval-3.8.tar.xz /rteval-3.8.tar.xz
/linux-6.12-rc4.tar.gz /linux-6.12-rc4.tar.gz
/rteval-3.9.tar.xz

109
Updated-rteval-man-page.patch
View File

@ -1,109 +0,0 @@
From 66b6c5851ea2241b6e368a1538dc10fd1f3f76f7 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Wed, 5 Jun 2024 14:04:54 -0400
Subject: [PATCH] Updated rteval man page
The rteval man page should reflect the functionality of the
current iteration of rteval. Currently the rteval man page
reflects the functionality of an outdated version. This patch
removes deprecated options and adds options that have been
added in recent updates.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
---
doc/rteval.8 | 50 +++++++++++++++++++++++++++++++++-----------------
1 file changed, 33 insertions(+), 17 deletions(-)
diff --git a/doc/rteval.8 b/doc/rteval.8
index 578de370effa..264bf5207716 100644
--- a/doc/rteval.8
+++ b/doc/rteval.8
@@ -63,6 +63,9 @@ for example to execute a ten minute run of rteval:
.B \-v, \-\-verbose
Increase the verbosity of output during the test run
.TP
+.B \-q, \-\-quiet
+Set rteval to run in quiet mode.
+.TP
.B \-w WORKDIR, \-\-workdir=WORKDIR
Tell rteval to change directory to WORKDIR before creating any
subdirectories for report files. The default WORKDIR is the directory
@@ -71,11 +74,8 @@ in which rteval was started.
.B \-l LOADDIR, \-\-loaddir=LOADDIR
Tell rteval where to find the source for the loads
.TP
-.B \-\-loads\-cpulist=CPULIST
-List of CPUs where loads will run
-.TP
-.B \-\-measurement-cpulist=CPULIST
-List of CPUs where measurement applciation will run
+.B \-i INSTALLDIR, \-\-installdir=INSTALLDIR
+Tell rteval where to locate installed templates.
.TP
.B \-s, \-\-sysreport
Have rteval run the sysreport utility after a run to gather
@@ -103,11 +103,25 @@ Log the output of the loads in the report directory
.B \-O, \-\-onlyload
Only run the loads (don't run measurement threads)
.TP
+.B \-V, \-\-version
+Print rteval version and exit.
+.TP
.B \-S KERNEL_VERSION, \-\-source\-download=KERNEL_VERSION
download a source kernel from kernel.org and exit
+.SH GROUP OPTIONS
+.TP
+.B \-\-loads\-cpulist=CPULIST
+List of CPUs where loads will run
+.TP
+.B \-\-measurement-cpulist=CPULIST
+List of CPUs where measurement applciation will run
+.TP
+.B \-\-measurement-run-on-isolcpus
+Include isolated CPUs in default cpulist
+
-.SH MODULE OPTIONS
+.SH CYCLICTEST OPTIONS
These are options that affect the execution behavior of the measurement and load modules.
.TP
.B \-\-cyclictest-priority=PRIORITY
@@ -116,21 +130,23 @@ SCHED_FIFO priority for measurement threads (default: 95)
.B \-\-cyclictest-interval=INTERVAL
Measurement thread interval in microseconds (default: 100)
.TP
-.B \-\-cyclictest-distance=DISTANCE
-Interval increment in microseconds (default: 0)
-.TP
.B \-\-cyclictest-buckets=NBUCKETS
Number of 1 microsecond histogram buckets (default: 3500)
.TP
-.B \-\-hackbench-jobspercore=N
-Number of jobs per online-core for hackbench load
+.B \-\-cyclictest-breaktrace=USEC
+Send a break trace command if latency exceeds the given number of microseconds.
+.TP
+.B \-\-cyclictest-threshold=USEC
+Exit rteval if latency exceeds the given number of microseconds.
+.TP
+.B \-\-stressng-option=OPTION
+Pass in command line options for the stress-ng package.
+.TP
+.B \-\-stressng-arg=ARG
+Pass in command line arguments for the stress-ng package.
.TP
-.B \-\-kcompile-jobspercore=N
-Number of jobs per online-core for kernel compile load
-.\" .SH SEE ALSO
-.\" .BR bar (1),
-.\" .BR baz (1).
-.\" .br
+.B \-\-stressng-timeout=T
+Timeout after T seconds.
.SH FILES
.BR rteval-YYYYMMDD-S
--
2.45.1

254
rteval-Add-module-for-tuned-state.patch
View File

@ -1,254 +0,0 @@
From 2b589a971f8895c81e13b5cfeb45879a28ea8cc6 Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 24 Jul 2024 09:37:42 +0200
Subject: [PATCH 1/2] rteval: Add module for tuned state
Add a sysinfo module for collecting the tuned state. Three properties
are collected:
- whether tuned is present
- what the active tuned profile is
- whether tuned profile verification passes
In case of a failed verification, the tuned log is also collected.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/sysinfo/__init__.py | 5 +-
rteval/sysinfo/tuned.py | 191 +++++++++++++++++++++++++++++++++++++
2 files changed, 195 insertions(+), 1 deletion(-)
create mode 100644 rteval/sysinfo/tuned.py
diff --git a/rteval/sysinfo/__init__.py b/rteval/sysinfo/__init__.py
index 09af52e..4b7b03c 100644
--- a/rteval/sysinfo/__init__.py
+++ b/rteval/sysinfo/__init__.py
@@ -15,10 +15,11 @@ from rteval.sysinfo.memory import MemoryInfo
from rteval.sysinfo.osinfo import OSInfo
from rteval.sysinfo.newnet import NetworkInfo
from rteval.sysinfo.cmdline import cmdlineInfo
+from rteval.sysinfo.tuned import TunedInfo
from rteval.sysinfo import dmi
class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
- MemoryInfo, OSInfo, NetworkInfo, cmdlineInfo):
+ MemoryInfo, OSInfo, NetworkInfo, cmdlineInfo, TunedInfo):
def __init__(self, config, logger=None):
self.__logger = logger
KernelInfo.__init__(self, logger=logger)
@@ -28,6 +29,7 @@ class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
OSInfo.__init__(self, logger=logger)
cmdlineInfo.__init__(self, logger=logger)
NetworkInfo.__init__(self, logger=logger)
+ TunedInfo.__init__(self, logger=logger)
# Parse initial DMI decoding errors
self.ProcessWarnings()
@@ -49,6 +51,7 @@ class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
report_n.addChild(MemoryInfo.MakeReport(self))
report_n.addChild(dmi.DMIinfo.MakeReport(self))
report_n.addChild(cmdlineInfo.MakeReport(self))
+ report_n.addChild(TunedInfo.MakeReport(self))
return report_n
diff --git a/rteval/sysinfo/tuned.py b/rteval/sysinfo/tuned.py
new file mode 100644
index 0000000..063fcbf
--- /dev/null
+++ b/rteval/sysinfo/tuned.py
@@ -0,0 +1,191 @@
+# -*- coding: utf-8 -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright 2024 Tomas Glozar <tglozar@redhat.com>
+#
+"""tuned sysinfo module"""
+import shutil
+import subprocess
+import sys
+import libxml2
+from rteval.Log import Log
+
+TUNED_ADM = "tuned-adm"
+TUNED_LOG_PATH = "/var/log/tuned/tuned.log"
+TUNED_VERIFY_START_LINE = "INFO tuned.daemon.daemon: verifying " \
+ "profile(s): realtime"
+
+
+def tuned_present():
+ """
+ Checks if tuned is present on the system
+ :return: True if tuned is present, False otherwise
+ """
+ return shutil.which(TUNED_ADM) is not None
+
+
+def tuned_active_profile():
+ """
+ Gets tuned active profile.
+ :return: Tuned profile (as a string) or "unknown"
+ """
+ try:
+ result = subprocess.check_output([TUNED_ADM, "active"])
+ except (OSError, subprocess.CalledProcessError):
+ return "unknown"
+ result = result.decode("utf-8")
+ split_result = result.split(": ")
+ if len(split_result) < 2:
+ return "unknown"
+ return split_result[1].strip()
+
+
+def tuned_verify():
+ """
+ Verifies if tuned profile is applied properly
+ :return: "success", "failure" or "unknown"
+ """
+ try:
+ result = subprocess.run([TUNED_ADM, "verify"],
+ stdout=subprocess.PIPE, check=False).stdout
+ except (OSError, subprocess.CalledProcessError):
+ return "unknown"
+ result = result.decode("utf-8")
+ if result.startswith("Verification succeeded"):
+ return "success"
+ if result.startswith("Verification failed"):
+ return "failure"
+ return "unknown"
+
+
+def tuned_get_log():
+ """
+ Read entries related to last profile verification from tuned log
+ :return: List of strings containing the entires, or None if no
+ verification is found in the log
+ """
+ try:
+ with open(TUNED_LOG_PATH, "r", encoding="utf-8") as file:
+ lines = file.readlines()
+ # Find start of last verification
+ start = None
+ for i in reversed(range(len(lines))):
+ if TUNED_VERIFY_START_LINE in lines[i]:
+ start = i
+ break
+ if start is None:
+ return None
+ return lines[start:]
+ except OSError:
+ return None
+
+
+class TunedInfo:
+ """
+ Gather information about tuned and make an XML report.
+ Collected information:
+ - whether tuned is present
+ - which tuned profile is active
+ - whether the tuned profile is applied correctly
+ - if not applied correctly, collect relevant info from log
+ """
+ def __init__(self, logger=None):
+ self.__logger = logger
+
+ def __log(self, logtype, msg):
+ if self.__logger:
+ self.__logger.log(logtype, msg)
+
+ def tuned_state_get(self):
+ """
+ Gets the state of tuned on the machine
+ :return: A dictionary describing the tuned state
+ """
+ result = {
+ "present": tuned_present()
+ }
+ if not result["present"]:
+ self.__log(Log.DEBUG, "tuned-adm not found; skipping tuned "
+ "sysinfo collection")
+ return result
+ result["active_profile"] = tuned_active_profile()
+ if result["active_profile"] == "unknown":
+ self.__log(Log.DEBUG, "could not retrieve tuned active profile")
+ return result
+ result["verified"] = tuned_verify()
+ if result["verified"] == "unknown":
+ self.__log(Log.DEBUG, "could not verify tuned state")
+ if result["verified"] == "failure":
+ # Include log to see cause to failure
+ result["verification_log"] = tuned_get_log()
+
+ return result
+
+ def MakeReport(self):
+ """
+ Create XML report
+ :return: libxml2 node containing the report
+ """
+ tuned = self.tuned_state_get()
+
+ rep_n = libxml2.newNode("Tuned")
+ rep_n.newProp("present", str(int(tuned["present"])))
+ for key, value in tuned.items():
+ if key == "present":
+ continue
+ child = libxml2.newNode(key)
+ if key == "verification_log":
+ if value is None:
+ self.__log(Log.WARN, "could not get verification log")
+ continue
+ for line in value:
+ # <date> <time> <log-level> <message>
+ line = line.split(" ", 3)
+ if len(line) != 4:
+ continue
+ line_child = libxml2.newNode("entry")
+ line_child.newProp("date", line[0])
+ line_child.newProp("time", line[1])
+ line_child.newProp("level", line[2])
+ line_child.setContent(line[3].strip())
+ child.addChild(line_child)
+ else:
+ child.setContent(value)
+ rep_n.addChild(child)
+
+ return rep_n
+
+
+def unit_test(rootdir):
+ try:
+ # Helper function tests
+ result = tuned_present()
+ print("tuned present:", result)
+ assert isinstance(result, bool), "__tuned_present() should return bool"
+ result = tuned_active_profile()
+ print("tuned active profile:", result)
+ assert isinstance(result, str), "__tuned_active_profile() should " \
+ "return string"
+ result = tuned_verify()
+ print("tuned verification state:", result)
+ assert isinstance(result, str), "__tuned_verify() should return string"
+ result = tuned_get_log()
+ assert isinstance(result, list) or result is None, \
+ "__tuned_get_log() should return list or None"
+
+ # Class tests
+ tuned = TunedInfo()
+ result = tuned.tuned_state_get()
+ print(result)
+ assert isinstance(result, dict), "TunedInfo.tuned_state_get() " \
+ "should return dict"
+ tuned.MakeReport()
+
+ return 0
+ except Exception as err:
+ print(f"** EXCEPTION: {str(err)}")
+ return 1
+
+
+if __name__ == '__main__':
+ sys.exit(unit_test(None))
--
2.45.2

190
rteval-Add-noload-option.patch
View File

@ -1,190 +0,0 @@
From 1a76277bc6a5e94d858497fdc8fa721540fa6af7 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 19 Jul 2024 14:58:41 -0500
Subject: [PATCH 5/7] rteval: Add --noload option
This option allows measurement to be done without any loads.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
doc/rteval.8 | 3 +++
rteval-cmd | 9 +++++++
rteval/__init__.py | 54 ++++++++++++++++++++++--------------------
rteval/rtevalReport.py | 3 ++-
rteval/rteval_text.xsl | 9 ++++++-
5 files changed, 50 insertions(+), 28 deletions(-)
diff --git a/doc/rteval.8 b/doc/rteval.8
index 9e2b377752e5..1981ac7b3913 100644
--- a/doc/rteval.8
+++ b/doc/rteval.8
@@ -108,6 +108,9 @@ Print rteval version and exit.
.TP
.B \-S KERNEL_VERSION, \-\-source\-download=KERNEL_VERSION
download a source kernel from kernel.org and exit
+.TP
+.B \-\-noload
+Only run the measurements (don't run loads)
.SH GROUP OPTIONS
.TP
diff --git a/rteval-cmd b/rteval-cmd
index 5cb6d7a44523..36b167a034e5 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -146,6 +146,9 @@ def parse_options(cfg, parser, cmdargs):
parser.add_argument("-S", "--source-download", nargs="*", dest="rteval___srcdownload",
type=str, default=None, metavar="KERNEL_VERSION",
help='download a source kernel from kernel.org and exit')
+ parser.add_argument("--noload", dest="rteval___noload",
+ action="store_true", default=False,
+ help="only run the measurements (don't run loads)")
if not cmdargs:
@@ -273,6 +276,12 @@ if __name__ == '__main__':
measuremods.SetupModuleOptions(parser)
cmd_args = parse_options(config, parser, sys.argv[1:])
+ if rtevcfg.noload:
+ if rtevcfg.onlyload:
+ # Make up your mind!
+ raise RuntimeError('The --noload and --onlyload options are incompatible.')
+ loadmods = None
+
# download kernel tarball
if rtevcfg.srcdownload:
logger.log(Log.DEBUG, f"Kernel Version to download = {rtevcfg.srcdownload}")
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 226d14f80f48..4d3e0c23e5ab 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -46,9 +46,6 @@ class RtEval(rtevalReport):
if not isinstance(config, rtevalConfig.rtevalConfig):
raise TypeError("config variable is not an rtevalConfig object")
- if not isinstance(loadmods, LoadModules):
- raise TypeError("loadmods variable is not a LoadModules object")
-
if not isinstance(measuremods, MeasurementModules):
raise TypeError("measuremods variable is not a MeasurementModules object")
@@ -111,20 +108,21 @@ class RtEval(rtevalReport):
except Exception as err:
raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]")
+ params = {'workdir':self.__rtevcfg.workdir,
+ 'reportdir':self.__reportdir and self.__reportdir or "",
+ 'builddir':builddir,
+ 'srcdir':self.__rtevcfg.srcdir,
+ 'verbose': self.__rtevcfg.verbose,
+ 'debugging': self.__rtevcfg.debugging,
+ 'numcores':self._sysinfo.cpu_getCores(True),
+ 'logging':self.__rtevcfg.logging,
+ 'memsize':self._sysinfo.mem_get_size(),
+ 'numanodes':self._sysinfo.mem_get_numa_nodes(),
+ 'duration': float(self.__rtevcfg.duration),
+ }
+
if self._loadmods:
self.__logger.log(Log.INFO, "Preparing load modules")
- params = {'workdir':self.__rtevcfg.workdir,
- 'reportdir':self.__reportdir and self.__reportdir or "",
- 'builddir':builddir,
- 'srcdir':self.__rtevcfg.srcdir,
- 'verbose': self.__rtevcfg.verbose,
- 'debugging': self.__rtevcfg.debugging,
- 'numcores':self._sysinfo.cpu_getCores(True),
- 'logging':self.__rtevcfg.logging,
- 'memsize':self._sysinfo.mem_get_size(),
- 'numanodes':self._sysinfo.mem_get_numa_nodes(),
- 'duration': float(self.__rtevcfg.duration),
- }
self._loadmods.Setup(params)
self.__logger.log(Log.INFO, "Preparing measurement modules")
@@ -144,15 +142,18 @@ class RtEval(rtevalReport):
print(f"rteval run on {os.uname()[2]} started at {time.asctime()}")
onlinecpus = self._sysinfo.cpu_getCores(True)
- cpulist = self._loadmods._cfg.GetSection("loads").cpulist
- if cpulist:
- print(f"started {self._loadmods.ModulesLoaded()} loads on cores {cpulist}", end=' ')
- else:
- print(f"started {self._loadmods.ModulesLoaded()} loads on {onlinecpus} cores", end=' ')
- if self._sysinfo.mem_get_numa_nodes() > 1:
- print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
- else:
- print("")
+ if self._loadmods:
+ cpulist = self._loadmods._cfg.GetSection("loads").cpulist
+ if cpulist:
+ print(f"started {self._loadmods.ModulesLoaded()} loads on cores {cpulist}",
+ end=' ')
+ else:
+ print(f"started {self._loadmods.ModulesLoaded()} loads on {onlinecpus} cores",
+ end=' ')
+ if self._sysinfo.mem_get_numa_nodes() > 1:
+ print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
+ else:
+ print("")
cpulist = self._measuremods._cfg.GetSection("measurement").cpulist
if cpulist:
print(f"started measurement threads on cores {cpulist}")
@@ -192,7 +193,7 @@ class RtEval(rtevalReport):
if threading.active_count() < nthreads:
raise RuntimeError("load thread died!")
- if not load_avg_checked:
+ if self._loadmods and not load_avg_checked:
self._loadmods.SaveLoadAvg()
load_avg_checked = 5
else:
@@ -202,7 +203,8 @@ class RtEval(rtevalReport):
left_to_run = stoptime - currtime
self.__show_remaining_time(left_to_run)
rpttime = currtime + report_interval
- print(f"load average: {self._loadmods.GetLoadAvg():.2f}")
+ if self._loadmods:
+ print(f"load average: {self._loadmods.GetLoadAvg():.2f}")
currtime = time.time()
self.__logger.log(Log.DEBUG, "out of measurement loop")
diff --git a/rteval/rtevalReport.py b/rteval/rtevalReport.py
index 57d99f520f50..7379a7904f3f 100644
--- a/rteval/rtevalReport.py
+++ b/rteval/rtevalReport.py
@@ -57,7 +57,8 @@ class rtevalReport:
self.__xmlreport.AppendXMLnodes(self._sysinfo.MakeReport())
# Add load info
- self.__xmlreport.AppendXMLnodes(self._loadmods.MakeReport())
+ if self._loadmods:
+ self.__xmlreport.AppendXMLnodes(self._loadmods.MakeReport())
# Add measurement data
self.__xmlreport.AppendXMLnodes(self._measuremods.MakeReport())
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index b801679abcc5..96846aaf4d54 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -15,7 +15,14 @@
<xsl:text>&#10;</xsl:text>
<xsl:text> Loads: </xsl:text>
- <xsl:value-of select="loads/@loads"/><xsl:text> loads run on cores </xsl:text><xsl:value-of select="loads/@loadcpus"/>
+ <xsl:choose>
+ <xsl:when test="loads">
+ <xsl:value-of select="loads/@loads"/><xsl:text> loads run on cores </xsl:text><xsl:value-of select="loads/@loadcpus"/>
+ </xsl:when>
+ <xsl:otherwise>
+ <xsl:text>none</xsl:text>
+ </xsl:otherwise>
+ </xsl:choose>
<xsl:text>&#10;</xsl:text>
<xsl:text> Measurement: </xsl:text>
--
2.45.2

46
rteval-Add-tuned-state-to-rteval-text-report.patch
View File

@ -1,46 +0,0 @@
From e97e27f37fb5a397969875787e062c4eb22323fa Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 7 Aug 2024 08:33:33 +0200
Subject: [PATCH 2/2] rteval: Add tuned state to rteval text report
Add a section "Tuned state" to the text report generated by rteval. Show
tuned active state if tuned is present, otherwise, show a message that
tuned is not present. If the active state is known, show also the
verification result.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/rteval_text.xsl | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 96846aa..70f3e0f 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -143,6 +143,22 @@
</xsl:when>
<xsl:otherwise>(unknown)</xsl:otherwise>
</xsl:choose>
+ <xsl:text>&#10;</xsl:text>
+
+ <xsl:text> Tuned state: </xsl:text>
+ <xsl:choose>
+ <xsl:when test="SystemInfo/Tuned/@present='1'">
+ <xsl:value-of select="SystemInfo/Tuned/active_profile"/>
+ <xsl:text> profile</xsl:text>
+ <xsl:if test="SystemInfo/Tuned/active_profile != 'unknown'">
+ <xsl:text>, verification: </xsl:text>
+ <xsl:value-of select="SystemInfo/Tuned/verified"/>
+ </xsl:if>
+ </xsl:when>
+ <xsl:otherwise>
+ <xsl:text>not present</xsl:text>
+ </xsl:otherwise>
+ </xsl:choose>
<xsl:text>&#10;&#10;</xsl:text>
<xsl:text> System load:&#10;</xsl:text>
--
2.45.2

221
rteval-Added-functionality-to-allow-user-to-set-the-.patch
View File

@ -1,221 +0,0 @@
From b3c96935c82b7410422ca2973398ac9d0272c844 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Fri, 2 Aug 2024 11:46:35 -0400
Subject: [PATCH 1/2] rteval: Added functionality to allow user to set the
cstate of specified cpus when running rteval
We would like to be able to set the idle states of CPUs while running
rteval.
This patch adds the file cpupower.py and option '--idle-set' within
rteval to use cpupower. The set idle state is applied to the cpulist
given by '--measurement-cpulist'.
cpupower.py provides the infrastructure to interface and execute the cpupower
command, and the options in rteval-cmd let the user specify the idle state
to be set and the CPUs to set it on.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 9 ++
rteval/cpupower.py | 125 +++++++++++++++++++++++++
rteval/modules/__init__.py | 2 +-
rteval/modules/measurement/__init__.py | 2 +
4 files changed, 137 insertions(+), 1 deletion(-)
create mode 100644 rteval/cpupower.py
diff --git a/rteval-cmd b/rteval-cmd
index 19c82a0b64b3..f440a8a22622 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -29,6 +29,7 @@ from rteval.Log import Log
from rteval import RtEval, rtevalConfig
from rteval.modules.loads import LoadModules
from rteval.modules.measurement import MeasurementModules
+from rteval import cpupower
from rteval.version import RTEVAL_VERSION
from rteval.systopology import SysTopology, parse_cpulist_from_config
from rteval.modules.loads.kcompile import ModuleParameters
@@ -402,6 +403,10 @@ if __name__ == '__main__':
if not os.path.isdir(rtevcfg.workdir):
raise RuntimeError(f"work directory {rtevcfg.workdir} does not exist")
+ # if idle-set has been specified, enable the idle state via cpupower
+ if msrcfg.idlestate:
+ cpupower_controller = cpupower.Cpupower(msrcfg.cpulist, msrcfg.idlestate, logger=logger)
+ cpupower_controller.enable_idle_state()
rteval = RtEval(config, loadmods, measuremods, logger)
rteval.Prepare(rtevcfg.onlyload)
@@ -421,6 +426,10 @@ if __name__ == '__main__':
ec = rteval.Measure()
logger.log(Log.DEBUG, f"exiting with exit code: {ec}")
+ # restore previous idle state settings
+ if msrcfg.idlestate:
+ cpupower_controller.restore_idle_states()
+
sys.exit(ec)
except KeyboardInterrupt:
sys.exit(0)
diff --git a/rteval/cpupower.py b/rteval/cpupower.py
new file mode 100644
index 000000000000..37c4d33f1df4
--- /dev/null
+++ b/rteval/cpupower.py
@@ -0,0 +1,125 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+# Copyright 2024 Anubhav Shelat <ashelat@redhat.com>
+""" Object to execute cpupower tool """
+
+import subprocess
+import os
+import shutil
+import sys
+from rteval.Log import Log
+from rteval.systopology import SysTopology as SysTop
+from rteval import cpulist_utils
+
+PATH = '/sys/devices/system/cpu/'
+
+class Cpupower:
+ """ class to store data for executing cpupower and restoring idle state configuration """
+ def __init__(self, cpulist, idlestate, logger=None):
+ if not self.cpupower_present():
+ print('cpupower not found')
+ sys.exit(1)
+
+ self.__idle_state = int(idlestate)
+ self.__states = os.listdir(PATH + 'cpu0/cpuidle/')
+ # self.__idle_states is a dict with cpus as keys,
+ # and another dict as the value. The value dict
+ # has idle states as keys and a boolean as the
+ # value indicating if the state is disabled.
+ self.__idle_states = {}
+ self.__name = "cpupower"
+ self.__online_cpus = SysTop().online_cpus()
+ self.__cpulist = cpulist
+ self.__logger = logger
+
+
+ def _log(self, logtype, msg):
+ """ Common log function for rteval modules """
+ if self.__logger:
+ self.__logger.log(logtype, f"[{self.__name}] {msg}")
+
+
+ def enable_idle_state(self):
+ """ Use cpupower to set the idle state """
+ self.get_idle_states()
+
+ # ensure that idle state is in range of available idle states
+ if self.__idle_state > len(self.__states) - 1 or self.__idle_state < 0:
+ print(f'Idle state {self.__idle_state} is out of range')
+ sys.exit(1)
+
+ # enable all idle states to a certain depth, and disable any deeper idle states
+ with open(os.devnull, 'wb') as buffer:
+ for state in self.__states:
+ s = state.strip("state")
+ if int(s) > self.__idle_state:
+ self.run_cpupower(['cpupower', '-c', self.__cpulist,'idle-set', '-d', s], buffer)
+ else:
+ self.run_cpupower(['cpupower', '-c', self.__cpulist,'idle-set', '-e', s], buffer)
+
+ self._log(Log.DEBUG, f'Idle state depth {self.__idle_state} enabled on CPUs {self.__cpulist}')
+
+
+ def run_cpupower(self, args, output_buffer=None):
+ """ execute cpupower """
+ try:
+ subprocess.run(args, check=True, stdout=output_buffer)
+ except subprocess.CalledProcessError:
+ print('cpupower failed')
+ sys.exit(1)
+
+
+ def get_idle_states(self):
+ """ Store the current idle state setting """
+ for cpu in self.__online_cpus:
+ self.__idle_states[cpu] = {}
+ for state in self.__states:
+ fp = os.path.join(PATH, 'cpu' + str(cpu) + '/cpuidle/' + state + '/disable')
+ self.__idle_states[cpu][state] = self.read_idle_state(fp)
+
+
+ def restore_idle_states(self):
+ """ restore the idle state setting """
+ for cpu, states in self.__idle_states.items():
+ for state, disabled in states.items():
+ fp = os.path.join(PATH, 'cpu' + str(cpu) + '/cpuidle/' + state + '/disable')
+ self.write_idle_state(fp, disabled)
+ self._log(Log.DEBUG, 'Idle state settings restored')
+
+
+ def read_idle_state(self, file):
+ """ read the disable value for an idle state """
+ with open(file, 'r', encoding='utf-8') as f:
+ return f.read(1)
+
+
+ def write_idle_state(self, file, state):
+ """ write the disable value for and idle state """
+ with open(file, 'w', encoding='utf-8') as f:
+ f.write(state)
+
+
+ def get_idle_info(self):
+ """ execute cpupower idle-info """
+ self.run_cpupower(['cpupower', 'idle-info'])
+
+
+ def cpupower_present(self):
+ """ check if cpupower is downloaded """
+ return shutil.which("cpupower") is not None
+
+
+if __name__ == '__main__':
+ l = Log()
+ l.SetLogVerbosity(Log.DEBUG)
+
+ online_cpus = cpulist_utils.collapse_cpulist(SysTop().online_cpus())
+ idlestate = '1'
+ info = True
+
+ cpupower = Cpupower(online_cpus, idlestate, logger=l)
+ if idlestate:
+ cpupower.enable_idle_state()
+ cpupower.restore_idle_states()
+ print()
+ cpupower.get_idle_info()
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index d7792108d5b8..acd6330788e2 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -282,7 +282,7 @@ reference from the first import"""
grparser.add_argument(f'--{self.__modtype}-cpulist',
dest=f'{self.__modtype}___cpulist', action='store', default="",
help=f'CPU list where {self.__modtype} modules will run',
- metavar='LIST')
+ metavar='CPULIST')
for (modname, mod) in list(self.__modsloaded.items()):
opts = mod.ModuleParameters()
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index ecadd0885991..9314d1cb6bbc 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -38,6 +38,8 @@ class MeasurementModules(RtEvalModules):
default=self._cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
== "true",
help="Include isolated CPUs in default cpulist")
+ grparser.add_argument('--idle-set', dest='measurement___idlestate', metavar='IDLESTATE',
+ default=None, help='Idle state depth to set on cpus running measurement modules')
def Setup(self, modparams):
--
2.45.2

83
rteval-Change-constant-name-to-uppercase.patch
View File

@ -1,83 +0,0 @@
From d6c6966026eb640d545ef7b5792c532b31a9f796 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Mon, 30 Sep 2024 15:10:51 -0400
Subject: [PATCH 1/2] rteval: Change constant name to uppercase
Change constant name earlystop to EARLYSTOP
add "from err" when re-raising RuntimeError
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/__init__.py | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 8ded374d287e..7c13e84e03bf 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -28,7 +28,7 @@ from rteval import version
RTEVAL_VERSION = version.RTEVAL_VERSION
-earlystop = False
+EARLYSTOP = False
stopsig = threading.Event()
def sig_handler(signum, frame):
@@ -106,7 +106,7 @@ class RtEval(rtevalReport):
if not onlyload or self.__rtevcfg.logging:
self.__reportdir = self._make_report_dir(self.__rtevcfg.workdir, "summary.xml")
except Exception as err:
- raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]")
+ raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]") from err
params = {'workdir':self.__rtevcfg.workdir,
'reportdir':self.__reportdir and self.__reportdir or "",
@@ -131,7 +131,7 @@ class RtEval(rtevalReport):
def __RunMeasurement(self):
- global earlystop
+ global EARLYSTOP
measure_start = None
try:
@@ -186,7 +186,7 @@ class RtEval(rtevalReport):
stopsig.wait(min(stoptime - currtime, 60.0))
if not self._measuremods.isAlive():
stoptime = currtime
- earlystop = True
+ EARLYSTOP = True
self.__logger.log(Log.WARN,
"Measurement threads did not use the full time slot. Doing a controlled stop.")
@@ -214,7 +214,7 @@ class RtEval(rtevalReport):
except RuntimeError as err:
if not stopsig.is_set():
- raise RuntimeError(f"appeared during measurement: {err}")
+ raise RuntimeError(f"appeared during measurement: {err}") from err
finally:
# stop measurement threads
@@ -234,7 +234,7 @@ class RtEval(rtevalReport):
def Measure(self):
""" Run the full measurement suite with reports """
- global earlystop
+ global EARLYSTOP
rtevalres = 0
measure_start = self.__RunMeasurement()
@@ -242,7 +242,7 @@ class RtEval(rtevalReport):
if self.__rtevcfg.sysreport:
self._sysinfo.run_sysreport(self.__reportdir)
- if earlystop:
+ if EARLYSTOP:
rtevalres = 1
self._sysinfo.copy_dmesg(self.__reportdir)
self._tar_results()
--
2.46.2

84
rteval-Enforce-only-one-latency-measurement-module-a.patch
View File

@ -1,84 +0,0 @@
From d6f62a5e52843e0b2651268e350a3c0ebe5c543b Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:17 -0500
Subject: [PATCH 4/7] rteval: Enforce only one latency measurement module at a
time
Latency modules will step on each other's toes if run at the same time
(on the same CPU, though that's an enhancement for later), so only
run one of them. A priority mechanism allows selecting
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 11 +++++++++++
rteval/modules/measurement/cyclictest.py | 1 +
rteval/modules/measurement/timerlat.py | 1 +
3 files changed, 13 insertions(+)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index de1ddc4628c1..2a4eafae71c7 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -40,6 +40,7 @@ class rtevalModulePrototype(threading.Thread):
"finished": threading.Event()}
self._donotrun = False
self._exclusive = False
+ self._latency = False
self.__timestamps = {}
self.__sleeptime = 2.0
@@ -67,6 +68,11 @@ class rtevalModulePrototype(threading.Thread):
self._exclusive = True
+ def set_latency(self):
+ """ Sets the module as an exclusive latency measurer """
+ self._latency = True
+
+
def set_donotrun(self):
""" set a module's donotrun field to True """
self._donotrun = True
@@ -412,9 +418,14 @@ class RtEvalModules:
self._logger.log(Log.INFO, f"Preparing {self._module_type} modules")
exclusive = 0
+ latency = False
for (modname, mod) in self.__modules:
if mod.is_exclusive() and mod.WorkloadWillRun():
exclusive += 1
+ if mod._latency:
+ if latency:
+ raise RuntimeError("More than one exclusive latency test")
+ latency = True
for (modname, mod) in self.__modules:
if exclusive >= 1:
if exclusive != 1:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 3a34c1b988d6..a9f5b0c4fba7 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -216,6 +216,7 @@ class Cyclictest(rtevalModulePrototype):
self.__started = False
self.__cyclicoutput = None
self.__breaktraceval = None
+ self.set_latency()
@staticmethod
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index f3bdc7098bc0..e4b80c33552e 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -216,6 +216,7 @@ class Timerlat(rtevalModulePrototype):
logfnc=self._log)
self.__timerlatdata['system'].description = (f"({self.__numcores} cores) ") + info['0']['model name']
self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
+ self.set_latency()
def _WorkloadSetup(self):
--
2.45.2

45
rteval-Fix-aNone-being-passed-to-cyclictest.patch
View File

@ -1,45 +0,0 @@
From 17a39863e5f89fbaf51dc158bc7e27f46676d46d Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Tue, 11 Jun 2024 16:45:53 +0200
Subject: [PATCH] rteval: Fix -aNone being passed to cyclictest
When rteval is called via the command line, cpulists for both
measurements and loads default to an empty string and are further
processed by parse_cpulist_from_config. However, this is not true when
rteval is used as a module: in that case, neither the default
command-line value is used nor parse_cpulist_from_config is run, leading
to None being set to the config property of measurements which is
explicitely passed down to the corresponding cyclictest config property.
After 64ce7848dfab ("rteval: Add relative cpulists for measurements"),
where the check for None was removed from the cyclictest module, rteval
passes "-aNone" to cyclictest when being used as a module.
Call parse_cpulist_from_config with an empty string to get the default
cpulist to pass to cyclictest module if cpulist is empty in
the measurement config.
Fixes: 64ce7848dfab ("rteval: Add relative cpulists for measurements")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/__init__.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 11bd7b0fce69..43c0fda30ce1 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -148,6 +148,9 @@ measurement profiles, based on their characteristics"""
modcfg = self.__cfg.GetSection("measurement")
cpulist = modcfg.cpulist
run_on_isolcpus = modcfg.run_on_isolcpus
+ if cpulist is None:
+ # Get default cpulist value
+ cpulist = cpulist_utils.collapse_cpulist(parse_cpulist_from_config("", run_on_isolcpus))
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
--
2.45.2

42
rteval-Fix-default-measurement-config.patch
View File

@ -1,42 +0,0 @@
From 9a3d515636fcea17690a4639554f8dae5ebd1ca0 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 19 Jul 2024 15:02:34 -0500
Subject: [PATCH 6/7] rteval: Fix default measurement config
Commit 761741d15d08e ("rteval: rteval-cmd: Some style changes suggested by pylint-3")
accidentally indented the check for missing measurement config, such that
it became part of the missing load conditional. This was harmless if
the cnofig file was missing entirely, but not if the config file
exists but the measurement section is empty.
Also, remove timerlat from the default as it conflicts with cyclictest,
and we're not quite ready to make timerlat the default.
Signed-off-by: Crystal Wood <crwood@redhat.com>
---
rteval-cmd | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/rteval-cmd b/rteval-cmd
index 36b167a034e5..19c82a0b64b3 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -247,11 +247,10 @@ if __name__ == '__main__':
'hackbench' : 'module',
'stressng' : 'module'})
- if not config.HasSection('measurement'):
- config.AppendConfig('measurement', {
- 'cyclictest' : 'module',
- 'timerlat' : 'module',
- 'sysstat' : 'module'})
+ if not config.HasSection('measurement'):
+ config.AppendConfig('measurement', {
+ 'cyclictest' : 'module',
+ 'sysstat' : 'module'})
# Prepare log levels before loading modules, not to have unwanted log messages
rtevcfg = config.GetSection('rteval')
--
2.45.2

102
rteval-Fix-parsing-in-kcompile-of-the-kernel-to-comp.patch
View File

@ -1,102 +0,0 @@
From 7e2fae40e551530f6d0ad8528c4de1ec2ae9e31b Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Sun, 18 Aug 2024 15:23:00 -0400
Subject: [PATCH 3/3] rteval: Fix parsing in kcompile of the kernel to compile
This patch does two things.
1. It allows you to create your own customer kernel with a dash -rteval
in the name, eg, linux-6.10.5-rteval
2. It fixes parsing of the kernel name so that if the user requests
linux-6.10.5 it doesn't use linux-6.10.6-rteval instead
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/loads/kcompile.py | 30 +++++++++++++++++++-----------
1 file changed, 19 insertions(+), 11 deletions(-)
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 80106812af0a..58c542201a1b 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -122,6 +122,8 @@ class Kcompile(CommandLineLoad):
self.cpulist = config.cpulist
CommandLineLoad.__init__(self, "kcompile", config, logger)
self.logger = logger
+ self._kernel_prefix = ""
+ self._log(Log.DEBUG, f'self._cfg.source = {self._cfg.source}')
def _extract_tarball(self):
if self.source is None:
@@ -152,22 +154,29 @@ class Kcompile(CommandLineLoad):
f"error removing builddir ({self.builddir}) (ret={ret})")
def _find_tarball(self):
- # If the user specifies the full kernel name, check if available
- tarfile = os.path.join(self.srcdir, self._cfg.source)
- if os.path.exists(tarfile):
- return tarfile
+ """ Find the tarball and self._kernel_prefix """
if 'rc' in self._cfg.source:
- tarfile_prefix = re.search(r"\d{1,2}\.\d{1,3}\-[a-z]*\d{1,2}", self._cfg.source).group(0)
+ tarfile_prefix = re.search(r"\d{1,2}\.\d{1,3}\-rc\d{1,2}", self._cfg.source).group(0)
+ elif 'rteval' in self._cfg.source:
+ tarfile_prefix = re.search(r"(\d{1,2}\.\d{1,3}\.\d{1,3}\-rteval)|(\d{1,2}\.\d{1,3}\-rteval)", self._cfg.source).group(0)
else:
tarfile_prefix = re.search(r"(\d{1,2}\.\d{1,3}\.\d{1,3})|(\d{1,2}\.\d{1,3})", self._cfg.source).group(0)
+ # Save the kernel prefix
+ self._kernel_prefix = "linux-" + tarfile_prefix
+
+ # If the user specifies the full kernel name, check if available
+ tarfile = os.path.join(self.srcdir, self._cfg.source)
+ if os.path.exists(tarfile):
+ return tarfile
+
# either a tar.xz or tar.gz might exist. Check for both.
xz_file = os.path.join(self.srcdir,"linux-" + tarfile_prefix + ".tar.xz" )
gz_file = os.path.join(self.srcdir,"linux-" + tarfile_prefix + ".tar.gz" )
if os.path.exists(xz_file):
return xz_file
- elif os.path.exists(gz_file):
+ if os.path.exists(gz_file):
return gz_file
raise rtevalRuntimeError(self, f"tarfile {tarfile} does not exist!")
@@ -178,21 +187,20 @@ class Kcompile(CommandLineLoad):
# find our source tarball
if self._cfg.source:
self.source = self._find_tarball()
- kernel_prefix = re.search(r"(linux-\d{1,2}\.\d{1,3}\.\d{1,3})|(linux-\d{1,2}\.\d{1,3})", self.source).group(0)
else:
tarfiles = glob.glob(os.path.join(self.srcdir, f"{DEFAULT_KERNEL_PREFIX}*"))
if tarfiles:
self.source = tarfiles[0]
else:
raise rtevalRuntimeError(self, f" no kernel tarballs found in {self.srcdir}")
- kernel_prefix = DEFAULT_KERNEL_PREFIX
- self._log(Log.DEBUG, f"kernel_prefix = {kernel_prefix}")
+ self._kernel_prefix = DEFAULT_KERNEL_PREFIX
+ self._log(Log.DEBUG, f"self._kernel_prefix = {self._kernel_prefix}")
# check for existing directory
kdir = None
names = os.listdir(self.builddir)
for d in names:
- if d.startswith(kernel_prefix):
+ if d == self._kernel_prefix:
kdir = d
break
if kdir is None:
@@ -200,7 +208,7 @@ class Kcompile(CommandLineLoad):
names = os.listdir(self.builddir)
for d in names:
self._log(Log.DEBUG, f"checking {d}")
- if d.startswith(kernel_prefix):
+ if d == self._kernel_prefix:
kdir = d
break
if kdir is None:
--
2.46.0

43
rteval-Fix-sysreport-traceback-when-utility-sos-not-.patch
View File

@ -1,43 +0,0 @@
From 2a0b5833be4f55dbbc00f1835a4ace554e498137 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Fri, 21 Jun 2024 13:20:26 -0400
Subject: [PATCH 3/4] rteval: Fix sysreport traceback when utility sos not
found
When rteval is run with
-s, --sysreport run sysreport to collect system data (default: False)
and sos, sosreport or sysreport cannot be found then rteval exits with
an error.
Fix this by adding /usr/bin to the places to search for this program.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/sysinfo/osinfo.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/rteval/sysinfo/osinfo.py b/rteval/sysinfo/osinfo.py
index 3d6d5f8daa96..3bdbcc81e068 100644
--- a/rteval/sysinfo/osinfo.py
+++ b/rteval/sysinfo/osinfo.py
@@ -45,10 +45,16 @@ class OSInfo:
def run_sysreport(self, repdir):
if os.path.exists('/usr/sbin/sos'):
exe = '/usr/sbin/sos report'
+ elif os.path.exists('/usr/bin/sos'):
+ exe = '/usr/bin/sos report'
elif os.path.exists('/usr/sbin/sosreport'):
exe = '/usr/sbin/sosreport'
+ elif os.path.exists('/usr/bin/sosreport'):
+ exe = '/usr/bin/sosreport'
elif os.path.exists('/usr/sbin/sysreport'):
exe = '/usr/sbin/sysreport'
+ elif os.path.exists('/usr/bin/sysreport'):
+ exe = '/usr/bin/sysreport'
else:
raise RuntimeError("Can't find sos/sosreport/sysreport")
--
2.45.2

463
rteval-Remove-MeasurementProfile.patch
View File

@ -1,463 +0,0 @@
From 70d84e566ddef917373a5eb20acf9d7cbccb1f97 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:15 -0500
Subject: [PATCH 2/7] rteval: Remove MeasurementProfile
Now that the ModuleInfo() flags are gone, remove the remaining infrastructure
around measurement profiles.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/__init__.py | 25 ++----
rteval/modules/measurement/__init__.py | 117 ++++---------------------
rteval/rteval_histogram_raw.xsl | 24 ++---
rteval/rteval_text.xsl | 41 +++------
4 files changed, 54 insertions(+), 153 deletions(-)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 4a6883e28e5b..226d14f80f48 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -20,7 +20,7 @@ import time
from datetime import datetime
import sysconfig
from rteval.modules.loads import LoadModules
-from rteval.modules.measurement import MeasurementModules, MeasurementProfile
+from rteval.modules.measurement import MeasurementModules
from rteval.rtevalReport import rtevalReport
from rteval.Log import Log
from rteval import rtevalConfig
@@ -131,10 +131,8 @@ class RtEval(rtevalReport):
self._measuremods.Setup(params)
- def __RunMeasurementProfile(self, measure_profile):
+ def __RunMeasurement(self):
global earlystop
- if not isinstance(measure_profile, MeasurementProfile):
- raise Exception("measure_profile is not an MeasurementProfile object")
measure_start = None
try:
@@ -155,15 +153,14 @@ class RtEval(rtevalReport):
print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
else:
print("")
- cpulist = self._measuremods._MeasurementModules__cfg.GetSection("measurement").cpulist
+ cpulist = self._measuremods._cfg.GetSection("measurement").cpulist
if cpulist:
print(f"started measurement threads on cores {cpulist}")
else:
print(f"started measurement threads on {onlinecpus} cores")
print(f"Run duration: {str(self.__rtevcfg.duration)} seconds")
- # start the cyclictest thread
- measure_profile.Start()
+ self._measuremods.Start()
# Unleash the loads and measurement threads
report_interval = int(self.__rtevcfg.report_interval)
@@ -172,7 +169,7 @@ class RtEval(rtevalReport):
nthreads = threading.active_count()
else:
nthreads = None
- measure_profile.Unleash()
+ self._measuremods.Unleash()
measure_start = datetime.now()
# wait for time to expire or thread to die
@@ -185,7 +182,7 @@ class RtEval(rtevalReport):
load_avg_checked = 5
while (currtime <= stoptime) and not stopsig.is_set():
stopsig.wait(min(stoptime - currtime, 60.0))
- if not measure_profile.isAlive():
+ if not self._measuremods.isAlive():
stoptime = currtime
earlystop = True
self.__logger.log(Log.WARN,
@@ -218,7 +215,7 @@ class RtEval(rtevalReport):
finally:
# stop measurement threads
- measure_profile.Stop()
+ self._measuremods.Stop()
# stop the loads
if self._loadmods:
@@ -227,7 +224,7 @@ class RtEval(rtevalReport):
print(f"stopping run at {time.asctime()}")
# wait for measurement modules to finish calculating stats
- measure_profile.WaitForCompletion()
+ self._measuremods.WaitForCompletion()
return measure_start
@@ -236,11 +233,7 @@ class RtEval(rtevalReport):
""" Run the full measurement suite with reports """
global earlystop
rtevalres = 0
- measure_start = None
- for meas_prf in self._measuremods:
- mstart = self.__RunMeasurementProfile(meas_prf)
- if measure_start is None:
- measure_start = mstart
+ measure_start = self.__RunMeasurement()
self._report(measure_start, self.__rtevcfg.xslt_report)
if self.__rtevcfg.sysreport:
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 7b1d84ef554d..ecadd0885991 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -8,43 +8,14 @@ from rteval.modules import RtEvalModules, ModuleContainer
from rteval.systopology import parse_cpulist_from_config
import rteval.cpulist_utils as cpulist_utils
-class MeasurementProfile(RtEvalModules):
- """Keeps and controls all the measurement modules with the same measurement profile"""
-
- def __init__(self, config, modules_root, logger):
- self._module_type = "measurement"
- self._module_config = "measurement"
- self._report_tag = "Profile"
- RtEvalModules.__init__(self, config, modules_root, logger)
-
-
- def ImportModule(self, module):
- "Imports an exported module from a ModuleContainer() class"
- return self._ImportModule(module)
-
-
- def Setup(self, modname):
- "Instantiates and prepares a measurement module"
-
- modobj = self._InstantiateModule(modname, self._cfg.GetSection(modname))
- self._RegisterModuleObject(modname, modobj)
-
-
-class MeasurementModules:
- """Class which takes care of all measurement modules and groups them into
-measurement profiles, based on their characteristics"""
+class MeasurementModules(RtEvalModules):
+ """Module container for measurement modules"""
def __init__(self, config, logger):
- self.__cfg = config
- self.__logger = logger
- self.__measureprofiles = []
- self.__modules_root = "modules.measurement"
- self.__iter_item = None
-
- # Temporary module container, which is used to evalute measurement modules.
- # This will container will be destroyed after Setup() has been called
- self.__container = ModuleContainer(self.__modules_root, self.__logger)
- self.__LoadModules(self.__cfg.GetSection("measurement"))
+ self._module_type = "measurement"
+ self._report_tag = "Measurements"
+ RtEvalModules.__init__(self, config, "modules.measurement", logger)
+ self.__LoadModules(self._cfg.GetSection("measurement"))
def __LoadModules(self, modcfg):
@@ -54,37 +25,28 @@ measurement profiles, based on their characteristics"""
# hope to eventually have different kinds but module is only on
# for now (jcw)
if m[1].lower() == 'module':
- self.__container.LoadModule(m[0])
-
-
- def GetProfile(self):
- "Returns the appropriate MeasurementProfile object, based on the profile type"
-
- for p in self.__measureprofiles:
- return p
- return None
-
+ self._LoadModule(m[0])
def SetupModuleOptions(self, parser):
"Sets up all the measurement modules' parameters for the option parser"
- grparser = self.__container.SetupModuleOptions(parser, self.__cfg)
+ grparser = super().SetupModuleOptions(parser)
# Set up options specific for measurement module group
grparser.add_argument("--measurement-run-on-isolcpus",
dest="measurement___run_on_isolcpus",
action="store_true",
- default=self.__cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
+ default=self._cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
== "true",
help="Include isolated CPUs in default cpulist")
def Setup(self, modparams):
- "Loads all measurement modules and group them into different measurement profiles"
+ "Loads all measurement modules"
if not isinstance(modparams, dict):
raise TypeError("modparams attribute is not of a dictionary type")
- modcfg = self.__cfg.GetSection("measurement")
+ modcfg = self._cfg.GetSection("measurement")
cpulist = modcfg.cpulist
run_on_isolcpus = modcfg.run_on_isolcpus
if cpulist is None:
@@ -93,61 +55,20 @@ measurement profiles, based on their characteristics"""
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
- self.__container.LoadModule(modname)
-
- # Get the correct measurement profile container for this module
- mp = self.GetProfile()
- if mp is None:
- # If not found, create a new measurement profile
- mp = MeasurementProfile(self.__cfg,
- self.__modules_root, self.__logger)
- self.__measureprofiles.append(mp)
-
- # Export the module imported here and transfer it to the
- # measurement profile
- mp.ImportModule(self.__container.ExportModule(modname))
+ self._cfg.AppendConfig(modname, modparams)
+ self._cfg.AppendConfig(modname, {'cpulist':cpulist})
+ self._cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
- # Setup this imported module inside the appropriate measurement profile
- self.__cfg.AppendConfig(modname, modparams)
- self.__cfg.AppendConfig(modname, {'cpulist':cpulist})
- self.__cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
- mp.Setup(modname)
-
- del self.__container
+ modobj = self._InstantiateModule(modname, self._cfg.GetSection(modname))
+ self._RegisterModuleObject(modname, modobj)
def MakeReport(self):
- "Generates an XML report for all measurement profiles"
+ rep_n = super().MakeReport()
- # Get the reports from all meaurement modules in all measurement profiles
- rep_n = libxml2.newNode("Measurements")
- cpulist = self.__cfg.GetSection("measurement").cpulist
- run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
+ cpulist = self._cfg.GetSection("measurement").cpulist
+ run_on_isolcpus = self._cfg.GetSection("measurement").run_on_isolcpus
cpulist = parse_cpulist_from_config(cpulist, run_on_isolcpus)
rep_n.newProp("measurecpus", cpulist_utils.collapse_cpulist(cpulist))
- for mp in self.__measureprofiles:
- mprep_n = mp.MakeReport()
- if mprep_n:
- rep_n.addChild(mprep_n)
-
return rep_n
-
-
- def __iter__(self):
- "Initiates an iteration loop for MeasurementProfile objects"
-
- self.__iter_item = len(self.__measureprofiles)
- return self
-
-
- def __next__(self):
- """Internal Python iterating method, returns the next
-MeasurementProfile object to be processed"""
-
- if self.__iter_item == 0:
- self.__iter_item = None
- raise StopIteration
-
- self.__iter_item -= 1
- return self.__measureprofiles[self.__iter_item]
diff --git a/rteval/rteval_histogram_raw.xsl b/rteval/rteval_histogram_raw.xsl
index 00b2be34f305..35d8e8461f74 100644
--- a/rteval/rteval_histogram_raw.xsl
+++ b/rteval/rteval_histogram_raw.xsl
@@ -11,25 +11,25 @@
<xsl:text>core&#09;index&#09;value&#10;</xsl:text>
<!-- Extract overall system histogram data -->
- <xsl:apply-templates select="Measurements/Profile/cyclictest/system/histogram/bucket">
+ <xsl:apply-templates select="Measurements/cyclictest/system/histogram/bucket">
<xsl:with-param name="label" select="'system'"/>
- <xsl:sort select="Measurements/Profile/cyclictest/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:sort select="Measurements/cyclictest/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
- <xsl:apply-templates select="Measurements/Profile/timerlat/system/histogram/bucket">
+ <xsl:apply-templates select="Measurements/timerlat/system/histogram/bucket">
<xsl:with-param name="label" select="'system'"/>
- <xsl:sort select="Measurements/Profile/timerlat/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:sort select="Measurements/timerlat/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
<!-- Extract per cpu core histogram data -->
- <xsl:apply-templates select="Measurements/Profile/cyclictest/core/histogram/bucket">
- <xsl:sort select="Measurements/Profile/cyclictest/core/@id" data-type="number"/>
- <xsl:sort select="Measurements/Profile/cyclictest/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:apply-templates select="Measurements/cyclictest/core/histogram/bucket">
+ <xsl:sort select="Measurements/cyclictest/core/@id" data-type="number"/>
+ <xsl:sort select="Measurements/cyclictest/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
- <xsl:apply-templates select="Measurements/Profile/timerlat/core/histogram/bucket">
- <xsl:sort select="Measurements/Profile/timerlat/core/@id" data-type="number"/>
- <xsl:sort select="Measurements/Profile/timerlat/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:apply-templates select="Measurements/timerlat/core/histogram/bucket">
+ <xsl:sort select="Measurements/timerlat/core/@id" data-type="number"/>
+ <xsl:sort select="Measurements/timerlat/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
</xsl:template>
@@ -38,7 +38,7 @@
<!-- -->
<!-- Record formatting -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/*/histogram/bucket">
+ <xsl:template match="/rteval/Measurements/cyclictest/*/histogram/bucket">
<xsl:param name="label"/>
<xsl:choose>
<!-- If we don't have a id tag in what should be a 'core' tag, use the given label -->
@@ -54,7 +54,7 @@
<xsl:text>&#10;</xsl:text>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/timerlat/*/histogram/bucket">
+ <xsl:template match="/rteval/Measurements/timerlat/*/histogram/bucket">
<xsl:param name="label"/>
<xsl:choose>
<!-- If we don't have a id tag in what should be a 'core' tag, use the given label -->
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 7ca0ae3a4c66..b801679abcc5 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -154,8 +154,8 @@
<xsl:value-of select="SystemInfo/cmdlineInfo/cmdline"/>
<xsl:text>&#10;</xsl:text>
- <!-- Generate a summary report for all measurement profiles -->
- <xsl:apply-templates select="Measurements/Profile"/>
+ <!-- Generate a summary report for all measurement modules -->
+ <xsl:apply-templates select="Measurements"/>
<xsl:text> ===================================================================&#10;</xsl:text>
</xsl:template>
<!-- -->
@@ -178,21 +178,8 @@
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile">
- <xsl:text> Measurement profile </xsl:text>
- <xsl:value-of select="position()"/><xsl:text>: </xsl:text>
- <xsl:choose>
- <xsl:when test="@loads = '1'"><xsl:text>With loads, </xsl:text></xsl:when>
- <xsl:otherwise><xsl:text>Without loads, </xsl:text></xsl:otherwise>
- </xsl:choose>
- <xsl:choose>
- <xsl:when test="@parallel = '1'">
- <xsl:text>measurements in parallel</xsl:text>
- </xsl:when>
- <xsl:otherwise>
- <xsl:text>measurements serialised</xsl:text>
- </xsl:otherwise>
- </xsl:choose>
+ <xsl:template match="/rteval/Measurements">
+ <xsl:text> Measurements: </xsl:text>
<xsl:text>&#10;</xsl:text>
<!-- Format other sections of the report, if they are found -->
@@ -206,7 +193,7 @@
</xsl:template>
<!-- Format the cyclictest section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest">
+ <xsl:template match="/rteval/Measurements/cyclictest">
<xsl:text> Latency test&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
@@ -238,7 +225,7 @@
<!-- Format the CPU core section in the cyclictest part -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/core">
+ <xsl:template match="/rteval/Measurements/cyclictest/core">
<xsl:text> CPU core </xsl:text>
<xsl:value-of select="@id"/>
<xsl:text> Priority: </xsl:text>
@@ -251,7 +238,7 @@
<!-- Generic formatting of statistics information -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/*/statistics">
+ <xsl:template match="/rteval/Measurements/cyclictest/*/statistics">
<xsl:text> Samples: </xsl:text>
<xsl:value-of select="samples"/>
<xsl:text>&#10;</xsl:text>
@@ -301,7 +288,7 @@
</xsl:template>
<!-- Format the timerlat section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat">
+ <xsl:template match="/rteval/Measurements/timerlat">
<xsl:text> Latency test&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
@@ -345,7 +332,7 @@
<!-- Format the CPU core section in the timerlat part -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat/core">
+ <xsl:template match="/rteval/Measurements/timerlat/core">
<xsl:text> CPU core </xsl:text>
<xsl:value-of select="@id"/>
<xsl:text> Priority: </xsl:text>
@@ -358,7 +345,7 @@
<!-- Generic formatting of statistics information -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat/*/statistics">
+ <xsl:template match="/rteval/Measurements/timerlat/*/statistics">
<xsl:text> Samples: </xsl:text>
<xsl:value-of select="samples"/>
<xsl:text>&#10;</xsl:text>
@@ -409,7 +396,7 @@
<!-- Format the hwlatdetect test section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0' and not(@aborted)]">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0' and not(@aborted)]">
<xsl:text> Hardware latency detector&#10;</xsl:text>
<xsl:text> Run duration: </xsl:text>
@@ -434,12 +421,12 @@
<xsl:apply-templates select="samples/sample"/>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0' and @aborted > 0]">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0' and @aborted > 0]">
<xsl:text> Hardware latency detector&#10;</xsl:text>
<xsl:text> ** WARNING ** hwlatedect failed to run&#10;</xsl:text>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0']/samples/sample">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0']/samples/sample">
<xsl:text> - @</xsl:text>
<xsl:value-of select="@timestamp"/>
<xsl:text> </xsl:text>
@@ -448,7 +435,7 @@
</xsl:template>
<!-- Format the cyclictest section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/sysstat">
+ <xsl:template match="/rteval/Measurements/sysstat">
<xsl:text> sysstat measurements&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
--
2.45.2

87
rteval-RtEvalModules-Remove-unused-methods.patch
View File

@ -1,87 +0,0 @@
From 7a0bd39fad4fb017f41e32a125e7f4ef4284b8f0 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:16 -0500
Subject: [PATCH 3/7] rteval: RtEvalModules: Remove unused methods
With measurement profiles gone, remove methods that no longer have any
users.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 37 -------------------------------------
1 file changed, 37 deletions(-)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 0c0ce7202f77..de1ddc4628c1 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -333,36 +333,11 @@ returned when a ModuleContainer object is iterated over"""
self.__modobjects[modname] = modobj
- def ExportModule(self, modname, modroot=None):
- "Export module info, used to transfer an imported module to another ModuleContainer"
- if modroot is None:
- modroot = self.__modules_root
-
- mod = f"{modroot}.{modname}"
- return (mod, self.__modsloaded[mod])
-
-
- def ImportModule(self, module):
- "Imports an exported module from another ModuleContainer"
- (modname, moduleimp) = module
- self.__modsloaded[modname] = moduleimp
-
-
def ModulesLoaded(self):
"Returns number of registered module objects"
return len(self.__modobjects)
- def GetModulesList(self):
- "Returns a list of module names"
- return list(self.__modobjects.keys())
-
-
- def GetNamedModuleObject(self, modname):
- "Looks up a named module and returns its registered module object"
- return self.__modobjects[modname]
-
-
def __iter__(self):
"Initiates the iterating process"
@@ -406,10 +381,6 @@ class RtEvalModules:
# Export some of the internal module container methods
# Primarily to have better control of the module containers
# iteration API
- def _ImportModule(self, module):
- "Imports a module exported by ModuleContainer::ExportModule()"
- return self.__modules.ImportModule(module)
-
def _InstantiateModule(self, modname, modcfg, modroot=None):
"Imports a module and returns an instantiated object from the module"
return self.__modules.InstantiateModule(modname, modcfg, modroot)
@@ -426,17 +397,9 @@ class RtEvalModules:
"Returns number of imported modules"
return self.__modules.ModulesLoaded()
- def GetModulesList(self):
- "Returns a list of module names"
- return self.__modules.GetModulesList()
-
def SetupModuleOptions(self, parser):
"Sets up argparse based argument groups for the loaded modules"
return self.__modules.SetupModuleOptions(parser, self._cfg)
-
- def GetNamedModuleObject(self, modname):
- "Returns a list of module names"
- return self.__modules.GetNamedModuleObject(modname)
# End of exports
--
2.45.2

88
rteval-Update-the-kcompile-kernel-to-linux-6.12-rc4.patch
View File

@ -1,88 +0,0 @@
From d7dc1a05dd60c0fe2ab3edaaea3786a8be592ade Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Tue, 22 Oct 2024 13:42:12 -0400
Subject: [PATCH 2/2] rteval: Update the kcompile kernel to linux-6.12-rc4
Update the kernel that rteval compiles as a load in the load module
kcompile
This kernel contains patches to address the fact that ENGINE API has
been deprecated since OpenSSL version 3.0. This is important because
some distros have been dropping the headers for the deprecated api
and this has been causing kcompile to fail, unless you supply a special
kernel with the patches to address this.
With this change the upstream kernel as is will work again.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
Dockerfile | 2 +-
Makefile | 2 +-
rteval/modules/loads/kcompile.py | 4 ++--
rteval/rteval.conf | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index f3ee516b57d5..b8f602b11703 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,7 +1,7 @@
# Use CentOS Stream 9 as base image
FROM centos:stream9
-ARG KERNEL_VERSION=linux-6.10.5.tar.xz
+ARG KERNEL_VERSION=linux-6.12-rc4.tar.gz
# Copy current directory to /opt/rteval/
diff --git a/Makefile b/Makefile
index e1a2bbba0373..a250b18611b4 100644
--- a/Makefile
+++ b/Makefile
@@ -14,7 +14,7 @@ PREFIX := /usr
DATADIR := $(DESTDIR)/$(PREFIX)/share
LOADDIR := loadsource
-KLOAD := $(LOADDIR)/linux-6.10.5.tar.xz
+KLOAD := $(LOADDIR)/linux-6.12-rc4.tar.gz
BLOAD := $(LOADDIR)/dbench-4.0.tar.gz
LOADS := $(KLOAD) $(BLOAD)
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 58c542201a1b..8a3a0e93fa14 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -21,7 +21,7 @@ expand_cpulist = cpulist_utils.expand_cpulist
compress_cpulist = cpulist_utils.compress_cpulist
nonisolated_cpulist = cpulist_utils.nonisolated_cpulist
-DEFAULT_KERNEL_PREFIX = "linux-6.10.5"
+DEFAULT_KERNEL_PREFIX = "linux-6.12-rc4"
class KBuildJob:
'''Class to manage a build job bound to a particular node'''
@@ -342,7 +342,7 @@ class Kcompile(CommandLineLoad):
def ModuleParameters():
return {"source": {"descr": "Source tar ball",
- "default": "linux-6.10.5.tar.xz",
+ "default": "linux-6.12-rc4.tar.gz",
"metavar": "TARBALL"},
"jobspercore": {"descr": "Number of working threads per core",
"default": 2,
diff --git a/rteval/rteval.conf b/rteval/rteval.conf
index 0611c031c2a0..5a49040d980f 100644
--- a/rteval/rteval.conf
+++ b/rteval/rteval.conf
@@ -18,7 +18,7 @@ dbench: external
stressng: module
[kcompile]
-source: linux-6.10.5.xz
+source: linux-6.12-rc4.tar.gz
jobspercore: 2
[hackbench]
--
2.46.2

80
rteval-Upgrade-load-kernel-to-linux-6.10.5.patch
View File

@ -1,80 +0,0 @@
From 0e6fa19365330937d54132a3077d5d9961877546 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Sun, 18 Aug 2024 12:10:55 -0400
Subject: [PATCH 20/23] rteval: Upgrade load kernel to linux-6.10.5
The older kernel fails to compile on newer tool chains, so upgrade
the kernel that kcompile compiles as a load to linux-6.10.5
Signed-off-by: John Kacur <jkacur@redhat.com>
---
Dockerfile | 2 +-
Makefile | 2 +-
rteval/modules/loads/kcompile.py | 4 ++--
rteval/rteval.conf | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 45f6434777d4..f3ee516b57d5 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,7 +1,7 @@
# Use CentOS Stream 9 as base image
FROM centos:stream9
-ARG KERNEL_VERSION=linux-6.6.1.tar.xz
+ARG KERNEL_VERSION=linux-6.10.5.tar.xz
# Copy current directory to /opt/rteval/
diff --git a/Makefile b/Makefile
index d9a6b9f116ac..e1a2bbba0373 100644
--- a/Makefile
+++ b/Makefile
@@ -14,7 +14,7 @@ PREFIX := /usr
DATADIR := $(DESTDIR)/$(PREFIX)/share
LOADDIR := loadsource
-KLOAD := $(LOADDIR)/linux-6.6.1.tar.xz
+KLOAD := $(LOADDIR)/linux-6.10.5.tar.xz
BLOAD := $(LOADDIR)/dbench-4.0.tar.gz
LOADS := $(KLOAD) $(BLOAD)
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index f7a9c0084805..80106812af0a 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -21,7 +21,7 @@ expand_cpulist = cpulist_utils.expand_cpulist
compress_cpulist = cpulist_utils.compress_cpulist
nonisolated_cpulist = cpulist_utils.nonisolated_cpulist
-DEFAULT_KERNEL_PREFIX = "linux-6.6"
+DEFAULT_KERNEL_PREFIX = "linux-6.10.5"
class KBuildJob:
'''Class to manage a build job bound to a particular node'''
@@ -334,7 +334,7 @@ class Kcompile(CommandLineLoad):
def ModuleParameters():
return {"source": {"descr": "Source tar ball",
- "default": "linux-6.6.1.tar.xz",
+ "default": "linux-6.10.5.tar.xz",
"metavar": "TARBALL"},
"jobspercore": {"descr": "Number of working threads per core",
"default": 2,
diff --git a/rteval/rteval.conf b/rteval/rteval.conf
index a4aad33e264f..0611c031c2a0 100644
--- a/rteval/rteval.conf
+++ b/rteval/rteval.conf
@@ -18,7 +18,7 @@ dbench: external
stressng: module
[kcompile]
-source: linux-6.6.1.xz
+source: linux-6.10.5.xz
jobspercore: 2
[hackbench]
--
2.46.2

31
rteval-fixed-manpage-to-include-stress-ng-header.patch
View File

@ -1,31 +0,0 @@
From 1b1d950e9a13914d1e7c158453c7bcd6e0bd4d22 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Tue, 9 Jul 2024 10:25:16 -0400
Subject: [PATCH 07/23] rteval: fixed manpage to include stress-ng header
Previously the manpage stress-ng options were listed with cyclictest options.
This patch puts them in a STRESS-NG section to more closely mirror
rteval help
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
- Rewrote the commit message a bit.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
doc/rteval.8 | 1 +
1 file changed, 1 insertion(+)
diff --git a/doc/rteval.8 b/doc/rteval.8
index bfdeb9d36d9e..9e2b377752e5 100644
--- a/doc/rteval.8
+++ b/doc/rteval.8
@@ -138,6 +138,7 @@ Send a break trace command if latency exceeds the given number of microseconds.
.TP
.B \-\-cyclictest-threshold=USEC
Exit rteval if latency exceeds the given number of microseconds.
+.SH STRESS-NG OPTIONS
.TP
.B \-\-stressng-option=OPTION
Pass in command line options for the stress-ng package.
--
2.46.2

89
rteval-measurement-Change-latency-flag-to-latency_te.patch
View File

@ -1,89 +0,0 @@
From 7cb8c1c14569426e867cbdbdb218d1d9bcd4d520 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 26 Jul 2024 13:32:37 -0500
Subject: [PATCH 7/7] rteval: measurement: Change latency flag to latency_test
As requested by John, change the name of the latency flag to avoid
looking like it holds an actual latency value.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 14 +++++++-------
rteval/modules/measurement/cyclictest.py | 2 +-
rteval/modules/measurement/timerlat.py | 2 +-
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 2a4eafae71c7..d7792108d5b8 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -40,7 +40,7 @@ class rtevalModulePrototype(threading.Thread):
"finished": threading.Event()}
self._donotrun = False
self._exclusive = False
- self._latency = False
+ self._latency_test = False
self.__timestamps = {}
self.__sleeptime = 2.0
@@ -68,9 +68,9 @@ class rtevalModulePrototype(threading.Thread):
self._exclusive = True
- def set_latency(self):
+ def set_latency_test(self):
""" Sets the module as an exclusive latency measurer """
- self._latency = True
+ self._latency_test = True
def set_donotrun(self):
@@ -418,14 +418,14 @@ class RtEvalModules:
self._logger.log(Log.INFO, f"Preparing {self._module_type} modules")
exclusive = 0
- latency = False
+ latency_test = False
for (modname, mod) in self.__modules:
if mod.is_exclusive() and mod.WorkloadWillRun():
exclusive += 1
- if mod._latency:
- if latency:
+ if mod._latency_test:
+ if latency_test:
raise RuntimeError("More than one exclusive latency test")
- latency = True
+ latency_test = True
for (modname, mod) in self.__modules:
if exclusive >= 1:
if exclusive != 1:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index a9f5b0c4fba7..d919058e927f 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -216,7 +216,7 @@ class Cyclictest(rtevalModulePrototype):
self.__started = False
self.__cyclicoutput = None
self.__breaktraceval = None
- self.set_latency()
+ self.set_latency_test()
@staticmethod
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index e4b80c33552e..92bc07051aa1 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -216,7 +216,7 @@ class Timerlat(rtevalModulePrototype):
logfnc=self._log)
self.__timerlatdata['system'].description = (f"({self.__numcores} cores) ") + info['0']['model name']
self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
- self.set_latency()
+ self.set_latency_test()
def _WorkloadSetup(self):
--
2.45.2

324
rteval-measurement-Remove-ModuleInfo.patch
View File

@ -1,324 +0,0 @@
From 5b55c62ff271e9d9278d25f027590aca05bff959 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:14 -0500
Subject: [PATCH 1/7] rteval: measurement: Remove ModuleInfo()
All of the measurement modules have identical settings, and the use cases
for deviating seem questionable. While we do have a desire to keep
cyclictest and timerlat from running together, turning off run_parallel
would prevent sysstat from running at the same time. And if there's a
desire to run without loads, that seems like it belongs in the realm of
user configuration, rather than anything inherent to a given measurement
module.
Any future module flags can be implemented similarly to what is done
in load modules (e.g. set_exclusive).
Places that checked with_loads now check whether a LoadModules container
is present at all, which can be wired up to a command line option in the
future.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/__init__.py | 39 ++++++-------
rteval/modules/__init__.py | 13 +----
rteval/modules/measurement/__init__.py | 73 ++----------------------
rteval/modules/measurement/cyclictest.py | 5 --
rteval/modules/measurement/sysstat.py | 7 ---
rteval/modules/measurement/timerlat.py | 5 --
6 files changed, 26 insertions(+), 116 deletions(-)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 1a61148ef327..4a6883e28e5b 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -111,20 +111,21 @@ class RtEval(rtevalReport):
except Exception as err:
raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]")
- self.__logger.log(Log.INFO, "Preparing load modules")
- params = {'workdir':self.__rtevcfg.workdir,
- 'reportdir':self.__reportdir and self.__reportdir or "",
- 'builddir':builddir,
- 'srcdir':self.__rtevcfg.srcdir,
- 'verbose': self.__rtevcfg.verbose,
- 'debugging': self.__rtevcfg.debugging,
- 'numcores':self._sysinfo.cpu_getCores(True),
- 'logging':self.__rtevcfg.logging,
- 'memsize':self._sysinfo.mem_get_size(),
- 'numanodes':self._sysinfo.mem_get_numa_nodes(),
- 'duration': float(self.__rtevcfg.duration),
- }
- self._loadmods.Setup(params)
+ if self._loadmods:
+ self.__logger.log(Log.INFO, "Preparing load modules")
+ params = {'workdir':self.__rtevcfg.workdir,
+ 'reportdir':self.__reportdir and self.__reportdir or "",
+ 'builddir':builddir,
+ 'srcdir':self.__rtevcfg.srcdir,
+ 'verbose': self.__rtevcfg.verbose,
+ 'debugging': self.__rtevcfg.debugging,
+ 'numcores':self._sysinfo.cpu_getCores(True),
+ 'logging':self.__rtevcfg.logging,
+ 'memsize':self._sysinfo.mem_get_size(),
+ 'numanodes':self._sysinfo.mem_get_numa_nodes(),
+ 'duration': float(self.__rtevcfg.duration),
+ }
+ self._loadmods.Setup(params)
self.__logger.log(Log.INFO, "Preparing measurement modules")
self._measuremods.Setup(params)
@@ -136,13 +137,11 @@ class RtEval(rtevalReport):
raise Exception("measure_profile is not an MeasurementProfile object")
measure_start = None
- (with_loads, run_parallel) = measure_profile.GetProfile()
- self.__logger.log(Log.INFO, f"Using measurement profile [loads: {with_loads} parallel: {run_parallel}]")
try:
nthreads = 0
# start the loads
- if with_loads:
+ if self._loadmods:
self._loadmods.Start()
print(f"rteval run on {os.uname()[2]} started at {time.asctime()}")
@@ -168,7 +167,7 @@ class RtEval(rtevalReport):
# Unleash the loads and measurement threads
report_interval = int(self.__rtevcfg.report_interval)
- if with_loads:
+ if self._loadmods:
self._loadmods.Unleash()
nthreads = threading.active_count()
else:
@@ -192,7 +191,7 @@ class RtEval(rtevalReport):
self.__logger.log(Log.WARN,
"Measurement threads did not use the full time slot. Doing a controlled stop.")
- if with_loads:
+ if nthreads:
if threading.active_count() < nthreads:
raise RuntimeError("load thread died!")
@@ -222,7 +221,7 @@ class RtEval(rtevalReport):
measure_profile.Stop()
# stop the loads
- if with_loads:
+ if self._loadmods:
self._loadmods.Stop()
print(f"stopping run at {time.asctime()}")
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 4330a839db6f..0c0ce7202f77 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -269,14 +269,6 @@ reference from the first import"""
return mod
- def ModuleInfo(self, modname, modroot=None):
- """Imports a module and calls the modules' ModuleInfo() function and returns
-the information provided by the module"""
-
- mod = self.LoadModule(modname, modroot)
- return mod.ModuleInfo()
-
-
def SetupModuleOptions(self, parser, config):
"""Sets up a separate argparse ArgumentGroup per module with its supported parameters"""
@@ -495,9 +487,8 @@ class RtEvalModules:
def Unleash(self):
- """Unleashes all the loaded modules workloads"""
+ """Unleashes all the loaded modules"""
- # turn loose the loads
nthreads = 0
self._logger.log(Log.INFO, f"Sending start event to all {self._module_type} modules")
for (modname, mod) in self.__modules:
@@ -508,7 +499,7 @@ class RtEvalModules:
return nthreads
- def _isAlive(self):
+ def isAlive(self):
"""Returns True if all modules are running"""
for (modname, mod) in self.__modules:
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 43c0fda30ce1..7b1d84ef554d 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -11,24 +11,13 @@ import rteval.cpulist_utils as cpulist_utils
class MeasurementProfile(RtEvalModules):
"""Keeps and controls all the measurement modules with the same measurement profile"""
- def __init__(self, config, with_load, run_parallel, modules_root, logger):
- self.__with_load = with_load
- self.__run_parallel = run_parallel
-
- # Only used when running modules serialised
- self.__run_serialised_mods = None
-
+ def __init__(self, config, modules_root, logger):
self._module_type = "measurement"
self._module_config = "measurement"
self._report_tag = "Profile"
RtEvalModules.__init__(self, config, modules_root, logger)
- def GetProfile(self):
- "Returns the profile characteristic as (with_load, run_parallel)"
- return (self.__with_load, self.__run_parallel)
-
-
def ImportModule(self, module):
"Imports an exported module from a ModuleContainer() class"
return self._ImportModule(module)
@@ -41,54 +30,6 @@ class MeasurementProfile(RtEvalModules):
self._RegisterModuleObject(modname, modobj)
- def Unleash(self):
- """Unleashes all the measurement modules"""
-
- if self.__run_parallel:
- # Use the inherrited method if running
- # measurements in parallel
- return RtEvalModules.Unleash(self)
-
- # Get a list of all registered modules,
- # and start the first one
- self.__serialised_mods = self.GetModulesList()
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- mod.setStart()
- return 1
-
-
- def MakeReport(self):
- "Generates an XML report for all run measurement modules in this profile"
- rep_n = RtEvalModules.MakeReport(self)
- rep_n.newProp("loads", self.__with_load and "1" or "0")
- rep_n.newProp("parallel", self.__run_parallel and "1" or "0")
- return rep_n
-
-
- def isAlive(self):
- """Returns True if all modules which are supposed to run runs"""
-
- if self.__run_parallel:
- return self._isAlive()
-
- if self.__serialised_mods:
- # If running serialised, first check if measurement is still running,
- # if so - return True.
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- if mod.WorkloadAlive():
- return True
-
- # If not, go to next on the list and kick it off
- self.__serialised_mods.remove(self.__serialised_mods[0])
- if self.__serialised_mods:
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- mod.setStart()
- return True
-
- # If we've been through everything, nothing is running
- return False
-
-
class MeasurementModules:
"""Class which takes care of all measurement modules and groups them into
measurement profiles, based on their characteristics"""
@@ -116,13 +57,11 @@ measurement profiles, based on their characteristics"""
self.__container.LoadModule(m[0])
- def GetProfile(self, with_load, run_parallel):
+ def GetProfile(self):
"Returns the appropriate MeasurementProfile object, based on the profile type"
for p in self.__measureprofiles:
- mp = p.GetProfile()
- if mp == (with_load, run_parallel):
- return p
+ return p
return None
@@ -154,15 +93,13 @@ measurement profiles, based on their characteristics"""
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
- # Extract the measurement modules info
- modinfo = self.__container.ModuleInfo(modname)
+ self.__container.LoadModule(modname)
# Get the correct measurement profile container for this module
- mp = self.GetProfile(modinfo["loads"], modinfo["parallel"])
+ mp = self.GetProfile()
if mp is None:
# If not found, create a new measurement profile
mp = MeasurementProfile(self.__cfg,
- modinfo["loads"], modinfo["parallel"],
self.__modules_root, self.__logger)
self.__measureprofiles.append(mp)
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 3301e1b45e11..3a34c1b988d6 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -394,11 +394,6 @@ class Cyclictest(rtevalModulePrototype):
return rep_n
-def ModuleInfo():
- return {"parallel": True,
- "loads": True}
-
-
def ModuleParameters():
""" default parameters """
return {"interval": {"descr": "Base interval of the threads in microseconds",
diff --git a/rteval/modules/measurement/sysstat.py b/rteval/modules/measurement/sysstat.py
index d4646c1646f4..a0efd8953659 100644
--- a/rteval/modules/measurement/sysstat.py
+++ b/rteval/modules/measurement/sysstat.py
@@ -93,13 +93,6 @@ class sysstat(rtevalModulePrototype):
-def ModuleInfo():
- # sysstat features - run in parallel with outher measurement modules with loads
- return {"parallel": True,
- "loads": True}
-
-
-
def ModuleParameters():
return {} # No arguments available
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index dc6226ccc991..f3bdc7098bc0 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -499,11 +499,6 @@ class Timerlat(rtevalModulePrototype):
return rep_n
-def ModuleInfo():
- """ Required measurement module information """
- return {"parallel": True,
- "loads": True}
-
def ModuleParameters():
""" default parameters """
return {"priority": {"descr": "Run rtla timerlat with this priority",
--
2.45.2

103
rteval-restore-all-load-module-options.patch
View File

@ -1,103 +0,0 @@
From 06acb385b2074f39146d5c1a41cb2133c43ade82 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Sun, 18 Aug 2024 09:48:49 -0400
Subject: [PATCH 1/3] rteval: restore all load module options
Commit 56c7cf63942d rteval: Allow arguments specific to module group
intended to allow group options to the overall measurement modules without
applying to the load modules. It inadvertently disabled options for most
load modules such as hackbench and kcompile.
This patch reworks the overall group mechanism a little bit to restore
these menus.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 22 +++++++++++++++++-----
rteval/modules/measurement/__init__.py | 12 +-----------
2 files changed, 18 insertions(+), 16 deletions(-)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index acd6330788e2..eb29db86ce7a 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -280,10 +280,24 @@ reference from the first import"""
grparser = parser.add_argument_group(f"Group Options for {self.__modtype} modules")
grparser.add_argument(f'--{self.__modtype}-cpulist',
- dest=f'{self.__modtype}___cpulist', action='store', default="",
+ dest=f'{self.__modtype}___cpulist', action='store',
+ default="",
help=f'CPU list where {self.__modtype} modules will run',
metavar='CPULIST')
+ # Set up options for measurement modules only
+ if self.__modtype == 'measurement':
+ grparser.add_argument(f'--{self.__modtype}-run-on-isolcpus',
+ dest = f'{self.__modtype}___run_on_isolcpus',
+ action = "store_true",
+ default = config.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower() == "true",
+ help = "Include isolated CPUs in default cpulist")
+ grparser.add_argument('--idle-set',
+ dest='measurement___idlestate',
+ metavar='IDLESTATE',
+ default=None,
+ help='Idle state depth to set on cpus running measurement modules')
+
for (modname, mod) in list(self.__modsloaded.items()):
opts = mod.ModuleParameters()
if len(opts) == 0:
@@ -296,7 +310,7 @@ reference from the first import"""
# Ignore if a section is not found
cfg = None
- modgrparser = parser.add_argument_group(f"Options for the {shortmod} module")
+ grparser = parser.add_argument_group(f"Options for the {shortmod} module")
for (o, s) in list(opts.items()):
descr = 'descr' in s and s['descr'] or ""
metavar = 'metavar' in s and s['metavar'] or None
@@ -311,7 +325,7 @@ reference from the first import"""
default = 'default' in s and s['default'] or None
- modgrparser.add_argument(f'--{shortmod}-{o}',
+ grparser.add_argument(f'--{shortmod}-{o}',
dest=f"{shortmod}___{o}",
action='store',
help='%s%s' % (descr,
@@ -319,8 +333,6 @@ reference from the first import"""
default=default,
metavar=metavar)
- return grparser
-
def InstantiateModule(self, modname, modcfg, modroot=None):
"""Imports a module and instantiates an object from the modules create() function.
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 9314d1cb6bbc..44708ce0b035 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -29,17 +29,7 @@ class MeasurementModules(RtEvalModules):
def SetupModuleOptions(self, parser):
"Sets up all the measurement modules' parameters for the option parser"
- grparser = super().SetupModuleOptions(parser)
-
- # Set up options specific for measurement module group
- grparser.add_argument("--measurement-run-on-isolcpus",
- dest="measurement___run_on_isolcpus",
- action="store_true",
- default=self._cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
- == "true",
- help="Include isolated CPUs in default cpulist")
- grparser.add_argument('--idle-set', dest='measurement___idlestate', metavar='IDLESTATE',
- default=None, help='Idle state depth to set on cpus running measurement modules')
+ super().SetupModuleOptions(parser)
def Setup(self, modparams):
--
2.46.0

71
rteval-run-cyclictest-using-default-system-when-sett.patch
View File

@ -1,71 +0,0 @@
From 034fb231ff06aa4615b7531a04e4c3e0ce4aa662 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Fri, 2 Aug 2024 11:46:36 -0400
Subject: [PATCH 2/2] rteval: run cyclictest using '--default-system' when
setting idle states
When running cyclictest in rteval, cyclictest automatically disables
idle states. This means whenever the user sets the idle state of a cpu
list using '--idle-set' it is overridden by cyclictest.
To fix this, the variable 'usingCpupower' is appended to the parameter
dictionary that's passed to the Cyclictest measurement object which executes
cyclictest in rteval.
If '--idle-set' is specified when running rteval,
'usingCpupower' is set to true and the '--default-system' option is
appended to the cyclictest command, which will prevent cyclictest from
disabling cstates.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 4 ++++
rteval/__init__.py | 1 +
rteval/modules/measurement/cyclictest.py | 3 +++
3 files changed, 8 insertions(+)
diff --git a/rteval-cmd b/rteval-cmd
index f440a8a22622..4e13d312a24a 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -266,6 +266,10 @@ if __name__ == '__main__':
| (rtevcfg.debugging and Log.DEBUG)
logger.SetLogVerbosity(loglev)
+ # check if cpupower is being used
+ if sys.argv.count('--idle-set') > 0:
+ rtevcfg.update({'usingCpupower': True})
+
# Load modules
loadmods = LoadModules(config, logger=logger)
measuremods = MeasurementModules(config, logger=logger)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 4d3e0c23e5ab..8ded374d287e 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -119,6 +119,7 @@ class RtEval(rtevalReport):
'memsize':self._sysinfo.mem_get_size(),
'numanodes':self._sysinfo.mem_get_numa_nodes(),
'duration': float(self.__rtevcfg.duration),
+ 'usingCpupower': self.__rtevcfg.usingCpupower
}
if self._loadmods:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index d919058e927f..2e8f6f1870ed 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -251,6 +251,9 @@ class Cyclictest(rtevalModulePrototype):
self.__cmd.append(f'-t{self.__numcores}')
self.__cmd.append(f'-a{self.__cpulist}')
+ if (self.__cfg.usingCpupower):
+ self.__cmd.append('--default-system')
+
if 'threads' in self.__cfg and self.__cfg.threads:
self.__cmd.append(f"-t{int(self.__cfg.threads)}")
--
2.45.2

65
rteval-sysstat-Convert-base64-data-to-text-before-wr.patch
View File

@ -1,65 +0,0 @@
From ca90d5aa7ae2ff6dac124c710fceadae028b5f4a Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:18:05 -0500
Subject: [PATCH 2/4] rteval: sysstat: Convert base64 data to text before
wrapping
As of Python 3, b64encode() returns data, not a string, causing this:
Traceback (most recent call last):
File "/home/crwood/git/rteval/./rteval-cmd", line 413, in <module>
ec = rteval.Measure()
^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/__init__.py", line 246, in Measure
self._report(measure_start, self.__rtevcfg.xslt_report)
File "/home/crwood/git/rteval/rteval/rtevalReport.py", line 63, in _report
self.__xmlreport.AppendXMLnodes(self._measuremods.MakeReport())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/__init__.py", line 190, in MakeReport
mprep_n = mp.MakeReport()
^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/__init__.py", line 62, in MakeReport
rep_n = RtEvalModules.MakeReport(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/__init__.py", line 559, in MakeReport
modrep_n = mod.MakeReport()
^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/sysstat.py", line 83, in MakeReport
data_n = rep_n.newTextChild(None, 'data', "\n"+"\n".join(textwrap.wrap(data, 75))+"\n")
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 384, in wrap
return w.wrap(text)
^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 356, in wrap
chunks = self._split_chunks(text)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 342, in _split_chunks
text = self._munge_whitespace(text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 153, in _munge_whitespace
text = text.translate(self.unicode_whitespace_trans)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'dict'
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/sysstat.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rteval/modules/measurement/sysstat.py b/rteval/modules/measurement/sysstat.py
index 57194a2b5f45..d4646c1646f4 100644
--- a/rteval/modules/measurement/sysstat.py
+++ b/rteval/modules/measurement/sysstat.py
@@ -79,7 +79,7 @@ class sysstat(rtevalModulePrototype):
fp = open(self.__datafile, "rb")
compr = bz2.BZ2Compressor(9)
cmpr = compr.compress(fp.read())
- data = base64.b64encode(cmpr + compr.flush())
+ data = base64.b64encode(cmpr + compr.flush()).decode('utf-8')
data_n = rep_n.newTextChild(None, 'data', "\n"+"\n".join(textwrap.wrap(data, 75))+"\n")
data_n.newProp('contents', 'sysstat/sar binary data')
data_n.newProp('encoding', 'base64')
--
2.45.2

526
rteval-timerlat-Add-timerlat-tracing-to-rteval.patch
View File

@ -1,526 +0,0 @@
From 5909521e06ed92ea60ffb247b25ca86c232f71bf Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Thu, 20 Jun 2024 21:34:24 -0400
Subject: [PATCH 1/4] rteval: timerlat: Add timerlat tracing to rteval
This patch adds tracing from timerlat to rteval.
These two options are added
--timerlat-stoptrace USEC
Stop trace if thread latency higher than USEC
--timerlat-trace FILE
File to save trace to
This also stores the summary from rtla timerlat in the rteval xml file.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/timerlat.py | 213 ++++++++++++++++++++++++-
rteval/rteval_text.xsl | 181 ++++++++++++++++++++-
2 files changed, 389 insertions(+), 5 deletions(-)
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index e8345fab1ad7..45adec1b33e1 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -16,6 +16,7 @@ from rteval.modules import rtevalModulePrototype
from rteval.systopology import cpuinfo, SysTopology
from rteval.cpulist_utils import expand_cpulist, collapse_cpulist
+
class TLRunData:
''' class to store instance data from a timerlat run '''
def __init__(self, coreid, datatype, priority, logfnc):
@@ -188,6 +189,15 @@ class Timerlat(rtevalModulePrototype):
self.__cpus = [str(c) for c in expand_cpulist(self.__cpulist)]
self.__numcores = len(self.__cpus)
+ # Has tracing been triggered
+ self.__stoptrace = False
+ # This stores the output from rtla
+ self.__posttrace = ""
+ # Stop Trace Data
+ self.__stdata = {}
+ # Stop Trace Cpu
+ self.stcpu = -1
+
self.__timerlat_out = None
self.__timerlat_err = None
self.__started = False
@@ -218,10 +228,21 @@ class Timerlat(rtevalModulePrototype):
self.__cmd = ['rtla', 'timerlat', 'hist', '-P', f'f:{int(self.__priority)}', '-u']
self.__cmd.append(f'-c{self.__cpulist}')
self.__cmd.append(f'-E{self.__buckets}')
+
+ if self.__cfg.stoptrace:
+ self.__cmd.append(f"-T{int(self.__cfg.stoptrace)}")
+
+ if self.__cfg.trace:
+ if not self.__cfg.stoptrace:
+ self._log(Log.WARN, f'Ignoring trace={self.__cfg.trace}, because stoptrace not invoked')
+ else:
+ self.__cmd.append(f'-t={self.__cfg.trace}')
+
self._log(Log.DEBUG, f'self.__cmd = {self.__cmd}')
self.__timerlat_out = tempfile.SpooledTemporaryFile(mode='w+b')
self.__timerlat_err = tempfile.SpooledTemporaryFile(mode='w+b')
+
def _WorkloadTask(self):
if self.__started:
return
@@ -252,6 +273,10 @@ class Timerlat(rtevalModulePrototype):
os.kill(self.__timerlat_process.pid, signal.SIGINT)
time.sleep(2)
+ blocking_thread_detected = False
+ softirq_interference_detected = False
+ irq_interference_detected = False
+
# Parse histogram output
self.__timerlat_out.seek(0)
for line in self.__timerlat_out:
@@ -261,6 +286,113 @@ class Timerlat(rtevalModulePrototype):
if not line:
continue
+ # Parsing if stoptrace has been invoked
+ if self.__stoptrace:
+ self.__posttrace += line
+ line = line.strip()
+ fields = line.split()
+ if line.startswith("##") and fields[1] == "CPU":
+ self.stcpu = int(fields[2])
+ self._log(Log.DEBUG, f"self.stcpu = {self.stcpu}")
+ self.__stdata[self.stcpu] = {}
+ continue
+ if self.stcpu == -1:
+ self._log(Log.WARN, "Stop trace has been invoked, but a stop cpu has not been identified.")
+ continue
+ if line.startswith('------------------'):
+ blocking_thread_detected = False
+ softirq_interference_detected = False
+ irq_interference_detected = False
+ continue
+ if line.startswith("Thread latency:"):
+ thread_latency_percent = fields[-1].strip('()%')
+ self._log(Log.DEBUG, f"thread_latency_percent = {thread_latency_percent}")
+ thread_latency = fields[-3]
+ self._log(Log.DEBUG, f"thread_latency = {thread_latency}")
+ self.__stdata[self.stcpu]["Thread_latency"] = (thread_latency, thread_latency_percent)
+ elif line.startswith("Previous IRQ interference"):
+ self._log(Log.DEBUG, f'Previous_IRQ_interference = {fields[-2]}')
+ self.__stdata[self.stcpu]["Previous_IRQ_interference"] = fields[-2]
+ elif line.startswith("IRQ handler delay:"):
+ irq_handler_delay_percent = fields[-2].strip('(')
+ irq_handler_delay = fields[-4]
+ # Do we have (exit from idle)?
+ if fields[3] == '(exit':
+ field_name = "IRQ_handler_delay_exit_from_idle"
+ else:
+ field_name = "IRQ_handler_delay"
+ self._log(Log.DEBUG, f"{field_name} = {irq_handler_delay}")
+ self._log(Log.DEBUG, f"{field_name}_percent = {irq_handler_delay_percent}")
+ self.__stdata[self.stcpu][field_name] = (irq_handler_delay, irq_handler_delay_percent)
+ elif line.startswith("IRQ latency:"):
+ self._log(Log.DEBUG, f"irq_latency = {fields[-2]}")
+ self.__stdata[self.stcpu]["IRQ_latency"] = fields[-2]
+ elif line.startswith("Timerlat IRQ duration"):
+ timerlat_irq_duration_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"timerlat_irq_duration_percent = {timerlat_irq_duration_percent}")
+ timerlat_irq_duration = fields[-4]
+ self._log(Log.DEBUG, f"timerlat_irq_duration = {timerlat_irq_duration}")
+ self.__stdata[self.stcpu]["Timerlat_IRQ_duration"] = (timerlat_irq_duration, timerlat_irq_duration_percent)
+ elif line.startswith("Blocking thread:"):
+ blocking_thread_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"blocking_thread_percent = {blocking_thread_percent}")
+ blocking_thread = fields[-4]
+ self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
+ self.__stdata[self.stcpu]["Blocking_Thread"] = (blocking_thread, blocking_thread_percent)
+ blocking_thread_detected = True
+ irq_interference_detected = False
+ softirq_interference_detected = False
+ elif line.startswith("IRQ interference"):
+ irq_interference_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"irq_interference_percent = {irq_interference_percent}")
+ irq_interference = fields[-4]
+ self._log(Log.DEBUG, f"irq_interference = {irq_interference}")
+ self.__stdata[self.stcpu]["IRQ_interference"] = (irq_interference, irq_interference_percent)
+ blocking_thread_detected = False
+ irq_interference_detected = True
+ softirq_interference_detected = False
+ elif line.startswith("Softirq interference"):
+ softirq_interference_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"softirq_interference_percent = {softirq_interference_percent}")
+ softirq_interference = fields[-4]
+ self._log(Log.DEBUG, f"softirq_interference = {softirq_interference}")
+ self.__stdata[self.stcpu]["Softirq_interference"] = (softirq_interference, softirq_interference_percent)
+ blocking_thread_detected = False
+ irq_interference_detected = False
+ softirq_interference_detected = True
+ elif blocking_thread_detected:
+ self._log(Log.DEBUG, f'line={line}')
+ blocking_thread = " ".join(fields[0:-2])
+ self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
+ blocking_threadus = fields[-2]
+ self._log(Log.DEBUG, f"blocking_threadus = {blocking_threadus}")
+ self.__stdata[self.stcpu].setdefault("blocking_thread", [])
+ self.__stdata[self.stcpu]["blocking_thread"] += [(blocking_thread, blocking_threadus)]
+ elif softirq_interference_detected:
+ softirq = " ".join(fields[0:-2])
+ softirq_latency = fields[-2]
+ self._log(Log.DEBUG, f'softirq = {softirq}')
+ self._log(Log.DEBUG, f'softirq_latency = {softirq_latency}')
+ self.__stdata[self.stcpu].setdefault("softirq_interference", [])
+ self.__stdata[self.stcpu]["softirq_interference"] += [(softirq, softirq_latency)]
+ elif irq_interference_detected:
+ irq_interference_name = " ".join(fields[0:-2])
+ irq_interference_latency = fields[-2]
+ self._log(Log.DEBUG, f'irq_interference = {irq_interference_name}, latency = {irq_interference_latency}')
+ self.__stdata[self.stcpu].setdefault("irq_interference", [])
+ self.__stdata[self.stcpu]["irq_interference"] += [(irq_interference_name, irq_interference_latency)]
+ elif line.startswith("Max timerlat IRQ latency"):
+ self._log(Log.DEBUG, f"line={line}")
+ max_timerlat_irq_latency = fields[-5]
+ self._log(Log.DEBUG, f"max_timerlat_irq_latency = {max_timerlat_irq_latency}")
+ max_timerlat_cpu = int(fields[-1])
+ self._log(Log.DEBUG, f"max_timerlat_cpu = {max_timerlat_cpu}")
+ self.__stdata.setdefault(max_timerlat_cpu, {})
+ self.__stdata[max_timerlat_cpu]["Max_timerlat_IRQ_latency_from_idle"] = max_timerlat_irq_latency
+ else:
+ self._log(Log.DEBUG, f'line = {line}')
+ continue
+
if line.startswith('#'):
if line.startswith('# Duration:'):
duration = line.split()[2]
@@ -285,9 +417,13 @@ class Timerlat(rtevalModulePrototype):
elif line.startswith('max:'):
#print(line)
continue
+ elif line.startswith('rtla timerlat hit stop tracing'):
+ self.__stoptrace = True
+ self.__posttrace += line
+ continue
else:
- pass
#print(line)
+ pass
vals = line.split()
if not vals:
@@ -301,12 +437,18 @@ class Timerlat(rtevalModulePrototype):
continue
for i, core in enumerate(self.__cpus):
- self.__timerlatdata[core].bucket(index, int(vals[i*3+1]),
+ # There might not be a count on every cpu if tracing invoked
+ if i*3 + 1 >= len(vals):
+ self.__timerlatdata[core].bucket(index, 0, 0, 0)
+ self.__timerlatdata['system'].bucket(index, 0, 0, 0)
+ else:
+ self.__timerlatdata[core].bucket(index, int(vals[i*3+1]),
int(vals[i*3+2]),
int(vals[i*3+3]))
- self.__timerlatdata['system'].bucket(index, int(vals[i*3+1]),
+ self.__timerlatdata['system'].bucket(index, int(vals[i*3+1]),
int(vals[i*3+2]),
int(vals[i*3+3]))
+
# Generate statistics for each RunData object
for n in list(self.__timerlatdata.keys()):
self.__timerlatdata[n].reduce()
@@ -320,6 +462,62 @@ class Timerlat(rtevalModulePrototype):
rep_n = libxml2.newNode('timerlat')
rep_n.newProp('command_line', ' '.join(self.__cmd))
+ stoptrace_invoked_n = libxml2.newNode('stoptrace_invoked')
+ if self.stcpu != -1:
+ stoptrace_invoked_n.newProp("invoked", "true")
+ else:
+ stoptrace_invoked_n.newProp("invoked", "")
+ rep_n.addChild(stoptrace_invoked_n)
+
+ if self.stcpu != -1:
+ self._log(Log.DEBUG, f'self.__stdata = {self.__stdata}')
+ for cpu in self.__stdata:
+ # This is Max timerlat IRQ latency from idle
+ # With no other data from that cpu, so don't create a
+ # stoptrace_report for this
+ if len(self.__stdata[cpu]) == 1:
+ continue
+ stoptrace_n = libxml2.newNode('stoptrace_report')
+ stoptrace_n.newProp("CPU", str(cpu))
+ for k, v in self.__stdata[cpu].items():
+ self._log(Log.DEBUG, f"cpu={cpu}, k={k}, v={v}")
+ if isinstance(v, tuple):
+ latency = str(v[0])
+ percent = str(v[1])
+ cpu_n = stoptrace_n.newTextChild(None, str(k), None)
+ n = cpu_n.newTextChild(None, "latency", latency)
+ n.newProp('unit', 'us')
+
+ n = cpu_n.newTextChild(None, "latency_percent", percent)
+ n.newProp('unit', '%')
+ elif isinstance(v, list):
+ if k in ("blocking_thread", "softirq_interference", "irq_interference"):
+ for name, latency in v:
+ cpu_n = stoptrace_n.newTextChild(None, k, None)
+ n = cpu_n.newTextChild(None, "name", name)
+ n = cpu_n.newTextChild(None, "latency", latency)
+ n.newProp('unit', 'us')
+ else:
+ if k == "Max_timerlat_IRQ_latency_from_idle":
+ continue
+ cpu_n = stoptrace_n.newTextChild(None, str(k), str(v))
+ cpu_n.newProp('unit', 'us')
+ rep_n.addChild(stoptrace_n)
+
+ self._log(Log.DEBUG, f'timerlat: posttrace = \n{self.__posttrace}')
+ self._log(Log.DEBUG, 'timerlat: posttrace END')
+ for cpu in self.__stdata:
+ for k, v in self.__stdata[cpu].items():
+ if isinstance(v, tuple):
+ continue
+ if k == "Max_timerlat_IRQ_latency_from_idle":
+ max_timerlat_n = libxml2.newNode('max_timerlat_report')
+ max_timerlat_n.newProp("CPU", str(cpu))
+ cpu_n = max_timerlat_n.newTextChild(None, k, str(v))
+ cpu_n.newProp('unit', 'us')
+ rep_n.addChild(max_timerlat_n)
+ return rep_n
+
rep_n.addChild(self.__timerlatdata['system'].MakeReport())
for thr in self.__cpus:
if str(thr) not in self.__timerlatdata:
@@ -341,7 +539,13 @@ def ModuleParameters():
"metavar": "PRIO" },
"buckets": {"descr": "Number of buckets",
"default": 3500,
- "metavar": "NUM"}
+ "metavar": "NUM" },
+ "stoptrace": {"descr": "Stop trace if thread latency higher than USEC",
+ "default": None,
+ "metavar": "USEC" },
+ "trace": {"descr": "File to save trace to",
+ "default": None,
+ "metavar": "FILE" },
}
def create(params, logger):
@@ -367,6 +571,7 @@ if __name__ == '__main__':
cfg_tl = cfg.GetSection('timerlat')
cfg_tl.cpulist = collapse_cpulist(SysTopology().online_cpus())
+ cfg_tl.stoptrace=50
RUNTIME = 10
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 2f03bda0bb55..70777354efa5 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -316,14 +316,26 @@
<xsl:value-of select="@command_line"/>
<xsl:text>&#10;&#10;</xsl:text>
- <xsl:apply-templates select="abort_report"/>
+ <xsl:if test="stoptrace_invoked">
+ <xsl:text>rtla timerlat hit stop tracing</xsl:text>
+ <xsl:text>
+</xsl:text>
+ <xsl:apply-templates select="stoptrace_report"/>
+ <xsl:apply-templates select="max_timerlat_report"/>
+ </xsl:if>
+ <!-- Make sure the description is available before printing System: -->
+ <xsl:if test="system/@description">
<xsl:text> System: </xsl:text>
<xsl:value-of select="system/@description"/>
<xsl:text>&#10;</xsl:text>
+ </xsl:if>
+ <!-- If stoptrace_invoked is true, no Statistics are available -->
+ <xsl:if test="stoptrace_invoked != true">
<xsl:text> Statistics: &#10;</xsl:text>
<xsl:apply-templates select="system/statistics"/>
+ </xsl:if>
<!-- Add CPU core info and stats-->
<xsl:apply-templates select="core">
@@ -470,4 +482,171 @@
</xsl:if>
</xsl:template>
+ <!-- Format posttrace information if present -->
+ <xsl:template match="stoptrace_report">
+ <xsl:text>## CPU </xsl:text>
+ <xsl:value-of select="@CPU"/>
+ <xsl:text> hit stop tracing, analyzing it ##</xsl:text>
+ <xsl:text>
+</xsl:text>
+
+
+ <xsl:if test="Previous_IRQ_interference">
+ <xsl:text>Previous IRQ interference: up to </xsl:text>
+ <xsl:value-of select="Previous_IRQ_interference"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Previous_IRQ_interference/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:if>
+
+ <xsl:if test="IRQ_handler_delay">
+ <xsl:text>IRQ handler delay: </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:if test="IRQ_handler_delay_exit_from_idle">
+ <xsl:text>IRQ handler delay: (exit from idle) </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:text>IRQ latency:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+
+ <xsl:text>Timerlat IRQ duration:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+
+ <xsl:if test="Blocking_Thread">
+ <xsl:text>Blocking thread:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="blocking_thread">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:if test="Softirq_interference">
+ <xsl:text>Softirq interference:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Softirq_interference/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="softirq_interference">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:if test="IRQ_interference">
+ <xsl:text>IRQ interference:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_interference/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="irq_interference">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:text>--------------------------------------------------------------------------------</xsl:text>
+ <xsl:text>Thread latency:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Thread_latency/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency_percent/@unit"/>
+ <xsl:text>)
+
+</xsl:text>
+ </xsl:template>
+
+ <xsl:template match="max_timerlat_report">
+ <xsl:text>
+</xsl:text>
+ <xsl:text>Max timerlat IRQ latency from idle:</xsl:text>
+ <xsl:value-of select="Max_timerlat_IRQ_latency_from_idle"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Max_timerlat_IRQ_latency_from_idle/@unit"/>
+ <xsl:text> in cpu </xsl:text>
+ <xsl:value-of select="@CPU"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:template>
+
</xsl:stylesheet>
--
2.45.2

217
rteval-timerlat-tracing-clean-up.patch
View File

@ -1,217 +0,0 @@
From 4fd5b77dc47f039551050925e442db9a17be5e75 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Mon, 24 Jun 2024 08:45:54 -0400
Subject: [PATCH 4/4] rteval: timerlat tracing clean-up
The patch cleans-up the code that stores the timerlat summary into xml,
after a trace is stopped. It does so by grouping similar kinds of output
together for parsing. It also reduces the amount of debug output.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/timerlat.py | 142 ++++++++++---------------
rteval/rteval_text.xsl | 10 +-
2 files changed, 62 insertions(+), 90 deletions(-)
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index 45adec1b33e1..dc6226ccc991 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -273,12 +273,14 @@ class Timerlat(rtevalModulePrototype):
os.kill(self.__timerlat_process.pid, signal.SIGINT)
time.sleep(2)
+
+ # Parse histogram output
+ self.__timerlat_out.seek(0)
+
blocking_thread_detected = False
softirq_interference_detected = False
irq_interference_detected = False
- # Parse histogram output
- self.__timerlat_out.seek(0)
for line in self.__timerlat_out:
line = bytes.decode(line)
@@ -291,6 +293,8 @@ class Timerlat(rtevalModulePrototype):
self.__posttrace += line
line = line.strip()
fields = line.split()
+ if not line:
+ continue
if line.startswith("##") and fields[1] == "CPU":
self.stcpu = int(fields[2])
self._log(Log.DEBUG, f"self.stcpu = {self.stcpu}")
@@ -304,91 +308,59 @@ class Timerlat(rtevalModulePrototype):
softirq_interference_detected = False
irq_interference_detected = False
continue
- if line.startswith("Thread latency:"):
- thread_latency_percent = fields[-1].strip('()%')
- self._log(Log.DEBUG, f"thread_latency_percent = {thread_latency_percent}")
- thread_latency = fields[-3]
- self._log(Log.DEBUG, f"thread_latency = {thread_latency}")
- self.__stdata[self.stcpu]["Thread_latency"] = (thread_latency, thread_latency_percent)
- elif line.startswith("Previous IRQ interference"):
- self._log(Log.DEBUG, f'Previous_IRQ_interference = {fields[-2]}')
- self.__stdata[self.stcpu]["Previous_IRQ_interference"] = fields[-2]
- elif line.startswith("IRQ handler delay:"):
- irq_handler_delay_percent = fields[-2].strip('(')
- irq_handler_delay = fields[-4]
- # Do we have (exit from idle)?
- if fields[3] == '(exit':
- field_name = "IRQ_handler_delay_exit_from_idle"
- else:
- field_name = "IRQ_handler_delay"
- self._log(Log.DEBUG, f"{field_name} = {irq_handler_delay}")
- self._log(Log.DEBUG, f"{field_name}_percent = {irq_handler_delay_percent}")
- self.__stdata[self.stcpu][field_name] = (irq_handler_delay, irq_handler_delay_percent)
- elif line.startswith("IRQ latency:"):
- self._log(Log.DEBUG, f"irq_latency = {fields[-2]}")
- self.__stdata[self.stcpu]["IRQ_latency"] = fields[-2]
- elif line.startswith("Timerlat IRQ duration"):
- timerlat_irq_duration_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"timerlat_irq_duration_percent = {timerlat_irq_duration_percent}")
- timerlat_irq_duration = fields[-4]
- self._log(Log.DEBUG, f"timerlat_irq_duration = {timerlat_irq_duration}")
- self.__stdata[self.stcpu]["Timerlat_IRQ_duration"] = (timerlat_irq_duration, timerlat_irq_duration_percent)
- elif line.startswith("Blocking thread:"):
- blocking_thread_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"blocking_thread_percent = {blocking_thread_percent}")
- blocking_thread = fields[-4]
- self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
- self.__stdata[self.stcpu]["Blocking_Thread"] = (blocking_thread, blocking_thread_percent)
- blocking_thread_detected = True
- irq_interference_detected = False
- softirq_interference_detected = False
- elif line.startswith("IRQ interference"):
- irq_interference_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"irq_interference_percent = {irq_interference_percent}")
- irq_interference = fields[-4]
- self._log(Log.DEBUG, f"irq_interference = {irq_interference}")
- self.__stdata[self.stcpu]["IRQ_interference"] = (irq_interference, irq_interference_percent)
- blocking_thread_detected = False
- irq_interference_detected = True
- softirq_interference_detected = False
- elif line.startswith("Softirq interference"):
- softirq_interference_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"softirq_interference_percent = {softirq_interference_percent}")
- softirq_interference = fields[-4]
- self._log(Log.DEBUG, f"softirq_interference = {softirq_interference}")
- self.__stdata[self.stcpu]["Softirq_interference"] = (softirq_interference, softirq_interference_percent)
- blocking_thread_detected = False
- irq_interference_detected = False
- softirq_interference_detected = True
- elif blocking_thread_detected:
- self._log(Log.DEBUG, f'line={line}')
- blocking_thread = " ".join(fields[0:-2])
- self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
- blocking_threadus = fields[-2]
- self._log(Log.DEBUG, f"blocking_threadus = {blocking_threadus}")
- self.__stdata[self.stcpu].setdefault("blocking_thread", [])
- self.__stdata[self.stcpu]["blocking_thread"] += [(blocking_thread, blocking_threadus)]
- elif softirq_interference_detected:
- softirq = " ".join(fields[0:-2])
- softirq_latency = fields[-2]
- self._log(Log.DEBUG, f'softirq = {softirq}')
- self._log(Log.DEBUG, f'softirq_latency = {softirq_latency}')
- self.__stdata[self.stcpu].setdefault("softirq_interference", [])
- self.__stdata[self.stcpu]["softirq_interference"] += [(softirq, softirq_latency)]
- elif irq_interference_detected:
- irq_interference_name = " ".join(fields[0:-2])
- irq_interference_latency = fields[-2]
- self._log(Log.DEBUG, f'irq_interference = {irq_interference_name}, latency = {irq_interference_latency}')
- self.__stdata[self.stcpu].setdefault("irq_interference", [])
- self.__stdata[self.stcpu]["irq_interference"] += [(irq_interference_name, irq_interference_latency)]
- elif line.startswith("Max timerlat IRQ latency"):
- self._log(Log.DEBUG, f"line={line}")
- max_timerlat_irq_latency = fields[-5]
- self._log(Log.DEBUG, f"max_timerlat_irq_latency = {max_timerlat_irq_latency}")
+
+ # work around rtla not printing ':' after all names
+ if line.startswith('Softirq interference'):
+ name = 'Softirq_interference'
+ elif line.startswith('IRQ interference'):
+ name = 'IRQ_interference'
+ else:
+ name = ''.join(line.split(':')[0]).replace(' ', '_')
+ self._log(Log.DEBUG, f"name={name}")
+
+ if name in ['Thread_latency']:
+ latency = fields[-3]
+ percent = fields[-1].strip('()%')
+ self._log(Log.DEBUG, f'{name} = ({latency}, {percent})')
+ self.__stdata[self.stcpu][name] = (latency, percent)
+ continue
+ if name in ['Timerlat_IRQ_duration', 'IRQ_handler_delay', 'Blocking_thread', 'IRQ_interference', 'Softirq_interference']:
+ latency = fields[-4]
+ percent = fields[-2].strip('(')
+ if name == 'IRQ_handler_delay' and fields[3] == '(exit':
+ name = 'IRQ_handler_delay_exit_from_idle'
+ self._log(Log.DEBUG, f'{name} = ({latency}, {percent})')
+ self.__stdata[self.stcpu][name] = (latency, percent)
+ detected = {'Blocking_thread' : (True, False, False),
+ 'IRQ_interference' : (False, True, False),
+ 'Softirq_interference' : (False, False, True) }
+ if name in ('Blocking_thread', 'IRQ_interference', 'Softirq_interference'):
+ blocking_thread_detected, irq_interference_detected, softirq_interference_detected = detected.get(name)
+ continue
+ if name in ["IRQ_latency", "Previous_IRQ_interference"]:
+ latency = fields[-2]
+ self._log(Log.DEBUG, f'{name} = {fields[-2]}')
+ self.__stdata[self.stcpu][name] = fields[-2]
+ continue
+ if blocking_thread_detected or softirq_interference_detected or irq_interference_detected:
+ if blocking_thread_detected:
+ field_name = "blocking_thread"
+ elif softirq_interference_detected:
+ field_name = "softirq_interference"
+ elif irq_interference_detected:
+ field_name = "irq_interference"
+ thread = " ".join(fields[0:-2])
+ latency = fields[-2]
+ self._log(Log.DEBUG, f"{field_name} += [({thread}, {latency})]")
+ self.__stdata[self.stcpu].setdefault(field_name, [])
+ self.__stdata[self.stcpu][field_name] += [(thread, latency)]
+ continue
+ if name == "Max_timerlat_IRQ_latency_from_idle":
+ latency = fields[-5]
max_timerlat_cpu = int(fields[-1])
- self._log(Log.DEBUG, f"max_timerlat_cpu = {max_timerlat_cpu}")
+ self._log(Log.DEBUG, f'self.__stdata[{max_timerlat_cpu}][{name}] = {latency}')
self.__stdata.setdefault(max_timerlat_cpu, {})
- self.__stdata[max_timerlat_cpu]["Max_timerlat_IRQ_latency_from_idle"] = max_timerlat_irq_latency
+ self.__stdata[max_timerlat_cpu][name] = latency
else:
self._log(Log.DEBUG, f'line = {line}')
continue
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 70777354efa5..7ca0ae3a4c66 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -546,16 +546,16 @@
<xsl:text>)
</xsl:text>
- <xsl:if test="Blocking_Thread">
+ <xsl:if test="Blocking_thread">
<xsl:text>Blocking thread:</xsl:text>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency"/>
+ <xsl:value-of select="Blocking_thread/latency"/>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency/@unit"/>
+ <xsl:value-of select="Blocking_thread/latency/@unit"/>
<xsl:text> (</xsl:text>
- <xsl:value-of select="Blocking_Thread/latency_percent"/>
+ <xsl:value-of select="Blocking_thread/latency_percent"/>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency_percent/@unit"/>
+ <xsl:value-of select="Blocking_thread/latency_percent/@unit"/>
<xsl:text>)
</xsl:text>
</xsl:if>
--
2.45.2

31
rteval.spec
View File

@ -1,6 +1,6 @@
Name: rteval Name: rteval
Version: 3.8 Version: 3.9
Release: 13%{?dist} Release: 1%{?dist}
Summary: Utility to evaluate system suitability for RT Linux Summary: Utility to evaluate system suitability for RT Linux
Group: Development/Tools Group: Development/Tools
@ -34,29 +34,6 @@ Requires: rtla
BuildArch: noarch BuildArch: noarch
# Patches # Patches
Patch1: Updated-rteval-man-page.patch
Patch2: rteval-Fix-aNone-being-passed-to-cyclictest.patch
Patch3: rteval-timerlat-Add-timerlat-tracing-to-rteval.patch
Patch4: rteval-sysstat-Convert-base64-data-to-text-before-wr.patch
Patch5: rteval-Fix-sysreport-traceback-when-utility-sos-not-.patch
Patch6: rteval-timerlat-tracing-clean-up.patch
Patch7: rteval-fixed-manpage-to-include-stress-ng-header.patch
Patch8: rteval-measurement-Remove-ModuleInfo.patch
Patch9: rteval-Remove-MeasurementProfile.patch
Patch10: rteval-RtEvalModules-Remove-unused-methods.patch
Patch11: rteval-Enforce-only-one-latency-measurement-module-a.patch
Patch12: rteval-Add-noload-option.patch
Patch13: rteval-Fix-default-measurement-config.patch
Patch14: rteval-measurement-Change-latency-flag-to-latency_te.patch
Patch15: rteval-Added-functionality-to-allow-user-to-set-the-.patch
Patch16: rteval-run-cyclictest-using-default-system-when-sett.patch
Patch17: rteval-Add-module-for-tuned-state.patch
Patch18: rteval-Add-tuned-state-to-rteval-text-report.patch
Patch19: rteval-restore-all-load-module-options.patch
Patch20: rteval-Upgrade-load-kernel-to-linux-6.10.5.patch
Patch21: rteval-Fix-parsing-in-kcompile-of-the-kernel-to-comp.patch
Patch22: rteval-Change-constant-name-to-uppercase.patch
Patch23: rteval-Update-the-kcompile-kernel-to-linux-6.12-rc4.patch
%description %description
The rteval script is a utility for measuring various aspects of The rteval script is a utility for measuring various aspects of
@ -89,6 +66,10 @@ to the screen.
%{_bindir}/rteval %{_bindir}/rteval
%changelog %changelog
* Fri Nov 15 2024 John Kacur <jkacur@redhat.com> - 3.9-1
- Rebase to upstream rteval-3.9
Resolves: RHEL-65483
* Tue Oct 29 2024 Troy Dawson <tdawson@redhat.com> - 3.8-13 * Tue Oct 29 2024 Troy Dawson <tdawson@redhat.com> - 3.8-13
- Bump release for October 2024 mass rebuild: - Bump release for October 2024 mass rebuild:
Resolves: RHEL-64018 Resolves: RHEL-64018

2
sources
View File

@ -1 +1 @@
SHA512 (rteval-3.8.tar.xz) = 25426b8f05d517b8b418975ff50bbc18ed406d3945eb0167af8e61687c872044a4fcfe39817f103bd4e2f4490c4613e64bfe28006c360698b039fbb10f1784c0 SHA512 (rteval-3.9.tar.xz) = 923fc11bdfa4c1f51ead0108184b7c2c708581a4b1adcdad168359cf916e85e9838adb1e65c923cead90e9644886c02bf663ab7ab14f3630d98cb2b67cddc541