rpms/rteval
7
0
Fork 0
Code Issues Pull Requests Releases Activity

import CS rteval-3.8-7.el9

Browse Source
This commit is contained in:
eabdullin 2024-09-30 16:32:41 +00:00
parent c2c84c0326
commit 5f4fb5e1a1
28 changed files with 2958 additions and 2007 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1 +1 @@
SOURCES/rteval-3.7.tar.xz
SOURCES/rteval-3.8.tar.xz

2
.rteval.metadata
View File

@ -1 +1 @@
fe85dce7852985e9ca916877b278feed31508f8f SOURCES/rteval-3.7.tar.xz
99d67b221f21f27366d48a23f2414894ccbdc002 SOURCES/rteval-3.8.tar.xz

109
SOURCES/Updated-rteval-man-page.patch Normal file
View File

@ -0,0 +1,109 @@
From 66b6c5851ea2241b6e368a1538dc10fd1f3f76f7 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Wed, 5 Jun 2024 14:04:54 -0400
Subject: [PATCH] Updated rteval man page
The rteval man page should reflect the functionality of the
current iteration of rteval. Currently the rteval man page
reflects the functionality of an outdated version. This patch
removes deprecated options and adds options that have been
added in recent updates.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
---
doc/rteval.8 | 50 +++++++++++++++++++++++++++++++++-----------------
1 file changed, 33 insertions(+), 17 deletions(-)
diff --git a/doc/rteval.8 b/doc/rteval.8
index 578de370effa..264bf5207716 100644
--- a/doc/rteval.8
+++ b/doc/rteval.8
@@ -63,6 +63,9 @@ for example to execute a ten minute run of rteval:
.B \-v, \-\-verbose
Increase the verbosity of output during the test run
.TP
+.B \-q, \-\-quiet
+Set rteval to run in quiet mode.
+.TP
.B \-w WORKDIR, \-\-workdir=WORKDIR
Tell rteval to change directory to WORKDIR before creating any
subdirectories for report files. The default WORKDIR is the directory
@@ -71,11 +74,8 @@ in which rteval was started.
.B \-l LOADDIR, \-\-loaddir=LOADDIR
Tell rteval where to find the source for the loads
.TP
-.B \-\-loads\-cpulist=CPULIST
-List of CPUs where loads will run
-.TP
-.B \-\-measurement-cpulist=CPULIST
-List of CPUs where measurement applciation will run
+.B \-i INSTALLDIR, \-\-installdir=INSTALLDIR
+Tell rteval where to locate installed templates.
.TP
.B \-s, \-\-sysreport
Have rteval run the sysreport utility after a run to gather
@@ -103,11 +103,25 @@ Log the output of the loads in the report directory
.B \-O, \-\-onlyload
Only run the loads (don't run measurement threads)
.TP
+.B \-V, \-\-version
+Print rteval version and exit.
+.TP
.B \-S KERNEL_VERSION, \-\-source\-download=KERNEL_VERSION
download a source kernel from kernel.org and exit
+.SH GROUP OPTIONS
+.TP
+.B \-\-loads\-cpulist=CPULIST
+List of CPUs where loads will run
+.TP
+.B \-\-measurement-cpulist=CPULIST
+List of CPUs where measurement applciation will run
+.TP
+.B \-\-measurement-run-on-isolcpus
+Include isolated CPUs in default cpulist
+
-.SH MODULE OPTIONS
+.SH CYCLICTEST OPTIONS
These are options that affect the execution behavior of the measurement and load modules.
.TP
.B \-\-cyclictest-priority=PRIORITY
@@ -116,21 +130,23 @@ SCHED_FIFO priority for measurement threads (default: 95)
.B \-\-cyclictest-interval=INTERVAL
Measurement thread interval in microseconds (default: 100)
.TP
-.B \-\-cyclictest-distance=DISTANCE
-Interval increment in microseconds (default: 0)
-.TP
.B \-\-cyclictest-buckets=NBUCKETS
Number of 1 microsecond histogram buckets (default: 3500)
.TP
-.B \-\-hackbench-jobspercore=N
-Number of jobs per online-core for hackbench load
+.B \-\-cyclictest-breaktrace=USEC
+Send a break trace command if latency exceeds the given number of microseconds.
+.TP
+.B \-\-cyclictest-threshold=USEC
+Exit rteval if latency exceeds the given number of microseconds.
+.TP
+.B \-\-stressng-option=OPTION
+Pass in command line options for the stress-ng package.
+.TP
+.B \-\-stressng-arg=ARG
+Pass in command line arguments for the stress-ng package.
.TP
-.B \-\-kcompile-jobspercore=N
-Number of jobs per online-core for kernel compile load
-.\" .SH SEE ALSO
-.\" .BR bar (1),
-.\" .BR baz (1).
-.\" .br
+.B \-\-stressng-timeout=T
+Timeout after T seconds.
.SH FILES
.BR rteval-YYYYMMDD-S
--
2.45.1

254
SOURCES/rteval-Add-module-for-tuned-state.patch Normal file
View File

@ -0,0 +1,254 @@
From 2b589a971f8895c81e13b5cfeb45879a28ea8cc6 Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 24 Jul 2024 09:37:42 +0200
Subject: [PATCH 1/2] rteval: Add module for tuned state
Add a sysinfo module for collecting the tuned state. Three properties
are collected:
- whether tuned is present
- what the active tuned profile is
- whether tuned profile verification passes
In case of a failed verification, the tuned log is also collected.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/sysinfo/__init__.py | 5 +-
rteval/sysinfo/tuned.py | 191 +++++++++++++++++++++++++++++++++++++
2 files changed, 195 insertions(+), 1 deletion(-)
create mode 100644 rteval/sysinfo/tuned.py
diff --git a/rteval/sysinfo/__init__.py b/rteval/sysinfo/__init__.py
index 09af52e..4b7b03c 100644
--- a/rteval/sysinfo/__init__.py
+++ b/rteval/sysinfo/__init__.py
@@ -15,10 +15,11 @@ from rteval.sysinfo.memory import MemoryInfo
from rteval.sysinfo.osinfo import OSInfo
from rteval.sysinfo.newnet import NetworkInfo
from rteval.sysinfo.cmdline import cmdlineInfo
+from rteval.sysinfo.tuned import TunedInfo
from rteval.sysinfo import dmi
class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
- MemoryInfo, OSInfo, NetworkInfo, cmdlineInfo):
+ MemoryInfo, OSInfo, NetworkInfo, cmdlineInfo, TunedInfo):
def __init__(self, config, logger=None):
self.__logger = logger
KernelInfo.__init__(self, logger=logger)
@@ -28,6 +29,7 @@ class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
OSInfo.__init__(self, logger=logger)
cmdlineInfo.__init__(self, logger=logger)
NetworkInfo.__init__(self, logger=logger)
+ TunedInfo.__init__(self, logger=logger)
# Parse initial DMI decoding errors
self.ProcessWarnings()
@@ -49,6 +51,7 @@ class SystemInfo(KernelInfo, SystemServices, dmi.DMIinfo, CPUtopology,
report_n.addChild(MemoryInfo.MakeReport(self))
report_n.addChild(dmi.DMIinfo.MakeReport(self))
report_n.addChild(cmdlineInfo.MakeReport(self))
+ report_n.addChild(TunedInfo.MakeReport(self))
return report_n
diff --git a/rteval/sysinfo/tuned.py b/rteval/sysinfo/tuned.py
new file mode 100644
index 0000000..063fcbf
--- /dev/null
+++ b/rteval/sysinfo/tuned.py
@@ -0,0 +1,191 @@
+# -*- coding: utf-8 -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright 2024 Tomas Glozar <tglozar@redhat.com>
+#
+"""tuned sysinfo module"""
+import shutil
+import subprocess
+import sys
+import libxml2
+from rteval.Log import Log
+
+TUNED_ADM = "tuned-adm"
+TUNED_LOG_PATH = "/var/log/tuned/tuned.log"
+TUNED_VERIFY_START_LINE = "INFO tuned.daemon.daemon: verifying " \
+ "profile(s): realtime"
+
+
+def tuned_present():
+ """
+ Checks if tuned is present on the system
+ :return: True if tuned is present, False otherwise
+ """
+ return shutil.which(TUNED_ADM) is not None
+
+
+def tuned_active_profile():
+ """
+ Gets tuned active profile.
+ :return: Tuned profile (as a string) or "unknown"
+ """
+ try:
+ result = subprocess.check_output([TUNED_ADM, "active"])
+ except (OSError, subprocess.CalledProcessError):
+ return "unknown"
+ result = result.decode("utf-8")
+ split_result = result.split(": ")
+ if len(split_result) < 2:
+ return "unknown"
+ return split_result[1].strip()
+
+
+def tuned_verify():
+ """
+ Verifies if tuned profile is applied properly
+ :return: "success", "failure" or "unknown"
+ """
+ try:
+ result = subprocess.run([TUNED_ADM, "verify"],
+ stdout=subprocess.PIPE, check=False).stdout
+ except (OSError, subprocess.CalledProcessError):
+ return "unknown"
+ result = result.decode("utf-8")
+ if result.startswith("Verification succeeded"):
+ return "success"
+ if result.startswith("Verification failed"):
+ return "failure"
+ return "unknown"
+
+
+def tuned_get_log():
+ """
+ Read entries related to last profile verification from tuned log
+ :return: List of strings containing the entires, or None if no
+ verification is found in the log
+ """
+ try:
+ with open(TUNED_LOG_PATH, "r", encoding="utf-8") as file:
+ lines = file.readlines()
+ # Find start of last verification
+ start = None
+ for i in reversed(range(len(lines))):
+ if TUNED_VERIFY_START_LINE in lines[i]:
+ start = i
+ break
+ if start is None:
+ return None
+ return lines[start:]
+ except OSError:
+ return None
+
+
+class TunedInfo:
+ """
+ Gather information about tuned and make an XML report.
+ Collected information:
+ - whether tuned is present
+ - which tuned profile is active
+ - whether the tuned profile is applied correctly
+ - if not applied correctly, collect relevant info from log
+ """
+ def __init__(self, logger=None):
+ self.__logger = logger
+
+ def __log(self, logtype, msg):
+ if self.__logger:
+ self.__logger.log(logtype, msg)
+
+ def tuned_state_get(self):
+ """
+ Gets the state of tuned on the machine
+ :return: A dictionary describing the tuned state
+ """
+ result = {
+ "present": tuned_present()
+ }
+ if not result["present"]:
+ self.__log(Log.DEBUG, "tuned-adm not found; skipping tuned "
+ "sysinfo collection")
+ return result
+ result["active_profile"] = tuned_active_profile()
+ if result["active_profile"] == "unknown":
+ self.__log(Log.DEBUG, "could not retrieve tuned active profile")
+ return result
+ result["verified"] = tuned_verify()
+ if result["verified"] == "unknown":
+ self.__log(Log.DEBUG, "could not verify tuned state")
+ if result["verified"] == "failure":
+ # Include log to see cause to failure
+ result["verification_log"] = tuned_get_log()
+
+ return result
+
+ def MakeReport(self):
+ """
+ Create XML report
+ :return: libxml2 node containing the report
+ """
+ tuned = self.tuned_state_get()
+
+ rep_n = libxml2.newNode("Tuned")
+ rep_n.newProp("present", str(int(tuned["present"])))
+ for key, value in tuned.items():
+ if key == "present":
+ continue
+ child = libxml2.newNode(key)
+ if key == "verification_log":
+ if value is None:
+ self.__log(Log.WARN, "could not get verification log")
+ continue
+ for line in value:
+ # <date> <time> <log-level> <message>
+ line = line.split(" ", 3)
+ if len(line) != 4:
+ continue
+ line_child = libxml2.newNode("entry")
+ line_child.newProp("date", line[0])
+ line_child.newProp("time", line[1])
+ line_child.newProp("level", line[2])
+ line_child.setContent(line[3].strip())
+ child.addChild(line_child)
+ else:
+ child.setContent(value)
+ rep_n.addChild(child)
+
+ return rep_n
+
+
+def unit_test(rootdir):
+ try:
+ # Helper function tests
+ result = tuned_present()
+ print("tuned present:", result)
+ assert isinstance(result, bool), "__tuned_present() should return bool"
+ result = tuned_active_profile()
+ print("tuned active profile:", result)
+ assert isinstance(result, str), "__tuned_active_profile() should " \
+ "return string"
+ result = tuned_verify()
+ print("tuned verification state:", result)
+ assert isinstance(result, str), "__tuned_verify() should return string"
+ result = tuned_get_log()
+ assert isinstance(result, list) or result is None, \
+ "__tuned_get_log() should return list or None"
+
+ # Class tests
+ tuned = TunedInfo()
+ result = tuned.tuned_state_get()
+ print(result)
+ assert isinstance(result, dict), "TunedInfo.tuned_state_get() " \
+ "should return dict"
+ tuned.MakeReport()
+
+ return 0
+ except Exception as err:
+ print(f"** EXCEPTION: {str(err)}")
+ return 1
+
+
+if __name__ == '__main__':
+ sys.exit(unit_test(None))
--
2.45.2

190
SOURCES/rteval-Add-noload-option.patch Normal file
View File

@ -0,0 +1,190 @@
From 1a76277bc6a5e94d858497fdc8fa721540fa6af7 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 19 Jul 2024 14:58:41 -0500
Subject: [PATCH 5/7] rteval: Add --noload option
This option allows measurement to be done without any loads.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
doc/rteval.8 | 3 +++
rteval-cmd | 9 +++++++
rteval/__init__.py | 54 ++++++++++++++++++++++--------------------
rteval/rtevalReport.py | 3 ++-
rteval/rteval_text.xsl | 9 ++++++-
5 files changed, 50 insertions(+), 28 deletions(-)
diff --git a/doc/rteval.8 b/doc/rteval.8
index 9e2b377752e5..1981ac7b3913 100644
--- a/doc/rteval.8
+++ b/doc/rteval.8
@@ -108,6 +108,9 @@ Print rteval version and exit.
.TP
.B \-S KERNEL_VERSION, \-\-source\-download=KERNEL_VERSION
download a source kernel from kernel.org and exit
+.TP
+.B \-\-noload
+Only run the measurements (don't run loads)
.SH GROUP OPTIONS
.TP
diff --git a/rteval-cmd b/rteval-cmd
index 5cb6d7a44523..36b167a034e5 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -146,6 +146,9 @@ def parse_options(cfg, parser, cmdargs):
parser.add_argument("-S", "--source-download", nargs="*", dest="rteval___srcdownload",
type=str, default=None, metavar="KERNEL_VERSION",
help='download a source kernel from kernel.org and exit')
+ parser.add_argument("--noload", dest="rteval___noload",
+ action="store_true", default=False,
+ help="only run the measurements (don't run loads)")
if not cmdargs:
@@ -273,6 +276,12 @@ if __name__ == '__main__':
measuremods.SetupModuleOptions(parser)
cmd_args = parse_options(config, parser, sys.argv[1:])
+ if rtevcfg.noload:
+ if rtevcfg.onlyload:
+ # Make up your mind!
+ raise RuntimeError('The --noload and --onlyload options are incompatible.')
+ loadmods = None
+
# download kernel tarball
if rtevcfg.srcdownload:
logger.log(Log.DEBUG, f"Kernel Version to download = {rtevcfg.srcdownload}")
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 226d14f80f48..4d3e0c23e5ab 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -46,9 +46,6 @@ class RtEval(rtevalReport):
if not isinstance(config, rtevalConfig.rtevalConfig):
raise TypeError("config variable is not an rtevalConfig object")
- if not isinstance(loadmods, LoadModules):
- raise TypeError("loadmods variable is not a LoadModules object")
-
if not isinstance(measuremods, MeasurementModules):
raise TypeError("measuremods variable is not a MeasurementModules object")
@@ -111,20 +108,21 @@ class RtEval(rtevalReport):
except Exception as err:
raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]")
+ params = {'workdir':self.__rtevcfg.workdir,
+ 'reportdir':self.__reportdir and self.__reportdir or "",
+ 'builddir':builddir,
+ 'srcdir':self.__rtevcfg.srcdir,
+ 'verbose': self.__rtevcfg.verbose,
+ 'debugging': self.__rtevcfg.debugging,
+ 'numcores':self._sysinfo.cpu_getCores(True),
+ 'logging':self.__rtevcfg.logging,
+ 'memsize':self._sysinfo.mem_get_size(),
+ 'numanodes':self._sysinfo.mem_get_numa_nodes(),
+ 'duration': float(self.__rtevcfg.duration),
+ }
+
if self._loadmods:
self.__logger.log(Log.INFO, "Preparing load modules")
- params = {'workdir':self.__rtevcfg.workdir,
- 'reportdir':self.__reportdir and self.__reportdir or "",
- 'builddir':builddir,
- 'srcdir':self.__rtevcfg.srcdir,
- 'verbose': self.__rtevcfg.verbose,
- 'debugging': self.__rtevcfg.debugging,
- 'numcores':self._sysinfo.cpu_getCores(True),
- 'logging':self.__rtevcfg.logging,
- 'memsize':self._sysinfo.mem_get_size(),
- 'numanodes':self._sysinfo.mem_get_numa_nodes(),
- 'duration': float(self.__rtevcfg.duration),
- }
self._loadmods.Setup(params)
self.__logger.log(Log.INFO, "Preparing measurement modules")
@@ -144,15 +142,18 @@ class RtEval(rtevalReport):
print(f"rteval run on {os.uname()[2]} started at {time.asctime()}")
onlinecpus = self._sysinfo.cpu_getCores(True)
- cpulist = self._loadmods._cfg.GetSection("loads").cpulist
- if cpulist:
- print(f"started {self._loadmods.ModulesLoaded()} loads on cores {cpulist}", end=' ')
- else:
- print(f"started {self._loadmods.ModulesLoaded()} loads on {onlinecpus} cores", end=' ')
- if self._sysinfo.mem_get_numa_nodes() > 1:
- print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
- else:
- print("")
+ if self._loadmods:
+ cpulist = self._loadmods._cfg.GetSection("loads").cpulist
+ if cpulist:
+ print(f"started {self._loadmods.ModulesLoaded()} loads on cores {cpulist}",
+ end=' ')
+ else:
+ print(f"started {self._loadmods.ModulesLoaded()} loads on {onlinecpus} cores",
+ end=' ')
+ if self._sysinfo.mem_get_numa_nodes() > 1:
+ print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
+ else:
+ print("")
cpulist = self._measuremods._cfg.GetSection("measurement").cpulist
if cpulist:
print(f"started measurement threads on cores {cpulist}")
@@ -192,7 +193,7 @@ class RtEval(rtevalReport):
if threading.active_count() < nthreads:
raise RuntimeError("load thread died!")
- if not load_avg_checked:
+ if self._loadmods and not load_avg_checked:
self._loadmods.SaveLoadAvg()
load_avg_checked = 5
else:
@@ -202,7 +203,8 @@ class RtEval(rtevalReport):
left_to_run = stoptime - currtime
self.__show_remaining_time(left_to_run)
rpttime = currtime + report_interval
- print(f"load average: {self._loadmods.GetLoadAvg():.2f}")
+ if self._loadmods:
+ print(f"load average: {self._loadmods.GetLoadAvg():.2f}")
currtime = time.time()
self.__logger.log(Log.DEBUG, "out of measurement loop")
diff --git a/rteval/rtevalReport.py b/rteval/rtevalReport.py
index 57d99f520f50..7379a7904f3f 100644
--- a/rteval/rtevalReport.py
+++ b/rteval/rtevalReport.py
@@ -57,7 +57,8 @@ class rtevalReport:
self.__xmlreport.AppendXMLnodes(self._sysinfo.MakeReport())
# Add load info
- self.__xmlreport.AppendXMLnodes(self._loadmods.MakeReport())
+ if self._loadmods:
+ self.__xmlreport.AppendXMLnodes(self._loadmods.MakeReport())
# Add measurement data
self.__xmlreport.AppendXMLnodes(self._measuremods.MakeReport())
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index b801679abcc5..96846aaf4d54 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -15,7 +15,14 @@
<xsl:text>&#10;</xsl:text>
<xsl:text> Loads: </xsl:text>
- <xsl:value-of select="loads/@loads"/><xsl:text> loads run on cores </xsl:text><xsl:value-of select="loads/@loadcpus"/>
+ <xsl:choose>
+ <xsl:when test="loads">
+ <xsl:value-of select="loads/@loads"/><xsl:text> loads run on cores </xsl:text><xsl:value-of select="loads/@loadcpus"/>
+ </xsl:when>
+ <xsl:otherwise>
+ <xsl:text>none</xsl:text>
+ </xsl:otherwise>
+ </xsl:choose>
<xsl:text>&#10;</xsl:text>
<xsl:text> Measurement: </xsl:text>
--
2.45.2

278
SOURCES/rteval-Add-relative-cpulists-for-measurements.patch
View File

@ -1,278 +0,0 @@
From 64ce7848dfabd2056d35c8a60f3354db45e36286 Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Thu, 18 Jan 2024 10:18:10 +0100
Subject: [PATCH 2/4] rteval: Add relative cpulists for measurements
Instead of specifying an absolute list of CPUs to run measurements on
in --measurement-cpulist, implement an option to specify a relative list
with respect to the current cpuset of rteval.
The relative cpulist can include CPUs both for addition and for removal,
e.g. +0,1,-7,8.
Also move the logic for processing cpulists specified by the user as
a string into cpulists usable by rteval to a single function.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 26 +++++++-----
rteval/cpulist_utils.py | 33 ++++++++++++++++
rteval/modules/measurement/__init__.py | 9 +----
rteval/modules/measurement/cyclictest.py | 50 +++---------------------
rteval/systopology.py | 33 ++++++++++++++++
5 files changed, 90 insertions(+), 61 deletions(-)
diff --git a/rteval-cmd b/rteval-cmd
index d224728..a5e8746 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -30,7 +30,7 @@ from rteval import RtEval, rtevalConfig
from rteval.modules.loads import LoadModules
from rteval.modules.measurement import MeasurementModules
from rteval.version import RTEVAL_VERSION
-from rteval.systopology import SysTopology
+from rteval.systopology import SysTopology, parse_cpulist_from_config
from rteval.modules.loads.kcompile import ModuleParameters
import rteval.cpulist_utils as cpulist_utils
@@ -339,26 +339,32 @@ if __name__ == '__main__':
ldcfg = config.GetSection('loads')
msrcfg = config.GetSection('measurement')
- if ldcfg.cpulist and msrcfg.cpulist:
+ msrcfg_cpulist_present = msrcfg.cpulist != ""
+ # Parse measurement cpulist using parse_cpulist_from_config to account for run-on-isolcpus
+ # and relative cpusets
+ cpulist = parse_cpulist_from_config(msrcfg.cpulist, msrcfg.run_on_isolcpus)
+ if msrcfg_cpulist_present and not cpulist_utils.is_relative(msrcfg.cpulist) and msrcfg.run_on_isolcpus:
+ logger.log(Log.WARN, "ignoring --measurement-run-on-isolcpus, since cpulist is specified")
+ msrcfg.cpulist = collapse_cpulist(cpulist)
+ if ldcfg.cpulist:
ldcfg.cpulist = remove_offline(ldcfg.cpulist)
- msrcfg.cpulist = remove_offline(msrcfg.cpulist)
# if we only specified one set of cpus (loads or measurement)
# default the other to the inverse of the specified list
- if not ldcfg.cpulist and msrcfg.cpulist:
+ if not ldcfg.cpulist and msrcfg_cpulist_present:
tmplist = expand_cpulist(msrcfg.cpulist)
tmplist = SysTopology().invert_cpulist(tmplist)
- ldcfg.cpulist = compress_cpulist(tmplist)
- msrcfg.cpulist = remove_offline(msrcfg.cpulist)
- if not msrcfg.cpulist and ldcfg.cpulist:
+ tmplist = cpulist_utils.online_cpulist(tmplist)
+ ldcfg.cpulist = collapse_cpulist(tmplist)
+ if not msrcfg_cpulist_present and ldcfg.cpulist:
tmplist = expand_cpulist(ldcfg.cpulist)
tmplist = SysTopology().invert_cpulist(tmplist)
- msrcfg.cpulist = compress_cpulist(tmplist)
- ldcfg.cpulist = remove_offline(ldcfg.cpulist)
+ tmplist = cpulist_utils.online_cpulist(tmplist)
+ msrcfg.cpulist = collapse_cpulist(tmplist)
if ldcfg.cpulist:
logger.log(Log.DEBUG, f"loads cpulist: {ldcfg.cpulist}")
# if --onlyload is specified msrcfg.cpulist is unused
- if msrcfg.cpulist and not rtevcfg.onlyload:
+ if msrcfg_cpulist_present and not rtevcfg.onlyload:
logger.log(Log.DEBUG, f"measurement cpulist: {msrcfg.cpulist}")
logger.log(Log.DEBUG, f"workdir: {rtevcfg.workdir}")
diff --git a/rteval/cpulist_utils.py b/rteval/cpulist_utils.py
index 402d579..7abc45a 100644
--- a/rteval/cpulist_utils.py
+++ b/rteval/cpulist_utils.py
@@ -126,3 +126,36 @@ def nonisolated_cpulist(cpulist):
isolated_cpulist = sysread(cpupath, "isolated")
isolated_cpulist = expand_cpulist(isolated_cpulist)
return list(set(cpulist).difference(set(isolated_cpulist)))
+
+
+def is_relative(cpulist):
+ return cpulist.startswith("+") or cpulist.startswith("-")
+
+
+def expand_relative_cpulist(cpulist):
+ """
+ Expand a relative cpulist into a tuple of lists.
+ :param cpulist: Relative cpulist of form +1,2,3,-4,5,6
+ :return: Tuple of two lists, one for added CPUs, one for removed CPUs
+ """
+ added_cpus = []
+ removed_cpus = []
+
+ if not cpulist:
+ return added_cpus, removed_cpus
+
+ cpus = None
+
+ for part in cpulist.split(','):
+ if part.startswith('+') or part.startswith('-'):
+ cpus = added_cpus if part[0] == '+' else removed_cpus
+ part = part[1:]
+ if '-' in part:
+ a, b = part.split('-')
+ a, b = int(a), int(b)
+ cpus.extend(list(range(a, b + 1)))
+ else:
+ a = int(part)
+ cpus.append(a)
+
+ return list(set(added_cpus)), list(set(removed_cpus))
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 66dc9c5..11bd7b0 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -5,7 +5,7 @@
import libxml2
from rteval.modules import RtEvalModules, ModuleContainer
-from rteval.systopology import SysTopology as SysTop
+from rteval.systopology import parse_cpulist_from_config
import rteval.cpulist_utils as cpulist_utils
class MeasurementProfile(RtEvalModules):
@@ -183,12 +183,7 @@ measurement profiles, based on their characteristics"""
rep_n = libxml2.newNode("Measurements")
cpulist = self.__cfg.GetSection("measurement").cpulist
run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
- if cpulist:
- # Convert str to list and remove offline cpus
- cpulist = cpulist_utils.expand_cpulist(cpulist)
- cpulist = cpulist_utils.online_cpulist(cpulist)
- else:
- cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
+ cpulist = parse_cpulist_from_config(cpulist, run_on_isolcpus)
rep_n.newProp("measurecpus", cpulist_utils.collapse_cpulist(cpulist))
for mp in self.__measureprofiles:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index fdca257..7224225 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -16,8 +16,7 @@ import math
import libxml2
from rteval.Log import Log
from rteval.modules import rtevalModulePrototype
-from rteval.systopology import cpuinfo
-from rteval.systopology import SysTopology
+from rteval.systopology import cpuinfo, parse_cpulist_from_config
import rteval.cpulist_utils as cpulist_utils
expand_cpulist = cpulist_utils.expand_cpulist
@@ -193,39 +192,9 @@ class Cyclictest(rtevalModulePrototype):
self.__priority = int(self.__cfg.setdefault('priority', 95))
self.__buckets = int(self.__cfg.setdefault('buckets', 2000))
self.__numcores = 0
- self.__cpus = []
self.__cyclicdata = {}
- self.__sparse = False
- self.__run_on_isolcpus = bool(self.__cfg.setdefault('run-on-isolcpus', False))
-
- if self.__cfg.cpulist:
- self.__cpulist = self.__cfg.cpulist
- self.__cpus = expand_cpulist(self.__cpulist)
- # Only include online cpus
- self.__cpus = cpulist_utils.online_cpulist(self.__cpus)
- # Reset cpulist from the newly calculated self.__cpus
- self.__cpulist = cpulist_utils.collapse_cpulist(self.__cpus)
- self.__cpus = [str(c) for c in self.__cpus]
- self.__sparse = True
- if self.__run_on_isolcpus:
- self._log(Log.WARN, "ignoring --measurement-run-on-isolcpus, since cpulist is specified")
- else:
- self.__cpus = SysTopology().online_cpus_str()
- # Get the cpuset from the environment
- cpuset = os.sched_getaffinity(0)
- # Convert the elements to strings
- cpuset = [str(c) for c in cpuset]
- # Get isolated CPU list
- isolcpus = [str(c) for c in SysTopology().isolated_cpus()]
- # Only include cpus that are in the cpuset and isolated CPUs if run_on_isolcpus is enabled
- self.__cpus = [c for c in self.__cpus if c in cpuset or self.__run_on_isolcpus and c in isolcpus]
- if self.__run_on_isolcpus:
- self.__sparse = True
- self.__cpulist = cpulist_utils.collapse_cpulist([int(c) for c in self.__cpus])
-
- # Sort the list of cpus to align with the order reported by cyclictest
- self.__cpus.sort(key=int)
-
+ self.__cpulist = self.__cfg.cpulist
+ self.__cpus = [str(c) for c in expand_cpulist(self.__cpulist)]
self.__numcores = len(self.__cpus)
info = cpuinfo()
@@ -242,10 +211,7 @@ class Cyclictest(rtevalModulePrototype):
logfnc=self._log)
self.__cyclicdata['system'].description = (f"({self.__numcores} cores) ") + info['0']['model name']
- if self.__sparse:
- self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
- else:
- self._log(Log.DEBUG, f"system has {self.__numcores} cpu cores")
+ self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
self.__started = False
self.__cyclicoutput = None
self.__breaktraceval = None
@@ -280,12 +246,8 @@ class Cyclictest(rtevalModulePrototype):
f'-h {self.__buckets}',
f"-p{int(self.__priority)}",
]
- if self.__sparse:
- self.__cmd.append(f'-t{self.__numcores}')
- self.__cmd.append(f'-a{self.__cpulist}')
- else:
- self.__cmd.append('-t')
- self.__cmd.append('-a')
+ self.__cmd.append(f'-t{self.__numcores}')
+ self.__cmd.append(f'-a{self.__cpulist}')
if 'threads' in self.__cfg and self.__cfg.threads:
self.__cmd.append(f"-t{int(self.__cfg.threads)}")
diff --git a/rteval/systopology.py b/rteval/systopology.py
index 9e45762..6bcfc77 100644
--- a/rteval/systopology.py
+++ b/rteval/systopology.py
@@ -241,6 +241,39 @@ class SysTopology:
""" return a list of online cpus in cpulist """
return [c for c in self.online_cpus() if c in cpulist]
+
+def parse_cpulist_from_config(cpulist, run_on_isolcpus=False):
+ """
+ Generates a cpulist based on --*-cpulist argument given by user
+ :param cpulist: Value of --*-cpulist argument
+ :param run_on_isolcpus: Value of --*-run-on-isolcpus argument
+ :return: Sorted list of CPUs as integers
+ """
+ if cpulist and not cpulist_utils.is_relative(cpulist):
+ result = cpulist_utils.expand_cpulist(cpulist)
+ # Only include online cpus
+ result = cpulist_utils.online_cpulist(result)
+ else:
+ result = SysTopology().online_cpus()
+ # Get the cpuset from the environment
+ cpuset = os.sched_getaffinity(0)
+ # Get isolated CPU list
+ isolcpus = SysTopology().isolated_cpus()
+ if cpulist and cpulist_utils.is_relative(cpulist):
+ # Include cpus that are not removed in relative cpuset and are either in cpuset from affinity,
+ # isolcpus (with run_on_isolcpus enabled, or added by relative cpuset
+ added_cpus, removed_cpus = cpulist_utils.expand_relative_cpulist(cpulist)
+ result = [c for c in result
+ if (c in cpuset or
+ c in added_cpus or
+ run_on_isolcpus and c in isolcpus) and
+ c not in removed_cpus]
+ else:
+ # Only include cpus that are in the cpuset and isolated CPUs if run_on_isolcpus is enabled
+ result = [c for c in result if c in cpuset or run_on_isolcpus and c in isolcpus]
+ return result
+
+
if __name__ == "__main__":
def unit_test():
--
2.43.0

46
SOURCES/rteval-Add-tuned-state-to-rteval-text-report.patch Normal file
View File

@ -0,0 +1,46 @@
From e97e27f37fb5a397969875787e062c4eb22323fa Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 7 Aug 2024 08:33:33 +0200
Subject: [PATCH 2/2] rteval: Add tuned state to rteval text report
Add a section "Tuned state" to the text report generated by rteval. Show
tuned active state if tuned is present, otherwise, show a message that
tuned is not present. If the active state is known, show also the
verification result.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/rteval_text.xsl | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 96846aa..70f3e0f 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -143,6 +143,22 @@
</xsl:when>
<xsl:otherwise>(unknown)</xsl:otherwise>
</xsl:choose>
+ <xsl:text>&#10;</xsl:text>
+
+ <xsl:text> Tuned state: </xsl:text>
+ <xsl:choose>
+ <xsl:when test="SystemInfo/Tuned/@present='1'">
+ <xsl:value-of select="SystemInfo/Tuned/active_profile"/>
+ <xsl:text> profile</xsl:text>
+ <xsl:if test="SystemInfo/Tuned/active_profile != 'unknown'">
+ <xsl:text>, verification: </xsl:text>
+ <xsl:value-of select="SystemInfo/Tuned/verified"/>
+ </xsl:if>
+ </xsl:when>
+ <xsl:otherwise>
+ <xsl:text>not present</xsl:text>
+ </xsl:otherwise>
+ </xsl:choose>
<xsl:text>&#10;&#10;</xsl:text>
<xsl:text> System load:&#10;</xsl:text>
--
2.45.2

221
SOURCES/rteval-Added-functionality-to-allow-user-to-set-the-.patch Normal file
View File

@ -0,0 +1,221 @@
From b3c96935c82b7410422ca2973398ac9d0272c844 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Fri, 2 Aug 2024 11:46:35 -0400
Subject: [PATCH 1/2] rteval: Added functionality to allow user to set the
cstate of specified cpus when running rteval
We would like to be able to set the idle states of CPUs while running
rteval.
This patch adds the file cpupower.py and option '--idle-set' within
rteval to use cpupower. The set idle state is applied to the cpulist
given by '--measurement-cpulist'.
cpupower.py provides the infrastructure to interface and execute the cpupower
command, and the options in rteval-cmd let the user specify the idle state
to be set and the CPUs to set it on.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 9 ++
rteval/cpupower.py | 125 +++++++++++++++++++++++++
rteval/modules/__init__.py | 2 +-
rteval/modules/measurement/__init__.py | 2 +
4 files changed, 137 insertions(+), 1 deletion(-)
create mode 100644 rteval/cpupower.py
diff --git a/rteval-cmd b/rteval-cmd
index 19c82a0b64b3..f440a8a22622 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -29,6 +29,7 @@ from rteval.Log import Log
from rteval import RtEval, rtevalConfig
from rteval.modules.loads import LoadModules
from rteval.modules.measurement import MeasurementModules
+from rteval import cpupower
from rteval.version import RTEVAL_VERSION
from rteval.systopology import SysTopology, parse_cpulist_from_config
from rteval.modules.loads.kcompile import ModuleParameters
@@ -402,6 +403,10 @@ if __name__ == '__main__':
if not os.path.isdir(rtevcfg.workdir):
raise RuntimeError(f"work directory {rtevcfg.workdir} does not exist")
+ # if idle-set has been specified, enable the idle state via cpupower
+ if msrcfg.idlestate:
+ cpupower_controller = cpupower.Cpupower(msrcfg.cpulist, msrcfg.idlestate, logger=logger)
+ cpupower_controller.enable_idle_state()
rteval = RtEval(config, loadmods, measuremods, logger)
rteval.Prepare(rtevcfg.onlyload)
@@ -421,6 +426,10 @@ if __name__ == '__main__':
ec = rteval.Measure()
logger.log(Log.DEBUG, f"exiting with exit code: {ec}")
+ # restore previous idle state settings
+ if msrcfg.idlestate:
+ cpupower_controller.restore_idle_states()
+
sys.exit(ec)
except KeyboardInterrupt:
sys.exit(0)
diff --git a/rteval/cpupower.py b/rteval/cpupower.py
new file mode 100644
index 000000000000..37c4d33f1df4
--- /dev/null
+++ b/rteval/cpupower.py
@@ -0,0 +1,125 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+# Copyright 2024 Anubhav Shelat <ashelat@redhat.com>
+""" Object to execute cpupower tool """
+
+import subprocess
+import os
+import shutil
+import sys
+from rteval.Log import Log
+from rteval.systopology import SysTopology as SysTop
+from rteval import cpulist_utils
+
+PATH = '/sys/devices/system/cpu/'
+
+class Cpupower:
+ """ class to store data for executing cpupower and restoring idle state configuration """
+ def __init__(self, cpulist, idlestate, logger=None):
+ if not self.cpupower_present():
+ print('cpupower not found')
+ sys.exit(1)
+
+ self.__idle_state = int(idlestate)
+ self.__states = os.listdir(PATH + 'cpu0/cpuidle/')
+ # self.__idle_states is a dict with cpus as keys,
+ # and another dict as the value. The value dict
+ # has idle states as keys and a boolean as the
+ # value indicating if the state is disabled.
+ self.__idle_states = {}
+ self.__name = "cpupower"
+ self.__online_cpus = SysTop().online_cpus()
+ self.__cpulist = cpulist
+ self.__logger = logger
+
+
+ def _log(self, logtype, msg):
+ """ Common log function for rteval modules """
+ if self.__logger:
+ self.__logger.log(logtype, f"[{self.__name}] {msg}")
+
+
+ def enable_idle_state(self):
+ """ Use cpupower to set the idle state """
+ self.get_idle_states()
+
+ # ensure that idle state is in range of available idle states
+ if self.__idle_state > len(self.__states) - 1 or self.__idle_state < 0:
+ print(f'Idle state {self.__idle_state} is out of range')
+ sys.exit(1)
+
+ # enable all idle states to a certain depth, and disable any deeper idle states
+ with open(os.devnull, 'wb') as buffer:
+ for state in self.__states:
+ s = state.strip("state")
+ if int(s) > self.__idle_state:
+ self.run_cpupower(['cpupower', '-c', self.__cpulist,'idle-set', '-d', s], buffer)
+ else:
+ self.run_cpupower(['cpupower', '-c', self.__cpulist,'idle-set', '-e', s], buffer)
+
+ self._log(Log.DEBUG, f'Idle state depth {self.__idle_state} enabled on CPUs {self.__cpulist}')
+
+
+ def run_cpupower(self, args, output_buffer=None):
+ """ execute cpupower """
+ try:
+ subprocess.run(args, check=True, stdout=output_buffer)
+ except subprocess.CalledProcessError:
+ print('cpupower failed')
+ sys.exit(1)
+
+
+ def get_idle_states(self):
+ """ Store the current idle state setting """
+ for cpu in self.__online_cpus:
+ self.__idle_states[cpu] = {}
+ for state in self.__states:
+ fp = os.path.join(PATH, 'cpu' + str(cpu) + '/cpuidle/' + state + '/disable')
+ self.__idle_states[cpu][state] = self.read_idle_state(fp)
+
+
+ def restore_idle_states(self):
+ """ restore the idle state setting """
+ for cpu, states in self.__idle_states.items():
+ for state, disabled in states.items():
+ fp = os.path.join(PATH, 'cpu' + str(cpu) + '/cpuidle/' + state + '/disable')
+ self.write_idle_state(fp, disabled)
+ self._log(Log.DEBUG, 'Idle state settings restored')
+
+
+ def read_idle_state(self, file):
+ """ read the disable value for an idle state """
+ with open(file, 'r', encoding='utf-8') as f:
+ return f.read(1)
+
+
+ def write_idle_state(self, file, state):
+ """ write the disable value for and idle state """
+ with open(file, 'w', encoding='utf-8') as f:
+ f.write(state)
+
+
+ def get_idle_info(self):
+ """ execute cpupower idle-info """
+ self.run_cpupower(['cpupower', 'idle-info'])
+
+
+ def cpupower_present(self):
+ """ check if cpupower is downloaded """
+ return shutil.which("cpupower") is not None
+
+
+if __name__ == '__main__':
+ l = Log()
+ l.SetLogVerbosity(Log.DEBUG)
+
+ online_cpus = cpulist_utils.collapse_cpulist(SysTop().online_cpus())
+ idlestate = '1'
+ info = True
+
+ cpupower = Cpupower(online_cpus, idlestate, logger=l)
+ if idlestate:
+ cpupower.enable_idle_state()
+ cpupower.restore_idle_states()
+ print()
+ cpupower.get_idle_info()
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index d7792108d5b8..acd6330788e2 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -282,7 +282,7 @@ reference from the first import"""
grparser.add_argument(f'--{self.__modtype}-cpulist',
dest=f'{self.__modtype}___cpulist', action='store', default="",
help=f'CPU list where {self.__modtype} modules will run',
- metavar='LIST')
+ metavar='CPULIST')
for (modname, mod) in list(self.__modsloaded.items()):
opts = mod.ModuleParameters()
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index ecadd0885991..9314d1cb6bbc 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -38,6 +38,8 @@ class MeasurementModules(RtEvalModules):
default=self._cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
== "true",
help="Include isolated CPUs in default cpulist")
+ grparser.add_argument('--idle-set', dest='measurement___idlestate', metavar='IDLESTATE',
+ default=None, help='Idle state depth to set on cpus running measurement modules')
def Setup(self, modparams):
--
2.45.2

65
SOURCES/rteval-Change-the-default-kernel-for-kcompile.patch
View File

@ -1,65 +0,0 @@
From c0ee73f00f6868e0ead5ace958a88a6a23db6ad3 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Thu, 9 Nov 2023 15:43:53 -0500
Subject: [PATCH] rteval: Change the default kernel for kcompile to linux-6.6.1
Change the default kernel for kcompile to linux-6.6.1
Signed-off-by: John Kacur <jkacur@redhat.com>
---
Makefile | 2 +-
rteval/modules/loads/kcompile.py | 4 ++--
rteval/rteval.conf | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Makefile b/Makefile
index b73e8c13f3e5..14f74e087eff 100644
--- a/Makefile
+++ b/Makefile
@@ -18,7 +18,7 @@ PREFIX := /usr
DATADIR := $(DESTDIR)/$(PREFIX)/share
LOADDIR := loadsource
-KLOAD := $(LOADDIR)/linux-6.1.8.tar.xz
+KLOAD := $(LOADDIR)/linux-6.6.1.tar.xz
BLOAD := $(LOADDIR)/dbench-4.0.tar.gz
LOADS := $(KLOAD) $(BLOAD)
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 8be79ce630d5..0d025771e90e 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -20,7 +20,7 @@ expand_cpulist = CpuList.expand_cpulist
compress_cpulist = CpuList.compress_cpulist
nonisolated_cpulist = CpuList.nonisolated_cpulist
-DEFAULT_KERNEL_PREFIX = "linux-6.1"
+DEFAULT_KERNEL_PREFIX = "linux-6.6"
class KBuildJob:
'''Class to manage a build job bound to a particular node'''
@@ -334,7 +334,7 @@ class Kcompile(CommandLineLoad):
def ModuleParameters():
return {"source": {"descr": "Source tar ball",
- "default": "linux-6.1.8.tar.xz",
+ "default": "linux-6.6.1.tar.xz",
"metavar": "TARBALL"},
"jobspercore": {"descr": "Number of working threads per core",
"default": 2,
diff --git a/rteval/rteval.conf b/rteval/rteval.conf
index 79e28032dc6b..a4aad33e264f 100644
--- a/rteval/rteval.conf
+++ b/rteval/rteval.conf
@@ -18,7 +18,7 @@ dbench: external
stressng: module
[kcompile]
-source: linux-6.1.8.xz
+source: linux-6.6.1.xz
jobspercore: 2
[hackbench]
--
2.41.0

616
SOURCES/rteval-Convert-CpuList-class-to-a-module.patch
View File

@ -1,616 +0,0 @@
From a788ea3ec976e51309f0bfae1a41cce704a77f2d Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Thu, 18 Jan 2024 10:18:09 +0100
Subject: [PATCH 1/4] rteval: Convert CpuList class to a module
Move out code from CpuList class in rteval.systopology into a separate
module named rteval.cpulist_utils and avoid wrapping CPU lists in
a CpuList object.
Almost all uses of CpuList in the code either use the static methods of
the class or its constructor to filter online CPUs by running
CpuList(...).cpulist. The only exception to this are NumaNode and
SimNumaNode classes in systopology; these store a CpuList object,
however their .getcpulist() method extracts the list out. Thus,
the class is completely unnecessary.
Note: A better name for the module would be cpulist, consistent with the
original name of the class, but this name is already used for variables
throughout the code, hence cpulist_utils is used instead.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 10 +-
rteval/cpulist_utils.py | 128 ++++++++++++++++++
rteval/modules/loads/__init__.py | 8 +-
rteval/modules/loads/hackbench.py | 9 +-
rteval/modules/loads/kcompile.py | 11 +-
rteval/modules/loads/stressng.py | 8 +-
rteval/modules/measurement/__init__.py | 8 +-
rteval/modules/measurement/cyclictest.py | 11 +-
rteval/systopology.py | 165 ++---------------------
9 files changed, 177 insertions(+), 181 deletions(-)
create mode 100644 rteval/cpulist_utils.py
diff --git a/rteval-cmd b/rteval-cmd
index 7c41429..d224728 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -30,11 +30,13 @@ from rteval import RtEval, rtevalConfig
from rteval.modules.loads import LoadModules
from rteval.modules.measurement import MeasurementModules
from rteval.version import RTEVAL_VERSION
-from rteval.systopology import CpuList, SysTopology
+from rteval.systopology import SysTopology
from rteval.modules.loads.kcompile import ModuleParameters
+import rteval.cpulist_utils as cpulist_utils
-compress_cpulist = CpuList.compress_cpulist
-expand_cpulist = CpuList.expand_cpulist
+compress_cpulist = cpulist_utils.compress_cpulist
+expand_cpulist = cpulist_utils.expand_cpulist
+collapse_cpulist = cpulist_utils.collapse_cpulist
def summarize(repfile, xslt):
""" Summarize an already existing XML report """
@@ -211,7 +213,7 @@ def remove_offline(cpulist):
""" return cpulist in collapsed compressed form with only online cpus """
tmplist = expand_cpulist(cpulist)
tmplist = SysTopology().online_cpulist(tmplist)
- return CpuList.collapse_cpulist(tmplist)
+ return collapse_cpulist(tmplist)
if __name__ == '__main__':
diff --git a/rteval/cpulist_utils.py b/rteval/cpulist_utils.py
new file mode 100644
index 0000000..402d579
--- /dev/null
+++ b/rteval/cpulist_utils.py
@@ -0,0 +1,128 @@
+# -*- coding: utf-8 -*-
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright 2016 - Clark Williams <williams@redhat.com>
+# Copyright 2021 - John Kacur <jkacur@redhat.com>
+# Copyright 2023 - Tomas Glozar <tglozar@redhat.com>
+#
+"""Module providing utility functions for working with CPU lists"""
+
+import os
+
+
+cpupath = "/sys/devices/system/cpu"
+
+
+def sysread(path, obj):
+ """ Helper function for reading system files """
+ with open(os.path.join(path, obj), "r") as fp:
+ return fp.readline().strip()
+
+
+def _online_file_exists():
+ """ Check whether machine / kernel is configured with online file """
+ # Note: some machines do not have cpu0/online so we check cpu1/online.
+ # In the case of machines with a single CPU, there is no cpu1, but
+ # that is not a problem, since a single CPU cannot be offline
+ return os.path.exists(os.path.join(cpupath, "cpu1/online"))
+
+
+def _isolated_file_exists():
+ """ Check whether machine / kernel is configured with isolated file """
+ return os.path.exists(os.path.join(cpupath, "isolated"))
+
+
+def collapse_cpulist(cpulist):
+ """
+ Collapse a list of cpu numbers into a string range
+ of cpus (e.g. 0-5, 7, 9)
+ """
+ cur_range = [None, None]
+ result = []
+ for cpu in cpulist + [None]:
+ if cur_range[0] is None:
+ cur_range[0] = cur_range[1] = cpu
+ continue
+ if cpu is not None and cpu == cur_range[1] + 1:
+ # Extend currently processed range
+ cur_range[1] += 1
+ else:
+ # Range processing finished, add range to string
+ result.append(f"{cur_range[0]}-{cur_range[1]}"
+ if cur_range[0] != cur_range[1]
+ else str(cur_range[0]))
+ # Reset
+ cur_range[0] = cur_range[1] = cpu
+ return ",".join(result)
+
+
+def compress_cpulist(cpulist):
+ """ return a string representation of cpulist """
+ if not cpulist:
+ return ""
+ if isinstance(cpulist[0], int):
+ return ",".join(str(e) for e in cpulist)
+ return ",".join(cpulist)
+
+
+def expand_cpulist(cpulist):
+ """ expand a range string into an array of cpu numbers
+ don't error check against online cpus
+ """
+ result = []
+
+ if not cpulist:
+ return result
+
+ for part in cpulist.split(','):
+ if '-' in part:
+ a, b = part.split('-')
+ a, b = int(a), int(b)
+ result.extend(list(range(a, b + 1)))
+ else:
+ a = int(part)
+ result.append(a)
+ return [int(i) for i in list(set(result))]
+
+
+def is_online(n):
+ """ check whether cpu n is online """
+ path = os.path.join(cpupath, f'cpu{n}')
+
+ # Some hardware doesn't allow cpu0 to be turned off
+ if not os.path.exists(path + '/online') and n == 0:
+ return True
+
+ return sysread(path, "online") == "1"
+
+
+def online_cpulist(cpulist):
+ """ Given a cpulist, return a cpulist of online cpus """
+ # This only works if the sys online files exist
+ if not _online_file_exists():
+ return cpulist
+ newlist = []
+ for cpu in cpulist:
+ if not _online_file_exists() and cpu == '0':
+ newlist.append(cpu)
+ elif is_online(int(cpu)):
+ newlist.append(cpu)
+ return newlist
+
+
+def isolated_cpulist(cpulist):
+ """Given a cpulist, return a cpulist of isolated CPUs"""
+ if not _isolated_file_exists():
+ return cpulist
+ isolated_cpulist = sysread(cpupath, "isolated")
+ isolated_cpulist = expand_cpulist(isolated_cpulist)
+ return list(set(isolated_cpulist) & set(cpulist))
+
+
+def nonisolated_cpulist(cpulist):
+ """Given a cpulist, return a cpulist of non-isolated CPUs"""
+ if not _isolated_file_exists():
+ return cpulist
+ isolated_cpulist = sysread(cpupath, "isolated")
+ isolated_cpulist = expand_cpulist(isolated_cpulist)
+ return list(set(cpulist).difference(set(isolated_cpulist)))
diff --git a/rteval/modules/loads/__init__.py b/rteval/modules/loads/__init__.py
index 13fba1e..0845742 100644
--- a/rteval/modules/loads/__init__.py
+++ b/rteval/modules/loads/__init__.py
@@ -11,7 +11,8 @@ import libxml2
from rteval.Log import Log
from rteval.rtevalConfig import rtevalCfgSection
from rteval.modules import RtEvalModules, rtevalModulePrototype
-from rteval.systopology import CpuList, SysTopology as SysTop
+from rteval.systopology import SysTopology as SysTop
+import rteval.cpulist_utils as cpulist_utils
class LoadThread(rtevalModulePrototype):
def __init__(self, name, config, logger=None):
@@ -117,10 +118,11 @@ class LoadModules(RtEvalModules):
cpulist = self._cfg.GetSection(self._module_config).cpulist
if cpulist:
# Convert str to list and remove offline cpus
- cpulist = CpuList(cpulist).cpulist
+ cpulist = cpulist_utils.expand_cpulist(cpulist)
+ cpulist = cpulist_utils.online_cpulist(cpulist)
else:
cpulist = SysTop().default_cpus()
- rep_n.newProp("loadcpus", CpuList.collapse_cpulist(cpulist))
+ rep_n.newProp("loadcpus", cpulist_utils.collapse_cpulist(cpulist))
return rep_n
diff --git a/rteval/modules/loads/hackbench.py b/rteval/modules/loads/hackbench.py
index cfc7063..a70fdb3 100644
--- a/rteval/modules/loads/hackbench.py
+++ b/rteval/modules/loads/hackbench.py
@@ -16,10 +16,11 @@ import errno
from signal import SIGKILL
from rteval.modules.loads import CommandLineLoad
from rteval.Log import Log
-from rteval.systopology import CpuList, SysTopology
+from rteval.systopology import SysTopology
+import rteval.cpulist_utils as cpulist_utils
-expand_cpulist = CpuList.expand_cpulist
-isolated_cpulist = CpuList.isolated_cpulist
+expand_cpulist = cpulist_utils.expand_cpulist
+isolated_cpulist = cpulist_utils.isolated_cpulist
class Hackbench(CommandLineLoad):
def __init__(self, config, logger):
@@ -61,7 +62,7 @@ class Hackbench(CommandLineLoad):
self.cpus[n] = [c for c in self.cpus[n] if c in expand_cpulist(self.cpulist)]
# if a cpulist was not specified, exclude isolated cpus
else:
- self.cpus[n] = CpuList.nonisolated_cpulist(self.cpus[n])
+ self.cpus[n] = cpulist_utils.nonisolated_cpulist(self.cpus[n])
# track largest number of cpus used on a node
node_biggest = len(self.cpus[n])
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 0d02577..b606f7a 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -14,11 +14,12 @@ import subprocess
from rteval.modules import rtevalRuntimeError
from rteval.modules.loads import CommandLineLoad
from rteval.Log import Log
-from rteval.systopology import CpuList, SysTopology
+from rteval.systopology import SysTopology
+import rteval.cpulist_utils as cpulist_utils
-expand_cpulist = CpuList.expand_cpulist
-compress_cpulist = CpuList.compress_cpulist
-nonisolated_cpulist = CpuList.nonisolated_cpulist
+expand_cpulist = cpulist_utils.expand_cpulist
+compress_cpulist = cpulist_utils.compress_cpulist
+nonisolated_cpulist = cpulist_utils.nonisolated_cpulist
DEFAULT_KERNEL_PREFIX = "linux-6.6"
@@ -38,7 +39,7 @@ class KBuildJob:
os.mkdir(self.objdir)
# Exclude isolated CPUs if cpulist not set
- cpus_available = len(nonisolated_cpulist(self.node.cpus.cpulist))
+ cpus_available = len(nonisolated_cpulist(self.node.cpus))
if os.path.exists('/usr/bin/numactl') and not cpulist:
# Use numactl
diff --git a/rteval/modules/loads/stressng.py b/rteval/modules/loads/stressng.py
index 7c9e63f..cbcf6b7 100644
--- a/rteval/modules/loads/stressng.py
+++ b/rteval/modules/loads/stressng.py
@@ -7,10 +7,10 @@ import subprocess
import signal
from rteval.modules.loads import CommandLineLoad
from rteval.Log import Log
-from rteval.systopology import CpuList, SysTopology
+from rteval.systopology import SysTopology
+import rteval.cpulist_utils as cpulist_utils
-expand_cpulist = CpuList.expand_cpulist
-nonisolated_cpulist = CpuList.nonisolated_cpulist
+expand_cpulist = cpulist_utils.expand_cpulist
class Stressng(CommandLineLoad):
" This class creates a load module that runs stress-ng "
@@ -73,7 +73,7 @@ class Stressng(CommandLineLoad):
cpus[n] = [c for c in cpus[n] if c in expand_cpulist(self.cpulist)]
# if a cpulist was not specified, exclude isolated cpus
else:
- cpus[n] = CpuList.nonisolated_cpulist(cpus[n])
+ cpus[n] = cpulist_utils.nonisolated_cpulist(cpus[n])
# remove nodes with no cpus available for running
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 41b8022..66dc9c5 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -5,7 +5,8 @@
import libxml2
from rteval.modules import RtEvalModules, ModuleContainer
-from rteval.systopology import CpuList, SysTopology as SysTop
+from rteval.systopology import SysTopology as SysTop
+import rteval.cpulist_utils as cpulist_utils
class MeasurementProfile(RtEvalModules):
"""Keeps and controls all the measurement modules with the same measurement profile"""
@@ -184,10 +185,11 @@ measurement profiles, based on their characteristics"""
run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
if cpulist:
# Convert str to list and remove offline cpus
- cpulist = CpuList(cpulist).cpulist
+ cpulist = cpulist_utils.expand_cpulist(cpulist)
+ cpulist = cpulist_utils.online_cpulist(cpulist)
else:
cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
- rep_n.newProp("measurecpus", CpuList.collapse_cpulist(cpulist))
+ rep_n.newProp("measurecpus", cpulist_utils.collapse_cpulist(cpulist))
for mp in self.__measureprofiles:
mprep_n = mp.MakeReport()
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 1b14e7e..fdca257 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -17,9 +17,10 @@ import libxml2
from rteval.Log import Log
from rteval.modules import rtevalModulePrototype
from rteval.systopology import cpuinfo
-from rteval.systopology import CpuList, SysTopology
+from rteval.systopology import SysTopology
+import rteval.cpulist_utils as cpulist_utils
-expand_cpulist = CpuList.expand_cpulist
+expand_cpulist = cpulist_utils.expand_cpulist
class RunData:
'''class to keep instance data from a cyclictest run'''
@@ -201,9 +202,9 @@ class Cyclictest(rtevalModulePrototype):
self.__cpulist = self.__cfg.cpulist
self.__cpus = expand_cpulist(self.__cpulist)
# Only include online cpus
- self.__cpus = CpuList(self.__cpus).cpulist
+ self.__cpus = cpulist_utils.online_cpulist(self.__cpus)
# Reset cpulist from the newly calculated self.__cpus
- self.__cpulist = CpuList.collapse_cpulist(self.__cpus)
+ self.__cpulist = cpulist_utils.collapse_cpulist(self.__cpus)
self.__cpus = [str(c) for c in self.__cpus]
self.__sparse = True
if self.__run_on_isolcpus:
@@ -220,7 +221,7 @@ class Cyclictest(rtevalModulePrototype):
self.__cpus = [c for c in self.__cpus if c in cpuset or self.__run_on_isolcpus and c in isolcpus]
if self.__run_on_isolcpus:
self.__sparse = True
- self.__cpulist = CpuList.collapse_cpulist([int(c) for c in self.__cpus])
+ self.__cpulist = cpulist_utils.collapse_cpulist([int(c) for c in self.__cpus])
# Sort the list of cpus to align with the order reported by cyclictest
self.__cpus.sort(key=int)
diff --git a/rteval/systopology.py b/rteval/systopology.py
index bed5192..9e45762 100644
--- a/rteval/systopology.py
+++ b/rteval/systopology.py
@@ -9,12 +9,8 @@
import os
import os.path
import glob
-
-
-def sysread(path, obj):
- """ Helper function for reading system files """
- with open(os.path.join(path, obj), "r") as fp:
- return fp.readline().strip()
+import rteval.cpulist_utils as cpulist_utils
+from rteval.cpulist_utils import sysread
def cpuinfo():
''' return a dictionary of cpu keys with various cpu information '''
@@ -56,145 +52,6 @@ def cpuinfo():
return info
-#
-# class to provide access to a list of cpus
-#
-
-class CpuList:
- "Object that represents a group of system cpus"
-
- cpupath = '/sys/devices/system/cpu'
-
- def __init__(self, cpulist):
- if isinstance(cpulist, list):
- self.cpulist = cpulist
- elif isinstance(cpulist, str):
- self.cpulist = self.expand_cpulist(cpulist)
- self.cpulist = self.online_cpulist(self.cpulist)
- self.cpulist.sort()
-
- def __str__(self):
- return self.collapse_cpulist(self.cpulist)
-
- def __contains__(self, cpu):
- return cpu in self.cpulist
-
- def __len__(self):
- return len(self.cpulist)
-
- def getcpulist(self):
- """ return the list of cpus tracked """
- return self.cpulist
-
- @staticmethod
- def online_file_exists():
- """ Check whether machine / kernel is configured with online file """
- # Note: some machines do not have cpu0/online so we check cpu1/online.
- # In the case of machines with a single CPU, there is no cpu1, but
- # that is not a problem, since a single CPU cannot be offline
- return os.path.exists(os.path.join(CpuList.cpupath, "cpu1/online"))
-
- @staticmethod
- def isolated_file_exists():
- """ Check whether machine / kernel is configured with isolated file """
- return os.path.exists(os.path.join(CpuList.cpupath, "isolated"))
-
- @staticmethod
- def collapse_cpulist(cpulist):
- """
- Collapse a list of cpu numbers into a string range
- of cpus (e.g. 0-5, 7, 9)
- """
- cur_range = [None, None]
- result = []
- for cpu in cpulist + [None]:
- if cur_range[0] is None:
- cur_range[0] = cur_range[1] = cpu
- continue
- if cpu is not None and cpu == cur_range[1] + 1:
- # Extend currently processed range
- cur_range[1] += 1
- else:
- # Range processing finished, add range to string
- result.append(f"{cur_range[0]}-{cur_range[1]}"
- if cur_range[0] != cur_range[1]
- else str(cur_range[0]))
- # Reset
- cur_range[0] = cur_range[1] = cpu
- return ",".join(result)
-
- @staticmethod
- def compress_cpulist(cpulist):
- """ return a string representation of cpulist """
- if not cpulist:
- return ""
- if isinstance(cpulist[0], int):
- return ",".join(str(e) for e in cpulist)
- return ",".join(cpulist)
-
- @staticmethod
- def expand_cpulist(cpulist):
- """ expand a range string into an array of cpu numbers
- don't error check against online cpus
- """
- result = []
-
- if not cpulist:
- return result
-
- for part in cpulist.split(','):
- if '-' in part:
- a, b = part.split('-')
- a, b = int(a), int(b)
- result.extend(list(range(a, b + 1)))
- else:
- a = int(part)
- result.append(a)
- return [int(i) for i in list(set(result))]
-
- @staticmethod
- def is_online(n):
- """ check whether cpu n is online """
- path = os.path.join(CpuList.cpupath, f'cpu{n}')
-
- # Some hardware doesn't allow cpu0 to be turned off
- if not os.path.exists(path + '/online') and n == 0:
- return True
-
- return sysread(path, "online") == "1"
-
- @staticmethod
- def online_cpulist(cpulist):
- """ Given a cpulist, return a cpulist of online cpus """
- # This only works if the sys online files exist
- if not CpuList.online_file_exists():
- return cpulist
- newlist = []
- for cpu in cpulist:
- if not CpuList.online_file_exists() and cpu == '0':
- newlist.append(cpu)
- elif CpuList.is_online(int(cpu)):
- newlist.append(cpu)
- return newlist
-
- @staticmethod
- def isolated_cpulist(cpulist):
- """Given a cpulist, return a cpulist of isolated CPUs"""
- if not CpuList.isolated_file_exists():
- return cpulist
- isolated_cpulist = sysread(CpuList.cpupath, "isolated")
- isolated_cpulist = CpuList.expand_cpulist(isolated_cpulist)
- return list(set(isolated_cpulist) & set(cpulist))
-
- @staticmethod
- def nonisolated_cpulist(cpulist):
- """Given a cpulist, return a cpulist of non-isolated CPUs"""
- if not CpuList.isolated_file_exists():
- return cpulist
- isolated_cpulist = sysread(CpuList.cpupath, "isolated")
- isolated_cpulist = CpuList.expand_cpulist(isolated_cpulist)
- return list(set(cpulist).difference(set(isolated_cpulist)))
-
#
# class to abstract access to NUMA nodes in /sys filesystem
#
@@ -208,7 +65,8 @@ class NumaNode:
"""
self.path = path
self.nodeid = int(os.path.basename(path)[4:].strip())
- self.cpus = CpuList(sysread(self.path, "cpulist"))
+ self.cpus = cpulist_utils.expand_cpulist(sysread(self.path, "cpulist"))
+ self.cpus = cpulist_utils.online_cpulist(self.cpus)
self.getmeminfo()
def __contains__(self, cpu):
@@ -240,11 +98,11 @@ class NumaNode:
def getcpustr(self):
""" return list of cpus for this node as a string """
- return str(self.cpus)
+ return cpulist_utils.collapse_cpulist(self.cpus)
def getcpulist(self):
""" return list of cpus for this node """
- return self.cpus.getcpulist()
+ return self.cpus
class SimNumaNode(NumaNode):
"""class representing a simulated NUMA node.
@@ -257,7 +115,8 @@ class SimNumaNode(NumaNode):
def __init__(self):
self.nodeid = 0
- self.cpus = CpuList(sysread(SimNumaNode.cpupath, "possible"))
+ self.cpus = cpulist_utils.expand_cpulist(sysread(SimNumaNode.cpupath, "possible"))
+ self.cpus = cpulist_utils.online_cpulist(self.cpus)
self.getmeminfo()
def getmeminfo(self):
@@ -339,7 +198,7 @@ class SysTopology:
""" return a list of integers of all online cpus """
cpulist = []
for n in self.nodes:
- cpulist += self.getcpus(n)
+ cpulist += cpulist_utils.online_cpulist(self.getcpus(n))
cpulist.sort()
return cpulist
@@ -347,7 +206,7 @@ class SysTopology:
""" return a list of integers of all isolated cpus """
cpulist = []
for n in self.nodes:
- cpulist += CpuList.isolated_cpulist(self.getcpus(n))
+ cpulist += cpulist_utils.isolated_cpulist(self.getcpus(n))
cpulist.sort()
return cpulist
@@ -355,7 +214,7 @@ class SysTopology:
""" return a list of integers of all default schedulable cpus, i.e. online non-isolated cpus """
cpulist = []
for n in self.nodes:
- cpulist += CpuList.nonisolated_cpulist(self.getcpus(n))
+ cpulist += cpulist_utils.nonisolated_cpulist(self.getcpus(n))
cpulist.sort()
return cpulist
@@ -403,7 +262,7 @@ if __name__ == "__main__":
onlcpus = s.online_cpus()
print(f'onlcpus = {onlcpus}')
- onlcpus = CpuList.collapse_cpulist(onlcpus)
+ onlcpus = cpulist_utils.collapse_cpulist(onlcpus)
print(f'onlcpus = {onlcpus}')
onlcpus_str = s.online_cpus_str()
--
2.43.0

87
SOURCES/rteval-Disable-use-of-python-dmidecode.patch
View File

@ -1,87 +0,0 @@
From d142f0d23d8df1cede3573c3d6cfbf16535b3475 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Wed, 20 Dec 2023 17:55:21 -0500
Subject: [PATCH 2/2] rteval: Disable use of python-dmidecode
python-dmidecode is not being maintained upstream.
For now just disable it's use in rteval since it is useful but not
essential information for running rteval
In the future look at generating this info directly using dmidecode.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
README | 6 ------
doc/installing.txt | 9 ---------
rteval/sysinfo/dmi.py | 4 ++--
3 files changed, 2 insertions(+), 17 deletions(-)
diff --git a/README b/README
index a5cf98344a46..b352d7f66ad2 100644
--- a/README
+++ b/README
@@ -19,15 +19,9 @@ Rteval requires the following packages to run:
Python >= 3.0
http://www.python.org/download/
-python-ethtool
- git://git.kernel.org/pub/scm/linux/kernel/git/acme/python-ethtool.git
-
python-lxml
http://lxml.de/
-python-dmidecode
- http://www.ohloh.net/p/python-dmidecode
-
libxml2-python
http://xmlsoft.org/
diff --git a/doc/installing.txt b/doc/installing.txt
index ff2d43cb9481..227249bbc9ed 100644
--- a/doc/installing.txt
+++ b/doc/installing.txt
@@ -1,18 +1,10 @@
The rteval utility requires some external software libraries to run
properly. These are:
-python-ethtool
- A python library to query network interfaces
- git://git.kernel.org/pub/scm/linux/kernel/git/acme/python-ethtool.git
-
python-lxml
A python library to parse XML files and XSLT stylesheets
http://lxml.de/
-python-dmidecode
- A python library used to access DMI table information
- http://www.autonomy.net.au/display/pydmi/Home
-
libxml2-python
A python library to parse XML files
http://xmlsoft.org/
@@ -22,7 +14,6 @@ rt-tests
git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git
-$ sudo yum install python-{dmidecode,ethtool)
$ git clone \
git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git
$ cd rt-tests && sudo make prefix=/usr install
diff --git a/rteval/sysinfo/dmi.py b/rteval/sysinfo/dmi.py
index e8285d263fe6..c01a0eef1435 100644
--- a/rteval/sysinfo/dmi.py
+++ b/rteval/sysinfo/dmi.py
@@ -15,8 +15,8 @@ from rteval import xmlout
from rteval import rtevalConfig
try:
- import dmidecode
- dmidecode_avail = True
+ # import dmidecode
+ dmidecode_avail = False
except ModuleNotFoundError:
dmidecode_avail = False
--
2.42.0

84
SOURCES/rteval-Enforce-only-one-latency-measurement-module-a.patch Normal file
View File

@ -0,0 +1,84 @@
From d6f62a5e52843e0b2651268e350a3c0ebe5c543b Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:17 -0500
Subject: [PATCH 4/7] rteval: Enforce only one latency measurement module at a
time
Latency modules will step on each other's toes if run at the same time
(on the same CPU, though that's an enhancement for later), so only
run one of them. A priority mechanism allows selecting
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 11 +++++++++++
rteval/modules/measurement/cyclictest.py | 1 +
rteval/modules/measurement/timerlat.py | 1 +
3 files changed, 13 insertions(+)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index de1ddc4628c1..2a4eafae71c7 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -40,6 +40,7 @@ class rtevalModulePrototype(threading.Thread):
"finished": threading.Event()}
self._donotrun = False
self._exclusive = False
+ self._latency = False
self.__timestamps = {}
self.__sleeptime = 2.0
@@ -67,6 +68,11 @@ class rtevalModulePrototype(threading.Thread):
self._exclusive = True
+ def set_latency(self):
+ """ Sets the module as an exclusive latency measurer """
+ self._latency = True
+
+
def set_donotrun(self):
""" set a module's donotrun field to True """
self._donotrun = True
@@ -412,9 +418,14 @@ class RtEvalModules:
self._logger.log(Log.INFO, f"Preparing {self._module_type} modules")
exclusive = 0
+ latency = False
for (modname, mod) in self.__modules:
if mod.is_exclusive() and mod.WorkloadWillRun():
exclusive += 1
+ if mod._latency:
+ if latency:
+ raise RuntimeError("More than one exclusive latency test")
+ latency = True
for (modname, mod) in self.__modules:
if exclusive >= 1:
if exclusive != 1:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 3a34c1b988d6..a9f5b0c4fba7 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -216,6 +216,7 @@ class Cyclictest(rtevalModulePrototype):
self.__started = False
self.__cyclicoutput = None
self.__breaktraceval = None
+ self.set_latency()
@staticmethod
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index f3bdc7098bc0..e4b80c33552e 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -216,6 +216,7 @@ class Timerlat(rtevalModulePrototype):
logfnc=self._log)
self.__timerlatdata['system'].description = (f"({self.__numcores} cores) ") + info['0']['model name']
self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
+ self.set_latency()
def _WorkloadSetup(self):
--
2.45.2

45
SOURCES/rteval-Fix-aNone-being-passed-to-cyclictest.patch Normal file
View File

@ -0,0 +1,45 @@
From 17a39863e5f89fbaf51dc158bc7e27f46676d46d Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Tue, 11 Jun 2024 16:45:53 +0200
Subject: [PATCH] rteval: Fix -aNone being passed to cyclictest
When rteval is called via the command line, cpulists for both
measurements and loads default to an empty string and are further
processed by parse_cpulist_from_config. However, this is not true when
rteval is used as a module: in that case, neither the default
command-line value is used nor parse_cpulist_from_config is run, leading
to None being set to the config property of measurements which is
explicitely passed down to the corresponding cyclictest config property.
After 64ce7848dfab ("rteval: Add relative cpulists for measurements"),
where the check for None was removed from the cyclictest module, rteval
passes "-aNone" to cyclictest when being used as a module.
Call parse_cpulist_from_config with an empty string to get the default
cpulist to pass to cyclictest module if cpulist is empty in
the measurement config.
Fixes: 64ce7848dfab ("rteval: Add relative cpulists for measurements")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/__init__.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 11bd7b0fce69..43c0fda30ce1 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -148,6 +148,9 @@ measurement profiles, based on their characteristics"""
modcfg = self.__cfg.GetSection("measurement")
cpulist = modcfg.cpulist
run_on_isolcpus = modcfg.run_on_isolcpus
+ if cpulist is None:
+ # Get default cpulist value
+ cpulist = cpulist_utils.collapse_cpulist(parse_cpulist_from_config("", run_on_isolcpus))
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
--
2.45.2

42
SOURCES/rteval-Fix-default-measurement-config.patch Normal file
View File

@ -0,0 +1,42 @@
From 9a3d515636fcea17690a4639554f8dae5ebd1ca0 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 19 Jul 2024 15:02:34 -0500
Subject: [PATCH 6/7] rteval: Fix default measurement config
Commit 761741d15d08e ("rteval: rteval-cmd: Some style changes suggested by pylint-3")
accidentally indented the check for missing measurement config, such that
it became part of the missing load conditional. This was harmless if
the cnofig file was missing entirely, but not if the config file
exists but the measurement section is empty.
Also, remove timerlat from the default as it conflicts with cyclictest,
and we're not quite ready to make timerlat the default.
Signed-off-by: Crystal Wood <crwood@redhat.com>
---
rteval-cmd | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/rteval-cmd b/rteval-cmd
index 36b167a034e5..19c82a0b64b3 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -247,11 +247,10 @@ if __name__ == '__main__':
'hackbench' : 'module',
'stressng' : 'module'})
- if not config.HasSection('measurement'):
- config.AppendConfig('measurement', {
- 'cyclictest' : 'module',
- 'timerlat' : 'module',
- 'sysstat' : 'module'})
+ if not config.HasSection('measurement'):
+ config.AppendConfig('measurement', {
+ 'cyclictest' : 'module',
+ 'sysstat' : 'module'})
# Prepare log levels before loading modules, not to have unwanted log messages
rtevcfg = config.GetSection('rteval')
--
2.45.2

42
SOURCES/rteval-Fix-sysreport-traceback-when-utility-sos-not-.patch Normal file
View File

@ -0,0 +1,42 @@
From 2a0b5833be4f55dbbc00f1835a4ace554e498137 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Fri, 21 Jun 2024 13:20:26 -0400
Subject: [PATCH] rteval: Fix sysreport traceback when utility sos not found
When rteval is run with
-s, --sysreport run sysreport to collect system data (default: False)
and sos, sosreport or sysreport cannot be found then rteval exits with
an error.
Fix this by adding /usr/bin to the places to search for this program.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/sysinfo/osinfo.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/rteval/sysinfo/osinfo.py b/rteval/sysinfo/osinfo.py
index 3d6d5f8daa96..3bdbcc81e068 100644
--- a/rteval/sysinfo/osinfo.py
+++ b/rteval/sysinfo/osinfo.py
@@ -45,10 +45,16 @@ class OSInfo:
def run_sysreport(self, repdir):
if os.path.exists('/usr/sbin/sos'):
exe = '/usr/sbin/sos report'
+ elif os.path.exists('/usr/bin/sos'):
+ exe = '/usr/bin/sos report'
elif os.path.exists('/usr/sbin/sosreport'):
exe = '/usr/sbin/sosreport'
+ elif os.path.exists('/usr/bin/sosreport'):
+ exe = '/usr/bin/sosreport'
elif os.path.exists('/usr/sbin/sysreport'):
exe = '/usr/sbin/sysreport'
+ elif os.path.exists('/usr/bin/sysreport'):
+ exe = '/usr/bin/sysreport'
else:
raise RuntimeError("Can't find sos/sosreport/sysreport")
--
2.45.2

64
SOURCES/rteval-Makefile-More-rpm-cleanups.patch
View File

@ -1,64 +0,0 @@
From 2d2e85c459d240926c99b1961bbef090aa80a1fc Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Wed, 20 Dec 2023 17:22:22 -0500
Subject: [PATCH 1/2] rteval: Makefile: More rpm cleanups
Afer having removed the upstream specfile, there were still a few
references to rpms in the Makefile. These are not necessary because any
work with rpms can be done with modern rpm tools and are typically done
by distributions
Also test whether directory 'run' exists since it may have been removed
by make realclean, and create it if it does not
Signed-off-by: John Kacur <jkacur@redhat.com>
---
Makefile | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/Makefile b/Makefile
index ee4cca555b95..b8bed643f760 100644
--- a/Makefile
+++ b/Makefile
@@ -31,6 +31,7 @@ load:
$(PYTHON) rteval-cmd --onlyload -D -L -v --workdir=./run --loaddir=$(HERE)/loadsource -f $(HERE)/rteval/rteval.conf -i $(HERE)/rteval
sysreport:
+ [ -d $(HERE)/run ] || mkdir run
$(PYTHON) rteval-cmd -D -v --workdir=$(HERE)/run --loaddir=$(HERE)/loadsource --duration=$(D) -i $(HERE)/rteval --sysreport
clean:
@@ -39,7 +40,7 @@ clean:
realclean: clean
[ -f $(XMLRPCDIR)/Makefile ] && make -C $(XMLRPCDIR) maintainer-clean || echo -n
- rm -rf run rpm
+ rm -rf run
install: install_loads install_rteval
@@ -73,13 +74,6 @@ rteval-xmlrpc-$(XMLRPCVER).tar.gz :
make distcheck
cp $(XMLRPCDIR)/rteval-xmlrpc-$(XMLRPCVER).tar.gz $(HERE)/
-rpm_prep:
- rm -rf rpm
- mkdir -p rpm/{BUILD,RPMS,SRPMS,SOURCES,SPECS}
-
-rpms rpm: rpm_prep rtevalrpm loadrpm
-
-
help:
@echo ""
@echo "rteval Makefile targets:"
@@ -88,6 +82,7 @@ help:
@echo " tarfile: create the source tarball"
@echo " install: install rteval locally"
@echo " clean: cleanup generated files"
+ @echo " realclean: Same as clean plus directory run"
@echo " sysreport: do a short testrun and generate sysreport data"
@echo " tags: generate a ctags file"
@echo " cleantags: remove the ctags file"
--
2.42.0

85
SOURCES/rteval-Minor-improvements-to-CpuList-class.patch
View File

@ -1,85 +0,0 @@
From 768daa63ec8e2299da53afe081e7304b37fc91cf Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 29 Nov 2023 10:34:55 +0100
Subject: [PATCH 2/9] rteval: Minor improvements to CpuList class
- Remove unnecessary if-else from online_file_exists
- Use cpupath in online_file_exists
- In is_online, remove check for n in cpuset and make it static
- Mark also the remaining methods static since they do not rely on
any fields of the class
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
- Removed incorrect line from commit message
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/systopology.py | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
diff --git a/rteval/systopology.py b/rteval/systopology.py
index ea8e242..60ac8e8 100644
--- a/rteval/systopology.py
+++ b/rteval/systopology.py
@@ -82,12 +82,17 @@ class CpuList:
def __len__(self):
return len(self.cpulist)
+ def getcpulist(self):
+ """ return the list of cpus tracked """
+ return self.cpulist
+
@staticmethod
def online_file_exists():
""" Check whether machine / kernel is configured with online file """
- if os.path.exists('/sys/devices/system/cpu/cpu1/online'):
- return True
- return False
+ # Note: some machines do not have cpu0/online so we check cpu1/online.
+ # In the case of machines with a single CPU, there is no cpu1, but
+ # that is not a problem, since a single CPU cannot be offline
+ return os.path.exists(os.path.join(CpuList.cpupath, "cpu1/online"))
@staticmethod
def isolated_file_exists():
@@ -147,14 +152,9 @@ class CpuList:
result.append(a)
return [int(i) for i in list(set(result))]
- def getcpulist(self):
- """ return the list of cpus tracked """
- return self.cpulist
-
- def is_online(self, n):
+ @staticmethod
+ def is_online(n):
""" check whether cpu n is online """
- if n not in self.cpulist:
- raise RuntimeError(f"invalid cpu number {n}")
path = os.path.join(CpuList.cpupath, f'cpu{n}')
# Some hardware doesn't allow cpu0 to be turned off
@@ -163,16 +163,17 @@ class CpuList:
return sysread(path, "online") == "1"
- def online_cpulist(self, cpulist):
+ @staticmethod
+ def online_cpulist(cpulist):
""" Given a cpulist, return a cpulist of online cpus """
# This only works if the sys online files exist
- if not self.online_file_exists():
+ if not CpuList.online_file_exists():
return cpulist
newlist = []
for cpu in cpulist:
- if not self.online_file_exists() and cpu == '0':
+ if not CpuList.online_file_exists() and cpu == '0':
newlist.append(cpu)
- elif self.is_online(int(cpu)):
+ elif CpuList.is_online(int(cpu)):
newlist.append(cpu)
return newlist
--
2.43.0

225
SOURCES/rteval-Refactor-collapse_cpulist-in-systopology.patch
View File

@ -1,225 +0,0 @@
From 4b4f50900e38a931ed9585c6cf200b74c0120a20 Mon Sep 17 00:00:00 2001
From: Tomas Glozar <tglozar@redhat.com>
Date: Wed, 29 Nov 2023 10:34:54 +0100
Subject: [PATCH 1/9] rteval: Refactor collapse_cpulist in systopology
Instead of having duplicate code in two functions, one top-level and
one member function of CpuList, have only one static function in
CpuList.
Additionally re-write the implementation to use a more straight forward
one-pass algorithm.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 4 +-
rteval/modules/loads/__init__.py | 4 +-
rteval/modules/measurement/__init__.py | 4 +-
rteval/modules/measurement/cyclictest.py | 6 +--
rteval/systopology.py | 68 ++++++++----------------
5 files changed, 30 insertions(+), 56 deletions(-)
diff --git a/rteval-cmd b/rteval-cmd
index 6f613a3..7c41429 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -30,7 +30,7 @@ from rteval import RtEval, rtevalConfig
from rteval.modules.loads import LoadModules
from rteval.modules.measurement import MeasurementModules
from rteval.version import RTEVAL_VERSION
-from rteval.systopology import CpuList, SysTopology, collapse_cpulist
+from rteval.systopology import CpuList, SysTopology
from rteval.modules.loads.kcompile import ModuleParameters
compress_cpulist = CpuList.compress_cpulist
@@ -211,7 +211,7 @@ def remove_offline(cpulist):
""" return cpulist in collapsed compressed form with only online cpus """
tmplist = expand_cpulist(cpulist)
tmplist = SysTopology().online_cpulist(tmplist)
- return collapse_cpulist(tmplist)
+ return CpuList.collapse_cpulist(tmplist)
if __name__ == '__main__':
diff --git a/rteval/modules/loads/__init__.py b/rteval/modules/loads/__init__.py
index aca0c9f..13fba1e 100644
--- a/rteval/modules/loads/__init__.py
+++ b/rteval/modules/loads/__init__.py
@@ -11,7 +11,7 @@ import libxml2
from rteval.Log import Log
from rteval.rtevalConfig import rtevalCfgSection
from rteval.modules import RtEvalModules, rtevalModulePrototype
-from rteval.systopology import collapse_cpulist, CpuList, SysTopology as SysTop
+from rteval.systopology import CpuList, SysTopology as SysTop
class LoadThread(rtevalModulePrototype):
def __init__(self, name, config, logger=None):
@@ -120,7 +120,7 @@ class LoadModules(RtEvalModules):
cpulist = CpuList(cpulist).cpulist
else:
cpulist = SysTop().default_cpus()
- rep_n.newProp("loadcpus", collapse_cpulist(cpulist))
+ rep_n.newProp("loadcpus", CpuList.collapse_cpulist(cpulist))
return rep_n
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 2a0556b..41b8022 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -5,7 +5,7 @@
import libxml2
from rteval.modules import RtEvalModules, ModuleContainer
-from rteval.systopology import collapse_cpulist, CpuList, SysTopology as SysTop
+from rteval.systopology import CpuList, SysTopology as SysTop
class MeasurementProfile(RtEvalModules):
"""Keeps and controls all the measurement modules with the same measurement profile"""
@@ -187,7 +187,7 @@ measurement profiles, based on their characteristics"""
cpulist = CpuList(cpulist).cpulist
else:
cpulist = SysTop().online_cpus() if run_on_isolcpus else SysTop().default_cpus()
- rep_n.newProp("measurecpus", collapse_cpulist(cpulist))
+ rep_n.newProp("measurecpus", CpuList.collapse_cpulist(cpulist))
for mp in self.__measureprofiles:
mprep_n = mp.MakeReport()
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 0af1d31..1b14e7e 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -17,7 +17,7 @@ import libxml2
from rteval.Log import Log
from rteval.modules import rtevalModulePrototype
from rteval.systopology import cpuinfo
-from rteval.systopology import CpuList, SysTopology, collapse_cpulist
+from rteval.systopology import CpuList, SysTopology
expand_cpulist = CpuList.expand_cpulist
@@ -203,7 +203,7 @@ class Cyclictest(rtevalModulePrototype):
# Only include online cpus
self.__cpus = CpuList(self.__cpus).cpulist
# Reset cpulist from the newly calculated self.__cpus
- self.__cpulist = collapse_cpulist(self.__cpus)
+ self.__cpulist = CpuList.collapse_cpulist(self.__cpus)
self.__cpus = [str(c) for c in self.__cpus]
self.__sparse = True
if self.__run_on_isolcpus:
@@ -220,7 +220,7 @@ class Cyclictest(rtevalModulePrototype):
self.__cpus = [c for c in self.__cpus if c in cpuset or self.__run_on_isolcpus and c in isolcpus]
if self.__run_on_isolcpus:
self.__sparse = True
- self.__cpulist = collapse_cpulist(self.__cpus)
+ self.__cpulist = CpuList.collapse_cpulist([int(c) for c in self.__cpus])
# Sort the list of cpus to align with the order reported by cyclictest
self.__cpus.sort(key=int)
diff --git a/rteval/systopology.py b/rteval/systopology.py
index 62ad355..ea8e242 100644
--- a/rteval/systopology.py
+++ b/rteval/systopology.py
@@ -10,25 +10,6 @@ import os
import os.path
import glob
-# Utility version of collapse_cpulist that doesn't require a CpuList object
-def collapse_cpulist(cpulist):
- """ Collapse a list of cpu numbers into a string range
- of cpus (e.g. 0-5, 7, 9) """
- if len(cpulist) == 0:
- return ""
- idx = CpuList.longest_sequence(cpulist)
- if idx == 0:
- seq = str(cpulist[0])
- else:
- if idx == 1:
- seq = f"{cpulist[0]},{cpulist[idx]}"
- else:
- seq = f"{cpulist[0]}-{cpulist[idx]}"
-
- rest = collapse_cpulist(cpulist[idx+1:])
- if rest == "":
- return seq
- return ",".join((seq, rest))
def sysread(path, obj):
""" Helper function for reading system files """
@@ -93,7 +74,7 @@ class CpuList:
self.cpulist.sort()
def __str__(self):
- return self.__collapse_cpulist(self.cpulist)
+ return self.collapse_cpulist(self.cpulist)
def __contains__(self, cpu):
return cpu in self.cpulist
@@ -114,35 +95,28 @@ class CpuList:
return os.path.exists(os.path.join(CpuList.cpupath, "isolated"))
@staticmethod
- def longest_sequence(cpulist):
- """ return index of last element of a sequence that steps by one """
- lim = len(cpulist)
- for idx, _ in enumerate(cpulist):
- if idx+1 == lim:
- break
- if int(cpulist[idx+1]) != (int(cpulist[idx])+1):
- return idx
- return lim - 1
-
- def __collapse_cpulist(self, cpulist):
- """ Collapse a list of cpu numbers into a string range
+ def collapse_cpulist(cpulist):
+ """
+ Collapse a list of cpu numbers into a string range
of cpus (e.g. 0-5, 7, 9)
"""
- if len(cpulist) == 0:
- return ""
- idx = self.longest_sequence(cpulist)
- if idx == 0:
- seq = str(cpulist[0])
- else:
- if idx == 1:
- seq = f"{cpulist[0]},{cpulist[idx]}"
+ cur_range = [None, None]
+ result = []
+ for cpu in cpulist + [None]:
+ if cur_range[0] is None:
+ cur_range[0] = cur_range[1] = cpu
+ continue
+ if cpu is not None and cpu == cur_range[1] + 1:
+ # Extend currently processed range
+ cur_range[1] += 1
else:
- seq = f"{cpulist[0]}-{cpulist[idx]}"
-
- rest = self.__collapse_cpulist(cpulist[idx+1:])
- if rest == "":
- return seq
- return ",".join((seq, rest))
+ # Range processing finished, add range to string
+ result.append(f"{cur_range[0]}-{cur_range[1]}"
+ if cur_range[0] != cur_range[1]
+ else str(cur_range[0]))
+ # Reset
+ cur_range[0] = cur_range[1] = cpu
+ return ",".join(result)
@staticmethod
def compress_cpulist(cpulist):
@@ -428,7 +402,7 @@ if __name__ == "__main__":
onlcpus = s.online_cpus()
print(f'onlcpus = {onlcpus}')
- onlcpus = collapse_cpulist(onlcpus)
+ onlcpus = CpuList.collapse_cpulist(onlcpus)
print(f'onlcpus = {onlcpus}')
onlcpus_str = s.online_cpus_str()
--
2.43.0

463
SOURCES/rteval-Remove-MeasurementProfile.patch Normal file
View File

@ -0,0 +1,463 @@
From 70d84e566ddef917373a5eb20acf9d7cbccb1f97 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:15 -0500
Subject: [PATCH 2/7] rteval: Remove MeasurementProfile
Now that the ModuleInfo() flags are gone, remove the remaining infrastructure
around measurement profiles.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/__init__.py | 25 ++----
rteval/modules/measurement/__init__.py | 117 ++++---------------------
rteval/rteval_histogram_raw.xsl | 24 ++---
rteval/rteval_text.xsl | 41 +++------
4 files changed, 54 insertions(+), 153 deletions(-)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 4a6883e28e5b..226d14f80f48 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -20,7 +20,7 @@ import time
from datetime import datetime
import sysconfig
from rteval.modules.loads import LoadModules
-from rteval.modules.measurement import MeasurementModules, MeasurementProfile
+from rteval.modules.measurement import MeasurementModules
from rteval.rtevalReport import rtevalReport
from rteval.Log import Log
from rteval import rtevalConfig
@@ -131,10 +131,8 @@ class RtEval(rtevalReport):
self._measuremods.Setup(params)
- def __RunMeasurementProfile(self, measure_profile):
+ def __RunMeasurement(self):
global earlystop
- if not isinstance(measure_profile, MeasurementProfile):
- raise Exception("measure_profile is not an MeasurementProfile object")
measure_start = None
try:
@@ -155,15 +153,14 @@ class RtEval(rtevalReport):
print(f" with {self._sysinfo.mem_get_numa_nodes()} numa nodes")
else:
print("")
- cpulist = self._measuremods._MeasurementModules__cfg.GetSection("measurement").cpulist
+ cpulist = self._measuremods._cfg.GetSection("measurement").cpulist
if cpulist:
print(f"started measurement threads on cores {cpulist}")
else:
print(f"started measurement threads on {onlinecpus} cores")
print(f"Run duration: {str(self.__rtevcfg.duration)} seconds")
- # start the cyclictest thread
- measure_profile.Start()
+ self._measuremods.Start()
# Unleash the loads and measurement threads
report_interval = int(self.__rtevcfg.report_interval)
@@ -172,7 +169,7 @@ class RtEval(rtevalReport):
nthreads = threading.active_count()
else:
nthreads = None
- measure_profile.Unleash()
+ self._measuremods.Unleash()
measure_start = datetime.now()
# wait for time to expire or thread to die
@@ -185,7 +182,7 @@ class RtEval(rtevalReport):
load_avg_checked = 5
while (currtime <= stoptime) and not stopsig.is_set():
stopsig.wait(min(stoptime - currtime, 60.0))
- if not measure_profile.isAlive():
+ if not self._measuremods.isAlive():
stoptime = currtime
earlystop = True
self.__logger.log(Log.WARN,
@@ -218,7 +215,7 @@ class RtEval(rtevalReport):
finally:
# stop measurement threads
- measure_profile.Stop()
+ self._measuremods.Stop()
# stop the loads
if self._loadmods:
@@ -227,7 +224,7 @@ class RtEval(rtevalReport):
print(f"stopping run at {time.asctime()}")
# wait for measurement modules to finish calculating stats
- measure_profile.WaitForCompletion()
+ self._measuremods.WaitForCompletion()
return measure_start
@@ -236,11 +233,7 @@ class RtEval(rtevalReport):
""" Run the full measurement suite with reports """
global earlystop
rtevalres = 0
- measure_start = None
- for meas_prf in self._measuremods:
- mstart = self.__RunMeasurementProfile(meas_prf)
- if measure_start is None:
- measure_start = mstart
+ measure_start = self.__RunMeasurement()
self._report(measure_start, self.__rtevcfg.xslt_report)
if self.__rtevcfg.sysreport:
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 7b1d84ef554d..ecadd0885991 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -8,43 +8,14 @@ from rteval.modules import RtEvalModules, ModuleContainer
from rteval.systopology import parse_cpulist_from_config
import rteval.cpulist_utils as cpulist_utils
-class MeasurementProfile(RtEvalModules):
- """Keeps and controls all the measurement modules with the same measurement profile"""
-
- def __init__(self, config, modules_root, logger):
- self._module_type = "measurement"
- self._module_config = "measurement"
- self._report_tag = "Profile"
- RtEvalModules.__init__(self, config, modules_root, logger)
-
-
- def ImportModule(self, module):
- "Imports an exported module from a ModuleContainer() class"
- return self._ImportModule(module)
-
-
- def Setup(self, modname):
- "Instantiates and prepares a measurement module"
-
- modobj = self._InstantiateModule(modname, self._cfg.GetSection(modname))
- self._RegisterModuleObject(modname, modobj)
-
-
-class MeasurementModules:
- """Class which takes care of all measurement modules and groups them into
-measurement profiles, based on their characteristics"""
+class MeasurementModules(RtEvalModules):
+ """Module container for measurement modules"""
def __init__(self, config, logger):
- self.__cfg = config
- self.__logger = logger
- self.__measureprofiles = []
- self.__modules_root = "modules.measurement"
- self.__iter_item = None
-
- # Temporary module container, which is used to evalute measurement modules.
- # This will container will be destroyed after Setup() has been called
- self.__container = ModuleContainer(self.__modules_root, self.__logger)
- self.__LoadModules(self.__cfg.GetSection("measurement"))
+ self._module_type = "measurement"
+ self._report_tag = "Measurements"
+ RtEvalModules.__init__(self, config, "modules.measurement", logger)
+ self.__LoadModules(self._cfg.GetSection("measurement"))
def __LoadModules(self, modcfg):
@@ -54,37 +25,28 @@ measurement profiles, based on their characteristics"""
# hope to eventually have different kinds but module is only on
# for now (jcw)
if m[1].lower() == 'module':
- self.__container.LoadModule(m[0])
-
-
- def GetProfile(self):
- "Returns the appropriate MeasurementProfile object, based on the profile type"
-
- for p in self.__measureprofiles:
- return p
- return None
-
+ self._LoadModule(m[0])
def SetupModuleOptions(self, parser):
"Sets up all the measurement modules' parameters for the option parser"
- grparser = self.__container.SetupModuleOptions(parser, self.__cfg)
+ grparser = super().SetupModuleOptions(parser)
# Set up options specific for measurement module group
grparser.add_argument("--measurement-run-on-isolcpus",
dest="measurement___run_on_isolcpus",
action="store_true",
- default=self.__cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
+ default=self._cfg.GetSection("measurement").setdefault("run-on-isolcpus", "false").lower()
== "true",
help="Include isolated CPUs in default cpulist")
def Setup(self, modparams):
- "Loads all measurement modules and group them into different measurement profiles"
+ "Loads all measurement modules"
if not isinstance(modparams, dict):
raise TypeError("modparams attribute is not of a dictionary type")
- modcfg = self.__cfg.GetSection("measurement")
+ modcfg = self._cfg.GetSection("measurement")
cpulist = modcfg.cpulist
run_on_isolcpus = modcfg.run_on_isolcpus
if cpulist is None:
@@ -93,61 +55,20 @@ measurement profiles, based on their characteristics"""
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
- self.__container.LoadModule(modname)
-
- # Get the correct measurement profile container for this module
- mp = self.GetProfile()
- if mp is None:
- # If not found, create a new measurement profile
- mp = MeasurementProfile(self.__cfg,
- self.__modules_root, self.__logger)
- self.__measureprofiles.append(mp)
-
- # Export the module imported here and transfer it to the
- # measurement profile
- mp.ImportModule(self.__container.ExportModule(modname))
+ self._cfg.AppendConfig(modname, modparams)
+ self._cfg.AppendConfig(modname, {'cpulist':cpulist})
+ self._cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
- # Setup this imported module inside the appropriate measurement profile
- self.__cfg.AppendConfig(modname, modparams)
- self.__cfg.AppendConfig(modname, {'cpulist':cpulist})
- self.__cfg.AppendConfig(modname, {'run-on-isolcpus':run_on_isolcpus})
- mp.Setup(modname)
-
- del self.__container
+ modobj = self._InstantiateModule(modname, self._cfg.GetSection(modname))
+ self._RegisterModuleObject(modname, modobj)
def MakeReport(self):
- "Generates an XML report for all measurement profiles"
+ rep_n = super().MakeReport()
- # Get the reports from all meaurement modules in all measurement profiles
- rep_n = libxml2.newNode("Measurements")
- cpulist = self.__cfg.GetSection("measurement").cpulist
- run_on_isolcpus = self.__cfg.GetSection("measurement").run_on_isolcpus
+ cpulist = self._cfg.GetSection("measurement").cpulist
+ run_on_isolcpus = self._cfg.GetSection("measurement").run_on_isolcpus
cpulist = parse_cpulist_from_config(cpulist, run_on_isolcpus)
rep_n.newProp("measurecpus", cpulist_utils.collapse_cpulist(cpulist))
- for mp in self.__measureprofiles:
- mprep_n = mp.MakeReport()
- if mprep_n:
- rep_n.addChild(mprep_n)
-
return rep_n
-
-
- def __iter__(self):
- "Initiates an iteration loop for MeasurementProfile objects"
-
- self.__iter_item = len(self.__measureprofiles)
- return self
-
-
- def __next__(self):
- """Internal Python iterating method, returns the next
-MeasurementProfile object to be processed"""
-
- if self.__iter_item == 0:
- self.__iter_item = None
- raise StopIteration
-
- self.__iter_item -= 1
- return self.__measureprofiles[self.__iter_item]
diff --git a/rteval/rteval_histogram_raw.xsl b/rteval/rteval_histogram_raw.xsl
index 00b2be34f305..35d8e8461f74 100644
--- a/rteval/rteval_histogram_raw.xsl
+++ b/rteval/rteval_histogram_raw.xsl
@@ -11,25 +11,25 @@
<xsl:text>core&#09;index&#09;value&#10;</xsl:text>
<!-- Extract overall system histogram data -->
- <xsl:apply-templates select="Measurements/Profile/cyclictest/system/histogram/bucket">
+ <xsl:apply-templates select="Measurements/cyclictest/system/histogram/bucket">
<xsl:with-param name="label" select="'system'"/>
- <xsl:sort select="Measurements/Profile/cyclictest/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:sort select="Measurements/cyclictest/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
- <xsl:apply-templates select="Measurements/Profile/timerlat/system/histogram/bucket">
+ <xsl:apply-templates select="Measurements/timerlat/system/histogram/bucket">
<xsl:with-param name="label" select="'system'"/>
- <xsl:sort select="Measurements/Profile/timerlat/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:sort select="Measurements/timerlat/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
<!-- Extract per cpu core histogram data -->
- <xsl:apply-templates select="Measurements/Profile/cyclictest/core/histogram/bucket">
- <xsl:sort select="Measurements/Profile/cyclictest/core/@id" data-type="number"/>
- <xsl:sort select="Measurements/Profile/cyclictest/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:apply-templates select="Measurements/cyclictest/core/histogram/bucket">
+ <xsl:sort select="Measurements/cyclictest/core/@id" data-type="number"/>
+ <xsl:sort select="Measurements/cyclictest/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
- <xsl:apply-templates select="Measurements/Profile/timerlat/core/histogram/bucket">
- <xsl:sort select="Measurements/Profile/timerlat/core/@id" data-type="number"/>
- <xsl:sort select="Measurements/Profile/timerlat/core/histogram/bucket/@index" data-type="number"/>
+ <xsl:apply-templates select="Measurements/timerlat/core/histogram/bucket">
+ <xsl:sort select="Measurements/timerlat/core/@id" data-type="number"/>
+ <xsl:sort select="Measurements/timerlat/core/histogram/bucket/@index" data-type="number"/>
</xsl:apply-templates>
</xsl:template>
@@ -38,7 +38,7 @@
<!-- -->
<!-- Record formatting -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/*/histogram/bucket">
+ <xsl:template match="/rteval/Measurements/cyclictest/*/histogram/bucket">
<xsl:param name="label"/>
<xsl:choose>
<!-- If we don't have a id tag in what should be a 'core' tag, use the given label -->
@@ -54,7 +54,7 @@
<xsl:text>&#10;</xsl:text>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/timerlat/*/histogram/bucket">
+ <xsl:template match="/rteval/Measurements/timerlat/*/histogram/bucket">
<xsl:param name="label"/>
<xsl:choose>
<!-- If we don't have a id tag in what should be a 'core' tag, use the given label -->
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 7ca0ae3a4c66..b801679abcc5 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -154,8 +154,8 @@
<xsl:value-of select="SystemInfo/cmdlineInfo/cmdline"/>
<xsl:text>&#10;</xsl:text>
- <!-- Generate a summary report for all measurement profiles -->
- <xsl:apply-templates select="Measurements/Profile"/>
+ <!-- Generate a summary report for all measurement modules -->
+ <xsl:apply-templates select="Measurements"/>
<xsl:text> ===================================================================&#10;</xsl:text>
</xsl:template>
<!-- -->
@@ -178,21 +178,8 @@
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile">
- <xsl:text> Measurement profile </xsl:text>
- <xsl:value-of select="position()"/><xsl:text>: </xsl:text>
- <xsl:choose>
- <xsl:when test="@loads = '1'"><xsl:text>With loads, </xsl:text></xsl:when>
- <xsl:otherwise><xsl:text>Without loads, </xsl:text></xsl:otherwise>
- </xsl:choose>
- <xsl:choose>
- <xsl:when test="@parallel = '1'">
- <xsl:text>measurements in parallel</xsl:text>
- </xsl:when>
- <xsl:otherwise>
- <xsl:text>measurements serialised</xsl:text>
- </xsl:otherwise>
- </xsl:choose>
+ <xsl:template match="/rteval/Measurements">
+ <xsl:text> Measurements: </xsl:text>
<xsl:text>&#10;</xsl:text>
<!-- Format other sections of the report, if they are found -->
@@ -206,7 +193,7 @@
</xsl:template>
<!-- Format the cyclictest section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest">
+ <xsl:template match="/rteval/Measurements/cyclictest">
<xsl:text> Latency test&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
@@ -238,7 +225,7 @@
<!-- Format the CPU core section in the cyclictest part -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/core">
+ <xsl:template match="/rteval/Measurements/cyclictest/core">
<xsl:text> CPU core </xsl:text>
<xsl:value-of select="@id"/>
<xsl:text> Priority: </xsl:text>
@@ -251,7 +238,7 @@
<!-- Generic formatting of statistics information -->
- <xsl:template match="/rteval/Measurements/Profile/cyclictest/*/statistics">
+ <xsl:template match="/rteval/Measurements/cyclictest/*/statistics">
<xsl:text> Samples: </xsl:text>
<xsl:value-of select="samples"/>
<xsl:text>&#10;</xsl:text>
@@ -301,7 +288,7 @@
</xsl:template>
<!-- Format the timerlat section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat">
+ <xsl:template match="/rteval/Measurements/timerlat">
<xsl:text> Latency test&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
@@ -345,7 +332,7 @@
<!-- Format the CPU core section in the timerlat part -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat/core">
+ <xsl:template match="/rteval/Measurements/timerlat/core">
<xsl:text> CPU core </xsl:text>
<xsl:value-of select="@id"/>
<xsl:text> Priority: </xsl:text>
@@ -358,7 +345,7 @@
<!-- Generic formatting of statistics information -->
- <xsl:template match="/rteval/Measurements/Profile/timerlat/*/statistics">
+ <xsl:template match="/rteval/Measurements/timerlat/*/statistics">
<xsl:text> Samples: </xsl:text>
<xsl:value-of select="samples"/>
<xsl:text>&#10;</xsl:text>
@@ -409,7 +396,7 @@
<!-- Format the hwlatdetect test section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0' and not(@aborted)]">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0' and not(@aborted)]">
<xsl:text> Hardware latency detector&#10;</xsl:text>
<xsl:text> Run duration: </xsl:text>
@@ -434,12 +421,12 @@
<xsl:apply-templates select="samples/sample"/>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0' and @aborted > 0]">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0' and @aborted > 0]">
<xsl:text> Hardware latency detector&#10;</xsl:text>
<xsl:text> ** WARNING ** hwlatedect failed to run&#10;</xsl:text>
</xsl:template>
- <xsl:template match="/rteval/Measurements/Profile/hwlatdetect[@format='1.0']/samples/sample">
+ <xsl:template match="/rteval/Measurements/hwlatdetect[@format='1.0']/samples/sample">
<xsl:text> - @</xsl:text>
<xsl:value-of select="@timestamp"/>
<xsl:text> </xsl:text>
@@ -448,7 +435,7 @@
</xsl:template>
<!-- Format the cyclictest section of the report -->
- <xsl:template match="/rteval/Measurements/Profile/sysstat">
+ <xsl:template match="/rteval/Measurements/sysstat">
<xsl:text> sysstat measurements&#10;</xsl:text>
<xsl:text> Started: </xsl:text>
--
2.45.2

566
SOURCES/rteval-Remove-upstream-spec-file.patch
View File

@ -1,566 +0,0 @@
From de8e25ff3a30dbdbba6fb1b68ea0921dff55cd91 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Mon, 13 Nov 2023 14:32:19 -0500
Subject: [PATCH] rteval: Remove upstream spec file
Specfiles should be maintained by distributions and not in the upstream
code. In practice they are not maintained upstream except for version
numbers, so just remove the specfile.
This also moves a lot of functionality around rpms in the Makefile, but
this functionality exists in tools such as rpmbuild and friends anyway.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
Makefile | 37 ----
rteval.spec | 484 ----------------------------------------------------
2 files changed, 521 deletions(-)
delete mode 100644 rteval.spec
diff --git a/Makefile b/Makefile
index 14f74e087eff..ee4cca555b95 100644
--- a/Makefile
+++ b/Makefile
@@ -79,49 +79,12 @@ rpm_prep:
rpms rpm: rpm_prep rtevalrpm loadrpm
-rtevalrpm: rteval-$(VERSION).tar.bz2
- cp $^ rpm/SOURCES
- cp rteval.spec rpm/SPECS
- rpmbuild -ba --define "_topdir $(HERE)/rpm" rpm/SPECS/rteval.spec
-
-rtevalsrpm: rteval-$(VERSION).tar.bz2
- cp $^ rpm/SOURCES
- cp rteval.spec rpm/SPECS
- rpmbuild -bs --define "_topdir $(HERE)/rpm" rpm/SPECS/rteval.spec
-
-
-xmlrpcrpm: rteval-xmlrpc-$(XMLRPCVER).tar.gz
- cp rteval-xmlrpc-$(XMLRPCVER).tar.gz rpm/SOURCES/
- cp server/rteval-parser.spec rpm/SPECS/
- rpmbuild -ba --define "_topdir $(HERE)/rpm" rpm/SPECS/rteval-parser.spec
-
-xmlsrpm: rteval-xmlrpc-$(XMLRPCVER).tar.gz
- cp rteval-xmlrpc-$(XMLRPCVER).tar.gz rpm/SOURCES/
- cp server/rteval-parser.spec rpm/SPECS/
- rpmbuild -bs --define "_topdir $(HERE)/rpm" rpm/SPECS/rteval-parser.spec
-
-loadrpm:
- rm -rf rpm-loads
- mkdir -p rpm-loads/{BUILD,RPMS,SRPMS,SOURCES,SPECS}
- cp rteval-loads.spec rpm-loads/SPECS
- cp $(LOADS) rpm-loads/SOURCES
- rpmbuild -ba --define "_topdir $(HERE)/rpm-loads" rpm-loads/SPECS/rteval-loads.spec
-
-rpmlint: rpms
- @echo "==============="
- @echo "running rpmlint"
- rpmlint -v $(shell find ./rpm -type f -name "*.rpm") \
- $(shell find ./rpm-loads -type f -name "*.rpm") \
- $(shell find ./rpm/SPECS -type f -name "rteval*.spec") \
- $(shell find ./rpm-loads/SPECS -type f -name "rteval*.spec" )
help:
@echo ""
@echo "rteval Makefile targets:"
@echo ""
@echo " runit: do a short testrun locally [default]"
- @echo " rpm: run rpmbuild for all rpms"
- @echo " rpmlint: run rpmlint against all rpms/srpms/specfiles"
@echo " tarfile: create the source tarball"
@echo " install: install rteval locally"
@echo " clean: cleanup generated files"
diff --git a/rteval.spec b/rteval.spec
deleted file mode 100644
index b5842f0d8206..000000000000
--- a/rteval.spec
+++ /dev/null
@@ -1,484 +0,0 @@
-%{!?python_sitelib: %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")}
-%{!?python_ver: %define python_ver %(%{__python} -c "import sys ; print sys.version[:3]")}
-
-Name: rteval
-Version: 3.7
-Release: 1%{?dist}
-Summary: Utility to evaluate system suitability for RT Linux
-
-Group: Development/Tools
-License: GPLv2
-URL: http://git.kernel.org/?p=linux/kernel/git/clrkwllms/rteval.git
-Source0: rteval-%{version}.tar.bz2
-BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
-
-BuildRequires: python3-devel
-Requires: platform-python
-Requires: python3-ethtool python3-lxml
-Requires: python3-dmidecode >= 3.10
-Requires: rt-tests >= 0.97
-Requires: rteval-loads >= 1.4
-Requires: rteval-common => %{version}-%{release}
-Requires: sysstat
-Requires: bzip2
-Requires: kernel-headers
-Requires: sos
-BuildArch: noarch
-Obsoletes: rteval <= 1.7
-Requires: numactl
-
-%description
-The rteval script is a utility for measuring various aspects of
-realtime behavior on a system under load. The script unpacks the
-kernel source, and then goes into a loop, running hackbench and
-compiling a kernel tree. During that loop the cyclictest program
-is run to measure event response time. After the run time completes,
-a statistical analysis of the event response times is done and printed
-to the screen.
-
-
-%package common
-Summary: Common rteval files
-BuildArch: noarch
-
-%description common
-Common files used by rteval, rteval-xmlrpc and rteval-parser
-
-%prep
-%setup -q
-
-# version sanity check (make sure specfile and rteval.py match)
-cp rteval/version.py rtevalversion.py
-srcver=$(%{__python} -c "from rtevalversion import RTEVAL_VERSION; print RTEVAL_VERSION")
-rm -rf rtevalversion.py
-if [ $srcver != %{version} ]; then
- printf "\n***\n*** rteval spec file version do not match the rteval/rteval.py version\n***\n\n"
- exit -1
-fi
-
-%build
-%{__python} setup.py build
-
-%install
-%{__python} setup.py install --root=$RPM_BUILD_ROOT
-
-%clean
-rm -rf $RPM_BUILD_ROOT
-
-%files common
-%doc COPYING
-%dir %{_datadir}/%{name}
-%{python_sitelib}/rteval/rtevalclient.py*
-%{python_sitelib}/rteval/rtevalConfig.py*
-%{python_sitelib}/rteval/rtevalXMLRPC.py*
-%{python_sitelib}/rteval/version.py*
-%{python_sitelib}/rteval/Log.py*
-%{python_sitelib}/rteval/misc.py*
-%{python_sitelib}/rteval/systopology.py*
-
-%files
-%defattr(-,root,root,-)
-%if "%{python_ver}" >= "2.5"
-%{python_sitelib}/*.egg-info
-%endif
-
-%doc COPYING README doc/rteval.txt
-%{_mandir}/man8/rteval.8.gz
-%config(noreplace) %{_sysconfdir}/rteval.conf
-%{_datadir}/%{name}/rteval_*.xsl
-%{python_sitelib}/rteval/__init__.py*
-%{python_sitelib}/rteval/rtevalMailer.py*
-%{python_sitelib}/rteval/rtevalReport.py*
-%{python_sitelib}/rteval/xmlout.py*
-%{python_sitelib}/rteval/modules
-%{python_sitelib}/rteval/sysinfo
-/usr/bin/rteval
-
-%changelog
-* Thu Mar 16 2017 Clark Williams <williams@redhat.com> - 2.14-1
-- removed leftover import of systopology from sysinfo
-
-* Wed Mar 15 2017 Clark Williams <williams@redhat.com> - 2.13-2
-- Updated specfile to correct version and bz [1382155]
-
-* Tue Sep 20 2016 Clark Williams <williams@rehdat.com> - 2.12-1
-- handle empty environment variables SUDO_USER and USER [1312057]
-
-* Tue Aug 30 2016 Clark Williams <williams@rehdat.com> - 2.11-1
-- make sure we return non-zero for early exit from tests
-
-* Wed Aug 3 2016 Clark Williams <williams@rehdat.com> - 2.10-1
-- bumped version for RHEL 7.3 release
-
-* Mon May 9 2016 Clark Williams <williams@redhat.com> - 2.9.1
-- default cpulist for modules if only one specified [1333831]
-
-* Tue Apr 26 2016 Clark Williams <williams@redhat.com> - 2.8.1
-- add the --version option to print the rteval version
-- made the --cyclictest-breaktrace option work properly [1209986]
-
-* Fri Apr 1 2016 Clark Williams <williams@redhat.com> - 2.7.1
-- treat SIGINT and SIGTERM as valid end-of-run events [1278757]
-- added cpulist options to man page
-
-* Thu Feb 11 2016 Clark Williams <williams@redhat.com> - 2.6.1
-- update to make --loads-cpulist and --measurement-cpulist work [1306437]
-
-* Thu Dec 10 2015 Clark Williams <williams@refhat.com> - 2.5-1
-- stop using old numactl --cpubind argument
-
-* Wed Dec 9 2015 Clark Williams <williams@refhat.com> - 2.4.2
-- added Require of package numactl
-
-* Tue Nov 17 2015 Clark Williams <williams@refhat.com> - 2.4.1
-- rework hackbench load to not generate cross-node traffic [1282826]
-
-* Wed Aug 12 2015 Clark Williams <williams@redhat.com> - 2.3-1
-- comment out HWLatDetect module from default config [1245699]
-
-* Wed Jun 10 2015 Clark Williams <williams@redhat.com> - 2.2-1
-- add --loads-cpulist and --measurement-cpulist to allow cpu placement [1230401]
-
-* Thu Apr 23 2015 Luis Claudio R. Goncalves <lgoncalv@redhat.com> - 2.1-8
-- load default configs when no config file is specified (Jiri kastner) [1212452]
-
-* Wed Jan 14 2015 Clark Williams <williams@redhat.com> - 2.1-7
-- added requires of bzip2 to specfile [1151567]
-
-* Thu Jan 8 2015 Clark Williams <williams@redhat.com> - 2.1-6
-- cleaned up product documentation [1173315]
-
-* Mon Nov 10 2014 Luis Claudio R. Goncalves <lgoncalv@redhat.com> - 2.1-5
-- rebuild for RHEL-7.1 (1151567)
-
-* Thu Mar 27 2014 Clark Williams <williams@redhat.com> - 2.1-4
-- cherry-picked old commit to deal with installdir problem
-
-* Wed Mar 26 2014 Clark Williams <williams@redhat.com> - 2.1-3
-- added sysstat requires to specfile
-
-* Tue Mar 12 2013 David Sommerseth <davids@redhat.com> - 2.1-2
-- Migrated from libxslt-python to python-lxml
-
-* Fri Jan 18 2013 David Sommerseth <davids@redhat.com> - 2.1-1
-- Made some log lines clearer
-- cyclictest: Added --cyclictest-breaktrace feature
-- cyclictest: Removed --cyclictest-distance option
-- cyclictest: Use a tempfile buffer for cyclictest's stdout data
-- cyclictest: Report if breaktrace was triggered
-- cyclictest: Make the unit test work again
-- cyclictest: Only log and show statistic data when samples are collected
-- Copyright updates
-
-* Thu Jan 17 2013 David Sommerseth <davids@redhat.com> - 2.0.1-1
-- Fix up type casting in the core module code
-- hwlatdetect: Add some more useful debug info
-- Reworked the run logic for modules - allow them to flag they won't run
-- Fixed a few log messages in load modules
-- Add a 30 seconds sleep before unleashing the measurement threads
-
-* Thu Jan 10 2013 David Sommerseth <davids@redhat.com> - 2.0-3
-- Separate out RTEVAL_VERSION into rteval.version, to avoid
- massive BuildRequirements
-
-* Fri Dec 21 2012 David Sommerseth <davids@redhat.com> - 2.0-2
-- Split out common files into rteval-common
-
-* Fri Dec 21 2012 David Sommerseth <davids@redhat.com> - 2.0-1
-- Updated to rteval v2.0 and reworked spec file to use setup.py directly
-
-* Tue Oct 23 2012 Clark Williams <williams@redhat.com> - 1.36-1
-- deal with system not having dmidecode python module
-- make sure to cast priority parameter to int
-- from Raphaël Beamonte <raphael.beamonte@gmail.com>:
- - Rewrite of the get_kthreads method to make it cross-distribution
- - Adds getcmdpath method to use which to locate the used commands
- - Rewrite of the get_services method to make it cross-distribution
-
-* Mon Apr 2 2012 Clark Williams <williams@redhat.com> - 1.35-1
-- fix thinko where SIGINT and SIGTERM handlers were commented out
-
-* Thu Jan 12 2012 Clark Williams <williams@redhat.com> - 1.34-1
-- fix missing config merge in rteval.py to pass parameters
- down to cyclictest
-- modify hackbench to use helper function to start process
-
-* Sat May 14 2011 Clark Williams <williams@redhat.com> - 1.33-1
-- modify hackbench cutoff to be 0.75GB/core
-
-* Mon Aug 23 2010 Clark Williams <williams@redhat.com> - 1.32-1
-- update docs
-- refactor some RTEval methods to utility functions
-- modify hackbench.py not to run under low memory conditions
-- clean up XML generation to deal with new hackbench code
-- clean up XSL code to deal with new XML 'run' attribute
-- from David Sommerseth <davids@redhat.com>:
- - improve CPU socket counting logic
- - delay log directory creation until actually needed
-- from Gowrishankar <gowrishankar.m@in.ibm.com>:
- - check if the core id really exists (multithreading fix)
-
-* Mon Jul 26 2010 Clark Williams <williams@redhat.com> - 1.31-1
-- from David Sommerseth <davids@redhat.com>:
- - Updated hackbench implementation to avoid overusing resources
- - Don't show NUMA node information if it's missing in the summary.xml
- - Show CPU cores properly
-
-* Wed Jul 21 2010 Clark Williams <williams@redhat.com> - 1.30-1
-- added code to hackbench to try to detect and ease memory pressure
-
-* Fri Jul 16 2010 Clark Williams <williams@redhat.com> - 1.29-1
-- fixed incorrect type value in kcompile.py
-
-* Fri Jul 16 2010 Clark Williams <williams@redhat.com> - 1.28-1
-- added logic to loads to adjust number of jobs based on ratio
- of memory per core
-
-* Wed Jul 14 2010 Clark Williams <williams@redhat.com> - 1.27-1
-- modified hackbench to go back to using threads rather than
- processes for units of work
-- added memory size, number of numa nodes and run duration to the
- parameter dictionary passed to all loads and cyclictest
-
-* Tue Jul 13 2010 Clark Williams <williams@redhat.com> - 1.26-1
-- modified hackbench parameters to reduce memory consumption
-
-* Mon Jul 12 2010 Clark Williams <williams@redhat.com> - 1.25-1
-- fixed cyclictest bug that caused everything to be uniprocessor
-- updated source copyrights to 2010
-
-* Fri Jul 9 2010 Clark Williams <williams@redhat.com> - 1.24-1
-- modified hackbench arguments and added new parameters for
- hackbench in rteval.conf
-
-* Thu Jul 8 2010 Clark Williams <williams@redhat.com> - 1.23-1
-- version bump to deal with out-of-sync cvs issue
-
-* Thu Jul 8 2010 Clark Williams <williams@redhat.com> - 1.22-1
-- merged David Sommerseth <davids@redhat.com> changes to use
- hackbench from rt-tests packages rather than carry local copy
-- converted all loads and cyclictest to pass __init__ parameters
- in a dictionary rather than as discrete parameters
-- added logging for load output
-
-* Tue Apr 13 2010 Clark Williams <williams@redhat.com> - 1.21-1
-- from Luis Claudio Goncalves <lgoncalv@redhat.com>:
- - remove unecessary wait() call in cyclictest.py
- - close /dev/null after using it
- - call subprocess.wait() when needed
- - remove delayloop code in hackbench.py
-- from David Sommerseth <davids@redhat.com>:
- - add SIGINT handler
- - handle non-root user case
- - process DMI warnings before command line arguments
- - added --annotate feature to rteval
- - updates to xmlrpc code
-
-* Tue Apr 6 2010 Clark Williams <williams@redhat.com> - 1.20-1
-- code fixes from Luis Claudio Goncalves <lgoncalv@redhat.com>
-- from David Sommerseth <davids@redhat.com>:
- - xmlrpc server updates
- - cputopology.py for recording topology in xml
- - added NUMA node recording for run data
- - rpmlint fixes
-- added start of rteval whitepaper in docs dir
-
-* Tue Mar 16 2010 Clark Williams <williams@redhat.com> - 1.19-1
-- add ability for --summarize to read tarfiles
-- from David Sommerseth <davids@redhat.com>
- - gather info about loaded kernel modules for XML file
- - added child tracking to hackbench to prevent zombies
-
-* Tue Feb 16 2010 Clark Williams <williams@redhat.com> - 1.18-1
-- fix usage of python 2.6 features on RHEL5 (python 2.4)
-
-* Tue Feb 16 2010 Clark Williams <williams@redhat.com> - 1.17-1
-- added logic to filter non-printables from service status output
- so that we have legal XML output
-- added logic to hackbench.py to cleanup properly at the end
- of the test
-
-* Thu Feb 11 2010 Clark Williams <williams@redhat.com> - 1.16-1
-- fix errors in show_remaining_time() introduced because
- time values are floats rather than ints
-
-* Thu Feb 11 2010 Clark Williams <williams@redhat.com> - 1.15-1
-- added logic to use --numa and --smp options of new cyclictest
-- added countdown report for time remaining in a run
-
-* Tue Feb 9 2010 Clark Williams <williams@redhat.com> - 1.14-1
-- David Sommerseth <davids@redhat.com>:
- merged XMLReport() changes for hwcert suite
-
-* Tue Dec 22 2009 Clark Williams <williams@redhat.com> - 1.13-1
-- added cyclictest default initializers
-- added sanity checks to statistics reduction code
-- updated release checklist to include origin push
-- updated Makefile clean and help targets
-- davids updates (mainly for v7 integration):
- - Add explicit sys.path directory to the python sitelib+
- '/rteval'
- - Send program arguments via RtEval() constructor
- - Added more DMI data into the summary.xml report
- - Fixed issue with not including all devices in the
- OnBoardDeviceInfo tag
-
-* Thu Dec 3 2009 David Sommerseth <davids@redhat.com> - 1.12-2
-- fixed Makefile and specfile to include and install the
- rteval/rteval_histogram_raw.py source file for gaining
- raw access to histogram data
-- Removed xmlrpc package during merge against master_ipv4 branch
-
-* Wed Nov 25 2009 Clark Williams <williams@redhat.com> - 1.12-1
-- fix incorrect reporting of measurement thread priorities
-
-* Mon Nov 16 2009 Clark Williams <williams@redhat.com> - 1.11-5
-- ensure that no double-slashes ("//") appear in the symlink
- path for /usr/bin/rteval (problem with rpmdiff)
-
-* Tue Nov 10 2009 Clark Williams <williams@redhat.com> - 1.11-4
-- changed symlink back to install and tracked by %%files
-
-* Mon Nov 9 2009 Clark Williams <williams@redhat.com> - 1.11-3
-- changed symlink generation from %%post to %%posttrans
-
-* Mon Nov 9 2009 Clark Williams <williams@redhat.com> - 1.11-2
-- fixed incorrect dependency for libxslt
-
-* Fri Nov 6 2009 Clark Williams <williams@redhat.com> - 1.11-1
-- added base OS info to XML file and XSL report
-- created new package rteval-loads for the load source code
-
-* Wed Nov 4 2009 Clark Williams <williams@redhat.com> - 1.10-1
-- added config file section for cyclictest and two settable
- parameters, buckets and interval
-
-* Thu Oct 29 2009 Clark Williams <williams@redhat.com> - 1.9-1
-- merged davids updates:
- -H option (raw histogram data)
- cleaned up xsl files
- fixed cpu sorting
-
-* Mon Oct 26 2009 David Sommerseth <davids@redhat.com> - 1.8-3
-- Fixed rpmlint complaints
-
-* Mon Oct 26 2009 David Sommerseth <davids@redhat.com> - 1.8-2
-- Added xmlrpc package, containing the XML-RPC mod_python modules
-
-* Tue Oct 20 2009 Clark Williams <williams@redhat.com> - 1.8-1
-- split kcompile and hackbench into sub-packages
-- reworked Makefile (and specfile) install/uninstall logic
-- fixed sysreport incorrect plugin option
-- catch failure when running on root-squashed NFS
-
-* Tue Oct 13 2009 Clark Williams <williams@redhat.com> - 1.7-1
-- added kthread status to xml file
-- merged davids changes for option processing and additions
- to xml summary
-
-* Tue Oct 13 2009 Clark Williams <williams@redhat.com> - 1.6-1
-- changed stat calculation to loop less
-- added methods to grab service and kthread status
-
-* Mon Oct 12 2009 Clark Williams <williams@redhat.com> - 1.5-1
-- changed cyclictest to use less memory when doing statisics
- calculations
-- updated debug output to use module name prefixes
-- changed option processing to only process config file once
-
-* Fri Oct 9 2009 Clark Williams <williams@redhat.com> - 1.4-1
-- changed cyclictest to use histogram rather than sample array
-- calcuated statistics directly from histogram
-- changed sample interval to 100us
-- added -a (affinity) argument to force cpu affinity for
- measurement threads
-
-* Thu Sep 24 2009 David Sommerseth <davids@redhat.com> - 1.3-3
-- Cleaned up the spec file and made rpmlint happy
-
-* Wed Sep 23 2009 David Sommerseth <davids@redhat.com> - 1.3-2
-- Removed version number from /usr/share/rteval path
-
-* Tue Sep 22 2009 Clark Williams <williams@redhat.com> - 1.3-1
-- changes from davids:
- * changed report code to sort by processor id
- * added report submission retry logic
- * added emailer class
-
-* Fri Sep 18 2009 Clark Williams <williams@redhat.com> - 1.2-1
-- added config file handling for modifying load behavior and
- setting defaults
-- added units in report per IBM request
-
-* Wed Aug 26 2009 Clark Williams <williams@redhat.com> - 1.1-2
-- missed a version change in rteval/rteval.py
-
-* Wed Aug 26 2009 Clark Williams <williams@redhat.com> - 1.1-1
-- modified cyclictest.py to start cyclictest threads with a
- 'distance' of zero, meaning they all have the same measurement
- interval
-
-* Tue Aug 25 2009 Clark Williams <williams@redhat.com> - 1.0-1
-- merged davids XMLRPC fixes
-- fixed --workdir option
-- verion bump to 1.0
-
-* Thu Aug 13 2009 Clark Williams <williams@redhat.com> - 0.9-2
-- fixed problem with incorrect version in rteval.py
-
-* Tue Aug 4 2009 Clark Williams <williams@redhat.com> - 0.9-1
-- merged dsommers XMLRPC and database changes
-- Specify minimum python-dmidecode version, which got native XML support
-- Added rteval_dmi.xsl
-- Fixed permission issues in /usr/share/rteval-x.xx
-
-* Wed Jul 22 2009 Clark Williams <williams@redhat.com> - 0.8-1
-- added code to capture clocksource info
-- added code to copy dmesg info to report directory
-- added code to display clocksource info in report
-- added --summarize option to display summary of existing report
-- added helpfile target to Makefile
-
-* Thu Mar 26 2009 Clark Williams <williams@torg> - 0.7-1
-- added require for python-schedutils to specfile
-- added default for cyclictest output file
-- added help parameter to option parser data
-- renamed xml output file to summary.xml
-- added routine to create tarfile of result files
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-6
-- added code to handle binary data coming from DMI tables
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-5
-- fixed logic for locating XSL template (williams)
-- fixed another stupid typo in specfile (williams)
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-4
-- fixed specfile to install rteval_text.xsl in /usr/share directory
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-3
-- added Requires for libxslt-python (williams)
-- fixed race condition in xmlout constructor/destructor (williams)
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-2
-- added Requires for libxslt (williams)
-- fixed stupid typo in rteval/rteval.py (williams)
-
-* Wed Mar 18 2009 Clark Williams <williams@torg> - 0.6-1
-- added xml output logic (williams, dsommers)
-- added xlst template for report generator (dsommers)
-- added dmi/smbios output to report (williams)
-- added __del__ method to hackbench to cleanup after run (williams)
-- modified to always keep run data (williams)
-
-* Fri Feb 20 2009 Clark Williams <williams@torg> - 0.5-1
-- fixed tab/space mix problem
-- added report path line to report
-
-* Fri Feb 20 2009 Clark Williams <williams@torg> - 0.4-1
-- reworked report output
-- handle keyboard interrupt better
-- removed duration mismatch between rteval and cyclictest
-
-* Mon Feb 2 2009 Clark Williams <williams@torg> - 0.3-1
-- initial checkin
--
2.41.0

87
SOURCES/rteval-RtEvalModules-Remove-unused-methods.patch Normal file
View File

@ -0,0 +1,87 @@
From 7a0bd39fad4fb017f41e32a125e7f4ef4284b8f0 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:16 -0500
Subject: [PATCH 3/7] rteval: RtEvalModules: Remove unused methods
With measurement profiles gone, remove methods that no longer have any
users.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 37 -------------------------------------
1 file changed, 37 deletions(-)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 0c0ce7202f77..de1ddc4628c1 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -333,36 +333,11 @@ returned when a ModuleContainer object is iterated over"""
self.__modobjects[modname] = modobj
- def ExportModule(self, modname, modroot=None):
- "Export module info, used to transfer an imported module to another ModuleContainer"
- if modroot is None:
- modroot = self.__modules_root
-
- mod = f"{modroot}.{modname}"
- return (mod, self.__modsloaded[mod])
-
-
- def ImportModule(self, module):
- "Imports an exported module from another ModuleContainer"
- (modname, moduleimp) = module
- self.__modsloaded[modname] = moduleimp
-
-
def ModulesLoaded(self):
"Returns number of registered module objects"
return len(self.__modobjects)
- def GetModulesList(self):
- "Returns a list of module names"
- return list(self.__modobjects.keys())
-
-
- def GetNamedModuleObject(self, modname):
- "Looks up a named module and returns its registered module object"
- return self.__modobjects[modname]
-
-
def __iter__(self):
"Initiates the iterating process"
@@ -406,10 +381,6 @@ class RtEvalModules:
# Export some of the internal module container methods
# Primarily to have better control of the module containers
# iteration API
- def _ImportModule(self, module):
- "Imports a module exported by ModuleContainer::ExportModule()"
- return self.__modules.ImportModule(module)
-
def _InstantiateModule(self, modname, modcfg, modroot=None):
"Imports a module and returns an instantiated object from the module"
return self.__modules.InstantiateModule(modname, modcfg, modroot)
@@ -426,17 +397,9 @@ class RtEvalModules:
"Returns number of imported modules"
return self.__modules.ModulesLoaded()
- def GetModulesList(self):
- "Returns a list of module names"
- return self.__modules.GetModulesList()
-
def SetupModuleOptions(self, parser):
"Sets up argparse based argument groups for the loaded modules"
return self.__modules.SetupModuleOptions(parser, self._cfg)
-
- def GetNamedModuleObject(self, modname):
- "Returns a list of module names"
- return self.__modules.GetNamedModuleObject(modname)
# End of exports
--
2.45.2

89
SOURCES/rteval-measurement-Change-latency-flag-to-latency_te.patch Normal file
View File

@ -0,0 +1,89 @@
From 7cb8c1c14569426e867cbdbdb218d1d9bcd4d520 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Fri, 26 Jul 2024 13:32:37 -0500
Subject: [PATCH 7/7] rteval: measurement: Change latency flag to latency_test
As requested by John, change the name of the latency flag to avoid
looking like it holds an actual latency value.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/__init__.py | 14 +++++++-------
rteval/modules/measurement/cyclictest.py | 2 +-
rteval/modules/measurement/timerlat.py | 2 +-
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 2a4eafae71c7..d7792108d5b8 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -40,7 +40,7 @@ class rtevalModulePrototype(threading.Thread):
"finished": threading.Event()}
self._donotrun = False
self._exclusive = False
- self._latency = False
+ self._latency_test = False
self.__timestamps = {}
self.__sleeptime = 2.0
@@ -68,9 +68,9 @@ class rtevalModulePrototype(threading.Thread):
self._exclusive = True
- def set_latency(self):
+ def set_latency_test(self):
""" Sets the module as an exclusive latency measurer """
- self._latency = True
+ self._latency_test = True
def set_donotrun(self):
@@ -418,14 +418,14 @@ class RtEvalModules:
self._logger.log(Log.INFO, f"Preparing {self._module_type} modules")
exclusive = 0
- latency = False
+ latency_test = False
for (modname, mod) in self.__modules:
if mod.is_exclusive() and mod.WorkloadWillRun():
exclusive += 1
- if mod._latency:
- if latency:
+ if mod._latency_test:
+ if latency_test:
raise RuntimeError("More than one exclusive latency test")
- latency = True
+ latency_test = True
for (modname, mod) in self.__modules:
if exclusive >= 1:
if exclusive != 1:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index a9f5b0c4fba7..d919058e927f 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -216,7 +216,7 @@ class Cyclictest(rtevalModulePrototype):
self.__started = False
self.__cyclicoutput = None
self.__breaktraceval = None
- self.set_latency()
+ self.set_latency_test()
@staticmethod
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index e4b80c33552e..92bc07051aa1 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -216,7 +216,7 @@ class Timerlat(rtevalModulePrototype):
logfnc=self._log)
self.__timerlatdata['system'].description = (f"({self.__numcores} cores) ") + info['0']['model name']
self._log(Log.DEBUG, f"system using {self.__numcores} cpu cores")
- self.set_latency()
+ self.set_latency_test()
def _WorkloadSetup(self):
--
2.45.2

324
SOURCES/rteval-measurement-Remove-ModuleInfo.patch Normal file
View File

@ -0,0 +1,324 @@
From 5b55c62ff271e9d9278d25f027590aca05bff959 Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:24:14 -0500
Subject: [PATCH 1/7] rteval: measurement: Remove ModuleInfo()
All of the measurement modules have identical settings, and the use cases
for deviating seem questionable. While we do have a desire to keep
cyclictest and timerlat from running together, turning off run_parallel
would prevent sysstat from running at the same time. And if there's a
desire to run without loads, that seems like it belongs in the realm of
user configuration, rather than anything inherent to a given measurement
module.
Any future module flags can be implemented similarly to what is done
in load modules (e.g. set_exclusive).
Places that checked with_loads now check whether a LoadModules container
is present at all, which can be wired up to a command line option in the
future.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/__init__.py | 39 ++++++-------
rteval/modules/__init__.py | 13 +----
rteval/modules/measurement/__init__.py | 73 ++----------------------
rteval/modules/measurement/cyclictest.py | 5 --
rteval/modules/measurement/sysstat.py | 7 ---
rteval/modules/measurement/timerlat.py | 5 --
6 files changed, 26 insertions(+), 116 deletions(-)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 1a61148ef327..4a6883e28e5b 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -111,20 +111,21 @@ class RtEval(rtevalReport):
except Exception as err:
raise RuntimeError(f"Cannot create report directory (NFS with rootsquash on?) [{err}]]")
- self.__logger.log(Log.INFO, "Preparing load modules")
- params = {'workdir':self.__rtevcfg.workdir,
- 'reportdir':self.__reportdir and self.__reportdir or "",
- 'builddir':builddir,
- 'srcdir':self.__rtevcfg.srcdir,
- 'verbose': self.__rtevcfg.verbose,
- 'debugging': self.__rtevcfg.debugging,
- 'numcores':self._sysinfo.cpu_getCores(True),
- 'logging':self.__rtevcfg.logging,
- 'memsize':self._sysinfo.mem_get_size(),
- 'numanodes':self._sysinfo.mem_get_numa_nodes(),
- 'duration': float(self.__rtevcfg.duration),
- }
- self._loadmods.Setup(params)
+ if self._loadmods:
+ self.__logger.log(Log.INFO, "Preparing load modules")
+ params = {'workdir':self.__rtevcfg.workdir,
+ 'reportdir':self.__reportdir and self.__reportdir or "",
+ 'builddir':builddir,
+ 'srcdir':self.__rtevcfg.srcdir,
+ 'verbose': self.__rtevcfg.verbose,
+ 'debugging': self.__rtevcfg.debugging,
+ 'numcores':self._sysinfo.cpu_getCores(True),
+ 'logging':self.__rtevcfg.logging,
+ 'memsize':self._sysinfo.mem_get_size(),
+ 'numanodes':self._sysinfo.mem_get_numa_nodes(),
+ 'duration': float(self.__rtevcfg.duration),
+ }
+ self._loadmods.Setup(params)
self.__logger.log(Log.INFO, "Preparing measurement modules")
self._measuremods.Setup(params)
@@ -136,13 +137,11 @@ class RtEval(rtevalReport):
raise Exception("measure_profile is not an MeasurementProfile object")
measure_start = None
- (with_loads, run_parallel) = measure_profile.GetProfile()
- self.__logger.log(Log.INFO, f"Using measurement profile [loads: {with_loads} parallel: {run_parallel}]")
try:
nthreads = 0
# start the loads
- if with_loads:
+ if self._loadmods:
self._loadmods.Start()
print(f"rteval run on {os.uname()[2]} started at {time.asctime()}")
@@ -168,7 +167,7 @@ class RtEval(rtevalReport):
# Unleash the loads and measurement threads
report_interval = int(self.__rtevcfg.report_interval)
- if with_loads:
+ if self._loadmods:
self._loadmods.Unleash()
nthreads = threading.active_count()
else:
@@ -192,7 +191,7 @@ class RtEval(rtevalReport):
self.__logger.log(Log.WARN,
"Measurement threads did not use the full time slot. Doing a controlled stop.")
- if with_loads:
+ if nthreads:
if threading.active_count() < nthreads:
raise RuntimeError("load thread died!")
@@ -222,7 +221,7 @@ class RtEval(rtevalReport):
measure_profile.Stop()
# stop the loads
- if with_loads:
+ if self._loadmods:
self._loadmods.Stop()
print(f"stopping run at {time.asctime()}")
diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 4330a839db6f..0c0ce7202f77 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -269,14 +269,6 @@ reference from the first import"""
return mod
- def ModuleInfo(self, modname, modroot=None):
- """Imports a module and calls the modules' ModuleInfo() function and returns
-the information provided by the module"""
-
- mod = self.LoadModule(modname, modroot)
- return mod.ModuleInfo()
-
-
def SetupModuleOptions(self, parser, config):
"""Sets up a separate argparse ArgumentGroup per module with its supported parameters"""
@@ -495,9 +487,8 @@ class RtEvalModules:
def Unleash(self):
- """Unleashes all the loaded modules workloads"""
+ """Unleashes all the loaded modules"""
- # turn loose the loads
nthreads = 0
self._logger.log(Log.INFO, f"Sending start event to all {self._module_type} modules")
for (modname, mod) in self.__modules:
@@ -508,7 +499,7 @@ class RtEvalModules:
return nthreads
- def _isAlive(self):
+ def isAlive(self):
"""Returns True if all modules are running"""
for (modname, mod) in self.__modules:
diff --git a/rteval/modules/measurement/__init__.py b/rteval/modules/measurement/__init__.py
index 43c0fda30ce1..7b1d84ef554d 100644
--- a/rteval/modules/measurement/__init__.py
+++ b/rteval/modules/measurement/__init__.py
@@ -11,24 +11,13 @@ import rteval.cpulist_utils as cpulist_utils
class MeasurementProfile(RtEvalModules):
"""Keeps and controls all the measurement modules with the same measurement profile"""
- def __init__(self, config, with_load, run_parallel, modules_root, logger):
- self.__with_load = with_load
- self.__run_parallel = run_parallel
-
- # Only used when running modules serialised
- self.__run_serialised_mods = None
-
+ def __init__(self, config, modules_root, logger):
self._module_type = "measurement"
self._module_config = "measurement"
self._report_tag = "Profile"
RtEvalModules.__init__(self, config, modules_root, logger)
- def GetProfile(self):
- "Returns the profile characteristic as (with_load, run_parallel)"
- return (self.__with_load, self.__run_parallel)
-
-
def ImportModule(self, module):
"Imports an exported module from a ModuleContainer() class"
return self._ImportModule(module)
@@ -41,54 +30,6 @@ class MeasurementProfile(RtEvalModules):
self._RegisterModuleObject(modname, modobj)
- def Unleash(self):
- """Unleashes all the measurement modules"""
-
- if self.__run_parallel:
- # Use the inherrited method if running
- # measurements in parallel
- return RtEvalModules.Unleash(self)
-
- # Get a list of all registered modules,
- # and start the first one
- self.__serialised_mods = self.GetModulesList()
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- mod.setStart()
- return 1
-
-
- def MakeReport(self):
- "Generates an XML report for all run measurement modules in this profile"
- rep_n = RtEvalModules.MakeReport(self)
- rep_n.newProp("loads", self.__with_load and "1" or "0")
- rep_n.newProp("parallel", self.__run_parallel and "1" or "0")
- return rep_n
-
-
- def isAlive(self):
- """Returns True if all modules which are supposed to run runs"""
-
- if self.__run_parallel:
- return self._isAlive()
-
- if self.__serialised_mods:
- # If running serialised, first check if measurement is still running,
- # if so - return True.
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- if mod.WorkloadAlive():
- return True
-
- # If not, go to next on the list and kick it off
- self.__serialised_mods.remove(self.__serialised_mods[0])
- if self.__serialised_mods:
- mod = self.GetNamedModuleObject(self.__serialised_mods[0])
- mod.setStart()
- return True
-
- # If we've been through everything, nothing is running
- return False
-
-
class MeasurementModules:
"""Class which takes care of all measurement modules and groups them into
measurement profiles, based on their characteristics"""
@@ -116,13 +57,11 @@ measurement profiles, based on their characteristics"""
self.__container.LoadModule(m[0])
- def GetProfile(self, with_load, run_parallel):
+ def GetProfile(self):
"Returns the appropriate MeasurementProfile object, based on the profile type"
for p in self.__measureprofiles:
- mp = p.GetProfile()
- if mp == (with_load, run_parallel):
- return p
+ return p
return None
@@ -154,15 +93,13 @@ measurement profiles, based on their characteristics"""
for (modname, modtype) in modcfg:
if isinstance(modtype, str) and modtype.lower() == 'module': # Only 'module' will be supported (ds)
- # Extract the measurement modules info
- modinfo = self.__container.ModuleInfo(modname)
+ self.__container.LoadModule(modname)
# Get the correct measurement profile container for this module
- mp = self.GetProfile(modinfo["loads"], modinfo["parallel"])
+ mp = self.GetProfile()
if mp is None:
# If not found, create a new measurement profile
mp = MeasurementProfile(self.__cfg,
- modinfo["loads"], modinfo["parallel"],
self.__modules_root, self.__logger)
self.__measureprofiles.append(mp)
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 3301e1b45e11..3a34c1b988d6 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -394,11 +394,6 @@ class Cyclictest(rtevalModulePrototype):
return rep_n
-def ModuleInfo():
- return {"parallel": True,
- "loads": True}
-
-
def ModuleParameters():
""" default parameters """
return {"interval": {"descr": "Base interval of the threads in microseconds",
diff --git a/rteval/modules/measurement/sysstat.py b/rteval/modules/measurement/sysstat.py
index d4646c1646f4..a0efd8953659 100644
--- a/rteval/modules/measurement/sysstat.py
+++ b/rteval/modules/measurement/sysstat.py
@@ -93,13 +93,6 @@ class sysstat(rtevalModulePrototype):
-def ModuleInfo():
- # sysstat features - run in parallel with outher measurement modules with loads
- return {"parallel": True,
- "loads": True}
-
-
-
def ModuleParameters():
return {} # No arguments available
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index dc6226ccc991..f3bdc7098bc0 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -499,11 +499,6 @@ class Timerlat(rtevalModulePrototype):
return rep_n
-def ModuleInfo():
- """ Required measurement module information """
- return {"parallel": True,
- "loads": True}
-
def ModuleParameters():
""" default parameters """
return {"priority": {"descr": "Run rtla timerlat with this priority",
--
2.45.2

71
SOURCES/rteval-run-cyclictest-using-default-system-when-sett.patch Normal file
View File

@ -0,0 +1,71 @@
From 034fb231ff06aa4615b7531a04e4c3e0ce4aa662 Mon Sep 17 00:00:00 2001
From: Anubhav Shelat <ashelat@redhat.com>
Date: Fri, 2 Aug 2024 11:46:36 -0400
Subject: [PATCH 2/2] rteval: run cyclictest using '--default-system' when
setting idle states
When running cyclictest in rteval, cyclictest automatically disables
idle states. This means whenever the user sets the idle state of a cpu
list using '--idle-set' it is overridden by cyclictest.
To fix this, the variable 'usingCpupower' is appended to the parameter
dictionary that's passed to the Cyclictest measurement object which executes
cyclictest in rteval.
If '--idle-set' is specified when running rteval,
'usingCpupower' is set to true and the '--default-system' option is
appended to the cyclictest command, which will prevent cyclictest from
disabling cstates.
Signed-off-by: Anubhav Shelat <ashelat@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval-cmd | 4 ++++
rteval/__init__.py | 1 +
rteval/modules/measurement/cyclictest.py | 3 +++
3 files changed, 8 insertions(+)
diff --git a/rteval-cmd b/rteval-cmd
index f440a8a22622..4e13d312a24a 100755
--- a/rteval-cmd
+++ b/rteval-cmd
@@ -266,6 +266,10 @@ if __name__ == '__main__':
| (rtevcfg.debugging and Log.DEBUG)
logger.SetLogVerbosity(loglev)
+ # check if cpupower is being used
+ if sys.argv.count('--idle-set') > 0:
+ rtevcfg.update({'usingCpupower': True})
+
# Load modules
loadmods = LoadModules(config, logger=logger)
measuremods = MeasurementModules(config, logger=logger)
diff --git a/rteval/__init__.py b/rteval/__init__.py
index 4d3e0c23e5ab..8ded374d287e 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -119,6 +119,7 @@ class RtEval(rtevalReport):
'memsize':self._sysinfo.mem_get_size(),
'numanodes':self._sysinfo.mem_get_numa_nodes(),
'duration': float(self.__rtevcfg.duration),
+ 'usingCpupower': self.__rtevcfg.usingCpupower
}
if self._loadmods:
diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index d919058e927f..2e8f6f1870ed 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -251,6 +251,9 @@ class Cyclictest(rtevalModulePrototype):
self.__cmd.append(f'-t{self.__numcores}')
self.__cmd.append(f'-a{self.__cpulist}')
+ if (self.__cfg.usingCpupower):
+ self.__cmd.append('--default-system')
+
if 'threads' in self.__cfg and self.__cfg.threads:
self.__cmd.append(f"-t{int(self.__cfg.threads)}")
--
2.45.2

65
SOURCES/rteval-sysstat-Convert-base64-data-to-text-before-wr.patch Normal file
View File

@ -0,0 +1,65 @@
From ca90d5aa7ae2ff6dac124c710fceadae028b5f4a Mon Sep 17 00:00:00 2001
From: Crystal Wood <crwood@redhat.com>
Date: Thu, 20 Jun 2024 21:18:05 -0500
Subject: [PATCH 2/3] rteval: sysstat: Convert base64 data to text before
wrapping
As of Python 3, b64encode() returns data, not a string, causing this:
Traceback (most recent call last):
File "/home/crwood/git/rteval/./rteval-cmd", line 413, in <module>
ec = rteval.Measure()
^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/__init__.py", line 246, in Measure
self._report(measure_start, self.__rtevcfg.xslt_report)
File "/home/crwood/git/rteval/rteval/rtevalReport.py", line 63, in _report
self.__xmlreport.AppendXMLnodes(self._measuremods.MakeReport())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/__init__.py", line 190, in MakeReport
mprep_n = mp.MakeReport()
^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/__init__.py", line 62, in MakeReport
rep_n = RtEvalModules.MakeReport(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/__init__.py", line 559, in MakeReport
modrep_n = mod.MakeReport()
^^^^^^^^^^^^^^^^
File "/home/crwood/git/rteval/rteval/modules/measurement/sysstat.py", line 83, in MakeReport
data_n = rep_n.newTextChild(None, 'data', "\n"+"\n".join(textwrap.wrap(data, 75))+"\n")
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 384, in wrap
return w.wrap(text)
^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 356, in wrap
chunks = self._split_chunks(text)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 342, in _split_chunks
text = self._munge_whitespace(text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/textwrap.py", line 153, in _munge_whitespace
text = text.translate(self.unicode_whitespace_trans)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'dict'
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/sysstat.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rteval/modules/measurement/sysstat.py b/rteval/modules/measurement/sysstat.py
index 57194a2b5f45..d4646c1646f4 100644
--- a/rteval/modules/measurement/sysstat.py
+++ b/rteval/modules/measurement/sysstat.py
@@ -79,7 +79,7 @@ class sysstat(rtevalModulePrototype):
fp = open(self.__datafile, "rb")
compr = bz2.BZ2Compressor(9)
cmpr = compr.compress(fp.read())
- data = base64.b64encode(cmpr + compr.flush())
+ data = base64.b64encode(cmpr + compr.flush()).decode('utf-8')
data_n = rep_n.newTextChild(None, 'data', "\n"+"\n".join(textwrap.wrap(data, 75))+"\n")
data_n.newProp('contents', 'sysstat/sar binary data')
data_n.newProp('encoding', 'base64')
--
2.45.2

526
SOURCES/rteval-timerlat-Add-timerlat-tracing-to-rteval.patch Normal file
View File

@ -0,0 +1,526 @@
From 5909521e06ed92ea60ffb247b25ca86c232f71bf Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Thu, 20 Jun 2024 21:34:24 -0400
Subject: [PATCH 1/4] rteval: timerlat: Add timerlat tracing to rteval
This patch adds tracing from timerlat to rteval.
These two options are added
--timerlat-stoptrace USEC
Stop trace if thread latency higher than USEC
--timerlat-trace FILE
File to save trace to
This also stores the summary from rtla timerlat in the rteval xml file.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/timerlat.py | 213 ++++++++++++++++++++++++-
rteval/rteval_text.xsl | 181 ++++++++++++++++++++-
2 files changed, 389 insertions(+), 5 deletions(-)
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index e8345fab1ad7..45adec1b33e1 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -16,6 +16,7 @@ from rteval.modules import rtevalModulePrototype
from rteval.systopology import cpuinfo, SysTopology
from rteval.cpulist_utils import expand_cpulist, collapse_cpulist
+
class TLRunData:
''' class to store instance data from a timerlat run '''
def __init__(self, coreid, datatype, priority, logfnc):
@@ -188,6 +189,15 @@ class Timerlat(rtevalModulePrototype):
self.__cpus = [str(c) for c in expand_cpulist(self.__cpulist)]
self.__numcores = len(self.__cpus)
+ # Has tracing been triggered
+ self.__stoptrace = False
+ # This stores the output from rtla
+ self.__posttrace = ""
+ # Stop Trace Data
+ self.__stdata = {}
+ # Stop Trace Cpu
+ self.stcpu = -1
+
self.__timerlat_out = None
self.__timerlat_err = None
self.__started = False
@@ -218,10 +228,21 @@ class Timerlat(rtevalModulePrototype):
self.__cmd = ['rtla', 'timerlat', 'hist', '-P', f'f:{int(self.__priority)}', '-u']
self.__cmd.append(f'-c{self.__cpulist}')
self.__cmd.append(f'-E{self.__buckets}')
+
+ if self.__cfg.stoptrace:
+ self.__cmd.append(f"-T{int(self.__cfg.stoptrace)}")
+
+ if self.__cfg.trace:
+ if not self.__cfg.stoptrace:
+ self._log(Log.WARN, f'Ignoring trace={self.__cfg.trace}, because stoptrace not invoked')
+ else:
+ self.__cmd.append(f'-t={self.__cfg.trace}')
+
self._log(Log.DEBUG, f'self.__cmd = {self.__cmd}')
self.__timerlat_out = tempfile.SpooledTemporaryFile(mode='w+b')
self.__timerlat_err = tempfile.SpooledTemporaryFile(mode='w+b')
+
def _WorkloadTask(self):
if self.__started:
return
@@ -252,6 +273,10 @@ class Timerlat(rtevalModulePrototype):
os.kill(self.__timerlat_process.pid, signal.SIGINT)
time.sleep(2)
+ blocking_thread_detected = False
+ softirq_interference_detected = False
+ irq_interference_detected = False
+
# Parse histogram output
self.__timerlat_out.seek(0)
for line in self.__timerlat_out:
@@ -261,6 +286,113 @@ class Timerlat(rtevalModulePrototype):
if not line:
continue
+ # Parsing if stoptrace has been invoked
+ if self.__stoptrace:
+ self.__posttrace += line
+ line = line.strip()
+ fields = line.split()
+ if line.startswith("##") and fields[1] == "CPU":
+ self.stcpu = int(fields[2])
+ self._log(Log.DEBUG, f"self.stcpu = {self.stcpu}")
+ self.__stdata[self.stcpu] = {}
+ continue
+ if self.stcpu == -1:
+ self._log(Log.WARN, "Stop trace has been invoked, but a stop cpu has not been identified.")
+ continue
+ if line.startswith('------------------'):
+ blocking_thread_detected = False
+ softirq_interference_detected = False
+ irq_interference_detected = False
+ continue
+ if line.startswith("Thread latency:"):
+ thread_latency_percent = fields[-1].strip('()%')
+ self._log(Log.DEBUG, f"thread_latency_percent = {thread_latency_percent}")
+ thread_latency = fields[-3]
+ self._log(Log.DEBUG, f"thread_latency = {thread_latency}")
+ self.__stdata[self.stcpu]["Thread_latency"] = (thread_latency, thread_latency_percent)
+ elif line.startswith("Previous IRQ interference"):
+ self._log(Log.DEBUG, f'Previous_IRQ_interference = {fields[-2]}')
+ self.__stdata[self.stcpu]["Previous_IRQ_interference"] = fields[-2]
+ elif line.startswith("IRQ handler delay:"):
+ irq_handler_delay_percent = fields[-2].strip('(')
+ irq_handler_delay = fields[-4]
+ # Do we have (exit from idle)?
+ if fields[3] == '(exit':
+ field_name = "IRQ_handler_delay_exit_from_idle"
+ else:
+ field_name = "IRQ_handler_delay"
+ self._log(Log.DEBUG, f"{field_name} = {irq_handler_delay}")
+ self._log(Log.DEBUG, f"{field_name}_percent = {irq_handler_delay_percent}")
+ self.__stdata[self.stcpu][field_name] = (irq_handler_delay, irq_handler_delay_percent)
+ elif line.startswith("IRQ latency:"):
+ self._log(Log.DEBUG, f"irq_latency = {fields[-2]}")
+ self.__stdata[self.stcpu]["IRQ_latency"] = fields[-2]
+ elif line.startswith("Timerlat IRQ duration"):
+ timerlat_irq_duration_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"timerlat_irq_duration_percent = {timerlat_irq_duration_percent}")
+ timerlat_irq_duration = fields[-4]
+ self._log(Log.DEBUG, f"timerlat_irq_duration = {timerlat_irq_duration}")
+ self.__stdata[self.stcpu]["Timerlat_IRQ_duration"] = (timerlat_irq_duration, timerlat_irq_duration_percent)
+ elif line.startswith("Blocking thread:"):
+ blocking_thread_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"blocking_thread_percent = {blocking_thread_percent}")
+ blocking_thread = fields[-4]
+ self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
+ self.__stdata[self.stcpu]["Blocking_Thread"] = (blocking_thread, blocking_thread_percent)
+ blocking_thread_detected = True
+ irq_interference_detected = False
+ softirq_interference_detected = False
+ elif line.startswith("IRQ interference"):
+ irq_interference_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"irq_interference_percent = {irq_interference_percent}")
+ irq_interference = fields[-4]
+ self._log(Log.DEBUG, f"irq_interference = {irq_interference}")
+ self.__stdata[self.stcpu]["IRQ_interference"] = (irq_interference, irq_interference_percent)
+ blocking_thread_detected = False
+ irq_interference_detected = True
+ softirq_interference_detected = False
+ elif line.startswith("Softirq interference"):
+ softirq_interference_percent = fields[-2].strip('(')
+ self._log(Log.DEBUG, f"softirq_interference_percent = {softirq_interference_percent}")
+ softirq_interference = fields[-4]
+ self._log(Log.DEBUG, f"softirq_interference = {softirq_interference}")
+ self.__stdata[self.stcpu]["Softirq_interference"] = (softirq_interference, softirq_interference_percent)
+ blocking_thread_detected = False
+ irq_interference_detected = False
+ softirq_interference_detected = True
+ elif blocking_thread_detected:
+ self._log(Log.DEBUG, f'line={line}')
+ blocking_thread = " ".join(fields[0:-2])
+ self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
+ blocking_threadus = fields[-2]
+ self._log(Log.DEBUG, f"blocking_threadus = {blocking_threadus}")
+ self.__stdata[self.stcpu].setdefault("blocking_thread", [])
+ self.__stdata[self.stcpu]["blocking_thread"] += [(blocking_thread, blocking_threadus)]
+ elif softirq_interference_detected:
+ softirq = " ".join(fields[0:-2])
+ softirq_latency = fields[-2]
+ self._log(Log.DEBUG, f'softirq = {softirq}')
+ self._log(Log.DEBUG, f'softirq_latency = {softirq_latency}')
+ self.__stdata[self.stcpu].setdefault("softirq_interference", [])
+ self.__stdata[self.stcpu]["softirq_interference"] += [(softirq, softirq_latency)]
+ elif irq_interference_detected:
+ irq_interference_name = " ".join(fields[0:-2])
+ irq_interference_latency = fields[-2]
+ self._log(Log.DEBUG, f'irq_interference = {irq_interference_name}, latency = {irq_interference_latency}')
+ self.__stdata[self.stcpu].setdefault("irq_interference", [])
+ self.__stdata[self.stcpu]["irq_interference"] += [(irq_interference_name, irq_interference_latency)]
+ elif line.startswith("Max timerlat IRQ latency"):
+ self._log(Log.DEBUG, f"line={line}")
+ max_timerlat_irq_latency = fields[-5]
+ self._log(Log.DEBUG, f"max_timerlat_irq_latency = {max_timerlat_irq_latency}")
+ max_timerlat_cpu = int(fields[-1])
+ self._log(Log.DEBUG, f"max_timerlat_cpu = {max_timerlat_cpu}")
+ self.__stdata.setdefault(max_timerlat_cpu, {})
+ self.__stdata[max_timerlat_cpu]["Max_timerlat_IRQ_latency_from_idle"] = max_timerlat_irq_latency
+ else:
+ self._log(Log.DEBUG, f'line = {line}')
+ continue
+
if line.startswith('#'):
if line.startswith('# Duration:'):
duration = line.split()[2]
@@ -285,9 +417,13 @@ class Timerlat(rtevalModulePrototype):
elif line.startswith('max:'):
#print(line)
continue
+ elif line.startswith('rtla timerlat hit stop tracing'):
+ self.__stoptrace = True
+ self.__posttrace += line
+ continue
else:
- pass
#print(line)
+ pass
vals = line.split()
if not vals:
@@ -301,12 +437,18 @@ class Timerlat(rtevalModulePrototype):
continue
for i, core in enumerate(self.__cpus):
- self.__timerlatdata[core].bucket(index, int(vals[i*3+1]),
+ # There might not be a count on every cpu if tracing invoked
+ if i*3 + 1 >= len(vals):
+ self.__timerlatdata[core].bucket(index, 0, 0, 0)
+ self.__timerlatdata['system'].bucket(index, 0, 0, 0)
+ else:
+ self.__timerlatdata[core].bucket(index, int(vals[i*3+1]),
int(vals[i*3+2]),
int(vals[i*3+3]))
- self.__timerlatdata['system'].bucket(index, int(vals[i*3+1]),
+ self.__timerlatdata['system'].bucket(index, int(vals[i*3+1]),
int(vals[i*3+2]),
int(vals[i*3+3]))
+
# Generate statistics for each RunData object
for n in list(self.__timerlatdata.keys()):
self.__timerlatdata[n].reduce()
@@ -320,6 +462,62 @@ class Timerlat(rtevalModulePrototype):
rep_n = libxml2.newNode('timerlat')
rep_n.newProp('command_line', ' '.join(self.__cmd))
+ stoptrace_invoked_n = libxml2.newNode('stoptrace_invoked')
+ if self.stcpu != -1:
+ stoptrace_invoked_n.newProp("invoked", "true")
+ else:
+ stoptrace_invoked_n.newProp("invoked", "")
+ rep_n.addChild(stoptrace_invoked_n)
+
+ if self.stcpu != -1:
+ self._log(Log.DEBUG, f'self.__stdata = {self.__stdata}')
+ for cpu in self.__stdata:
+ # This is Max timerlat IRQ latency from idle
+ # With no other data from that cpu, so don't create a
+ # stoptrace_report for this
+ if len(self.__stdata[cpu]) == 1:
+ continue
+ stoptrace_n = libxml2.newNode('stoptrace_report')
+ stoptrace_n.newProp("CPU", str(cpu))
+ for k, v in self.__stdata[cpu].items():
+ self._log(Log.DEBUG, f"cpu={cpu}, k={k}, v={v}")
+ if isinstance(v, tuple):
+ latency = str(v[0])
+ percent = str(v[1])
+ cpu_n = stoptrace_n.newTextChild(None, str(k), None)
+ n = cpu_n.newTextChild(None, "latency", latency)
+ n.newProp('unit', 'us')
+
+ n = cpu_n.newTextChild(None, "latency_percent", percent)
+ n.newProp('unit', '%')
+ elif isinstance(v, list):
+ if k in ("blocking_thread", "softirq_interference", "irq_interference"):
+ for name, latency in v:
+ cpu_n = stoptrace_n.newTextChild(None, k, None)
+ n = cpu_n.newTextChild(None, "name", name)
+ n = cpu_n.newTextChild(None, "latency", latency)
+ n.newProp('unit', 'us')
+ else:
+ if k == "Max_timerlat_IRQ_latency_from_idle":
+ continue
+ cpu_n = stoptrace_n.newTextChild(None, str(k), str(v))
+ cpu_n.newProp('unit', 'us')
+ rep_n.addChild(stoptrace_n)
+
+ self._log(Log.DEBUG, f'timerlat: posttrace = \n{self.__posttrace}')
+ self._log(Log.DEBUG, 'timerlat: posttrace END')
+ for cpu in self.__stdata:
+ for k, v in self.__stdata[cpu].items():
+ if isinstance(v, tuple):
+ continue
+ if k == "Max_timerlat_IRQ_latency_from_idle":
+ max_timerlat_n = libxml2.newNode('max_timerlat_report')
+ max_timerlat_n.newProp("CPU", str(cpu))
+ cpu_n = max_timerlat_n.newTextChild(None, k, str(v))
+ cpu_n.newProp('unit', 'us')
+ rep_n.addChild(max_timerlat_n)
+ return rep_n
+
rep_n.addChild(self.__timerlatdata['system'].MakeReport())
for thr in self.__cpus:
if str(thr) not in self.__timerlatdata:
@@ -341,7 +539,13 @@ def ModuleParameters():
"metavar": "PRIO" },
"buckets": {"descr": "Number of buckets",
"default": 3500,
- "metavar": "NUM"}
+ "metavar": "NUM" },
+ "stoptrace": {"descr": "Stop trace if thread latency higher than USEC",
+ "default": None,
+ "metavar": "USEC" },
+ "trace": {"descr": "File to save trace to",
+ "default": None,
+ "metavar": "FILE" },
}
def create(params, logger):
@@ -367,6 +571,7 @@ if __name__ == '__main__':
cfg_tl = cfg.GetSection('timerlat')
cfg_tl.cpulist = collapse_cpulist(SysTopology().online_cpus())
+ cfg_tl.stoptrace=50
RUNTIME = 10
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 2f03bda0bb55..70777354efa5 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -316,14 +316,26 @@
<xsl:value-of select="@command_line"/>
<xsl:text>&#10;&#10;</xsl:text>
- <xsl:apply-templates select="abort_report"/>
+ <xsl:if test="stoptrace_invoked">
+ <xsl:text>rtla timerlat hit stop tracing</xsl:text>
+ <xsl:text>
+</xsl:text>
+ <xsl:apply-templates select="stoptrace_report"/>
+ <xsl:apply-templates select="max_timerlat_report"/>
+ </xsl:if>
+ <!-- Make sure the description is available before printing System: -->
+ <xsl:if test="system/@description">
<xsl:text> System: </xsl:text>
<xsl:value-of select="system/@description"/>
<xsl:text>&#10;</xsl:text>
+ </xsl:if>
+ <!-- If stoptrace_invoked is true, no Statistics are available -->
+ <xsl:if test="stoptrace_invoked != true">
<xsl:text> Statistics: &#10;</xsl:text>
<xsl:apply-templates select="system/statistics"/>
+ </xsl:if>
<!-- Add CPU core info and stats-->
<xsl:apply-templates select="core">
@@ -470,4 +482,171 @@
</xsl:if>
</xsl:template>
+ <!-- Format posttrace information if present -->
+ <xsl:template match="stoptrace_report">
+ <xsl:text>## CPU </xsl:text>
+ <xsl:value-of select="@CPU"/>
+ <xsl:text> hit stop tracing, analyzing it ##</xsl:text>
+ <xsl:text>
+</xsl:text>
+
+
+ <xsl:if test="Previous_IRQ_interference">
+ <xsl:text>Previous IRQ interference: up to </xsl:text>
+ <xsl:value-of select="Previous_IRQ_interference"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Previous_IRQ_interference/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:if>
+
+ <xsl:if test="IRQ_handler_delay">
+ <xsl:text>IRQ handler delay: </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:if test="IRQ_handler_delay_exit_from_idle">
+ <xsl:text>IRQ handler delay: (exit from idle) </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_handler_delay_exit_from_idle/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:text>IRQ latency:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+
+ <xsl:text>Timerlat IRQ duration:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Timerlat_IRQ_duration/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+
+ <xsl:if test="Blocking_Thread">
+ <xsl:text>Blocking thread:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Blocking_Thread/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="blocking_thread">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:if test="Softirq_interference">
+ <xsl:text>Softirq interference:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Softirq_interference/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Softirq_interference/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="softirq_interference">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:if test="IRQ_interference">
+ <xsl:text>IRQ interference:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="IRQ_interference/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="IRQ_interference/latency_percent/@unit"/>
+ <xsl:text>)
+</xsl:text>
+ </xsl:if>
+
+ <xsl:for-each select="irq_interference">
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="name"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="latency/@unit"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:for-each>
+
+ <xsl:text>--------------------------------------------------------------------------------</xsl:text>
+ <xsl:text>Thread latency:</xsl:text>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency/@unit"/>
+ <xsl:text> (</xsl:text>
+ <xsl:value-of select="Thread_latency/latency_percent"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Thread_latency/latency_percent/@unit"/>
+ <xsl:text>)
+
+</xsl:text>
+ </xsl:template>
+
+ <xsl:template match="max_timerlat_report">
+ <xsl:text>
+</xsl:text>
+ <xsl:text>Max timerlat IRQ latency from idle:</xsl:text>
+ <xsl:value-of select="Max_timerlat_IRQ_latency_from_idle"/>
+ <xsl:text> </xsl:text>
+ <xsl:value-of select="Max_timerlat_IRQ_latency_from_idle/@unit"/>
+ <xsl:text> in cpu </xsl:text>
+ <xsl:value-of select="@CPU"/>
+ <xsl:text>
+</xsl:text>
+ </xsl:template>
+
</xsl:stylesheet>
--
2.45.2

217
SOURCES/rteval-timerlat-tracing-clean-up.patch Normal file
View File

@ -0,0 +1,217 @@
From 4fd5b77dc47f039551050925e442db9a17be5e75 Mon Sep 17 00:00:00 2001
From: John Kacur <jkacur@redhat.com>
Date: Mon, 24 Jun 2024 08:45:54 -0400
Subject: [PATCH 4/4] rteval: timerlat tracing clean-up
The patch cleans-up the code that stores the timerlat summary into xml,
after a trace is stopped. It does so by grouping similar kinds of output
together for parsing. It also reduces the amount of debug output.
Signed-off-by: John Kacur <jkacur@redhat.com>
---
rteval/modules/measurement/timerlat.py | 142 ++++++++++---------------
rteval/rteval_text.xsl | 10 +-
2 files changed, 62 insertions(+), 90 deletions(-)
diff --git a/rteval/modules/measurement/timerlat.py b/rteval/modules/measurement/timerlat.py
index 45adec1b33e1..dc6226ccc991 100644
--- a/rteval/modules/measurement/timerlat.py
+++ b/rteval/modules/measurement/timerlat.py
@@ -273,12 +273,14 @@ class Timerlat(rtevalModulePrototype):
os.kill(self.__timerlat_process.pid, signal.SIGINT)
time.sleep(2)
+
+ # Parse histogram output
+ self.__timerlat_out.seek(0)
+
blocking_thread_detected = False
softirq_interference_detected = False
irq_interference_detected = False
- # Parse histogram output
- self.__timerlat_out.seek(0)
for line in self.__timerlat_out:
line = bytes.decode(line)
@@ -291,6 +293,8 @@ class Timerlat(rtevalModulePrototype):
self.__posttrace += line
line = line.strip()
fields = line.split()
+ if not line:
+ continue
if line.startswith("##") and fields[1] == "CPU":
self.stcpu = int(fields[2])
self._log(Log.DEBUG, f"self.stcpu = {self.stcpu}")
@@ -304,91 +308,59 @@ class Timerlat(rtevalModulePrototype):
softirq_interference_detected = False
irq_interference_detected = False
continue
- if line.startswith("Thread latency:"):
- thread_latency_percent = fields[-1].strip('()%')
- self._log(Log.DEBUG, f"thread_latency_percent = {thread_latency_percent}")
- thread_latency = fields[-3]
- self._log(Log.DEBUG, f"thread_latency = {thread_latency}")
- self.__stdata[self.stcpu]["Thread_latency"] = (thread_latency, thread_latency_percent)
- elif line.startswith("Previous IRQ interference"):
- self._log(Log.DEBUG, f'Previous_IRQ_interference = {fields[-2]}')
- self.__stdata[self.stcpu]["Previous_IRQ_interference"] = fields[-2]
- elif line.startswith("IRQ handler delay:"):
- irq_handler_delay_percent = fields[-2].strip('(')
- irq_handler_delay = fields[-4]
- # Do we have (exit from idle)?
- if fields[3] == '(exit':
- field_name = "IRQ_handler_delay_exit_from_idle"
- else:
- field_name = "IRQ_handler_delay"
- self._log(Log.DEBUG, f"{field_name} = {irq_handler_delay}")
- self._log(Log.DEBUG, f"{field_name}_percent = {irq_handler_delay_percent}")
- self.__stdata[self.stcpu][field_name] = (irq_handler_delay, irq_handler_delay_percent)
- elif line.startswith("IRQ latency:"):
- self._log(Log.DEBUG, f"irq_latency = {fields[-2]}")
- self.__stdata[self.stcpu]["IRQ_latency"] = fields[-2]
- elif line.startswith("Timerlat IRQ duration"):
- timerlat_irq_duration_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"timerlat_irq_duration_percent = {timerlat_irq_duration_percent}")
- timerlat_irq_duration = fields[-4]
- self._log(Log.DEBUG, f"timerlat_irq_duration = {timerlat_irq_duration}")
- self.__stdata[self.stcpu]["Timerlat_IRQ_duration"] = (timerlat_irq_duration, timerlat_irq_duration_percent)
- elif line.startswith("Blocking thread:"):
- blocking_thread_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"blocking_thread_percent = {blocking_thread_percent}")
- blocking_thread = fields[-4]
- self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
- self.__stdata[self.stcpu]["Blocking_Thread"] = (blocking_thread, blocking_thread_percent)
- blocking_thread_detected = True
- irq_interference_detected = False
- softirq_interference_detected = False
- elif line.startswith("IRQ interference"):
- irq_interference_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"irq_interference_percent = {irq_interference_percent}")
- irq_interference = fields[-4]
- self._log(Log.DEBUG, f"irq_interference = {irq_interference}")
- self.__stdata[self.stcpu]["IRQ_interference"] = (irq_interference, irq_interference_percent)
- blocking_thread_detected = False
- irq_interference_detected = True
- softirq_interference_detected = False
- elif line.startswith("Softirq interference"):
- softirq_interference_percent = fields[-2].strip('(')
- self._log(Log.DEBUG, f"softirq_interference_percent = {softirq_interference_percent}")
- softirq_interference = fields[-4]
- self._log(Log.DEBUG, f"softirq_interference = {softirq_interference}")
- self.__stdata[self.stcpu]["Softirq_interference"] = (softirq_interference, softirq_interference_percent)
- blocking_thread_detected = False
- irq_interference_detected = False
- softirq_interference_detected = True
- elif blocking_thread_detected:
- self._log(Log.DEBUG, f'line={line}')
- blocking_thread = " ".join(fields[0:-2])
- self._log(Log.DEBUG, f"blocking_thread = {blocking_thread}")
- blocking_threadus = fields[-2]
- self._log(Log.DEBUG, f"blocking_threadus = {blocking_threadus}")
- self.__stdata[self.stcpu].setdefault("blocking_thread", [])
- self.__stdata[self.stcpu]["blocking_thread"] += [(blocking_thread, blocking_threadus)]
- elif softirq_interference_detected:
- softirq = " ".join(fields[0:-2])
- softirq_latency = fields[-2]
- self._log(Log.DEBUG, f'softirq = {softirq}')
- self._log(Log.DEBUG, f'softirq_latency = {softirq_latency}')
- self.__stdata[self.stcpu].setdefault("softirq_interference", [])
- self.__stdata[self.stcpu]["softirq_interference"] += [(softirq, softirq_latency)]
- elif irq_interference_detected:
- irq_interference_name = " ".join(fields[0:-2])
- irq_interference_latency = fields[-2]
- self._log(Log.DEBUG, f'irq_interference = {irq_interference_name}, latency = {irq_interference_latency}')
- self.__stdata[self.stcpu].setdefault("irq_interference", [])
- self.__stdata[self.stcpu]["irq_interference"] += [(irq_interference_name, irq_interference_latency)]
- elif line.startswith("Max timerlat IRQ latency"):
- self._log(Log.DEBUG, f"line={line}")
- max_timerlat_irq_latency = fields[-5]
- self._log(Log.DEBUG, f"max_timerlat_irq_latency = {max_timerlat_irq_latency}")
+
+ # work around rtla not printing ':' after all names
+ if line.startswith('Softirq interference'):
+ name = 'Softirq_interference'
+ elif line.startswith('IRQ interference'):
+ name = 'IRQ_interference'
+ else:
+ name = ''.join(line.split(':')[0]).replace(' ', '_')
+ self._log(Log.DEBUG, f"name={name}")
+
+ if name in ['Thread_latency']:
+ latency = fields[-3]
+ percent = fields[-1].strip('()%')
+ self._log(Log.DEBUG, f'{name} = ({latency}, {percent})')
+ self.__stdata[self.stcpu][name] = (latency, percent)
+ continue
+ if name in ['Timerlat_IRQ_duration', 'IRQ_handler_delay', 'Blocking_thread', 'IRQ_interference', 'Softirq_interference']:
+ latency = fields[-4]
+ percent = fields[-2].strip('(')
+ if name == 'IRQ_handler_delay' and fields[3] == '(exit':
+ name = 'IRQ_handler_delay_exit_from_idle'
+ self._log(Log.DEBUG, f'{name} = ({latency}, {percent})')
+ self.__stdata[self.stcpu][name] = (latency, percent)
+ detected = {'Blocking_thread' : (True, False, False),
+ 'IRQ_interference' : (False, True, False),
+ 'Softirq_interference' : (False, False, True) }
+ if name in ('Blocking_thread', 'IRQ_interference', 'Softirq_interference'):
+ blocking_thread_detected, irq_interference_detected, softirq_interference_detected = detected.get(name)
+ continue
+ if name in ["IRQ_latency", "Previous_IRQ_interference"]:
+ latency = fields[-2]
+ self._log(Log.DEBUG, f'{name} = {fields[-2]}')
+ self.__stdata[self.stcpu][name] = fields[-2]
+ continue
+ if blocking_thread_detected or softirq_interference_detected or irq_interference_detected:
+ if blocking_thread_detected:
+ field_name = "blocking_thread"
+ elif softirq_interference_detected:
+ field_name = "softirq_interference"
+ elif irq_interference_detected:
+ field_name = "irq_interference"
+ thread = " ".join(fields[0:-2])
+ latency = fields[-2]
+ self._log(Log.DEBUG, f"{field_name} += [({thread}, {latency})]")
+ self.__stdata[self.stcpu].setdefault(field_name, [])
+ self.__stdata[self.stcpu][field_name] += [(thread, latency)]
+ continue
+ if name == "Max_timerlat_IRQ_latency_from_idle":
+ latency = fields[-5]
max_timerlat_cpu = int(fields[-1])
- self._log(Log.DEBUG, f"max_timerlat_cpu = {max_timerlat_cpu}")
+ self._log(Log.DEBUG, f'self.__stdata[{max_timerlat_cpu}][{name}] = {latency}')
self.__stdata.setdefault(max_timerlat_cpu, {})
- self.__stdata[max_timerlat_cpu]["Max_timerlat_IRQ_latency_from_idle"] = max_timerlat_irq_latency
+ self.__stdata[max_timerlat_cpu][name] = latency
else:
self._log(Log.DEBUG, f'line = {line}')
continue
diff --git a/rteval/rteval_text.xsl b/rteval/rteval_text.xsl
index 70777354efa5..7ca0ae3a4c66 100644
--- a/rteval/rteval_text.xsl
+++ b/rteval/rteval_text.xsl
@@ -546,16 +546,16 @@
<xsl:text>)
</xsl:text>
- <xsl:if test="Blocking_Thread">
+ <xsl:if test="Blocking_thread">
<xsl:text>Blocking thread:</xsl:text>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency"/>
+ <xsl:value-of select="Blocking_thread/latency"/>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency/@unit"/>
+ <xsl:value-of select="Blocking_thread/latency/@unit"/>
<xsl:text> (</xsl:text>
- <xsl:value-of select="Blocking_Thread/latency_percent"/>
+ <xsl:value-of select="Blocking_thread/latency_percent"/>
<xsl:text> </xsl:text>
- <xsl:value-of select="Blocking_Thread/latency_percent/@unit"/>
+ <xsl:value-of select="Blocking_thread/latency_percent/@unit"/>
<xsl:text>)
</xsl:text>
</xsl:if>
--
2.45.2

100
SPECS/rteval.spec
View File

@ -1,5 +1,5 @@
Name: rteval
Version: 3.7
Version: 3.8
Release: 7%{?dist}
Summary: Utility to evaluate system suitability for RT Linux
@ -13,7 +13,7 @@ BuildRequires: python3-devel
Requires: python3-lxml
Requires: python3-libxml2
Requires: python3-requests
Requires: realtime-tests >= 2.5-1
Requires: realtime-tests >= 2.6-5
Requires: rteval-loads >= 1.6-2
Requires: sysstat
Requires: xz bzip2 tar gzip m4 make gawk
@ -27,17 +27,29 @@ Requires: stress-ng
Requires: perl-interpreter, perl-devel, perl-generators
Requires: libmpc, libmpc-devel
Requires: dwarves
Requires: dmidecode
Requires: procps-ng
Requires: rtla
BuildArch: noarch
#Patches
Patch1: rteval-Change-the-default-kernel-for-kcompile.patch
Patch2: rteval-Remove-upstream-spec-file.patch
Patch3: rteval-Makefile-More-rpm-cleanups.patch
Patch4: rteval-Disable-use-of-python-dmidecode.patch
Patch5: rteval-Refactor-collapse_cpulist-in-systopology.patch
Patch6: rteval-Minor-improvements-to-CpuList-class.patch
Patch7: rteval-Convert-CpuList-class-to-a-module.patch
Patch8: rteval-Add-relative-cpulists-for-measurements.patch
Patch1: Updated-rteval-man-page.patch
Patch2: rteval-Fix-aNone-being-passed-to-cyclictest.patch
Patch3: rteval-Fix-sysreport-traceback-when-utility-sos-not-.patch
Patch4: rteval-sysstat-Convert-base64-data-to-text-before-wr.patch
Patch5: rteval-timerlat-Add-timerlat-tracing-to-rteval.patch
Patch6: rteval-timerlat-tracing-clean-up.patch
Patch7: rteval-measurement-Remove-ModuleInfo.patch
Patch8: rteval-Remove-MeasurementProfile.patch
Patch9: rteval-RtEvalModules-Remove-unused-methods.patch
Patch10: rteval-Enforce-only-one-latency-measurement-module-a.patch
Patch11: rteval-Add-noload-option.patch
Patch12: rteval-Fix-default-measurement-config.patch
Patch13: rteval-measurement-Change-latency-flag-to-latency_te.patch
Patch14: rteval-Added-functionality-to-allow-user-to-set-the-.patch
Patch15: rteval-run-cyclictest-using-default-system-when-sett.patch
Patch16: rteval-Add-module-for-tuned-state.patch
Patch17: rteval-Add-tuned-state-to-rteval-text-report.patch
%description
The rteval script is a utility for measuring various aspects of
@ -49,15 +61,7 @@ a statistical analysis of the event response times is done and printed
to the screen.
%prep
%setup -q
%patch1 -p1
%patch2 -p1
%patch3 -p1
%patch4 -p1
%patch5 -p1
%patch6 -p1
%patch7 -p1
%patch8 -p1
%autosetup -p1
%build
%{__python3} setup.py build
@ -79,6 +83,64 @@ to the screen.
%{_bindir}/rteval
%changelog
* Mon Aug 12 2024 Tomas Glozar <tglozar@redhat.com> - 3.8-7
- Collect tuned state in sysinfo
Resolves: RHEL-33303
* Tue Aug 6 2024 Anubhav Shelat <ashelat@redhat.com> - 3.8-6
- Add functionality to allow user to execute cpupower tool to
set idle state depth while running rteval.
- Disable latency trick by using --default-system option with
cyclictest when setting idle state depth.
Resolves: RHEL-37645
* Mon Jul 29 2024 Crystal Wood <crwood@redhat.com> - 3.8-5
- Prevent using cyclictest and timerlat and the same time
- Add a --noload option
- Switch to autosetup, picking up the updates that should have been in 3.8-4
Resolves: RHEL-35506
Resolves: RHEL-50323
* Mon Jun 24 2024 John Kacur <jkacur@redhat.com> - 3.8-4
- Add timerlat tracing
- Add Fix -aNone being passed to cyclictest
- Add rtla as a requires for timerlat
Resolves: RHEL-35461
* Mon Jun 24 2024 Anubhav Shelat <ashelat@redhat.com> - 3.8-3
- Allow rteval to search /usr/bin/ for sysreport
- Convert base64 data to text before wrapping.
- Add sysreport to gating tests and check the exit status.
Resolves: RHEL-44437
* Mon Jun 10 2024 Anubhav Shelat <ashelat@redhat.com> - 3.8-2
- Updated rteval man page
Resolves: RHEL-37634
* Tue May 07 2024 John Kacur <jkacur@redhat.com> - 3.8-1
- Rebase to upstream rteval-3.8
Resolves: RHEL-30164
* Mon May 06 2024 John Kacur <jkacur@redhat.com> - 3.7-12
- Add timerlat as a measurement module
Resolves:RHEL-8682
* Wed Apr 10 2024 Tomas Glozar <tglozar@redhat.com> - 3.7-11
- Add relative cpulists for loads
Resolves: RHEL-25206
* Tue Apr 09 2024 Tomas Glozar <tglozar@redhat.com> - 3.7-10
- Add dependency on procps-ng which was missing in minimal environments
Resolves: RHEL-31921
* Fri Apr 05 2024 Tomas Glozar <tglozar@redhat.com> - 3.7-9
- Add dependency on dmidecode
Resolves: RHEL-20019
* Thu Mar 21 2024 Tomas Glozar <tglozar@redhat.com> - 3.7-8
- Replace python-dmidecode with initial dmidecode output parsing
Resolves: RHEL-20019
* Wed Jan 31 2024 Tomas Glozar <tglozar@redhat.com> - 3.7-7
- Added patchset for relative cpuset functionality from upstream
Resolves: RHEL-9912