Compare commits

...

No commits in common. "c8" and "c9-beta" have entirely different histories.
c8 ... c9-beta

6 changed files with 184 additions and 102 deletions

2
.gitignore vendored
View File

@ -1 +1 @@
SOURCES/v0.2.5.tar.gz
SOURCES/v0.2.4.tar.gz

1
.memstrack.metadata Normal file
View File

@ -0,0 +1 @@
4c54a54025a506f5b01a20b377884013459c0e42 SOURCES/v0.2.4.tar.gz

View File

@ -1,74 +0,0 @@
From e2d91c4823b29ddd17e9b7ea966fe1dec53ae278 Mon Sep 17 00:00:00 2001
From: Tao Liu <ltao@redhat.com>
Date: Tue, 28 Mar 2023 15:08:04 +0800
Subject: [PATCH] Enlarge rlimits if maximum open fds reached for
sys_perf_event_open
The sys_perf_event_open may fail due to reaching the maximum number of open
file descriptors. In fedora the ulimit -n gives 1024 by default, and
sys_perf_event_open() failing can be seen on systems which have a large cpu
number:
$ lscpu
Architecture: aarch64
CPU(s): 224
On-line CPU(s) list: 0-223
...
In this case, for function perf_handling_init(), we will have
perf_event_ring_num = perf_event_enabled_num * 224, i.e. 6 * 224 = 1344, which
is larger than 1024. As a result, perf_ring_setup() will be called 1344 times.
And each time a sys_perf_event_open() will return a perf_fd. So the maximun
number of fds will exceed limits and fail.
In this patch, we will try to enlarge rlimit if we have reached the maximum
number of open fds.
Signed-off-by: Tao Liu <ltao@redhat.com>
---
src/backend/perf-events.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/src/backend/perf-events.c b/src/backend/perf-events.c
index 67f1dad..7199925 100644
--- a/src/backend/perf-events.c
+++ b/src/backend/perf-events.c
@@ -29,6 +29,7 @@
#include <sys/sysinfo.h>
#include <linux/perf_event.h>
#include <sys/syscall.h>
+#include <sys/resource.h>
#include "perf-internal.h"
#include "perf-events-define.h"
@@ -357,9 +358,27 @@ int perf_ring_setup(struct PerfEventRing *ring) {
attr.wakeup_watermark = WAKEUP_WATERMARK;
attr.watermark = 1;
+retry:
perf_fd = sys_perf_event_open(&attr, -1, ring->cpu, -1, 0);
if (perf_fd <= 0) {
+ if (errno == EMFILE) {
+ struct rlimit rl;
+ if (getrlimit(RLIMIT_NOFILE, &rl) != 0) {
+ log_error("Error getting rlimit value: %s\n",
+ strerror(errno));
+ return errno;
+ }
+ rl.rlim_cur <<= 1;
+ rl.rlim_max <<= 1;
+ if (setrlimit(RLIMIT_NOFILE, &rl) != 0) {
+ log_error("Error setting rlimit value to be"
+ " rlim_cur(%lu) rlim_max(%lu): %s\n",
+ rl.rlim_cur, rl.rlim_max, strerror(errno));
+ return errno;
+ }
+ goto retry;
+ }
log_error("Error calling perf_event_open: %s\n", strerror(errno));
return errno;
}
--
2.40.1

View File

@ -0,0 +1,88 @@
From 2d27b3694bf0996767b6e5282b9d39784d1524c1 Mon Sep 17 00:00:00 2001
From: Tao Liu <ltao@redhat.com>
Date: Mon, 11 Jul 2022 15:28:25 +0800
Subject: [PATCH 1/2] Skip memcg info in __process_stacktrace for page_owner
backend
Kernel patch set [1] extended page_owner to show memcg information:
Page allocated via order 0, mask 0x0(), pid 1, tgid 1 (swapper/0), ts 158908732 ns, free_ts 0 ns
PFN 4540 type Unmovable Block 8 type Unmovable Flags 0xfffffc0010200(slab|head|node=0|zone=1|lastcpupid=0x1fffff)
register_early_stack+0x28/0x57
init_page_owner+0x1d/0x2f
kernel_init_freeable+0x138/0x1a2
kernel_init+0x16/0x120
Slab cache page
Page allocated via order 0, mask 0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), pid 737, tgid 737 (NetworkManager), ts 3964670439 ns, free_ts 3961489285 ns
PFN 11172 type Unmovable Block 21 type Unmovable Flags 0xfffffc0000000(node=0|zone=1|lastcpupid=0x1fffff)
get_page_from_freelist+0x3f0/0x500
__alloc_pages+0xe6/0x230
pte_alloc_one+0x15/0x50
...
asm_exc_page_fault+0x1e/0x30
Charged (via objcg) to memcg NetworkManager.service
"^Slab|^Charged" lines are unexpected for __process_stacktrace.
As a result, error messages as "Page owner stacktrace malformed"
will be output and fail.
This patch fix the issue by skip the memcg info lines when iterating
stacktrace.
[1]: https://lore.kernel.org/all/20220202203036.744010-3-longman@redhat.com/T/#mcd67a65c84092f71efb766d0d509f0e8255d4dc2
Signed-off-by: Tao Liu <ltao@redhat.com>
---
src/backend/page_owner.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/src/backend/page_owner.c b/src/backend/page_owner.c
index f66d77d..156f688 100644
--- a/src/backend/page_owner.c
+++ b/src/backend/page_owner.c
@@ -37,6 +37,23 @@
static char *page_owner_file;
+static char *memcg_info[] = {
+ "Slab cache page",
+ "Charged ",
+};
+
+static int is_memcg_info(char *str)
+{
+ for (int i = 0;
+ i < sizeof(memcg_info) / sizeof(__typeof__(memcg_info[0]));
+ i++) {
+ if (!strncmp(str, memcg_info[i], strlen(memcg_info[i]))) {
+ return true;
+ }
+ }
+ return false;
+}
+
static struct Tracenode* __process_stacktrace(
struct Tracenode *tn, struct PageEvent *pe, char *line, FILE *file)
{
@@ -45,6 +62,7 @@ static struct Tracenode* __process_stacktrace(
int callsite_len;
unsigned long len;
+retry:
if (!fgets(line, MAX_LINE, file)) {
log_error("Page owner file ended unexpectly before stacktrace.\n");
return NULL;
@@ -56,6 +74,9 @@ static struct Tracenode* __process_stacktrace(
return NULL;
}
+ if (is_memcg_info(line))
+ goto retry;
+
/* Empty line, end of a stacktrace */
if (line[0] == '\n' || line[0] == '\r')
return tn;
--
2.33.1

View File

@ -0,0 +1,53 @@
From 8c42fcfa92998da170ee84cd24c2377db2e90f33 Mon Sep 17 00:00:00 2001
From: Tao Liu <ltao@redhat.com>
Date: Tue, 12 Jul 2022 19:41:25 +0800
Subject: [PATCH 2/2] Fix data type error in perf_handle_mm_page_alloc
For systems as s390, unsigned long and unsigned int are different in
size. After expanding the macro, the code will be:
unsigned int order = *((unsigned long*)
(((const unsigned char*)(&raw->data)) +
((struct __perf_event_field_table_mm_page_alloc*)(perf_event_mm_page_alloc.fields))->order_info.offset));
or simply:
unsigned int order = *((unsigned long*)
(((const unsigned char*)(&raw->data)) + 16));
If we have the following data array:
Raw data[16]: 0x0
Raw data[17]: 0x0
Raw data[18]: 0x0
Raw data[19]: 0x2
Raw data[20]: 0x0
Raw data[21]: 0x0
Raw data[22]: 0xc
Raw data[23]: 0xc0
The order will be: 0x0cc0, instead of 0x2.
This patch will fix the error data type.
Signed-off-by: Tao Liu <ltao@redhat.com>
---
src/backend/perf-events.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/perf-events.c b/src/backend/perf-events.c
index 0252991..67f1dad 100644
--- a/src/backend/perf-events.c
+++ b/src/backend/perf-events.c
@@ -156,7 +156,7 @@ static int perf_handle_mm_page_alloc(const unsigned char* header) {
sizeof(callchain->ips) * callchain->nr);
unsigned long pfn = read_data_from_perf_raw(mm_page_alloc, pfn, unsigned long, raw);
- unsigned int order = read_data_from_perf_raw(mm_page_alloc, order, unsigned long, raw);
+ unsigned int order = read_data_from_perf_raw(mm_page_alloc, order, unsigned int, raw);
int pid = read_data_from_perf_raw(mm_page_alloc, common_pid, int, raw);
// TODO: pfn == -1?
--
2.33.1

View File

@ -1,18 +1,20 @@
# vim: syntax=spec
Name: memstrack
Version: 0.2.5
Release: 2%{?dist}
Version: 0.2.4
Release: 1%{?dist}
Summary: A memory allocation tracer, like a hot spot analyzer for memory allocation
License: GPLv3
URL: https://github.com/ryncsn/memstrack
VCS: git+git@github.com:ryncsn/memstrack.git
BuildRequires: make
BuildRequires: gcc
BuildRequires: ncurses-devel
Source: https://github.com/ryncsn/memstrack/archive/refs/tags/v%{version}.tar.gz
Patch1: 0001-Enlarge-rlimits-if-maximum-open-fds-reached-for-sys_.patch
Patch1: 0001-Skip-memcg-info-in-__process_stacktrace-for-page_own.patch
Patch2: 0002-Fix-data-type-error-in-perf_handle_mm_page_alloc.patch
%description
A memory allocation tracer, like a hot spot analyzer for memory allocation
@ -20,10 +22,11 @@ A memory allocation tracer, like a hot spot analyzer for memory allocation
%prep
%setup -q -n memstrack-%{version}
%patch1 -p1
%patch2 -p1
%build
%{set_build_flags}
make %{?_smp_mflags}
%{make_build}
%install
mkdir -p %{buildroot}/%{_bindir}
@ -35,35 +38,46 @@ install -p -m 755 memstrack %{buildroot}/%{_bindir}
%{_bindir}/memstrack
%changelog
* Tue Jun 27 2023 Tao Liu <ltao@redhat.com> - 0.2.5-2
- Add missing tests directory
* Mon Jun 26 2023 Tao Liu <ltao@redhat.com> - 0.2.5-1
- Rebase to latest upstream(e2d91c4823b)
* Fri Jul 15 2022 Tao Liu <ltao@redhat.com> - 0.2.4-2
- Revert bz2107730
* Fri Jul 15 2022 Tao Liu <ltao@redhat.com> - 0.2.4-1
- Rebase to latest upstream(813c2feaa2f)
* Tue Aug 6 2020 Kairui Song <kasong@redhat.com> - 0.1.11-1
- Fix memstrack failure with large memory hole
* Mon Aug 09 2021 Mohan Boddu <mboddu@redhat.com> - 0.2.3-2
- Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
Related: rhbz#1991688
* Tue Jul 14 2020 Kairui Song <kasong@redhat.com> - 0.1.8-2
- Fix TUI not quiting issue
* Fri Jul 09 2021 Kairui Song <kasong@redhat.com> - 0.2.3-1
- Update to upstream latest release
* Sun Jun 28 2020 Kairui Song <kasong@redhat.com> - 0.1.8-1
- Bump to latest upstream version, fix some issues and introduce unit test
* Fri Apr 16 2021 Mohan Boddu <mboddu@redhat.com> - 0.2.2-2
- Rebuilt for RHEL 9 BETA on Apr 15th 2021. Related: rhbz#1947937
* Wed Jun 24 2020 Kairui Song <kasong@redhat.com> - 0.1.7-1
- Bump to latest upstream version to fix multiple bugs
* Mon Feb 08 2021 Kairui Song <kasong@redhat.com> - 0.2.2-1
- Update to upstream latest release
* Wed May 20 2020 Kairui Song <kasong@redhat.com> - 0.1.4-1
- Bump to latest upstream version to fix some other issue found in early testing
* Tue Jan 26 2021 Fedora Release Engineering <releng@fedoraproject.org> - 0.1.12-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild
* Tue May 19 2020 Kairui Song <kasong@redhat.com> - 0.1.3-1
- Bump to latest upstream version to fix problems found by rpmdiff
* Mon Jan 11 2021 Timm Bäder <tbaeder@redhat.com> - 0.1.12-2
- Use %%make_build macro
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_parallel_make
* Thu May 07 2020 Kairui Song <kasong@redhat.com> - 0.1.2-1
- Initial Release
* Sun Aug 30 2020 Kairui Song <kasong@redhat.com> - 0.1.12-1
- Update to upstream latest release
* Thu Jul 30 2020 Kairui Song <kasong@redhat.com> - 0.1.9-1
- Update to upstream latest release
* Tue Jul 28 2020 Fedora Release Engineering <releng@fedoraproject.org> - 0.1.8-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
* Sat Jun 06 2020 Kairui Song <kasong@redhat.com> - 0.1.8-1
- Update to upstream latest release
* Sat May 30 2020 Kairui Song <ryncsn@gmail.com> - 0.1.5-1
- Update to upstream latest release
* Tue Apr 21 2020 Kairui Song <ryncsn@gmail.com> - 0.1.2-1
- Update to upstream latest release
* Sun Mar 15 2020 Kairui Song <ryncsn@gmail.com> - 0-1.20200310gitee02de2
- First release