rpms/pcp
7
0
Fork 0
Code Issues Pull Requests Releases Activity

import pcp-5.3.5-8.el8

Browse Source
This commit is contained in:
CentOS Sources 2022-05-10 02:59:42 -04:00 committed by Stepan Oksanichenko
parent 4bd3c021d2
commit 41bf8c0c70
17 changed files with 3732 additions and 13538 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1 +1 @@
SOURCES/pcp-5.3.1.src.tar.gz
SOURCES/pcp-5.3.5.src.tar.gz

2
.pcp.metadata
View File

@ -1 +1 @@
5b693868cb11c09b87880ca2d57e672a3bfd63e6 SOURCES/pcp-5.3.1.src.tar.gz
ee69b4a4e33cb5366d30c4c006ba41ccd6386d87 SOURCES/pcp-5.3.5.src.tar.gz

6966
SOURCES/redhat-bugzilla-1879350.patch
View File

File diff suppressed because it is too large Load Diff

849
SOURCES/redhat-bugzilla-1947989.patch
View File

@ -1,849 +0,0 @@
commit 2bad6aef10339f000f7cb578108db5ee80bd640c
Author: Mark Goodwin <mgoodwin@redhat.com>
Date: Wed Jun 9 17:04:33 2021 +1000
pmproxy: add mutex for client req lists, fix https/tls support, QA
Add a new mutext to struct proxy and use it to protect parallel
multithreaded updates to the proxy->first client list.
Also use the same mutext to protect updates to the pending_writes
client list and avoid the doubly linked list corruption that was
causing parallel https/tls requests to get stuck spinning in
flush_secure_module(), as reported in BZ#1947989.
qa/1457 is extensively updated to test parallel http, https/tls
(and combinations of http and https/tls) RESTAPI calls. Previously
it only tested a single https/tls call.
With these changes, parallel https/tls RESTAPI requests from the
grafana-pcp datasource to pmproxy now work correctly whereas previously
pmproxy would hang/spin.
Resolves: RHBZ#1947989 - pmproxy hangs and consume 100% cpu if the
redis datasource is configured with TLS.
Related: https://github.com/performancecopilot/pcp/issues/1311
diff --git a/qa/1457 b/qa/1457
index 94969f6e0..8bf395944 100755
--- a/qa/1457
+++ b/qa/1457
@@ -2,7 +2,7 @@
# PCP QA Test No. 1457
# Exercise HTTPS access to the PMWEBAPI(3).
#
-# Copyright (c) 2019 Red Hat.
+# Copyright (c) 2019,2021 Red Hat.
#
seq=`basename $0`
@@ -138,14 +138,59 @@ else
fi
date >>$seq.full
-echo "=== checking TLS operation ===" | tee -a $seq.full
-# (-k) allows us to use self-signed (insecure) certificates, so for testing only
-# (-v) provides very detailed TLS connection information, for debugging only
-curl -k --get 2>$tmp.err \
- "https://localhost:$port/pmapi/metric?name=sample.long.ten" \
- | _filter_json
-cat $tmp.err >>$seq.full
+echo "=== checking serial http operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -Gs "http://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i
+done
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking parallel http operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -Gs "http://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i & 2>/dev/null eval pid$i=$!
+done
+wait $pid1 $pid2 $pid3 $pid4
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking serial https/TLS operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -k -Gs "https://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i
+done
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
date >>$seq.full
+echo "=== checking parallel https/TLS operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -k -Gs "https://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i & 2>/dev/null eval pid$i=$!
+done
+wait $pid1 $pid2 $pid3 $pid4
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking parallel mixed http and https/TLS operations ===" | tee -a $seq.full
+for i in 1 3 5 7; do
+ j=`expr $i + 1`
+ curl -k -Gs "http://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i & 2>/dev/null eval pid$i=$!
+ curl -k -Gs "https://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$j >$tmp.out$j & 2>/dev/null eval pid$j=$!
+done
+wait $pid1 $pid2 $pid3 $pid4 $pid5 $pid6 $pid7 $pid8
+for i in 1 2 3 4 5 6 7 8; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
echo "=== check pmproxy is running ==="
pminfo -v -h localhost@localhost:$port hinv.ncpu
@@ -156,7 +201,7 @@ else
fi
# valgrind takes awhile to shutdown too
-pmsignal $pid
+pmsignal $pid >/dev/null 2>&1
pmsleep 3.5
echo "=== valgrind stdout ===" | tee -a $seq.full
cat $tmp.valout | _filter_valgrind
@@ -164,6 +209,9 @@ cat $tmp.valout | _filter_valgrind
echo "=== valgrind stderr ===" | tee -a $seq.full
cat $tmp.valerr | _filter_pmproxy_log | _filter_port
+# final kill if it's spinning
+$sudo kill -9 $pid >/dev/null 2>&1
+
# success, all done
status=0
exit
diff --git a/qa/1457.out b/qa/1457.out
index a7b64cdc5..422176db2 100644
--- a/qa/1457.out
+++ b/qa/1457.out
@@ -1,5 +1,539 @@
QA output created by 1457
-=== checking TLS operation ===
+=== checking serial http operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking parallel http operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking serial https/TLS operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking parallel https/TLS operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking parallel mixed http and https/TLS operations ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out5 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out6 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out7 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out8 ===
{
"context": "CONTEXT"
"metrics": [
diff --git a/qa/group b/qa/group
index 462dffaaa..77cac788d 100644
--- a/qa/group
+++ b/qa/group
@@ -1818,7 +1818,7 @@ x11
1436 pmda.postgresql local
1437 pmda.kvm local
1455 pmlogrewrite labels pmdumplog local
-1457 pmproxy local
+1457 pmproxy libpcp_web threads secure local
1480 pmda.lmsensors local
1489 python pmrep pmimport local
1490 python local labels
diff --git a/src/pmproxy/src/secure.c b/src/pmproxy/src/secure.c
index 77265894c..072e2a085 100644
--- a/src/pmproxy/src/secure.c
+++ b/src/pmproxy/src/secure.c
@@ -16,13 +16,25 @@
#include <openssl/opensslv.h>
#include <openssl/ssl.h>
+/* called with proxy->mutex locked */
static void
remove_connection_from_queue(struct client *client)
{
+ struct proxy *proxy = client->proxy;
+
if (client->secure.pending.writes_buffer != NULL)
free(client->secure.pending.writes_buffer);
- if (client->secure.pending.prev != NULL)
- *client->secure.pending.prev = client->secure.pending.next;
+ if (client->secure.pending.prev == NULL) {
+ /* next (if any) becomes first in pending_writes list */
+ proxy->pending_writes = client->secure.pending.next;
+ if (proxy->pending_writes)
+ proxy->pending_writes->secure.pending.prev = NULL;
+ }
+ else {
+ /* link next and prev */
+ client->secure.pending.prev->secure.pending.next = client->secure.pending.next;
+ client->secure.pending.next->secure.pending.prev = client->secure.pending.prev;
+ }
memset(&client->secure.pending, 0, sizeof(client->secure.pending));
}
@@ -32,7 +44,9 @@ on_secure_client_close(struct client *client)
if (pmDebugOptions.auth || pmDebugOptions.http)
fprintf(stderr, "%s: client %p\n", "on_secure_client_close", client);
+ uv_mutex_lock(&client->proxy->mutex);
remove_connection_from_queue(client);
+ uv_mutex_unlock(&client->proxy->mutex);
/* client->read and client->write freed by SSL_free */
SSL_free(client->secure.ssl);
}
@@ -40,6 +54,8 @@ on_secure_client_close(struct client *client)
static void
maybe_flush_ssl(struct proxy *proxy, struct client *client)
{
+ struct client *c;
+
if (client->secure.pending.queued)
return;
@@ -47,13 +63,19 @@ maybe_flush_ssl(struct proxy *proxy, struct client *client)
client->secure.pending.writes_count > 0)
return;
- client->secure.pending.next = proxy->pending_writes;
- if (client->secure.pending.next != NULL)
- client->secure.pending.next->secure.pending.prev = &client->secure.pending.next;
- client->secure.pending.prev = &proxy->pending_writes;
+ uv_mutex_lock(&proxy->mutex);
+ if (proxy->pending_writes == NULL) {
+ proxy->pending_writes = client;
+ client->secure.pending.prev = client->secure.pending.next = NULL;
+ }
+ else {
+ for (c=proxy->pending_writes; c->secure.pending.next; c = c->secure.pending.next)
+ ; /**/
+ c->secure.pending.next = client;
+ client->secure.pending.prev = c;
+ }
client->secure.pending.queued = 1;
-
- proxy->pending_writes = client;
+ uv_mutex_unlock(&proxy->mutex);
}
static void
@@ -135,10 +157,12 @@ flush_ssl_buffer(struct client *client)
void
flush_secure_module(struct proxy *proxy)
{
- struct client *client, **head = &proxy->pending_writes;
+ struct client *client, **head;
size_t i, used;
int sts;
+ uv_mutex_lock(&proxy->mutex);
+ head = &proxy->pending_writes;
while ((client = *head) != NULL) {
flush_ssl_buffer(client);
@@ -188,6 +212,7 @@ flush_secure_module(struct proxy *proxy)
sizeof(uv_buf_t) * client->secure.pending.writes_count);
}
}
+ uv_mutex_unlock(&proxy->mutex);
}
void
diff --git a/src/pmproxy/src/server.c b/src/pmproxy/src/server.c
index 612d3613b..b5c5d84de 100644
--- a/src/pmproxy/src/server.c
+++ b/src/pmproxy/src/server.c
@@ -149,6 +149,7 @@ server_init(int portcount, const char *localpath)
pmGetProgname());
return NULL;
}
+ uv_mutex_init(&proxy->mutex);
count = portcount + (*localpath ? 1 : 0);
if (count) {
@@ -251,6 +252,7 @@ void
client_put(struct client *client)
{
unsigned int refcount;
+ struct proxy *proxy = client->proxy;
uv_mutex_lock(&client->mutex);
assert(client->refcount);
@@ -259,9 +261,11 @@ client_put(struct client *client)
if (refcount == 0) {
/* remove client from the doubly-linked list */
+ uv_mutex_lock(&proxy->mutex);
if (client->next != NULL)
client->next->prev = client->prev;
*client->prev = client->next;
+ uv_mutex_unlock(&proxy->mutex);
if (client->protocol & STREAM_PCP)
on_pcp_client_close(client);
@@ -514,10 +518,12 @@ on_client_connection(uv_stream_t *stream, int status)
client->proxy = proxy;
/* insert client into doubly-linked list at the head */
+ uv_mutex_lock(&proxy->mutex);
if ((client->next = proxy->first) != NULL)
proxy->first->prev = &client->next;
proxy->first = client;
client->prev = &proxy->first;
+ uv_mutex_unlock(&proxy->mutex);
status = uv_read_start((uv_stream_t *)&client->stream.u.tcp,
on_buffer_alloc, on_client_read);
diff --git a/src/pmproxy/src/server.h b/src/pmproxy/src/server.h
index f0b7a5f68..f93daeff4 100644
--- a/src/pmproxy/src/server.h
+++ b/src/pmproxy/src/server.h
@@ -118,7 +118,7 @@ typedef struct secure_client {
BIO *write;
struct {
struct client *next;
- struct client **prev;
+ struct client *prev;
unsigned int queued;
size_t writes_count;
uv_buf_t *writes_buffer;
@@ -166,6 +166,7 @@ typedef struct proxy {
struct dict *config; /* configuration dictionary */
uv_loop_t *events; /* global, async event loop */
uv_callback_t write_callbacks;
+ uv_mutex_t mutex; /* protects client lists and pending writes */
} proxy;
extern void proxylog(pmLogLevel, sds, void *);

2242
SOURCES/redhat-bugzilla-1962019.patch
View File

File diff suppressed because it is too large Load Diff

193
SOURCES/redhat-bugzilla-1973833.patch Normal file
View File

@ -0,0 +1,193 @@
commit 6b907e90c74fce82d6b712493d8b362bdd1a1ec1
Author: Nathan Scott <nathans@redhat.com>
Date: Tue Dec 14 08:54:14 2021 +1100
pmlogconf: switch to the bulk pmLookupDescs(3) interface
No functional change, all existing regression tests pass.
Related to Red Hat BZ #1973833.
diff --git a/src/pmlogconf/pmlogconf.c b/src/pmlogconf/pmlogconf.c
index fa1156859d..ef4fc08bbd 100644
--- a/src/pmlogconf/pmlogconf.c
+++ b/src/pmlogconf/pmlogconf.c
@@ -473,13 +473,19 @@ fetch_groups(void)
{
static pmResult *result;
const char **names;
+ pmDesc *descs;
pmID *pmids;
- int i, n, sts;
+ int i, n, sts, count;
- /* prepare arrays of names and identifiers for PMAPI metric lookup */
+ /* prepare arrays of names, descriptors and IDs for PMAPI metric lookup */
if ((names = calloc(ngroups, sizeof(char *))) == NULL)
return -ENOMEM;
+ if ((descs = calloc(ngroups, sizeof(pmDesc))) == NULL) {
+ free(names);
+ return -ENOMEM;
+ }
if ((pmids = calloc(ngroups, sizeof(pmID))) == NULL) {
+ free(descs);
free(names);
return -ENOMEM;
}
@@ -490,15 +496,16 @@ fetch_groups(void)
continue;
names[n++] = (const char *)groups[i].metric;
}
+ count = n;
- if ((sts = pmLookupName(n, names, pmids)) < 0) {
- if (n == 1)
+ if ((sts = pmLookupName(count, names, pmids)) < 0) {
+ if (count == 1)
groups[0].pmid = PM_ID_NULL;
else
fprintf(stderr, "%s: cannot lookup metric names: %s\n",
pmGetProgname(), pmErrStr(sts));
}
- else if ((sts = pmFetch(n, pmids, &result)) < 0) {
+ else if ((sts = pmFetch(count, pmids, &result)) < 0) {
fprintf(stderr, "%s: cannot fetch metric values: %s\n",
pmGetProgname(), pmErrStr(sts));
}
@@ -510,6 +517,13 @@ fetch_groups(void)
else
groups[i].pmid = pmids[n++];
}
+ /* descriptor lookup, descs_hash handles failure here */
+ (void) pmLookupDescs(count, pmids, descs);
+
+ /* create a hash over the descs for quick PMID lookup */
+ if ((sts = descs_hash(count, descs)) < 0)
+ fprintf(stderr, "%s: cannot hash metric descs: %s\n",
+ pmGetProgname(), pmErrStr(sts));
/* create a hash over the result for quick PMID lookup */
if ((sts = values_hash(result)) < 0)
fprintf(stderr, "%s: cannot hash metric values: %s\n",
@@ -806,14 +820,16 @@ evaluate_string_regexp(group_t *group, regex_cmp_t compare)
int i, found;
pmValueSet *vsp;
pmValue *vp;
+ pmDesc *dp;
pmAtomValue atom;
regex_t regex;
int sts, type;
- if ((vsp = metric_values(group->pmid)) == NULL)
+ if ((vsp = metric_values(group->pmid)) == NULL ||
+ (dp = metric_desc(group->pmid)) == NULL)
return 0;
- type = metric_type(group->pmid);
+ type = dp->type;
if (type < 0 || type > PM_TYPE_STRING) {
fprintf(stderr, "%s: %s uses regular expression on non-scalar metric\n",
pmGetProgname(), group->tag);
@@ -849,11 +865,14 @@ evaluate_string_regexp(group_t *group, regex_cmp_t compare)
static int
evaluate_values(group_t *group, numeric_cmp_t ncmp, string_cmp_t scmp)
{
- int type = metric_type(group->pmid);
+ pmDesc *dp;
+
+ if ((dp = metric_desc(group->pmid)) == NULL)
+ return 0;
- if (type == PM_TYPE_STRING)
+ if (dp->type == PM_TYPE_STRING)
return evaluate_string_values(group, scmp);
- return evaluate_number_values(group, type, ncmp);
+ return evaluate_number_values(group, dp->type, ncmp);
}
int
diff --git a/src/pmlogconf/util.c b/src/pmlogconf/util.c
index d44c2e529a..293eb2eca3 100644
--- a/src/pmlogconf/util.c
+++ b/src/pmlogconf/util.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2020 Red Hat. All Rights Reserved.
+ * Copyright (c) 2020-2021 Red Hat. All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -14,7 +14,7 @@
#include "util.h"
static __pmHashCtl valuesctl; /* pointers to values in pmResult */
-static __pmHashCtl typesctl; /* metric types from pmLookupDesc */
+static __pmHashCtl descsctl; /* metric descs from pmLookupDesc */
int
values_hash(pmResult *result)
@@ -47,27 +47,33 @@ metric_values(pmID pmid)
}
int
-metric_type(pmID pmid)
+descs_hash(int numpmid, pmDesc *descs)
{
- __pmHashNode *node;
- pmDesc desc;
- int sts, *data;
+ unsigned int i;
+ pmDesc *dp;
+ int sts;
- if (pmid == PM_IN_NULL)
- return PM_TYPE_UNKNOWN;
- if ((node = __pmHashSearch(pmid, &typesctl)) == NULL) {
- if ((sts = pmLookupDesc(pmid, &desc)) < 0)
- return sts;
- if ((data = malloc(sizeof(int))) == NULL)
- return sts;
- *data = desc.type;
- if ((sts = __pmHashAdd(pmid, data, &typesctl)) < 0) {
- free(data);
+ if ((sts = __pmHashPreAlloc(numpmid, &descsctl)) < 0)
+ return sts;
+
+ for (i = 0; i < numpmid; i++) {
+ dp = &descs[i];
+ if ((sts = __pmHashAdd(dp->pmid, dp, &descsctl)) < 0)
return sts;
- }
- return *data;
}
- return *(int *)node->data;
+ return numpmid;
+}
+
+pmDesc *
+metric_desc(pmID pmid)
+{
+ __pmHashNode *node;
+
+ if (pmid == PM_IN_NULL)
+ return NULL;
+ if ((node = __pmHashSearch(pmid, &descsctl)) == NULL)
+ return NULL;
+ return (pmDesc *)node->data;
}
int
diff --git a/src/pmlogconf/util.h b/src/pmlogconf/util.h
index 17d856a0d7..a11350d899 100644
--- a/src/pmlogconf/util.h
+++ b/src/pmlogconf/util.h
@@ -34,7 +34,9 @@ extern void fmt(const char *, char *, size_t, int, int, fmt_t, void *);
extern int values_hash(pmResult *);
extern pmValueSet *metric_values(pmID);
-extern int metric_type(pmID);
+
+extern int descs_hash(int, pmDesc *);
+extern pmDesc *metric_desc(pmID);
typedef int (*numeric_cmp_t)(double, double);
extern int number_equal(double, double);

2478
SOURCES/redhat-bugzilla-1974266.patch
View File

File diff suppressed because it is too large Load Diff

922
SOURCES/redhat-bugzilla-1975069.patch
View File

@ -1,922 +0,0 @@
3f5ba2218 libpcp_web: add mutex to struct webgroup protecting the context dict
107633192 src/libpcp: be more careful when calling __pmLogChangeVol()
49bdfdfff libpcp: redefine __pmLogSetTime()
5e3b792d3 libpcp_web: plug mem leak in redisMapInsert during daily log-rolling
2a00a90b0 libpcp_web/discovery: improve lock handling and scalability
commit 3f5ba221842e6a02e9fb22e23c754854271c3c9a
Author: Mark Goodwin <mgoodwin@redhat.com>
Date: Wed Jun 9 16:44:30 2021 +1000
libpcp_web: add mutex to struct webgroup protecting the context dict
Add a mutex to the local webgroups structure in libpcp_web and
use it to protect multithreaded parallel updates (dictAdd,
dictDelete) to the groups->contexts dict and the dict traversal
in the timer driven garbage collector.
Tested by qa/297 and related tests and also an updated version
of qa/1457 (which now stress tests parallel http and https/tls
pmproxy RESTAPI calls .. in a later commit).
Related: RHBZ#1947989
Resolves: https://github.com/performancecopilot/pcp/issues/1311
diff --git a/src/libpcp_web/src/webgroup.c b/src/libpcp_web/src/webgroup.c
index 08c2518ed..35f05441b 100644
--- a/src/libpcp_web/src/webgroup.c
+++ b/src/libpcp_web/src/webgroup.c
@@ -51,14 +51,20 @@ typedef struct webgroups {
uv_loop_t *events;
unsigned int active;
uv_timer_t timer;
+ uv_mutex_t mutex;
} webgroups;
static struct webgroups *
webgroups_lookup(pmWebGroupModule *module)
{
- if (module->privdata == NULL)
+ struct webgroups *groups = module->privdata;
+
+ if (module->privdata == NULL) {
module->privdata = calloc(1, sizeof(struct webgroups));
- return (struct webgroups *)module->privdata;
+ groups = (struct webgroups *)module->privdata;
+ uv_mutex_init(&groups->mutex);
+ }
+ return groups;
}
static int
@@ -94,8 +100,11 @@ webgroup_drop_context(struct context *context, struct webgroups *groups)
context->garbage = 1;
uv_timer_stop(&context->timer);
}
- if (groups)
+ if (groups) {
+ uv_mutex_lock(&groups->mutex);
dictDelete(groups->contexts, &context->randomid);
+ uv_mutex_unlock(&groups->mutex);
+ }
uv_close((uv_handle_t *)&context->timer, webgroup_release_context);
}
}
@@ -207,13 +216,16 @@ webgroup_new_context(pmWebGroupSettings *sp, dict *params,
cp->context = -1;
cp->timeout = polltime;
+ uv_mutex_lock(&groups->mutex);
if ((cp->randomid = random()) < 0 ||
dictFind(groups->contexts, &cp->randomid) != NULL) {
infofmt(*message, "random number failure on new web context");
pmwebapi_free_context(cp);
*status = -ESRCH;
+ uv_mutex_unlock(&groups->mutex);
return NULL;
}
+ uv_mutex_unlock(&groups->mutex);
cp->origin = sdscatfmt(sdsempty(), "%i", cp->randomid);
cp->name.sds = sdsdup(hostspec ? hostspec : LOCALHOST);
cp->realm = sdscatfmt(sdsempty(), "pmapi/%i", cp->randomid);
@@ -242,7 +254,9 @@ webgroup_new_context(pmWebGroupSettings *sp, dict *params,
pmwebapi_free_context(cp);
return NULL;
}
+ uv_mutex_lock(&groups->mutex);
dictAdd(groups->contexts, &cp->randomid, cp);
+ uv_mutex_unlock(&groups->mutex);
/* leave until the end because uv_timer_init makes this visible in uv_run */
handle = (uv_handle_t *)&cp->timer;
@@ -261,25 +275,34 @@ webgroup_new_context(pmWebGroupSettings *sp, dict *params,
static void
webgroup_garbage_collect(struct webgroups *groups)
{
- dictIterator *iterator = dictGetSafeIterator(groups->contexts);
+ dictIterator *iterator;
dictEntry *entry;
context_t *cp;
if (pmDebugOptions.http || pmDebugOptions.libweb)
fprintf(stderr, "%s: started\n", "webgroup_garbage_collect");
- while ((entry = dictNext(iterator)) != NULL) {
- cp = (context_t *)dictGetVal(entry);
- if (cp->garbage && cp->privdata == groups) {
- if (pmDebugOptions.http || pmDebugOptions.libweb)
- fprintf(stderr, "GC context %u (%p)\n", cp->randomid, cp);
- webgroup_drop_context(cp, groups);
+ /* do context GC if we get the lock (else don't block here) */
+ if (uv_mutex_trylock(&groups->mutex) == 0) {
+ iterator = dictGetSafeIterator(groups->contexts);
+ for (entry = dictNext(iterator); entry;) {
+ cp = (context_t *)dictGetVal(entry);
+ entry = dictNext(iterator);
+ if (cp->garbage && cp->privdata == groups) {
+ if (pmDebugOptions.http || pmDebugOptions.libweb)
+ fprintf(stderr, "GC context %u (%p)\n", cp->randomid, cp);
+ uv_mutex_unlock(&groups->mutex);
+ webgroup_drop_context(cp, groups);
+ uv_mutex_lock(&groups->mutex);
+ }
}
+ dictReleaseIterator(iterator);
+ uv_mutex_unlock(&groups->mutex);
}
- dictReleaseIterator(iterator);
/* TODO - trim maps, particularly instmap if proc metrics are not excluded */
+ /* TODO move the following to a new stats timer */
if (groups->metrics_handle) {
mmv_stats_set(groups->metrics_handle, "contextmap.size",
NULL, dictSize(contextmap));
commit 107633192326b27ae571d4d4955052b8d86222c2
Author: Ken McDonell <kenj@kenj.id.au>
Date: Fri Jul 2 16:52:48 2021 +1000
src/libpcp: be more careful when calling __pmLogChangeVol()
Mark observed a SEGV which looks like __pmLogFetch() died because
ctxp->c_archctl->ac_mfp was (unexpectedly) NULL.
See: https://github.com/performancecopilot/pcp/issues/1338
Initial guess is that a physical file was removed by concurrent
activity (like pmlogger_check or pmlogger_daily), causing
__pmLogChangeVol() to fail ... and this was not being checked for
on the __pmLogFetch() path and in a couple of other places.
modified: interp.c
modified: logutil.c
diff --git a/src/libpcp/src/interp.c b/src/libpcp/src/interp.c
index d7effbc1e..c8f6fe382 100644
--- a/src/libpcp/src/interp.c
+++ b/src/libpcp/src/interp.c
@@ -1312,7 +1312,9 @@ __pmLogFetchInterp(__pmContext *ctxp, int numpmid, pmID pmidlist[], pmResult **r
}
/* get to the last remembered place */
- __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ sts = __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ if (sts < 0)
+ goto all_done;
__pmFseek(ctxp->c_archctl->ac_mfp, ctxp->c_archctl->ac_offset, SEEK_SET);
seen_mark = 0; /* interested in <mark> records seen from here on */
@@ -1397,7 +1399,9 @@ __pmLogFetchInterp(__pmContext *ctxp, int numpmid, pmID pmidlist[], pmResult **r
* at least one metric requires a bound from earlier in the log ...
* position ourselves, ... and search
*/
- __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ sts = __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ if (sts < 0)
+ goto all_done;
__pmFseek(ctxp->c_archctl->ac_mfp, ctxp->c_archctl->ac_offset, SEEK_SET);
done = 0;
@@ -1542,7 +1546,9 @@ __pmLogFetchInterp(__pmContext *ctxp, int numpmid, pmID pmidlist[], pmResult **r
* at least one metric requires a bound from later in the log ...
* position ourselves ... and search
*/
- __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ sts = __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ if (sts < 0)
+ goto all_done;
__pmFseek(ctxp->c_archctl->ac_mfp, ctxp->c_archctl->ac_offset, SEEK_SET);
done = 0;
diff --git a/src/libpcp/src/logutil.c b/src/libpcp/src/logutil.c
index fe35ed422..0ef76de25 100644
--- a/src/libpcp/src/logutil.c
+++ b/src/libpcp/src/logutil.c
@@ -1992,7 +1992,10 @@ __pmLogFetch(__pmContext *ctxp, int numpmid, pmID pmidlist[], pmResult **result)
all_derived = check_all_derived(numpmid, pmidlist);
/* re-establish position */
- __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ sts = __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
+ if (sts < 0)
+ goto func_return;
+ assert(ctxp->c_archctl->ac_mfp != NULL);
__pmFseek(ctxp->c_archctl->ac_mfp,
(long)ctxp->c_archctl->ac_offset, SEEK_SET);
@@ -2489,10 +2492,12 @@ __pmLogSetTime(__pmContext *ctxp)
/* index either not available, or not useful */
if (mode == PM_MODE_FORW) {
__pmLogChangeVol(acp, lcp->l_minvol);
+ assert(acp->ac_mfp != NULL);
__pmFseek(acp->ac_mfp, (long)(sizeof(__pmLogLabel) + 2*sizeof(int)), SEEK_SET);
}
else if (mode == PM_MODE_BACK) {
__pmLogChangeVol(acp, lcp->l_maxvol);
+ assert(acp->ac_mfp != NULL);
__pmFseek(acp->ac_mfp, (long)0, SEEK_END);
}
@@ -3141,6 +3146,7 @@ LogChangeToPreviousArchive(__pmContext *ctxp)
/* Set up to scan backwards from the end of the archive. */
__pmLogChangeVol(acp, lcp->l_maxvol);
+ assert(acp->ac_mfp != NULL);
__pmFseek(acp->ac_mfp, (long)0, SEEK_END);
ctxp->c_archctl->ac_offset = __pmFtell(acp->ac_mfp);
assert(ctxp->c_archctl->ac_offset >= 0);
commit 49bdfdfff83ac165de2bdc9a40e61a56512585d8
Author: Ken McDonell <kenj@kenj.id.au>
Date: Sun Jul 4 10:07:09 2021 +1000
libpcp: redefine __pmLogSetTime()
The problem is that if physical files for the data volumes of an
archive are removed (asynchronously by someone else) while we're
trying to switch volumes then we don't handle this safely.
The previous commit 10763319 was as stop-gap to address Mark's SEGV
issue at https://github.com/performancecopilot/pcp/issues/1338 and
simply handled direct calls to __pmLogChangeVol() and ensured the
return status was checked.
I was aware, then Coverity made a lot more people aware, that this
"fix" was incomplete, specifically the calls to __pmLogChangeVol()
from within __pmLogSetTime() were not checked.
To fix the latter we have to change the type of __pmLogSetTime() from
void to int so we can return status to indicate that __pmLogChangeVol()
has failed. And then make sure all the callers of __pmLogSetTime()
check the status returned from that function.
modified: src/libpcp/src/fetch.c
modified: src/libpcp/src/internal.h
modified: src/libpcp/src/logutil.c
Because this introduces some new -Dlog diagnostics, qa/251 needed
a bit of a make-over.
diff --git a/qa/251 b/qa/251
index 2b8a07917..f9b293e98 100755
--- a/qa/251
+++ b/qa/251
@@ -37,7 +37,7 @@ _filter()
status=1 # failure is the default!
$sudo rm -rf $tmp.* $seq.full
-trap "cd $here; rm -rf $tmp; exit \$status" 0 1 2 3 15
+trap "cd $here; rm -rf $tmp $tmp.*; exit \$status" 0 1 2 3 15
# real QA test starts here
mkdir $tmp
@@ -50,56 +50,62 @@ cd $tmp
for inst in "bin-100" "bin-100,bin-500,bin-900"
do
echo
- echo "All volumes present ... $inst ..."
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ echo "All volumes present ... $inst ..." | tee -a $here/$seq.full
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
echo
- echo "First volume missing ... $inst ..."
+ echo "First volume missing ... $inst ..." | tee -a $here/$seq.full
mv ok-mv-bar.0 foo.0
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
mv foo.0 ok-mv-bar.0
echo
- echo "Last volume missing ... $inst ..."
+ echo "Last volume missing ... $inst ..." | tee -a $here/$seq.full
mv ok-mv-bar.3 foo.3
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
mv foo.3 ok-mv-bar.3
echo
- echo "Second volume missing ... $inst ..."
+ echo "Second volume missing ... $inst ..." | tee -a $here/$seq.full
mv ok-mv-bar.1 foo.1
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
mv foo.1 ok-mv-bar.1
echo
- echo "Second last volume missing ... $inst ..."
+ echo "Second last volume missing ... $inst ..." | tee -a $here/$seq.full
mv ok-mv-bar.2 foo.2
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
mv foo.2 ok-mv-bar.2
echo
- echo "All volumes but second missing ... $inst ..."
+ echo "All volumes but second missing ... $inst ..." | tee -a $here/$seq.full
mv ok-mv-bar.0 foo.0
mv ok-mv-bar.2 foo.2
mv ok-mv-bar.3 foo.3
- pmval -z -O $offset -D128 -t2 -a ok-mv-bar -i $inst sampledso.bin 2>err >out
- egrep 'Skip|Change' err
- _filter <out
+ pmval -z -O $offset -Dlog -t2 -a ok-mv-bar -i $inst sampledso.bin 2>$tmp.err >$tmp.out
+ cat $tmp.err >>$here/$seq.full
+ grep '^__pmLogChangeVol:' $tmp.err
+ _filter <$tmp.out
[ -f die ] && exit
mv foo.0 ok-mv-bar.0
mv foo.2 ok-mv-bar.2
diff --git a/src/libpcp/src/fetch.c b/src/libpcp/src/fetch.c
index 5328a2807..01d5bf7fc 100644
--- a/src/libpcp/src/fetch.c
+++ b/src/libpcp/src/fetch.c
@@ -458,6 +458,7 @@ pmSetMode(int mode, const struct timeval *when, int delta)
/* assume PM_CONTEXT_ARCHIVE */
if (l_mode == PM_MODE_INTERP ||
l_mode == PM_MODE_FORW || l_mode == PM_MODE_BACK) {
+ int lsts;
if (when != NULL) {
/*
* special case of NULL for timestamp
@@ -468,7 +469,18 @@ pmSetMode(int mode, const struct timeval *when, int delta)
}
ctxp->c_mode = mode;
ctxp->c_delta = delta;
- __pmLogSetTime(ctxp);
+ lsts = __pmLogSetTime(ctxp);
+ if (lsts < 0) {
+ /*
+ * most unlikely; not much we can do here but expect
+ * PMAPI error to be returned once pmFetch's start
+ */
+ if (pmDebugOptions.log) {
+ char errmsg[PM_MAXERRMSGLEN];
+ fprintf(stderr, "pmSetMode: __pmLogSetTime failed: %s\n",
+ pmErrStr_r(lsts, errmsg, sizeof(errmsg)));
+ }
+ }
__pmLogResetInterp(ctxp);
sts = 0;
}
diff --git a/src/libpcp/src/internal.h b/src/libpcp/src/internal.h
index 977efdcf6..fd8d6e740 100644
--- a/src/libpcp/src/internal.h
+++ b/src/libpcp/src/internal.h
@@ -407,7 +407,7 @@ extern int __pmLogGenerateMark(__pmLogCtl *, int, pmResult **) _PCP_HIDDEN;
extern int __pmLogFetchInterp(__pmContext *, int, pmID *, pmResult **) _PCP_HIDDEN;
extern int __pmGetArchiveLabel(__pmLogCtl *, pmLogLabel *) _PCP_HIDDEN;
extern pmTimeval *__pmLogStartTime(__pmArchCtl *) _PCP_HIDDEN;
-extern void __pmLogSetTime(__pmContext *) _PCP_HIDDEN;
+extern int __pmLogSetTime(__pmContext *) _PCP_HIDDEN;
extern void __pmLogResetInterp(__pmContext *) _PCP_HIDDEN;
extern void __pmArchCtlFree(__pmArchCtl *) _PCP_HIDDEN;
extern int __pmLogChangeArchive(__pmContext *, int) _PCP_HIDDEN;
diff --git a/src/libpcp/src/logutil.c b/src/libpcp/src/logutil.c
index 0ef76de25..2ea559bfe 100644
--- a/src/libpcp/src/logutil.c
+++ b/src/libpcp/src/logutil.c
@@ -1995,7 +1995,6 @@ __pmLogFetch(__pmContext *ctxp, int numpmid, pmID pmidlist[], pmResult **result)
sts = __pmLogChangeVol(ctxp->c_archctl, ctxp->c_archctl->ac_vol);
if (sts < 0)
goto func_return;
- assert(ctxp->c_archctl->ac_mfp != NULL);
__pmFseek(ctxp->c_archctl->ac_mfp,
(long)ctxp->c_archctl->ac_offset, SEEK_SET);
@@ -2010,7 +2009,9 @@ more:
* no serial access, so need to make sure we are
* starting in the correct place
*/
- __pmLogSetTime(ctxp);
+ sts = __pmLogSetTime(ctxp);
+ if (sts < 0)
+ goto func_return;
ctxp->c_archctl->ac_offset = __pmFtell(ctxp->c_archctl->ac_mfp);
ctxp->c_archctl->ac_vol = ctxp->c_archctl->ac_curvol;
/*
@@ -2299,7 +2300,7 @@ VolSkip(__pmArchCtl *acp, int mode, int j)
return PM_ERR_EOL;
}
-void
+int
__pmLogSetTime(__pmContext *ctxp)
{
__pmArchCtl *acp = ctxp->c_archctl;
@@ -2356,6 +2357,7 @@ __pmLogSetTime(__pmContext *ctxp)
if (lcp->l_numti) {
/* we have a temporal index, use it! */
int j = -1;
+ int try;
int toobig = 0;
int match = 0;
int vol;
@@ -2406,9 +2408,13 @@ __pmLogSetTime(__pmContext *ctxp)
acp->ac_serial = 1;
if (match) {
+ try = j;
j = VolSkip(acp, mode, j);
- if (j < 0)
- return;
+ if (j < 0) {
+ if (pmDebugOptions.log)
+ fprintf(stderr, "__pmLogSetTime: VolSkip mode=%d vol=%d failed #1\n", mode, try);
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)lcp->l_ti[j].ti_log, SEEK_SET);
if (mode == PM_MODE_BACK)
acp->ac_serial = 0;
@@ -2418,9 +2424,13 @@ __pmLogSetTime(__pmContext *ctxp)
}
}
else if (j < 1) {
+ try = 0;
j = VolSkip(acp, PM_MODE_FORW, 0);
- if (j < 0)
- return;
+ if (j < 0) {
+ if (pmDebugOptions.log)
+ fprintf(stderr, "__pmLogSetTime: VolSkip mode=%d vol=%d failed #2\n", PM_MODE_FORW, try);
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)lcp->l_ti[j].ti_log, SEEK_SET);
if (pmDebugOptions.log) {
fprintf(stderr, " before start ti@");
@@ -2428,9 +2438,13 @@ __pmLogSetTime(__pmContext *ctxp)
}
}
else if (j == numti) {
+ try = numti-1;
j = VolSkip(acp, PM_MODE_BACK, numti-1);
- if (j < 0)
- return;
+ if (j < 0) {
+ if (pmDebugOptions.log)
+ fprintf(stderr, "__pmLogSetTime: VolSkip mode=%d vol=%d failed #3\n", PM_MODE_BACK, try);
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)lcp->l_ti[j].ti_log, SEEK_SET);
if (mode == PM_MODE_BACK)
acp->ac_serial = 0;
@@ -2450,9 +2464,13 @@ __pmLogSetTime(__pmContext *ctxp)
t_hi = __pmTimevalSub(&lcp->l_ti[j].ti_stamp, &ctxp->c_origin);
t_lo = __pmTimevalSub(&ctxp->c_origin, &lcp->l_ti[j-1].ti_stamp);
if (t_hi <= t_lo && !toobig) {
+ try = j;
j = VolSkip(acp, mode, j);
- if (j < 0)
- return;
+ if (j < 0) {
+ if (pmDebugOptions.log)
+ fprintf(stderr, "__pmLogSetTime: VolSkip mode=%d vol=%d failed #4\n", mode, try);
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)lcp->l_ti[j].ti_log, SEEK_SET);
if (mode == PM_MODE_FORW)
acp->ac_serial = 0;
@@ -2462,9 +2480,13 @@ __pmLogSetTime(__pmContext *ctxp)
}
}
else {
+ try = j-1;
j = VolSkip(acp, mode, j-1);
- if (j < 0)
- return;
+ if (j < 0) {
+ if (pmDebugOptions.log)
+ fprintf(stderr, "__pmLogSetTime: VolSkip mode=%d vol=%d failed #5\n", mode, try);
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)lcp->l_ti[j].ti_log, SEEK_SET);
if (mode == PM_MODE_BACK)
acp->ac_serial = 0;
@@ -2490,14 +2512,37 @@ __pmLogSetTime(__pmContext *ctxp)
}
else {
/* index either not available, or not useful */
+ int j;
if (mode == PM_MODE_FORW) {
- __pmLogChangeVol(acp, lcp->l_minvol);
- assert(acp->ac_mfp != NULL);
+ for (j = lcp->l_minvol; j <= lcp->l_maxvol; j++) {
+ if (__pmLogChangeVol(acp, j) >= 0)
+ break;
+ }
+ if (j > lcp->l_maxvol) {
+ /* no volume found */
+ if (pmDebugOptions.log)
+ fprintf(stderr, " index not useful, no volume between %d...%d\n",
+ lcp->l_minvol, lcp->l_maxvol);
+ acp->ac_curvol = -1;
+ acp->ac_mfp = NULL;
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)(sizeof(__pmLogLabel) + 2*sizeof(int)), SEEK_SET);
}
else if (mode == PM_MODE_BACK) {
- __pmLogChangeVol(acp, lcp->l_maxvol);
- assert(acp->ac_mfp != NULL);
+ for (j = lcp->l_maxvol; j >= lcp->l_minvol; j--) {
+ if (__pmLogChangeVol(acp, j) >= 0)
+ break;
+ }
+ if (j < lcp->l_minvol) {
+ /* no volume found */
+ if (pmDebugOptions.log)
+ fprintf(stderr, " index not useful, no volume between %d...%d\n",
+ lcp->l_maxvol, lcp->l_minvol);
+ acp->ac_curvol = -1;
+ acp->ac_mfp = NULL;
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)0, SEEK_END);
}
@@ -2513,6 +2558,8 @@ __pmLogSetTime(__pmContext *ctxp)
acp->ac_offset = __pmFtell(acp->ac_mfp);
assert(acp->ac_offset >= 0);
acp->ac_vol = acp->ac_curvol;
+
+ return 0;
}
/* Read the label of the current archive. */
@@ -3100,6 +3147,7 @@ LogChangeToPreviousArchive(__pmContext *ctxp)
pmTimeval save_origin;
int save_mode;
int sts;
+ int j;
/*
* Check whether there is a previous archive to switch to.
@@ -3145,12 +3193,23 @@ LogChangeToPreviousArchive(__pmContext *ctxp)
}
/* Set up to scan backwards from the end of the archive. */
- __pmLogChangeVol(acp, lcp->l_maxvol);
- assert(acp->ac_mfp != NULL);
+ for (j = lcp->l_maxvol; j >= lcp->l_minvol; j--) {
+ if (__pmLogChangeVol(acp, j) >= 0)
+ break;
+ }
+ if (j < lcp->l_minvol) {
+ /* no volume found */
+ if (pmDebugOptions.log)
+ fprintf(stderr, "LogChangeToPreviousArchive: no volume between %d...%d\n",
+ lcp->l_maxvol, lcp->l_minvol);
+ acp->ac_curvol = -1;
+ acp->ac_mfp = NULL;
+ return PM_ERR_LOGFILE;
+ }
__pmFseek(acp->ac_mfp, (long)0, SEEK_END);
- ctxp->c_archctl->ac_offset = __pmFtell(acp->ac_mfp);
- assert(ctxp->c_archctl->ac_offset >= 0);
- ctxp->c_archctl->ac_vol = ctxp->c_archctl->ac_curvol;
+ acp->ac_offset = __pmFtell(acp->ac_mfp);
+ assert(acp->ac_offset >= 0);
+ acp->ac_vol = acp->ac_curvol;
/*
* Check for temporal overlap here. Do this last in case the API client
commit 5e3b792d3d8ae60f2cebbd51c37b9b0722c3b26e
Author: Mark Goodwin <mgoodwin@redhat.com>
Date: Tue Jul 6 20:09:28 2021 +1000
libpcp_web: plug mem leak in redisMapInsert during daily log-rolling
When pmlogger_daily processes daily archives, the resulting
merged archive(s) are discovered and processed by pmproxy
(if the discovery module is enabled). Since the metadata and
logvol data in each merged archive is likely to have already
been previously processed (but discovery doesn't know this),
we see a lot of dict updates for existing keys and values that
are already mapped.
Static analysis by Coverity (CID323605 Resource Leak) shows
when redisMapInsert calls dictAdd for an existing key, the
new value field is assigned but the old value is not free'd,
and so it leaks.
Related: RHBZ1975069 and https://github.com/performancecopilot/pcp/issues/1318
diff --git a/src/libpcp_web/src/maps.c b/src/libpcp_web/src/maps.c
index 013ef02d3..ce20476c9 100644
--- a/src/libpcp_web/src/maps.c
+++ b/src/libpcp_web/src/maps.c
@@ -160,6 +160,12 @@ redisMapLookup(redisMap *map, sds key)
void
redisMapInsert(redisMap *map, sds key, sds value)
{
+ redisMapEntry *entry = redisMapLookup(map, key);
+
+ if (entry) {
+ /* fix for Coverity CID323605 Resource Leak */
+ dictDelete(map, key);
+ }
dictAdd(map, key, value);
}
commit 2a00a90b0bc3aecb8465fd32aef1ddbe745b2c91
Author: Mark Goodwin <mgoodwin@redhat.com>
Date: Tue Jul 6 20:43:01 2021 +1000
libpcp_web/discovery: improve lock handling and scalability
Rework the global log-rolling lock detection with finer grain
(per-pmlogger directory) detection, and break early in
process_meta() and process_logvol() if a lock file is found
in the same directory as a monitored archive. This is much
more scalable since archive directories that are not locked
can continue to be processed and ingested whilst log-rolling
progresses elsewhere. Also uses much less CPU time since we
don't need a traversal of all monitored archives looking for
locks on every fs change event.
Also improve process_logvol/meta handling for archives that
are deleted whilst being processed by the discovery module.
In conjunction with Kenj's changes in libpcp - stop processing
metadata and logvols if pmFetchArchive returns -ENOENT .. the
archive has been deleted so there is no point further ingesting
it's data.
Related: RHBZ1975069
Related: https://github.com/performancecopilot/pcp/issues/1338
diff --git a/src/libpcp_web/src/discover.c b/src/libpcp_web/src/discover.c
index 991055ce5..964813f66 100644
--- a/src/libpcp_web/src/discover.c
+++ b/src/libpcp_web/src/discover.c
@@ -33,9 +33,6 @@ static char *pmDiscoverFlagsStr(pmDiscover *);
#define PM_DISCOVER_HASHTAB_SIZE 32
static pmDiscover *discover_hashtable[PM_DISCOVER_HASHTAB_SIZE];
-/* pmlogger_daily log-roll lock count */
-static int logrolling = 0;
-
/* number of archives or directories currently being monitored */
static int n_monitored = 0;
@@ -426,28 +423,6 @@ is_deleted(pmDiscover *p, struct stat *sbuf)
return ret;
}
-static int
-check_for_locks()
-{
- int i;
- pmDiscover *p;
- char sep = pmPathSeparator();
- char path[MAXNAMELEN];
-
- for (i=0; i < PM_DISCOVER_HASHTAB_SIZE; i++) {
- for (p = discover_hashtable[i]; p; p = p->next) {
- if (p->flags & PM_DISCOVER_FLAGS_DIRECTORY) {
- pmsprintf(path, sizeof(path), "%s%c%s", p->context.name, sep, "lock");
- if (access(path, F_OK) == 0)
- return 1;
- }
- }
- }
-
- /* no locks */
- return 0;
-}
-
static void
check_deleted(pmDiscover *p)
{
@@ -465,37 +440,8 @@ fs_change_callBack(uv_fs_event_t *handle, const char *filename, int events, int
pmDiscover *p;
char *s;
sds path;
- int locksfound = 0;
struct stat statbuf;
- /*
- * check if logs are currently being rolled by pmlogger_daily et al
- * in any of the directories we are tracking. For mutex, the log control
- * scripts use a 'lock' file in each directory as it is processed.
- */
- locksfound = check_for_locks();
-
- if (!logrolling && locksfound) {
- /* log-rolling has started */
- if (pmDebugOptions.discovery)
- fprintf(stderr, "%s discovery callback: log-rolling in progress\n", stamp());
- logrolling = locksfound;
- return;
- }
-
- if (logrolling && locksfound) {
- logrolling = locksfound;
- return; /* still in progress */
- }
-
- if (logrolling && !locksfound) {
- /* log-rolling is finished: check what got deleted, and then purge */
- if (pmDebugOptions.discovery)
- fprintf(stderr, "%s discovery callback: finished log-rolling\n", stamp());
- pmDiscoverTraverse(PM_DISCOVER_FLAGS_META|PM_DISCOVER_FLAGS_DATAVOL, check_deleted);
- }
- logrolling = locksfound;
-
uv_fs_event_getpath(handle, buffer, &bytes);
path = sdsnewlen(buffer, bytes);
@@ -1037,6 +983,17 @@ pmDiscoverNewSource(pmDiscover *p, int context)
pmDiscoverInvokeSourceCallBacks(p, &timestamp);
}
+static char *
+archive_dir_lock_path(pmDiscover *p)
+{
+ char path[MAXNAMELEN], lockpath[MAXNAMELEN];
+ int sep = pmPathSeparator();
+
+ strncpy(path, p->context.name, sizeof(path)-1);
+ pmsprintf(lockpath, sizeof(lockpath), "%s%c%s", dirname(path), sep, "lock");
+ return strndup(lockpath, sizeof(lockpath));
+}
+
/*
* Process metadata records until EOF. That can span multiple
* callbacks if we get a partial record read.
@@ -1059,6 +1016,7 @@ process_metadata(pmDiscover *p)
__pmLogHdr hdr;
sds msg, source;
static uint32_t *buf = NULL;
+ char *lock_path;
int deleted;
struct stat sbuf;
static int buflen = 0;
@@ -1073,7 +1031,10 @@ process_metadata(pmDiscover *p)
fprintf(stderr, "process_metadata: %s in progress %s\n",
p->context.name, pmDiscoverFlagsStr(p));
pmDiscoverStatsAdd(p->module, "metadata.callbacks", NULL, 1);
+ lock_path = archive_dir_lock_path(p);
for (;;) {
+ if (lock_path && access(lock_path, F_OK) == 0)
+ break;
pmDiscoverStatsAdd(p->module, "metadata.loops", NULL, 1);
off = lseek(p->fd, 0, SEEK_CUR);
nb = read(p->fd, &hdr, sizeof(__pmLogHdr));
@@ -1240,6 +1201,9 @@ process_metadata(pmDiscover *p)
/* flag that all available metadata has now been read */
p->flags &= ~PM_DISCOVER_FLAGS_META_IN_PROGRESS;
+ if (lock_path)
+ free(lock_path);
+
if (pmDebugOptions.discovery)
fprintf(stderr, "%s: completed, partial=%d %s %s\n",
"process_metadata", partial, p->context.name, pmDiscoverFlagsStr(p));
@@ -1266,14 +1230,18 @@ static void
process_logvol(pmDiscover *p)
{
int sts;
- pmResult *r;
+ pmResult *r = NULL;
pmTimespec ts;
int oldcurvol;
__pmContext *ctxp;
__pmArchCtl *acp;
+ char *lock_path;
pmDiscoverStatsAdd(p->module, "logvol.callbacks", NULL, 1);
+ lock_path = archive_dir_lock_path(p);
for (;;) {
+ if (lock_path && access(lock_path, F_OK) == 0)
+ break;
pmDiscoverStatsAdd(p->module, "logvol.loops", NULL, 1);
pmUseContext(p->ctx);
ctxp = __pmHandleToPtr(p->ctx);
@@ -1312,6 +1280,7 @@ process_logvol(pmDiscover *p)
}
/* we are done - return and wait for another callback */
+ r = NULL;
break;
}
@@ -1328,14 +1297,15 @@ process_logvol(pmDiscover *p)
}
/*
- * TODO: persistently save current timestamp, so after being restarted,
- * pmproxy can resume where it left off for each archive.
+ * TODO (perhaps): persistently save current timestamp, so after being
+ * restarted, pmproxy can resume where it left off for each archive.
*/
ts.tv_sec = r->timestamp.tv_sec;
ts.tv_nsec = r->timestamp.tv_usec * 1000;
bump_logvol_decode_stats(p, r);
pmDiscoverInvokeValuesCallBack(p, &ts, r);
pmFreeResult(r);
+ r = NULL;
}
if (r) {
@@ -1348,6 +1318,9 @@ process_logvol(pmDiscover *p)
/* datavol is now up-to-date and at EOF */
p->flags &= ~PM_DISCOVER_FLAGS_DATAVOL_READY;
+
+ if (lock_path)
+ free(lock_path);
}
static void
@@ -1357,6 +1330,10 @@ pmDiscoverInvokeCallBacks(pmDiscover *p)
sds msg;
sds metaname;
+ check_deleted(p);
+ if (p->flags & PM_DISCOVER_FLAGS_DELETED)
+ return; /* ignore deleted archive */
+
if (p->ctx < 0) {
/*
* once off initialization on the first event
@@ -1366,16 +1343,23 @@ pmDiscoverInvokeCallBacks(pmDiscover *p)
/* create the PMAPI context (once off) */
if ((sts = pmNewContext(p->context.type, p->context.name)) < 0) {
- /*
- * Likely an early callback on a new (still empty) archive.
- * If so, just ignore the callback and don't log any scary
- * looking messages. We'll get another CB soon.
- */
- if (sts != PM_ERR_NODATA || pmDebugOptions.desperate) {
- infofmt(msg, "pmNewContext failed for %s: %s\n",
- p->context.name, pmErrStr(sts));
- moduleinfo(p->module, PMLOG_ERROR, msg, p->data);
+ if (sts == -ENOENT) {
+ /* newly deleted archive */
+ p->flags |= PM_DISCOVER_FLAGS_DELETED;
}
+ else {
+ /*
+ * Likely an early callback on a new (still empty) archive.
+ * If so, just ignore the callback and don't log any scary
+ * looking messages. We'll get another CB soon.
+ */
+ if (sts != PM_ERR_NODATA || pmDebugOptions.desperate) {
+ infofmt(msg, "pmNewContext failed for %s: %s\n",
+ p->context.name, pmErrStr(sts));
+ moduleinfo(p->module, PMLOG_ERROR, msg, p->data);
+ }
+ }
+ /* no further processing for this archive */
return;
}
pmDiscoverStatsAdd(p->module, "logvol.new_contexts", NULL, 1);
@@ -1410,8 +1394,12 @@ pmDiscoverInvokeCallBacks(pmDiscover *p)
metaname = sdsnew(p->context.name);
metaname = sdscat(metaname, ".meta");
if ((p->fd = open(metaname, O_RDONLY)) < 0) {
- infofmt(msg, "open failed for %s: %s\n", metaname, osstrerror());
- moduleinfo(p->module, PMLOG_ERROR, msg, p->data);
+ if (p->fd == -ENOENT)
+ p->flags |= PM_DISCOVER_FLAGS_DELETED;
+ else {
+ infofmt(msg, "open failed for %s: %s\n", metaname, osstrerror());
+ moduleinfo(p->module, PMLOG_ERROR, msg, p->data);
+ }
sdsfree(metaname);
return;
}

83
SOURCES/redhat-bugzilla-1991763.patch Normal file
View File

@ -0,0 +1,83 @@
commit 69c7d9bf5ac24bda51f8c876dc258bfe054b7cf8
Author: Sunil Mohan Adapa <sunil@medhas.org>
Date: Tue Feb 1 09:55:23 2022 -0800
pmlogger: zeroconf: Prioritize user configuration over zeroconf
In 2c17ba0cc16f58de511dff1e3122096d60c50bb0, zeroconf provided
defaults (/usr/share/pcp/zeroconf/pmlogger, which was actually
/etc/sysconf/pmlogger_zeroconf at the time of the change) were prioritized over
user configuration (/etc/sysconf/pmlogger). This lead to regression in clients
which edited the user configuration and expected the changes to be given
priority over zeroconf configuration. This was identified at least in
ansible-pcp[1].
Undo the changes in this commit so that the final priority is as follows:
User configuration (/etc/sysconfig/pmlogger)
(priority over)
Zeroconf defaults (/usr/share/pcp/zeroconf/pmlogger)
(priority over)
Code defaults (pmlogger.c)
Links:
1) https://github.com/performancecopilot/pcp/pull/1462#issuecomment-1022714960
Tests:
- Install pcp. Ensure pmlogger is running. Notice that there is no
PMLOGGER_INTERVAL set in the pmlogger daemon's environment.
- Install pcp-zeroconf. Restart pmlogger. Notice that PMLOGGER_INTERVAL
environment is set in the pmlogger daemon's environment. The value is 10.
- Edit /etc/sysconfig/pmlogger and set the value of PMLOGGER_INTERVAL to 15.
Restart pmlogger and notice that PMLOGGER_INTERVAL is set to 15 in pmlogger
daemon's environment.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
diff --git a/src/pmlogger/pmlogger.defaults b/src/pmlogger/pmlogger.defaults
index 1765760b5..1e96cd6ff 100644
--- a/src/pmlogger/pmlogger.defaults
+++ b/src/pmlogger/pmlogger.defaults
@@ -1,7 +1,7 @@
# Environment variables for the primary pmlogger daemon. See also
# the pmlogger control file and pmlogconf(1) for additional details.
-# Settings defined in this file will be overridden by any settings
-# in the pmlogger zeroconf file (if present).
+# Settings defined in this file will override any settings in the
+# pmlogger zeroconf file (if present).
# Behaviour regarding listening on external-facing interfaces;
# unset PMLOGGER_LOCAL to allow connections from remote hosts.
diff --git a/src/pmlogger/pmlogger.zeroconf b/src/pmlogger/pmlogger.zeroconf
index 9defc6e3d..fe86dedcc 100644
--- a/src/pmlogger/pmlogger.zeroconf
+++ b/src/pmlogger/pmlogger.zeroconf
@@ -1,6 +1,6 @@
#
-# PMLOGGER environment variables defined in this file take precedence over
-# the same variables defined in the standard pmlogger config file.
+# PMLOGGER environment variables defined in the standard pmlogger config file
+# take precedence over the same variables defined in this file.
#
# The PMLOGGER_INTERVAL setting affects the default primary pmlogger recording
# frequency. This only affects the *default* interval setting when specified
diff --git a/src/pmlogger/pmlogger_check.sh b/src/pmlogger/pmlogger_check.sh
index 6cc2a8ed7..64750cb5f 100755
--- a/src/pmlogger/pmlogger_check.sh
+++ b/src/pmlogger/pmlogger_check.sh
@@ -983,8 +983,8 @@ END { print m }'`
then
if [ "X$primary" = Xy ]
then
- # pcp-zeroconf environment variables (if present) take precedence
- envs=`grep -h ^PMLOGGER "$PMLOGGERENVS" "$PMLOGGERZEROCONFENVS" 2>/dev/null`
+ # User configuration takes precedence over pcp-zeroconf
+ envs=`grep -h ^PMLOGGER "$PMLOGGERZEROCONFENVS" "$PMLOGGERENVS" 2>/dev/null`
args="-P $args"
iam=" primary"
# clean up port-map, just in case

20
SOURCES/redhat-bugzilla-2003956-pmdabcc-update-kernel-version-check-due-to-backporting.patch Normal file
View File

@ -0,0 +1,20 @@
bcc included in RHEL 8.6 doesn't support the kernel_struct_has_field function.
The 4.18.x kernel in RHEL 8.6 did backport the `state` to `__state` rename (upstream:
change was in kernel v5.14+), and now we're in a situation where we can't test for
the existence of this kernel struct member and also can't rely on a kernel version check.
Therefore, let's patch it here for RHEL 8.x only:
diff --git a/src/pmdas/bcc/modules/runqlat.python b/src/pmdas/bcc/modules/runqlat.python
index 1c6c6b4b0..efc30e958 100644
--- a/src/pmdas/bcc/modules/runqlat.python
+++ b/src/pmdas/bcc/modules/runqlat.python
@@ -100,7 +100,7 @@ class PCPBCCModule(PCPBCCBase):
if (
hasattr(BPF, "kernel_struct_has_field")
and BPF.kernel_struct_has_field(b"task_struct", b"__state") == 1
- ) or self.kernel_version() >= (5, 14, 0):
+ ) or self.kernel_version() >= (4, 18, 0):
self.bpf_text = self.bpf_text.replace('STATE_FIELD', '__state')
else:
self.bpf_text = self.bpf_text.replace('STATE_FIELD', 'state')

1433
SOURCES/redhat-bugzilla-2003956.patch Normal file
View File

File diff suppressed because it is too large Load Diff

126
SOURCES/redhat-bugzilla-2017632.patch Normal file
View File

@ -0,0 +1,126 @@
commit 343aef25472baa74c3a4669cf86b670d93d41dcb
Author: Nathan Scott <nathans@redhat.com>
Date: Tue Nov 23 16:11:24 2021 +1100
pmlogconf: fix reprobing with interactive changes
In testing an earlier persistance fix it was discovered
that reprobe was inadvertently overwriting user requests
for changes interactively.
Related to Red Hat BZ #2017632
diff --git a/qa/368.out b/qa/368.out
index 829162f859..179a1142cf 100644
--- a/qa/368.out
+++ b/qa/368.out
@@ -80,44 +80,10 @@ Differences ...
Keep changes? [y]
Group: qa group one
Log this group? [y] ...
-Differences ...
-*** TMP.orig ...
---- TMP.conf.new ...
-***************
-*** 22,35 ****
- sample.long.million
- }
- #----
-! #+ 02:x::
-! #----
-! #+ 03:y:default:
-! ## qa group three
- log advisory on default {
-! sample.longlong.ten
- }
- #----
- #+ 04:n:default:
- ## qa group four
- #----
---- 22,36 ----
- sample.long.million
- }
- #----
-! #+ 02:y:default:
-! ## qa group two
- log advisory on default {
-! sample.ulong.ten
- }
- #----
-+ #+ 03:n:default:
-+ ## qa group three
-+ #----
- #+ 04:n:default:
- ## qa group four
- #----
-Keep changes? [y]
+No changes
+
Group: qa group one
-Log this group? [y] ......
+Log this group? [y] .....
Differences ...
*** TMP.orig ...
--- TMP.conf.new ...
@@ -132,18 +98,8 @@ Differences ...
#----
--- 18,23 ----
***************
-*** 26,31 ****
---- 25,31 ----
- ## qa group two
- log advisory on default {
- sample.ulong.ten
-+ sample.ulong.million
- }
- #----
- #+ 03:n:default:
-***************
-*** 37,42 ****
---- 37,53 ----
+*** 36,41 ****
+--- 35,51 ----
#+ 05:n:default:
## qa group five
#----
@@ -185,15 +141,13 @@ log advisory on default {
sample.long.million
}
#----
-#+ 02:y:default:
-## qa group two
-log advisory on default {
- sample.ulong.ten
- sample.ulong.million
-}
+#+ 02:x::
#----
-#+ 03:n:default:
+#+ 03:y:default:
## qa group three
+log advisory on default {
+ sample.longlong.ten
+}
#----
#+ 04:n:default:
## qa group four
diff --git a/src/pmlogconf/pmlogconf.c b/src/pmlogconf/pmlogconf.c
index c03c61e797..fa1156859d 100644
--- a/src/pmlogconf/pmlogconf.c
+++ b/src/pmlogconf/pmlogconf.c
@@ -913,13 +913,13 @@ evaluate_state(group_t *group)
if ((group->pmlogger || group->pmrep) && !group->pmlogconf) {
state = group->saved_state;
} else if (evaluate_group(group)) { /* probe */
- if (reprobe == 0 && group->saved_state != 0)
+ if (group->saved_state != 0)
state = group->saved_state;
else
state = group->true_state;
group->success = 1;
} else {
- if (reprobe == 0 && group->saved_state != 0)
+ if (group->saved_state != 0)
state = group->saved_state;
else
state = group->false_state;

841
SOURCES/redhat-bugzilla-2027753.patch Normal file
View File

@ -0,0 +1,841 @@
diff -Naurp pcp-5.3.5.orig/qa/032 pcp-5.3.5/qa/032
--- pcp-5.3.5.orig/qa/032 2021-11-01 13:02:26.000000000 +1100
+++ pcp-5.3.5/qa/032 2022-01-21 10:55:30.286602172 +1100
@@ -34,13 +34,6 @@ trap "_cleanup" 0 1 2 3 15
_stop_auto_restart pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_service pcp stop | _filter_pcp_stop
_wait_pmcd_end
_writable_primary_logger
@@ -48,7 +41,7 @@ _writable_primary_logger
_service pmcd start 2>&1 | _filter_pcp_start
_wait_for_pmcd
_service pmlogger start 2>&1 | _filter_pcp_start
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# real QA test starts here
_echo "expect this to be off"
diff -Naurp pcp-5.3.5.orig/qa/041 pcp-5.3.5/qa/041
--- pcp-5.3.5.orig/qa/041 2021-09-01 08:58:41.000000000 +1000
+++ pcp-5.3.5/qa/041 2022-01-21 10:55:30.286602172 +1100
@@ -40,12 +40,11 @@ _expect()
echo "" | tee -a $seq.full
}
+status=1
_needclean=true
TAG=000666000magic
-status=1
-[ -z "$PCP_PMLOGGERCONTROL_PATH" ] && \
- PCP_PMLOGGERCONTROL_PATH="$PCP_SYSCONF_DIR/pmlogger/control"
-
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
+rm -f $seq.full
trap "_cleanup" 0 1 2 3 15
_stop_auto_restart pmcd
@@ -68,15 +67,6 @@ _cleanup()
exit $status
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
-rm -f $seq.full
-
# real QA test starts here
# disable all pmloggers ...
diff -Naurp pcp-5.3.5.orig/qa/066 pcp-5.3.5/qa/066
--- pcp-5.3.5.orig/qa/066 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/066 2022-01-21 10:55:30.286602172 +1100
@@ -42,6 +42,7 @@ signal=$PCP_BINADM_DIR/pmsignal
log=$PCP_PMCDLOG_PATH
_needclean=true
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_filter_log()
{
@@ -80,13 +81,6 @@ interrupt()
exit
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
cleanup()
{
if [ $_needclean ]
diff -Naurp pcp-5.3.5.orig/qa/067 pcp-5.3.5/qa/067
--- pcp-5.3.5.orig/qa/067 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/067 2022-01-21 10:55:30.287602155 +1100
@@ -31,6 +31,7 @@ trap "rm -f $tmp.*; exit" 0 1 2 3 15
signal=$PCP_BINADM_DIR/pmsignal
log=$PCP_PMCDLOG_PATH
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_filter_log()
{
@@ -69,13 +70,6 @@ interrupt()
exit
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
cleanup()
{
_restore_config $PCP_PMCDCONF_PATH
diff -Naurp pcp-5.3.5.orig/qa/068 pcp-5.3.5/qa/068
--- pcp-5.3.5.orig/qa/068 2019-01-13 14:14:12.000000000 +1100
+++ pcp-5.3.5/qa/068 2022-01-21 10:55:30.287602155 +1100
@@ -19,22 +19,11 @@ echo "QA output created by $seq"
_needclean=true
LOCALHOST=`hostname`
-
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
trap "_cleanup" 0 1 2 3 15
-# don't need to regenerate config.default with pmlogconf
-#
-export PMLOGGER_CHECK_SKIP_LOGCONF=yes
-
_stop_auto_restart pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_cleanup()
{
echo >>$seq.full
diff -Naurp pcp-5.3.5.orig/qa/069 pcp-5.3.5/qa/069
--- pcp-5.3.5.orig/qa/069 2021-11-08 09:45:56.000000000 +1100
+++ pcp-5.3.5/qa/069 2022-01-21 10:55:30.287602155 +1100
@@ -45,6 +45,7 @@ nconfig=$tmp.pmcd.conf.new
log=$PCP_PMCDLOG_PATH
LOCALHOST=`hostname`
LOCALHOST_FULL=`pmhostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_needclean=true
rm -f $seq.full
@@ -104,13 +105,6 @@ skip > 0 { skip--; next }
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
cleanup()
{
if $_needclean
diff -Naurp pcp-5.3.5.orig/qa/1055 pcp-5.3.5/qa/1055
--- pcp-5.3.5.orig/qa/1055 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/1055 2022-01-21 10:55:30.287602155 +1100
@@ -17,6 +17,7 @@ signal=$PCP_BINADM_DIR/pmsignal
status=1
done_clean=false
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
@@ -66,13 +67,7 @@ $sudo cp $tmp.tmp $PCP_PMCDCONF_PATH
_writable_primary_logger
_service pcp restart 2>&1 | _filter_pcp_start
_wait_for_pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# Reset pmlogger
echo "log sample.dynamic.meta.metric"
diff -Naurp pcp-5.3.5.orig/qa/172 pcp-5.3.5/qa/172
--- pcp-5.3.5.orig/qa/172 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/172 2022-01-21 10:55:30.287602155 +1100
@@ -39,6 +39,7 @@ else
esac
fi
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
LOCALHOST=`hostname`
_needclean=true
status=0
@@ -48,13 +49,6 @@ _interrupt()
status=1
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_cleanup()
{
if $_needclean
diff -Naurp pcp-5.3.5.orig/qa/192 pcp-5.3.5/qa/192
--- pcp-5.3.5.orig/qa/192 2019-01-13 14:14:12.000000000 +1100
+++ pcp-5.3.5/qa/192 2022-01-21 10:55:30.287602155 +1100
@@ -13,6 +13,7 @@ echo "QA output created by $seq"
. ./common.filter
. ./common.check
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
LOCALHOST=`hostname`
_needclean=true
status=0
@@ -22,13 +23,6 @@ _interrupt()
status=1
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_cleanup()
{
if $_needclean
diff -Naurp pcp-5.3.5.orig/qa/220 pcp-5.3.5/qa/220
--- pcp-5.3.5.orig/qa/220 2021-07-13 09:34:38.000000000 +1000
+++ pcp-5.3.5/qa/220 2022-01-21 10:55:30.287602155 +1100
@@ -18,13 +18,7 @@ which netstat >/dev/null 2>&1 || _notrun
status=0
clean=false
LOCALHOST=`hostname`
-
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
diff -Naurp pcp-5.3.5.orig/qa/258 pcp-5.3.5/qa/258
--- pcp-5.3.5.orig/qa/258 2019-01-30 14:22:38.000000000 +1100
+++ pcp-5.3.5/qa/258 2022-01-21 10:55:30.287602155 +1100
@@ -19,6 +19,7 @@ echo "QA output created by $seq"
signal=$PCP_BINADM_DIR/pmsignal
status=1
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_needclean=true
_interrupt()
@@ -26,13 +27,6 @@ _interrupt()
status=1
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_cleanup()
{
cd $here
diff -Naurp pcp-5.3.5.orig/qa/279 pcp-5.3.5/qa/279
--- pcp-5.3.5.orig/qa/279 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/279 2022-01-21 10:55:30.287602155 +1100
@@ -17,6 +17,7 @@ echo "QA output created by $seq"
status=1 # failure is the default!
killer=`pwd`/src/killparent
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_needclean=true
rm -f $seq.full
@@ -76,13 +77,6 @@ _filter_pmcd()
# end
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_save_config $PCP_PMCDCONF_PATH
_disable_loggers
pmafm $LOGGING_DIR/$LOCALHOST/Latest remove >$tmp.cmd 2>&1 \
diff -Naurp pcp-5.3.5.orig/qa/280 pcp-5.3.5/qa/280
--- pcp-5.3.5.orig/qa/280 2021-08-16 14:12:25.000000000 +1000
+++ pcp-5.3.5/qa/280 2022-01-21 10:55:30.287602155 +1100
@@ -13,16 +13,6 @@ echo "QA output created by $seq"
. ./common.filter
. ./common.check
-if [ -d "$PCP_ARCHIVE_DIR" ]
-then
- LOGGING_DIR=$PCP_ARCHIVE_DIR
-elif [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_filter()
{
[ -z "$PCP_PMLOGGERCONTROL_PATH" ] && \
@@ -52,6 +42,7 @@ _filter()
status=1 # failure is the default!
signal=$PCP_BINADM_DIR/pmsignal
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
trap "_cleanup; $sudo rm -f $tmp.* $LOGGING_DIR/$LOCALHOST/lock; exit \$status" 0 1 2 3 15
_stop_auto_restart pmcd
diff -Naurp pcp-5.3.5.orig/qa/282 pcp-5.3.5/qa/282
--- pcp-5.3.5.orig/qa/282 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/282 2022-01-21 10:55:30.287602155 +1100
@@ -32,7 +32,7 @@ _needclean=true
sleepy=false
LOCALHOST=`hostname`
-LOGGING_DIR=$PCP_LOG_DIR/pmlogger
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
CHECK=`which pmlogger_check`
[ -z "$CHECK" -o ! -x "$CHECK" ] && \
_notrun "Cannot find an executable pmlogger_check: \"$CHECK\""
diff -Naurp pcp-5.3.5.orig/qa/336 pcp-5.3.5/qa/336
--- pcp-5.3.5.orig/qa/336 2018-10-23 07:37:45.000000000 +1100
+++ pcp-5.3.5/qa/336 2022-01-21 10:55:30.288602138 +1100
@@ -19,6 +19,7 @@ echo "QA output created by $seq"
signal=$PCP_BINADM_DIR/pmsignal
status=1 # failure is the default!
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_needclean=true
if [ -n "$PCP_TRACE_HOST" ]
@@ -31,13 +32,6 @@ _interrupt()
status=1
}
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-
_cleanup()
{
cd $here
diff -Naurp pcp-5.3.5.orig/qa/854 pcp-5.3.5/qa/854
--- pcp-5.3.5.orig/qa/854 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/854 2022-01-21 10:55:30.288602138 +1100
@@ -17,6 +17,7 @@ signal=$PCP_BINADM_DIR/pmsignal
status=1
done_clean=false
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
@@ -59,20 +60,14 @@ cat <<End-of-File >$tmp.tmp
# Installed by PCP QA test $seq on `date`
pmcd 2 dso pmcd_init $PMDA_PMCD_PATH
sample 29 pipe binary $PCP_PMDAS_DIR/sample/pmdasample -d 29
-simple 253 pipe binary /var/lib/pcp/pmdas/simple/pmdasimple -d 253
+simple 253 pipe binary $PCP_PMDAS_DIR/simple/pmdasimple -d 253
End-of-File
$sudo cp $tmp.tmp $PCP_PMCDCONF_PATH
_writable_primary_logger
_service pcp restart 2>&1 | _filter_pcp_start
_wait_for_pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# Reset pmlogger
echo "log sample.dynamic.meta.metric"
diff -Naurp pcp-5.3.5.orig/qa/856 pcp-5.3.5/qa/856
--- pcp-5.3.5.orig/qa/856 2019-01-13 14:14:12.000000000 +1100
+++ pcp-5.3.5/qa/856 2022-01-21 10:55:30.288602138 +1100
@@ -17,6 +17,7 @@ signal=$PCP_BINADM_DIR/pmsignal
status=1
done_clean=false
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
@@ -69,13 +70,7 @@ $sudo cp $tmp.tmp $PCP_PMCDCONF_PATH
_writable_primary_logger
_service pcp restart 2>&1 | _filter_pcp_start
_wait_for_pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# Reset pmlogger
echo "log sample.dynamic.meta.metric"
diff -Naurp pcp-5.3.5.orig/qa/882 pcp-5.3.5/qa/882
--- pcp-5.3.5.orig/qa/882 2019-01-13 14:14:12.000000000 +1100
+++ pcp-5.3.5/qa/882 2022-01-21 10:55:30.288602138 +1100
@@ -17,6 +17,7 @@ signal=$PCP_BINADM_DIR/pmsignal
status=1
done_clean=false
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
@@ -69,13 +70,7 @@ $sudo cp $tmp.tmp $PCP_PMCDCONF_PATH
_writable_primary_logger
_service pcp restart 2>&1 | _filter_pcp_start
_wait_for_pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# Reset pmlogger
echo "log sample.dynamic.meta.metric"
diff -Naurp pcp-5.3.5.orig/qa/932 pcp-5.3.5/qa/932
--- pcp-5.3.5.orig/qa/932 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/qa/932 2022-01-21 10:55:30.288602138 +1100
@@ -17,6 +17,7 @@ signal=$PCP_BINADM_DIR/pmsignal
status=1
done_clean=false
LOCALHOST=`hostname`
+LOGGING_DIR="$PCP_ARCHIVE_DIR"
_cleanup()
{
@@ -67,13 +68,7 @@ $sudo cp $tmp.tmp $PCP_PMCDCONF_PATH
_writable_primary_logger
_service pcp restart 2>&1 | _filter_pcp_start
_wait_for_pmcd
-if [ -d $PCP_LOG_DIR/pmlogger ]
-then
- LOGGING_DIR=$PCP_LOG_DIR/pmlogger
-else
- LOGGING_DIR=$PCP_LOG_DIR
-fi
-_wait_for_pmlogger -P $LOGGING_DIR/$LOCALHOST/pmlogger.log
+_wait_for_pmlogger
# Reset pmlogger
echo "log sample.dynamic.meta.metric"
diff -Naurp pcp-5.3.5.orig/qa/common.check pcp-5.3.5/qa/common.check
--- pcp-5.3.5.orig/qa/common.check 2021-11-05 09:54:05.000000000 +1100
+++ pcp-5.3.5/qa/common.check 2022-01-21 10:55:30.288602138 +1100
@@ -262,7 +262,7 @@ _service()
# good reason for this)
# ditto for pmlogger_daily and pmlogger_daily-poll
#
- for svc in pmlogger_check pmlogger_daily pmlogger_daily-poll
+ for svc in pmlogger_check pmlogger_daily pmlogger_farm_check pmlogger_daily-poll
do
if systemctl show --property=ActiveState $svc.timer 2>&1 \
| grep '=active$' >/dev/null
@@ -1940,9 +1940,11 @@ _remove_job_scheduler()
$rc_sudo rm -f $rc_cron_backup $rc_systemd_state
if systemctl cat pmie_daily.timer >/dev/null 2>&1; then
- for i in pmie.service pmie_daily.timer pmie_check.timer pmlogger_daily.timer \
+ for i in pmie.service pmie_daily.timer \
+ pmie_check.timer pmie_farm_check.timer \
pmlogger_daily_report.timer pmlogger_daily_report-poll.timer \
- pmlogger_daily-poll.timer pmlogger_check.timer ; do
+ pmlogger_daily-poll.timer pmlogger_daily.timer \
+ pmlogger_check.timer pmlogger_farm_check.timer ; do
$rc_sudo systemctl is-active "$i" > /dev/null || continue
$rc_sudo systemctl stop $i >/dev/null
echo "$i" >> $rc_systemd_state
diff -Naurp pcp-5.3.5.orig/src/pmie/GNUmakefile pcp-5.3.5/src/pmie/GNUmakefile
--- pcp-5.3.5.orig/src/pmie/GNUmakefile 2021-11-09 10:50:58.000000000 +1100
+++ pcp-5.3.5/src/pmie/GNUmakefile 2022-01-21 10:55:54.918186303 +1100
@@ -1,5 +1,5 @@
#
-# Copyright (c) 2013-2015,2020-2021 Red Hat.
+# Copyright (c) 2013-2015,2020-2022 Red Hat.
# Copyright (c) 2000,2004 Silicon Graphics, Inc. All Rights Reserved.
#
# This program is free software; you can redistribute it and/or modify it
@@ -84,7 +84,6 @@ pmie.service : pmie.service.in
pmie_farm.service : pmie_farm.service.in
$(SED) <$< >$@ \
- -e 's;@CRONTAB_PATH@;'$(CRONTAB_PATH)';' \
-e 's;@PCP_SYSCONFIG_DIR@;'$(PCP_SYSCONFIG_DIR)';' \
-e 's;@PCP_BINADM_DIR@;'$(PCP_BINADM_DIR)';' \
-e 's;@PCP_VAR_DIR@;'$(PCP_VAR_DIR)';' \
@@ -95,7 +94,10 @@ pmie_farm.service : pmie_farm.service.in
pmie_farm_check.service : pmie_farm_check.service.in
$(SED) <$< >$@ \
+ -e 's;@CRONTAB_PATH@;'$(CRONTAB_PATH)';' \
-e 's;@PCP_BIN_DIR@;'$(PCP_BIN_DIR)';' \
+ -e 's;@PCP_VAR_DIR@;'$(PCP_VAR_DIR)';' \
+ -e 's;@SD_SERVICE_TYPE@;'$(SD_SERVICE_TYPE)';' \
# END
pmie_check.service : pmie_check.service.in
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_check.service.in pcp-5.3.5/src/pmie/pmie_check.service.in
--- pcp-5.3.5.orig/src/pmie/pmie_check.service.in 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_check.service.in 2022-01-21 10:55:30.288602138 +1100
@@ -2,7 +2,6 @@
Description=Check PMIE instances are running
Documentation=man:pmie_check(1)
ConditionPathExists=!@CRONTAB_PATH@
-PartOf=pmie.service
[Service]
Type=@SD_SERVICE_TYPE@
@@ -15,6 +14,3 @@ ExecStart=@PCP_BINADM_DIR@/pmie_check $P
WorkingDirectory=@PCP_VAR_DIR@
Group=@PCP_GROUP@
User=@PCP_USER@
-
-[Install]
-RequiredBy=pmie.service
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_check.timer pcp-5.3.5/src/pmie/pmie_check.timer
--- pcp-5.3.5.orig/src/pmie/pmie_check.timer 2021-02-17 15:27:41.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_check.timer 2022-01-21 10:55:30.288602138 +1100
@@ -1,6 +1,5 @@
[Unit]
Description=Half-hourly check of PMIE instances
-PartOf=pmie.service
[Timer]
# if enabled, runs 1m after boot and every half hour
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_daily.service.in pcp-5.3.5/src/pmie/pmie_daily.service.in
--- pcp-5.3.5.orig/src/pmie/pmie_daily.service.in 2021-10-07 14:28:56.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_daily.service.in 2022-01-21 10:55:30.288602138 +1100
@@ -6,6 +6,7 @@ ConditionPathExists=!@CRONTAB_PATH@
[Service]
Type=@SD_SERVICE_TYPE@
Restart=no
+TimeoutStartSec=1h
Environment="PMIE_DAILY_PARAMS=-X xz -x 3"
EnvironmentFile=-@PCP_SYSCONFIG_DIR@/pmie_timers
ExecStart=@PCP_BINADM_DIR@/pmie_daily $PMIE_DAILY_PARAMS
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_daily.timer pcp-5.3.5/src/pmie/pmie_daily.timer
--- pcp-5.3.5.orig/src/pmie/pmie_daily.timer 2019-02-20 18:03:00.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_daily.timer 2022-01-21 10:55:30.288602138 +1100
@@ -2,6 +2,7 @@
Description=Daily processing of PMIE logs
[Timer]
+Persistent=true
OnCalendar=*-*-* 00:08:00
[Install]
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_farm_check.service.in pcp-5.3.5/src/pmie/pmie_farm_check.service.in
--- pcp-5.3.5.orig/src/pmie/pmie_farm_check.service.in 2021-11-09 10:50:58.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_farm_check.service.in 2022-01-21 10:55:54.918186303 +1100
@@ -1,19 +1,15 @@
[Unit]
-Description=Check and migrate non-primary pmie instances to pmie_farm
-Documentation=man:pmie_check(1)
-# TODO non-systemd ConditionPathExists=!/etc/cron.d/pcp-pmie
+Description=Check and migrate non-primary pmie farm instances
+Documentation=man:pmiectl(1)
+ConditionPathExists=!@CRONTAB_PATH@
[Service]
-Type=exec
+Type=@SD_SERVICE_TYPE@
Restart=no
TimeoutStartSec=4h
TimeoutStopSec=120
ExecStart=@PCP_BIN_DIR@/pmiectl -m check
-WorkingDirectory=/var/lib/pcp
-
+WorkingDirectory=@PCP_VAR_DIR@
# root so pmiectl can migrate pmie processes to the pmie_farm service
Group=root
User=root
-
-[Install]
-RequiredBy=pmie_farm.service
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_farm_check.timer pcp-5.3.5/src/pmie/pmie_farm_check.timer
--- pcp-5.3.5.orig/src/pmie/pmie_farm_check.timer 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_farm_check.timer 2022-01-21 10:49:07.985980678 +1100
@@ -1,10 +1,11 @@
[Unit]
-Description=5 minute check of pmie farm instances
+Description=Half-hourly check of pmie farm instances
[Timer]
-# if enabled, runs 1m after boot and every 5 mins
+# if enabled, runs 1m after boot and every half hour
OnBootSec=1min
-OnCalendar=*:00/5
+OnCalendar=*-*-* *:28:10
+OnCalendar=*-*-* *:58:10
[Install]
WantedBy=timers.target
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_farm.service.in pcp-5.3.5/src/pmie/pmie_farm.service.in
--- pcp-5.3.5.orig/src/pmie/pmie_farm.service.in 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_farm.service.in 2022-01-21 10:49:07.986980661 +1100
@@ -1,9 +1,9 @@
[Unit]
Description=pmie farm service
-Documentation=man:pmie(1)
-After=network-online.target pmcd.service
-Before=pmie_check.timer pmie_daily.timer
-BindsTo=pmie_farm_check.timer pmie_check.timer pmie_daily.timer
+Documentation=man:pmie_check(1)
+Before=pmie_farm_check.timer
+BindsTo=pmie_farm_check.timer
+PartOf=pmie.service
[Service]
Type=@SD_SERVICE_TYPE@
@@ -15,13 +15,9 @@ TimeoutStopSec=120
Environment="PMIE_CHECK_PARAMS=--skip-primary"
EnvironmentFile=-@PCP_SYSCONFIG_DIR@/pmie_timers
ExecStart=@PCP_BINADM_DIR@/pmie_farm $PMIE_CHECK_PARAMS
-
WorkingDirectory=@PCP_VAR_DIR@
Group=@PCP_GROUP@
User=@PCP_USER@
[Install]
-WantedBy=multi-user.target
-
-# This dependency will be removed in PCPv6.
-WantedBy=pmie.service
+RequiredBy=pmie.service
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie_farm.sh pcp-5.3.5/src/pmie/pmie_farm.sh
--- pcp-5.3.5.orig/src/pmie/pmie_farm.sh 2021-11-05 17:02:47.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie_farm.sh 2022-01-21 10:49:07.986980661 +1100
@@ -12,8 +12,8 @@
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
-# Administrative script to start the pmlogger_farm service.
-# All arguments to pmlogger_check are passed from pmlogger_farm.service.
+# Administrative script to start the pmie_farm service.
+# All arguments to pmie_check are passed from pmie_farm.service.
#
. $PCP_DIR/etc/pcp.env
diff -Naurp pcp-5.3.5.orig/src/pmie/pmie.service.in pcp-5.3.5/src/pmie/pmie.service.in
--- pcp-5.3.5.orig/src/pmie/pmie.service.in 2021-02-22 11:32:05.000000000 +1100
+++ pcp-5.3.5/src/pmie/pmie.service.in 2022-01-21 10:49:07.986980661 +1100
@@ -2,8 +2,8 @@
Description=Performance Metrics Inference Engine
Documentation=man:pmie(1)
After=network-online.target pmcd.service
-Before=pmie_check.timer pmie_daily.timer
-BindsTo=pmie_check.timer pmie_daily.timer
+Before=pmie_farm.service pmie_check.timer pmie_daily.timer
+BindsTo=pmie_farm.service pmie_check.timer pmie_daily.timer
Wants=pmcd.service
[Service]
diff -Naurp pcp-5.3.5.orig/src/pmlogger/GNUmakefile pcp-5.3.5/src/pmlogger/GNUmakefile
--- pcp-5.3.5.orig/src/pmlogger/GNUmakefile 2021-11-09 09:08:40.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/GNUmakefile 2022-01-21 10:55:54.918186303 +1100
@@ -1,5 +1,5 @@
#
-# Copyright (c) 2013-2021 Red Hat.
+# Copyright (c) 2013-2022 Red Hat.
# Copyright (c) 2000,2004 Silicon Graphics, Inc. All Rights Reserved.
#
# This program is free software; you can redistribute it and/or modify it
@@ -99,7 +99,6 @@ pmlogger.service : pmlogger.service.in
pmlogger_farm.service : pmlogger_farm.service.in
$(SED) <$< >$@ \
- -e 's;@CRONTAB_PATH@;'$(CRONTAB_PATH)';' \
-e 's;@PCP_SYSCONFIG_DIR@;'$(PCP_SYSCONFIG_DIR)';' \
-e 's;@PCP_BINADM_DIR@;'$(PCP_BINADM_DIR)';' \
-e 's;@PCP_VAR_DIR@;'$(PCP_VAR_DIR)';' \
@@ -110,7 +109,10 @@ pmlogger_farm.service : pmlogger_farm.se
pmlogger_farm_check.service : pmlogger_farm_check.service.in
$(SED) <$< >$@ \
+ -e 's;@CRONTAB_PATH@;'$(CRONTAB_PATH)';' \
-e 's;@PCP_BIN_DIR@;'$(PCP_BIN_DIR)';' \
+ -e 's;@PCP_VAR_DIR@;'$(PCP_VAR_DIR)';' \
+ -e 's;@SD_SERVICE_TYPE@;'$(SD_SERVICE_TYPE)';' \
# END
pmlogger_daily.service : pmlogger_daily.service.in
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger_check.service.in pcp-5.3.5/src/pmlogger/pmlogger_check.service.in
--- pcp-5.3.5.orig/src/pmlogger/pmlogger_check.service.in 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger_check.service.in 2022-01-21 10:55:30.289602121 +1100
@@ -15,6 +15,3 @@ ExecStart=@PCP_BINADM_DIR@/pmlogger_chec
WorkingDirectory=@PCP_VAR_DIR@
Group=@PCP_GROUP@
User=@PCP_USER@
-
-[Install]
-RequiredBy=pmlogger.service
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger_daily_report.service.in pcp-5.3.5/src/pmlogger/pmlogger_daily_report.service.in
--- pcp-5.3.5.orig/src/pmlogger/pmlogger_daily_report.service.in 2021-10-07 14:28:56.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger_daily_report.service.in 2022-01-21 10:55:30.289602121 +1100
@@ -13,6 +13,3 @@ ExecStart=@PCP_BINADM_DIR@/pmlogger_dail
WorkingDirectory=@PCP_VAR_DIR@
Group=@PCP_GROUP@
User=@PCP_USER@
-
-[Install]
-WantedBy=pmlogger.service
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger_farm_check.service.in pcp-5.3.5/src/pmlogger/pmlogger_farm_check.service.in
--- pcp-5.3.5.orig/src/pmlogger/pmlogger_farm_check.service.in 2021-11-09 09:08:40.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger_farm_check.service.in 2022-01-21 10:55:54.918186303 +1100
@@ -1,19 +1,15 @@
[Unit]
-Description=Check and migrate non-primary pmlogger instances to pmlogger_farm
-Documentation=man:pmlogger_check(1)
-# TODO non-systemd ConditionPathExists=!/etc/cron.d/pcp-pmlogger
+Description=Check and migrate non-primary pmlogger farm instances
+Documentation=man:pmlogctl(1)
+ConditionPathExists=!@CRONTAB_PATH@
[Service]
-Type=exec
+Type=@SD_SERVICE_TYPE@
Restart=no
TimeoutStartSec=4h
TimeoutStopSec=120
ExecStart=@PCP_BIN_DIR@/pmlogctl -m check
-WorkingDirectory=/var/lib/pcp
-
-# root so pmlogctl can migrate pmloggers to the pmlogger_farm service
+WorkingDirectory=@PCP_VAR_DIR@
+# root so pmlogctl can migrate pmlogger processes to the pmlogger_farm service
Group=root
User=root
-
-[Install]
-RequiredBy=pmlogger_farm.service
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger_farm_check.timer pcp-5.3.5/src/pmlogger/pmlogger_farm_check.timer
--- pcp-5.3.5.orig/src/pmlogger/pmlogger_farm_check.timer 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger_farm_check.timer 2022-01-21 10:49:07.986980661 +1100
@@ -1,10 +1,11 @@
[Unit]
-Description=5 minute check of pmlogger farm instances
+Description=Half-hourly check of pmlogger farm instances
[Timer]
-# if enabled, runs 1m after boot and every 5 mins
+# if enabled, runs 1m after boot and every half hour
OnBootSec=1min
-OnCalendar=*:00/5
+OnCalendar=*-*-* *:25:10
+OnCalendar=*-*-* *:55:10
[Install]
WantedBy=timers.target
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger_farm.service.in pcp-5.3.5/src/pmlogger/pmlogger_farm.service.in
--- pcp-5.3.5.orig/src/pmlogger/pmlogger_farm.service.in 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger_farm.service.in 2022-01-21 10:55:30.289602121 +1100
@@ -1,9 +1,9 @@
[Unit]
Description=pmlogger farm service
-Documentation=man:pmlogger(1)
-After=network-online.target pmcd.service
-Before=pmlogger_check.timer pmlogger_daily.timer
-BindsTo=pmlogger_farm_check.timer pmlogger_check.timer pmlogger_daily.timer
+Documentation=man:pmlogger_check(1)
+Before=pmlogger_farm_check.timer
+BindsTo=pmlogger_farm_check.timer
+PartOf=pmlogger.service
[Service]
Type=@SD_SERVICE_TYPE@
@@ -12,16 +12,12 @@ Restart=always
TimeoutStartSec=4h
TimeoutStopSec=120
# the pmlogger_farm service manages all pmloggers except the primary
-Environment="PMLOGGER_CHECK_PARAMS=--skip-primary"
+Environment="PMLOGGER_CHECK_PARAMS=--skip-primary --quick"
EnvironmentFile=-@PCP_SYSCONFIG_DIR@/pmlogger_timers
ExecStart=@PCP_BINADM_DIR@/pmlogger_farm $PMLOGGER_CHECK_PARAMS
-
WorkingDirectory=@PCP_VAR_DIR@
Group=@PCP_GROUP@
User=@PCP_USER@
[Install]
-WantedBy=multi-user.target
-
-# This dependency will be removed in PCPv6.
-WantedBy=pmlogger.service
+RequiredBy=pmlogger.service
diff -Naurp pcp-5.3.5.orig/src/pmlogger/pmlogger.service.in pcp-5.3.5/src/pmlogger/pmlogger.service.in
--- pcp-5.3.5.orig/src/pmlogger/pmlogger.service.in 2021-11-04 08:26:15.000000000 +1100
+++ pcp-5.3.5/src/pmlogger/pmlogger.service.in 2022-01-21 10:49:07.986980661 +1100
@@ -2,8 +2,8 @@
Description=Performance Metrics Archive Logger
Documentation=man:pmlogger(1)
After=network-online.target pmcd.service
-Before=pmlogger_check.timer pmlogger_daily.timer
-BindsTo=pmlogger_check.timer pmlogger_daily.timer
+Before=pmlogger_farm.service pmlogger_check.timer pmlogger_daily.timer
+BindsTo=pmlogger_farm.service pmlogger_check.timer pmlogger_daily.timer
Wants=pmcd.service
[Service]

38
SOURCES/redhat-bugzilla-2029301.patch Normal file
View File

@ -0,0 +1,38 @@
commit 9d9adc9d6c8eb24a6884da81c18b927ea706a68e
Author: Nathan Scott <nathans@redhat.com>
Date: Tue Dec 7 11:18:11 2021 +1100
pmdanvidia: fix mishandling of zero-byte size passed to realloc
Picked up during QA of recent nvidia changes - some hardware lacks
support for per-process metrics, or the hardware (GPU) has not yet
been accessed by a process using its resources, which had the side
effect that a zero-byte size argument was passed into realloc. In
turn, this passes back something that can be freed and an issue in
the logic meant this would happen on subsequent calls also.
Resolves the QA failure and Red Hat BZ #2029301
diff --git a/src/pmdas/nvidia/nvidia.c b/src/pmdas/nvidia/nvidia.c
index f1c12f2275..dc5bb93a0d 100644
--- a/src/pmdas/nvidia/nvidia.c
+++ b/src/pmdas/nvidia/nvidia.c
@@ -617,11 +617,16 @@ refresh(pcp_nvinfo_t *nvinfo, int need_processes)
/* update indoms, cull old entries that remain inactive */
if (need_processes) {
pmdaIndom *proc_indomp = &indomtab[PROC_INDOM];
- pmdaInstid *it_set = proc_indomp->it_set;
+ pmdaInstid *it_set = NULL;
size_t bytes = nproc * sizeof(pmdaInstid);
- if ((it_set = (pmdaInstid *)realloc(it_set, bytes)) == NULL)
+ if (bytes > 0) {
+ it_set = (pmdaInstid *)realloc(proc_indomp->it_set, bytes);
+ if (it_set == NULL)
+ free(proc_indomp->it_set);
+ } else if (proc_indomp->it_set != NULL) {
free(proc_indomp->it_set);
+ }
if ((proc_indomp->it_set = it_set) != NULL) {
for (i = j = 0; i < processes.hsize && j < nproc; i++) {

50
SOURCES/redhat-bugzilla-2030121.patch Normal file
View File

@ -0,0 +1,50 @@
commit df7b7bf64eb354114e6c519e3e03ffc446afa8ba
Author: Nathan Scott <nathans@redhat.com>
Date: Fri Nov 26 09:17:23 2021 +1100
libpcp_pmda: add indom cache fast-paths for inst lookup beyond max
We encountered a situation where indom cache loading consumed vast
CPU resources for an indom of size ~150k instances. Profiling was
used to identify the insert loop that ensures the inst linked list
within the cache hash tables is sorted - this loop is O(N*2) as we
potentially walk this list from the start on every insert during a
cache load. Because cache loading happens from a sorted file, the
worst-case scenario happened every time - each new instance insert
occurs beyond the current maximum. Fortunately we maintain a last
entry pointer, so the new fast path uses that first and falls back
to the original behaviour for an out-of-order insertion.
A second opportunity for the same optimization was identified when
auditing the rest of cache.c - in the find_inst() routine for inst
identifier lookups beyond the current maximum observed instance.
Resolves Red Hat BZ #2024648
diff --git a/src/libpcp_pmda/src/cache.c b/src/libpcp_pmda/src/cache.c
index 0e66506d74..196ffc1da9 100644
--- a/src/libpcp_pmda/src/cache.c
+++ b/src/libpcp_pmda/src/cache.c
@@ -328,6 +328,9 @@ find_inst(hdr_t *h, int inst)
{
entry_t *e;
+ if ((e = h->last) != NULL && e->inst < inst)
+ return NULL;
+
for (e = h->first; e != NULL; e = e->next) {
if (e->inst == inst && e->state != PMDA_CACHE_EMPTY)
break;
@@ -621,7 +624,11 @@ insert_cache(hdr_t *h, const char *name, int inst, int *sts)
*sts = PM_ERR_INST;
return e;
}
- for (e = h->first; e != NULL; e = e->next) {
+ /* if this entry is beyond the (sorted) list end, avoid linear scan */
+ if ((e = h->last) == NULL || e->inst > inst)
+ e = h->first;
+ /* linear search over linked list, starting at either first or last */
+ for (; e != NULL; e = e->next) {
if (e->inst < inst)
last_e = e;
else if (e->inst > inst)

742
SOURCES/redhat-bugzilla-2030140.patch Normal file
View File

@ -0,0 +1,742 @@
diff -Naurp pcp-5.3.5.orig/qa/1458 pcp-5.3.5/qa/1458
--- pcp-5.3.5.orig/qa/1458 1970-01-01 10:00:00.000000000 +1000
+++ pcp-5.3.5/qa/1458 2021-12-09 11:25:01.973327231 +1100
@@ -0,0 +1,219 @@
+#!/bin/sh
+# PCP QA Test No. 1458
+# Exercise access pmproxy with secure.enabled = false
+#
+# The main purpose of this is to test that the component works correctly
+# when secure.enabled = false; we can expect the https URLs to fail.
+#
+# See https://github.com/performancecopilot/pcp/issues/1490
+
+# Copyright (c) 2019,2021 Red Hat
+# Modified by Netflix, Inc.
+#
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+# get standard environment, filters and checks
+. ./common.product
+. ./common.filter
+. ./common.check
+
+_check_series # pmseries availability means libuv is in use
+_check_valgrind
+openssl help 2>/dev/null || _notrun "No openssl binary found"
+
+if [ -f /etc/lsb-release ]
+then
+ . /etc/lsb-release
+ if [ "$DISTRIB_ID" = Ubuntu ]
+ then
+ # This test fails for Ubuntu 19.10 with a myriad of errors involving
+ # the use of uninitialized values. The code paths very but typically
+ # involve libuv -> libssl -> libcrypto
+ #
+ case "$DISTRIB_RELEASE"
+ in
+ 19.10)
+ _notrun "problems with libuv, libssl, libcrypto and valgrind on Ubuntu $DISTRIB_RELEASE"
+ ;;
+ esac
+ fi
+fi
+
+_cleanup()
+{
+ cd $here
+ if $need_restore
+ then
+ need_restore=false
+ _restore_config $PCP_SYSCONF_DIR/labels
+ _sighup_pmcd
+ fi
+ $sudo rm -rf $tmp $tmp.*
+}
+
+status=1 # failure is the default!
+need_restore=false
+username=`id -u -n`
+$sudo rm -rf $tmp $tmp.* $seq.full
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_check_empty()
+{
+ tee -a $seq.full > $tmp.unfiltered
+ if [ -s $tmp.unfiltered ]
+ then
+ echo "Botch: got output from curl"
+ else
+ echo "Good!, empty output from curl"
+ fi
+}
+
+_filter_json()
+{
+ tee -a $seq.full > $tmp.unfiltered
+ if [ -s $tmp.unfiltered ]
+ then
+ pmjson < $tmp.unfiltered > $tmp.filtered
+ status=$?
+ if [ $status -eq 0 ]; then
+ cat $tmp.filtered | \
+ sed \
+ -e '/"machineid": .*/d' \
+ -e 's,"series": .*,"series": "SERIES",g' \
+ -e 's,"context": .*,"context": "CONTEXT",g' \
+ -e 's,"hostname": .*,"hostname": "HOSTNAME",g' \
+ -e 's,"domainname": .*,"domainname": "DOMAINNAME",g' \
+ #end
+ else
+ echo "Invalid JSON: $status"
+ cat $tmp.unfiltered
+ rm -f $tmp.context
+ fi
+ else
+ echo "Botch: no output from curl"
+ fi
+}
+
+_filter_port()
+{
+ sed \
+ -e '/ ipv6 /d' \
+ -e "s/ $port / PORT /g" \
+ #end
+}
+
+# real QA test starts here
+_save_config $PCP_SYSCONF_DIR/labels
+need_restore=true
+
+$sudo rm -rf $PCP_SYSCONF_DIR/labels/*
+_sighup_pmcd
+
+openssl req \
+ -new -newkey rsa:4096 -days 365 -nodes -x509 \
+ -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.pcpqa.com" \
+ -keyout $tmp.key -out $tmp.cert >$seq.full 2>&1
+# creates a self-signed (insecure) certificate, so for testing only
+
+echo "[pmproxy]" >> $tmp.conf
+echo "pcp.enabled = true" >> $tmp.conf
+echo "http.enabled = true" >> $tmp.conf
+echo "redis.enabled = false" >> $tmp.conf
+echo "secure.enabled = false" >> $tmp.conf
+
+port=`_find_free_port`
+mkdir -p $tmp.pmproxy/pmproxy
+export PCP_RUN_DIR=$tmp.pmproxy
+export PCP_TMP_DIR=$tmp.pmproxy
+
+$_valgrind_clean_assert pmproxy -f -l- --timeseries \
+ -c $tmp.conf -p $port -U $username \
+ >$tmp.valout 2>$tmp.valerr &
+pid=$!
+
+echo "valgrind pid: $pid" >>$seq.full
+echo "pmproxy port: $port" >>$seq.full
+
+# valgrind takes awhile to fire up
+i=0
+while [ $i -lt 40 ]
+do
+ $PCP_BINADM_DIR/telnet-probe -c localhost $port && break
+ sleep 1
+ i=`expr $i + 1`
+done
+if $PCP_BINADM_DIR/telnet-probe -c localhost $port
+then
+ echo "Startup took $i secs" >>$seq.full
+else
+ echo "Arrgh: valgrind failed start pmproxy and get port $port ready after 30 secs"
+ exit
+fi
+
+date >>$seq.full
+echo "=== checking serial http operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -Gs "http://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i
+done
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking parallel http operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -Gs "http://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i & 2>/dev/null eval pid$i=$!
+done
+wait $pid1 $pid2 $pid3 $pid4
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_filter_json < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking serial https/TLS operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -k -Gs "https://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i
+done
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_check_empty < $tmp.out$i
+done
+
+date >>$seq.full
+echo "=== checking parallel https/TLS operation ===" | tee -a $seq.full
+for i in 1 2 3 4; do
+ curl -k -Gs "https://localhost:$port/pmapi/metric?name=sample.long.ten" 2>$tmp.err$i >$tmp.out$i & 2>/dev/null eval pid$i=$!
+done
+wait $pid1 $pid2 $pid3 $pid4
+for i in 1 2 3 4; do
+echo === out$i === | tee -a $seq.full
+_check_empty < $tmp.out$i
+done
+
+echo "=== check pmproxy is running ==="
+pminfo -v -h localhost@localhost:$port hinv.ncpu
+if [ $? -eq 0 ]; then
+ echo "pmproxy check passed"
+else
+ echo "pmproxy check failed"
+fi
+
+# valgrind takes awhile to shutdown too
+pmsignal $pid >/dev/null 2>&1
+pmsleep 3.5
+echo "=== valgrind stdout ===" | tee -a $seq.full
+cat $tmp.valout | _filter_valgrind
+
+echo "=== valgrind stderr ===" | tee -a $seq.full
+cat $tmp.valerr | _filter_pmproxy_log | _filter_port
+
+# final kill if it's spinning
+$sudo kill -9 $pid >/dev/null 2>&1
+
+# success, all done
+status=0
+exit
diff -Naurp pcp-5.3.5.orig/qa/1458.out pcp-5.3.5/qa/1458.out
--- pcp-5.3.5.orig/qa/1458.out 1970-01-01 10:00:00.000000000 +1000
+++ pcp-5.3.5/qa/1458.out 2021-12-09 11:25:01.973327231 +1100
@@ -0,0 +1,221 @@
+QA output created by 1458
+=== checking serial http operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking parallel http operation ===
+=== out1 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out2 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out3 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== out4 ===
+{
+ "context": "CONTEXT"
+ "metrics": [
+ {
+ "name": "sample.long.ten",
+ "series": "SERIES"
+ "pmid": "29.0.11",
+ "type": "32",
+ "sem": "instant",
+ "units": "none",
+ "labels": {
+ "agent": "sample",
+ "cluster": "zero",
+ "domainname": "DOMAINNAME"
+ "hostname": "HOSTNAME"
+ "role": "testing"
+ },
+ "text-oneline": "10 as a 32-bit integer",
+ "text-help": "10 as a 32-bit integer"
+ }
+ ]
+}
+=== checking serial https/TLS operation ===
+=== out1 ===
+Good!, empty output from curl
+=== out2 ===
+Good!, empty output from curl
+=== out3 ===
+Good!, empty output from curl
+=== out4 ===
+Good!, empty output from curl
+=== checking parallel https/TLS operation ===
+=== out1 ===
+Good!, empty output from curl
+=== out2 ===
+Good!, empty output from curl
+=== out3 ===
+Good!, empty output from curl
+=== out4 ===
+Good!, empty output from curl
+=== check pmproxy is running ===
+pmproxy check passed
+=== valgrind stdout ===
+=== valgrind stderr ===
+Log for pmproxy on HOST started DATE
+
+pmproxy: PID = PID
+pmproxy request port(s):
+ sts fd port family address
+ === ==== ===== ====== =======
+ok FD unix UNIX_DOMAIN_SOCKET
+ok FD PORT inet INADDR_ANY
+[DATE] pmproxy(PID) Info: pmproxy caught SIGTERM
+[DATE] pmproxy(PID) Info: pmproxy Shutdown
+
+Log finished DATE
diff -Naurp pcp-5.3.5.orig/src/pmproxy/src/pcp.c pcp-5.3.5/src/pmproxy/src/pcp.c
--- pcp-5.3.5.orig/src/pmproxy/src/pcp.c 2021-09-24 09:33:06.000000000 +1000
+++ pcp-5.3.5/src/pmproxy/src/pcp.c 2021-12-09 11:22:09.829321418 +1100
@@ -1,6 +1,6 @@
/*
- * Copyright (c) 2018-2019 Red Hat.
- *
+ * Copyright (c) 2018-2019,2021 Red Hat.
+ *
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2.1 of the License, or
@@ -19,22 +19,13 @@
#define PDU_MAXLENGTH (MAXHOSTNAMELEN + HEADER_LENGTH + sizeof("65536")-1)
static void
-client_free(struct client *client)
-{
- if (client->u.pcp.hostname)
- sdsfree(client->u.pcp.hostname);
- if (client->buffer)
- sdsfree(client->buffer);
-}
-
-static void
on_server_close(uv_handle_t *handle)
{
struct client *client = (struct client *)handle;
if (pmDebugOptions.pdu)
fprintf(stderr, "client %p pmcd connection closed\n", client);
- client_free(client);
+ client_put(client);
}
static void
@@ -92,12 +83,11 @@ on_server_read(uv_stream_t *stream, ssiz
void
on_pcp_client_close(struct client *client)
{
- if (client->u.pcp.connected) {
+ if (client->u.pcp.connected)
uv_close((uv_handle_t *)&client->u.pcp.socket, on_server_close);
- memset(&client->u.pcp, 0, sizeof(client->u.pcp));
- } else {
- client_free(client);
- }
+ if (client->u.pcp.hostname)
+ sdsfree(client->u.pcp.hostname);
+ memset(&client->u.pcp, 0, sizeof(client->u.pcp));
}
static void
@@ -118,6 +108,8 @@ on_pcp_client_connect(uv_connect_t *conn
/* socket connection to pmcd successfully established */
client->u.pcp.state = PCP_PROXY_SETUP;
+ client->u.pcp.connected = 1;
+ client_get(client);
/* if we have already received PDUs, send them on now */
if ((buffer = client->buffer) != NULL) {
diff -Naurp pcp-5.3.5.orig/src/pmproxy/src/secure.c pcp-5.3.5/src/pmproxy/src/secure.c
--- pcp-5.3.5.orig/src/pmproxy/src/secure.c 2021-11-01 13:02:26.000000000 +1100
+++ pcp-5.3.5/src/pmproxy/src/secure.c 2021-12-09 11:22:09.831321384 +1100
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2019 Red Hat.
+ * Copyright (c) 2019,2021 Red Hat.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU Lesser General Public License as published
@@ -16,7 +16,7 @@
#include <openssl/opensslv.h>
#include <openssl/ssl.h>
-/* called with proxy->mutex locked */
+/* called with proxy->write_mutex locked */
static void
remove_connection_from_queue(struct client *client)
{
@@ -44,9 +44,9 @@ on_secure_client_close(struct client *cl
if (pmDebugOptions.auth || pmDebugOptions.http)
fprintf(stderr, "%s: client %p\n", "on_secure_client_close", client);
- uv_mutex_lock(&client->proxy->mutex);
+ uv_mutex_lock(&client->proxy->write_mutex);
remove_connection_from_queue(client);
- uv_mutex_unlock(&client->proxy->mutex);
+ uv_mutex_unlock(&client->proxy->write_mutex);
/* client->read and client->write freed by SSL_free */
SSL_free(client->secure.ssl);
}
@@ -63,7 +63,7 @@ maybe_flush_ssl(struct proxy *proxy, str
client->secure.pending.writes_count > 0)
return;
- uv_mutex_lock(&proxy->mutex);
+ uv_mutex_lock(&proxy->write_mutex);
if (proxy->pending_writes == NULL) {
proxy->pending_writes = client;
client->secure.pending.prev = client->secure.pending.next = NULL;
@@ -75,7 +75,7 @@ maybe_flush_ssl(struct proxy *proxy, str
client->secure.pending.prev = c;
}
client->secure.pending.queued = 1;
- uv_mutex_unlock(&proxy->mutex);
+ uv_mutex_unlock(&proxy->write_mutex);
}
static void
@@ -161,7 +161,7 @@ flush_secure_module(struct proxy *proxy)
size_t i, used;
int sts;
- uv_mutex_lock(&proxy->mutex);
+ uv_mutex_lock(&proxy->write_mutex);
head = &proxy->pending_writes;
while ((client = *head) != NULL) {
flush_ssl_buffer(client);
@@ -212,7 +212,7 @@ flush_secure_module(struct proxy *proxy)
sizeof(uv_buf_t) * client->secure.pending.writes_count);
}
}
- uv_mutex_unlock(&proxy->mutex);
+ uv_mutex_unlock(&proxy->write_mutex);
}
void
@@ -221,7 +221,8 @@ secure_client_write(struct client *clien
struct proxy *proxy = client->proxy;
uv_buf_t *dup;
size_t count, bytes;
- int i, sts, defer = 0, maybe = 0;
+ unsigned int i;
+ int sts, defer = 0, maybe = 0;
for (i = 0; i < request->nbuffers; i++) {
if (defer == 0) {
diff -Naurp pcp-5.3.5.orig/src/pmproxy/src/server.c pcp-5.3.5/src/pmproxy/src/server.c
--- pcp-5.3.5.orig/src/pmproxy/src/server.c 2021-11-01 13:02:26.000000000 +1100
+++ pcp-5.3.5/src/pmproxy/src/server.c 2021-12-09 11:22:09.831321384 +1100
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2018-2019 Red Hat.
+ * Copyright (c) 2018-2019,2021 Red Hat.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU Lesser General Public License as published
@@ -125,7 +125,7 @@ server_init(int portcount, const char *l
pmGetProgname());
return NULL;
}
- uv_mutex_init(&proxy->mutex);
+ uv_mutex_init(&proxy->write_mutex);
count = portcount + (*localpath ? 1 : 0);
if (count) {
@@ -229,7 +229,6 @@ void
client_put(struct client *client)
{
unsigned int refcount;
- struct proxy *proxy = client->proxy;
uv_mutex_lock(&client->mutex);
assert(client->refcount);
@@ -237,22 +236,16 @@ client_put(struct client *client)
uv_mutex_unlock(&client->mutex);
if (refcount == 0) {
- /* remove client from the doubly-linked list */
- uv_mutex_lock(&proxy->mutex);
- if (client->next != NULL)
- client->next->prev = client->prev;
- *client->prev = client->next;
- uv_mutex_unlock(&proxy->mutex);
-
if (client->protocol & STREAM_PCP)
on_pcp_client_close(client);
if (client->protocol & STREAM_HTTP)
on_http_client_close(client);
if (client->protocol & STREAM_REDIS)
on_redis_client_close(client);
- if (client->protocol & STREAM_SECURE)
+ if ((client->protocol & STREAM_SECURE) && client->stream.secure)
on_secure_client_close(client);
-
+ if (client->buffer)
+ sdsfree(client->buffer);
memset(client, 0, sizeof(*client));
free(client);
}
@@ -284,7 +277,7 @@ on_client_write(uv_write_t *writer, int
"on_client_write", status, client);
if (status == 0) {
- if (client->protocol & STREAM_SECURE)
+ if ((client->protocol & STREAM_SECURE) && client->stream.secure)
on_secure_client_write(client);
if (client->protocol & STREAM_PCP)
on_pcp_client_write(client);
@@ -434,10 +427,16 @@ on_client_read(uv_stream_t *stream, ssiz
if (nread > 0) {
if (client->protocol == STREAM_UNKNOWN)
client->protocol |= client_protocol(*buf->base);
- if (client->protocol & STREAM_SECURE)
+
+#ifdef HAVE_OPENSSL
+ if ((client->protocol & STREAM_SECURE) && (proxy->ssl != NULL))
on_secure_client_read(proxy, client, nread, buf);
else
on_protocol_read(stream, nread, buf);
+#else
+ on_protocol_read(stream, nread, buf);
+#endif
+
} else if (nread < 0) {
if (pmDebugOptions.af)
fprintf(stderr, "%s: read error %ld "
@@ -494,14 +493,6 @@ on_client_connection(uv_stream_t *stream
handle->data = (void *)proxy;
client->proxy = proxy;
- /* insert client into doubly-linked list at the head */
- uv_mutex_lock(&proxy->mutex);
- if ((client->next = proxy->first) != NULL)
- proxy->first->prev = &client->next;
- proxy->first = client;
- client->prev = &proxy->first;
- uv_mutex_unlock(&proxy->mutex);
-
status = uv_read_start((uv_stream_t *)&client->stream.u.tcp,
on_buffer_alloc, on_client_read);
if (status != 0) {
@@ -719,7 +710,7 @@ shutdown_ports(void *arg)
struct proxy *proxy = (struct proxy *)arg;
struct server *server;
struct stream *stream;
- int i;
+ unsigned int i;
for (i = 0; i < proxy->nservers; i++) {
server = &proxy->servers[i];
@@ -756,7 +747,8 @@ dump_request_ports(FILE *output, void *a
struct proxy *proxy = (struct proxy *)arg;
struct stream *stream;
uv_os_fd_t uv_fd;
- int i, fd;
+ unsigned int i;
+ int fd;
fprintf(output, "%s request port(s):\n"
" sts fd port family address\n"
diff -Naurp pcp-5.3.5.orig/src/pmproxy/src/server.h pcp-5.3.5/src/pmproxy/src/server.h
--- pcp-5.3.5.orig/src/pmproxy/src/server.h 2021-09-24 09:33:06.000000000 +1000
+++ pcp-5.3.5/src/pmproxy/src/server.h 2021-12-09 11:22:09.830321401 +1100
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2018-2019 Red Hat.
+ * Copyright (c) 2018-2019,2021 Red Hat.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU Lesser General Public License as published
@@ -97,11 +97,11 @@ typedef struct http_client {
typedef struct pcp_client {
pcp_proxy_state_t state;
- sds hostname;
unsigned int port : 16;
unsigned int certreq : 1;
unsigned int connected : 1;
unsigned int pad : 14;
+ sds hostname;
uv_connect_t pmcd;
uv_tcp_t socket;
} pcp_client_t;
@@ -136,8 +136,6 @@ typedef struct client {
pcp_client_t pcp;
} u;
struct proxy *proxy;
- struct client *next;
- struct client **prev;
sds buffer;
} client_t;
@@ -161,7 +159,7 @@ typedef struct proxy {
struct dict *config; /* configuration dictionary */
uv_loop_t *events; /* global, async event loop */
uv_callback_t write_callbacks;
- uv_mutex_t mutex; /* protects client lists and pending writes */
+ uv_mutex_t write_mutex; /* protects pending writes */
} proxy_t;
extern void proxylog(pmLogLevel, sds, void *);

283
SPECS/pcp.spec
View File

@ -1,6 +1,6 @@
Name: pcp
Version: 5.3.1
Release: 5%{?dist}
Version: 5.3.5
Release: 8%{?dist}
Summary: System-level performance monitoring and performance management
License: GPLv2+ and LGPLv2+ and CC-BY
URL: https://pcp.io
@ -8,11 +8,15 @@ URL: https://pcp.io
%global artifactory https://performancecopilot.jfrog.io/artifactory
Source0: %{artifactory}/pcp-source-release/pcp-%{version}.src.tar.gz
Patch000: redhat-bugzilla-1947989.patch
Patch001: redhat-bugzilla-1974266.patch
Patch002: redhat-bugzilla-1975069.patch
Patch003: redhat-bugzilla-1879350.patch
Patch004: redhat-bugzilla-1962019.patch
Patch0: redhat-bugzilla-2017632.patch
Patch1: redhat-bugzilla-2029301.patch
Patch2: redhat-bugzilla-2030121.patch
Patch3: redhat-bugzilla-2027753.patch
Patch4: redhat-bugzilla-2030140.patch
Patch5: redhat-bugzilla-2003956.patch
Patch6: redhat-bugzilla-2003956-pmdabcc-update-kernel-version-check-due-to-backporting.patch
Patch7: redhat-bugzilla-1973833.patch
Patch8: redhat-bugzilla-1991763.patch
%if 0%{?fedora} >= 26 || 0%{?rhel} > 7
%global __python2 python2
@ -96,6 +100,17 @@ Patch004: redhat-bugzilla-1962019.patch
%global disable_bcc 1
%endif
# support for pmdabpf, check bpf.spec for supported architectures of bpf
%if 0%{?fedora} >= 33 || 0%{?rhel} > 8
%ifarch x86_64 %{power64} aarch64 s390x
%global disable_bpf 0
%else
%global disable_bpf 1
%endif
%else
%global disable_bpf 1
%endif
# support for pmdabpftrace, check bpftrace.spec for supported architectures of bpftrace
%if 0%{?fedora} >= 30 || 0%{?rhel} > 7
%ifarch x86_64 %{power64} aarch64 s390x
@ -361,6 +376,12 @@ Requires: pcp-selinux = %{version}-%{release}
%global _with_bcc --with-pmdabcc=yes
%endif
%if %{disable_bpf}
%global _with_bpf --with-pmdabpf=no
%else
%global _with_bpf --with-pmdabpf=yes
%endif
%if %{disable_bpftrace}
%global _with_bpftrace --with-pmdabpftrace=no
%else
@ -503,11 +524,11 @@ Requires: pcp-pmda-activemq pcp-pmda-bonding pcp-pmda-dbping pcp-pmda-ds389 pcp-
Requires: pcp-pmda-elasticsearch pcp-pmda-gpfs pcp-pmda-gpsd pcp-pmda-lustre
Requires: pcp-pmda-memcache pcp-pmda-mysql pcp-pmda-named pcp-pmda-netfilter pcp-pmda-news
Requires: pcp-pmda-nginx pcp-pmda-nfsclient pcp-pmda-pdns pcp-pmda-postfix pcp-pmda-postgresql pcp-pmda-oracle
Requires: pcp-pmda-samba pcp-pmda-slurm pcp-pmda-vmware pcp-pmda-zimbra
Requires: pcp-pmda-samba pcp-pmda-slurm pcp-pmda-zimbra
Requires: pcp-pmda-dm pcp-pmda-apache
Requires: pcp-pmda-bash pcp-pmda-cisco pcp-pmda-gfs2 pcp-pmda-mailq pcp-pmda-mounts
Requires: pcp-pmda-nvidia-gpu pcp-pmda-roomtemp pcp-pmda-sendmail pcp-pmda-shping pcp-pmda-smart
Requires: pcp-pmda-hacluster pcp-pmda-lustrecomm pcp-pmda-logger pcp-pmda-docker pcp-pmda-bind2
Requires: pcp-pmda-hacluster pcp-pmda-lustrecomm pcp-pmda-logger pcp-pmda-denki pcp-pmda-docker pcp-pmda-bind2
Requires: pcp-pmda-sockets
%if !%{disable_podman}
Requires: pcp-pmda-podman
@ -521,6 +542,9 @@ Requires: pcp-pmda-nutcracker
%if !%{disable_bcc}
Requires: pcp-pmda-bcc
%endif
%if !%{disable_bpf}
Requires: pcp-pmda-bpf
%endif
%if !%{disable_bpftrace}
Requires: pcp-pmda-bpftrace
%endif
@ -528,7 +552,7 @@ Requires: pcp-pmda-bpftrace
Requires: pcp-pmda-gluster pcp-pmda-zswap pcp-pmda-unbound pcp-pmda-mic
Requires: pcp-pmda-libvirt pcp-pmda-lio pcp-pmda-openmetrics pcp-pmda-haproxy
Requires: pcp-pmda-lmsensors pcp-pmda-netcheck pcp-pmda-rabbitmq
Requires: pcp-pmda-openvswitch
Requires: pcp-pmda-openvswitch pcp-pmda-mongodb
%endif
%if !%{disable_mssql}
Requires: pcp-pmda-mssql
@ -540,9 +564,7 @@ Requires: pcp-pmda-snmp
Requires: pcp-pmda-json
%endif
Requires: pcp-pmda-summary pcp-pmda-trace pcp-pmda-weblog
%if !%{disable_python2} || !%{disable_python3}
Requires: pcp-system-tools
%endif
%if !%{disable_qt}
Requires: pcp-gui
%endif
@ -1110,6 +1132,20 @@ This package contains the PCP Performance Metrics Domain Agent (PMDA) for
collecting metrics about a GPS Daemon.
#end pcp-pmda-gpsd
#
# pcp-pmda-denki
#
%package pmda-denki
License: GPLv2+
Summary: Performance Co-Pilot (PCP) metrics dealing with electrical power
URL: https://pcp.io
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
%description pmda-denki
This package contains the PCP Performance Metrics Domain Agent (PMDA) for
collecting metrics related to the electrical power consumed by and inside
the system.
# end pcp-pmda-denki
#
# pcp-pmda-docker
#
@ -1373,21 +1409,6 @@ collecting metrics about SNMP.
#end pcp-pmda-snmp
%endif
#
# pcp-pmda-vmware
#
%package pmda-vmware
License: GPLv2+
Summary: Performance Co-Pilot (PCP) metrics for VMware
URL: https://pcp.io
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
Requires: perl-PCP-PMDA = %{version}-%{release}
%description pmda-vmware
This package contains the PCP Performance Metrics Domain Agent (PMDA) for
collecting metrics for VMware.
#end pcp-pmda-vmware
#
# pcp-pmda-zimbra
#
@ -1435,6 +1456,23 @@ extracting performance metrics from eBPF/BCC Python modules.
# end pcp-pmda-bcc
%endif
%if !%{disable_bpf}
#
# pcp-pmda-bpf
#
%package pmda-bpf
License: ASL 2.0 and GPLv2+
Summary: Performance Co-Pilot (PCP) metrics from eBPF ELF modules
URL: https://pcp.io
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
Requires: libbpf
BuildRequires: libbpf-devel clang llvm bpftool
%description pmda-bpf
This package contains the PCP Performance Metrics Domain Agent (PMDA) for
extracting performance metrics from eBPF ELF modules.
# end pcp-pmda-bpf
%endif
%if !%{disable_bpftrace}
#
# pcp-pmda-bpftrace
@ -1712,6 +1750,7 @@ BuildRequires: %{__python2}-requests
%endif
Obsoletes: pcp-pmda-prometheus < 5.0.0
Provides: pcp-pmda-prometheus < 5.0.0
Obsoletes: pcp-pmda-vmware < 5.3.5
%description pmda-openmetrics
This package contains the PCP Performance Metrics Domain Agent (PMDA) for
@ -1757,6 +1796,29 @@ This package contains the PCP Performance Metrics Domain Agent (PMDA) for
collecting metrics from simple network checks.
# end pcp-pmda-netcheck
#
# pcp-pmda-mongodb
#
%package pmda-mongodb
License: GPLv2+
Summary: Performance Co-Pilot (PCP) metrics for MongoDB
URL: https://pcp.io
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
%if !%{disable_python3}
Requires: python3-pcp
%if 0%{?rhel} == 0
Requires: python3-pymongo
%endif
%else
Requires: %{__python2}-pcp
%if 0%{?rhel} == 0
Requires: %{__python2}-pymongo
%endif
%endif
%description pmda-mongodb
This package contains the PCP Performance Metrics Domain Agent (PMDA) for
collecting metrics from MongoDB.
# end pcp-pmda-mongodb
%endif
%if !%{disable_mssql}
@ -2124,7 +2186,6 @@ Performance Metric API (PMAPI) monitor tools and Performance
Metric Domain Agent (PMDA) collector tools written in Python3.
%endif
%if !%{disable_python2} || !%{disable_python3}
#
# pcp-system-tools
#
@ -2132,23 +2193,24 @@ Metric Domain Agent (PMDA) collector tools written in Python3.
License: GPLv2+
Summary: Performance Co-Pilot (PCP) System and Monitoring Tools
URL: https://pcp.io
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
%if !%{disable_python2} || !%{disable_python3}
%if !%{disable_python3}
Requires: python3-pcp = %{version}-%{release}
%else
Requires: %{__python2}-pcp = %{version}-%{release}
%endif
Requires: pcp = %{version}-%{release} pcp-libs = %{version}-%{release}
%if !%{disable_dstat}
# https://fedoraproject.org/wiki/Packaging:Guidelines "Renaming/Replacing Existing Packages"
Provides: dstat = %{version}-%{release}
Provides: /usr/bin/dstat
Obsoletes: dstat <= 0.8
%endif
%endif
%description system-tools
This PCP module contains additional system monitoring tools written
in the Python language.
%endif
%if !%{disable_qt}
#
@ -2217,15 +2279,23 @@ interface rules, type enforcement and file context adjustments for an
updated policy package.
%endif
%prep
%setup -q
%patch000 -p1
%patch001 -p1
%patch002 -p1
%patch003 -p1
%patch004 -p1
%patch0 -p1
%patch1 -p1
%patch2 -p1
%patch3 -p1
%patch4 -p1
%patch5 -p1
%patch6 -p1
%patch7 -p1
%patch8 -p1
%build
# the buildsubdir macro gets defined in %setup and is apparently only available in the next step (i.e. the %build step)
%global __strip %{_builddir}/%{?buildsubdir}/build/rpm/custom-strip
# fix up build version
_build=`echo %{release} | sed -e 's/\..*$//'`
sed -i "/PACKAGE_BUILD/s/=[0-9]*/=$_build/" VERSION.pcp
@ -2233,7 +2303,7 @@ sed -i "/PACKAGE_BUILD/s/=[0-9]*/=$_build/" VERSION.pcp
%if !%{disable_python2} && 0%{?default_python} != 3
export PYTHON=python%{?default_python}
%endif
%configure %{?_with_initd} %{?_with_doc} %{?_with_dstat} %{?_with_ib} %{?_with_podman} %{?_with_statsd} %{?_with_perfevent} %{?_with_bcc} %{?_with_bpftrace} %{?_with_json} %{?_with_snmp} %{?_with_nutcracker} %{?_with_python2}
%configure %{?_with_initd} %{?_with_doc} %{?_with_dstat} %{?_with_ib} %{?_with_podman} %{?_with_statsd} %{?_with_perfevent} %{?_with_bcc} %{?_with_bpf} %{?_with_bpftrace} %{?_with_json} %{?_with_snmp} %{?_with_nutcracker} %{?_with_python2}
make %{?_smp_mflags} default_pcp
%install
@ -2365,12 +2435,12 @@ basic_manifest() {
# Likewise, for the pcp-pmda and pcp-testsuite subpackages.
#
total_manifest | keep 'tutorials|/html/|pcp-doc|man.*\.[1-9].*' | cull 'out' >pcp-doc-files
total_manifest | keep 'testsuite|etc/systemd/system' >pcp-testsuite-files
total_manifest | keep 'testsuite|etc/systemd/system|libpcp_fault|pcp/fault.h' >pcp-testsuite-files
basic_manifest | keep "$PCP_GUI|pcp-gui|applications|pixmaps|hicolor" | cull 'pmtime.h' >pcp-gui-files
basic_manifest | keep 'selinux' | cull 'tmp|GNUselinuxdefs' >pcp-selinux-files
basic_manifest | keep 'zeroconf|daily[-_]report|/sa$' >pcp-zeroconf-files
basic_manifest | grep -E -e 'pmiostat|pmrep|dstat|pcp2csv' \
basic_manifest | grep -E -e 'pmiostat|pmrep|dstat|htop|pcp2csv' \
-e 'pcp-atop|pcp-dmcache|pcp-dstat|pcp-free|pcp-htop' \
-e 'pcp-ipcs|pcp-iostat|pcp-lvmcache|pcp-mpstat' \
-e 'pcp-numastat|pcp-pidstat|pcp-shping|pcp-tapestat' \
@ -2398,11 +2468,13 @@ basic_manifest | keep '(etc/pcp|pmdas)/bash(/|$)' >pcp-pmda-bash-files
basic_manifest | keep '(etc/pcp|pmdas)/bcc(/|$)' >pcp-pmda-bcc-files
basic_manifest | keep '(etc/pcp|pmdas)/bind2(/|$)' >pcp-pmda-bind2-files
basic_manifest | keep '(etc/pcp|pmdas)/bonding(/|$)' >pcp-pmda-bonding-files
basic_manifest | keep '(etc/pcp|pmdas)/bpf(/|$)' >pcp-pmda-bpf-files
basic_manifest | keep '(etc/pcp|pmdas)/bpftrace(/|$)' >pcp-pmda-bpftrace-files
basic_manifest | keep '(etc/pcp|pmdas)/cifs(/|$)' >pcp-pmda-cifs-files
basic_manifest | keep '(etc/pcp|pmdas)/cisco(/|$)' >pcp-pmda-cisco-files
basic_manifest | keep '(etc/pcp|pmdas)/dbping(/|$)' >pcp-pmda-dbping-files
basic_manifest | keep '(etc/pcp|pmdas|pmieconf)/dm(/|$)' >pcp-pmda-dm-files
basic_manifest | keep '(etc/pcp|pmdas)/denki(/|$)' >pcp-pmda-denki-files
basic_manifest | keep '(etc/pcp|pmdas)/docker(/|$)' >pcp-pmda-docker-files
basic_manifest | keep '(etc/pcp|pmdas)/ds389log(/|$)' >pcp-pmda-ds389log-files
basic_manifest | keep '(etc/pcp|pmdas)/ds389(/|$)' >pcp-pmda-ds389-files
@ -2425,6 +2497,7 @@ basic_manifest | keep '(etc/pcp|pmdas)/memcache(/|$)' >pcp-pmda-memcache-files
basic_manifest | keep '(etc/pcp|pmdas)/mailq(/|$)' >pcp-pmda-mailq-files
basic_manifest | keep '(etc/pcp|pmdas)/mic(/|$)' >pcp-pmda-mic-files
basic_manifest | keep '(etc/pcp|pmdas)/mounts(/|$)' >pcp-pmda-mounts-files
basic_manifest | keep '(etc/pcp|pmdas)/mongodb(/|$)' >pcp-pmda-mongodb-files
basic_manifest | keep '(etc/pcp|pmdas|pmieconf)/mssql(/|$)' >pcp-pmda-mssql-files
basic_manifest | keep '(etc/pcp|pmdas)/mysql(/|$)' >pcp-pmda-mysql-files
basic_manifest | keep '(etc/pcp|pmdas)/named(/|$)' >pcp-pmda-named-files
@ -2461,23 +2534,22 @@ basic_manifest | keep '(etc/pcp|pmdas)/systemd(/|$)' >pcp-pmda-systemd-files
basic_manifest | keep '(etc/pcp|pmdas)/trace(/|$)' >pcp-pmda-trace-files
basic_manifest | keep '(etc/pcp|pmdas)/unbound(/|$)' >pcp-pmda-unbound-files
basic_manifest | keep '(etc/pcp|pmdas)/weblog(/|$)' >pcp-pmda-weblog-files
basic_manifest | keep '(etc/pcp|pmdas)/vmware(/|$)' >pcp-pmda-vmware-files
basic_manifest | keep '(etc/pcp|pmdas)/zimbra(/|$)' >pcp-pmda-zimbra-files
basic_manifest | keep '(etc/pcp|pmdas)/zswap(/|$)' >pcp-pmda-zswap-files
rm -f packages.list
for pmda_package in \
activemq apache \
bash bcc bind2 bonding bpftrace \
bash bcc bind2 bonding bpf bpftrace \
cifs cisco \
dbping docker dm ds389 ds389log \
dbping denki docker dm ds389 ds389log \
elasticsearch \
gfs2 gluster gpfs gpsd \
hacluster haproxy \
infiniband \
json \
libvirt lio lmsensors logger lustre lustrecomm \
mailq memcache mic mounts mssql mysql \
mailq memcache mic mounts mongodb mssql mysql \
named netcheck netfilter news nfsclient nginx \
nutcracker nvidia \
openmetrics openvswitch oracle \
@ -2487,7 +2559,6 @@ for pmda_package in \
sockets statsd summary systemd \
unbound \
trace \
vmware \
weblog \
zimbra zswap ; \
do \
@ -2622,10 +2693,8 @@ exit 0
chown -R pcpqa:pcpqa %{_testsdir} 2>/dev/null
%if 0%{?rhel}
%if !%{disable_systemd}
systemctl restart pmcd >/dev/null 2>&1
systemctl restart pmlogger >/dev/null 2>&1
systemctl enable pmcd >/dev/null 2>&1
systemctl enable pmlogger >/dev/null 2>&1
systemctl restart pmcd pmlogger >/dev/null 2>&1
systemctl enable pmcd pmlogger >/dev/null 2>&1
%else
/sbin/chkconfig --add pmcd >/dev/null 2>&1
/sbin/chkconfig --add pmlogger >/dev/null 2>&1
@ -2712,6 +2781,9 @@ exit 0
%preun pmda-dbping
%{pmda_remove "$1" "dbping"}
%preun pmda-denki
%{pmda_remove "$1" "denki"}
%preun pmda-docker
%{pmda_remove "$1" "docker"}
@ -2771,9 +2843,6 @@ exit 0
%preun pmda-samba
%{pmda_remove "$1" "samba"}
%preun pmda-vmware
%{pmda_remove "$1" "vmware"}
%preun pmda-zimbra
%{pmda_remove "$1" "zimbra"}
@ -2785,6 +2854,11 @@ exit 0
%{pmda_remove "$1" "bcc"}
%endif
%if !%{disable_bpf}
%preun pmda-bpf
%{pmda_remove "$1" "bpf"}
%endif
%if !%{disable_bpftrace}
%preun pmda-bpftrace
%{pmda_remove "$1" "bpftrace"}
@ -2812,6 +2886,9 @@ exit 0
%preun pmda-lmsensors
%{pmda_remove "$1" "lmsensors"}
%preun pmda-mongodb
%{pmda_remove "$1" "mongodb"}
%if !%{disable_mssql}
%preun pmda-mssql
%{pmda_remove "$1" "mssql"}
@ -2880,10 +2957,7 @@ exit 0
%preun zeroconf
if [ "$1" -eq 0 ]
then
%systemd_preun pmlogger_daily_report.timer
%systemd_preun pmlogger_daily_report.service
%systemd_preun pmlogger_daily_report-poll.timer
%systemd_preun pmlogger_daily_report-poll.service
%systemd_preun pmlogger_daily_report.timer pmlogger_daily_report.service
fi
%endif
@ -2892,19 +2966,9 @@ if [ "$1" -eq 0 ]
then
# stop daemons before erasing the package
%if !%{disable_systemd}
%systemd_preun pmlogger.service
%systemd_preun pmie.service
%systemd_preun pmproxy.service
%systemd_preun pmcd.service
%systemd_preun pmie_daily.timer
%systemd_preun pmlogger_daily.timer
%systemd_preun pmlogger_daily-poll.timer
%systemd_preun pmlogger_check.timer
%systemd_preun pmlogger_check.timer pmlogger_daily.timer pmlogger_farm_check.timer pmlogger_farm_check.service pmlogger_farm.service pmlogger.service pmie_check.timer pmie_daily.timer pmie_farm_check.timer pmie_farm_check.service pmie_farm.service pmie.service pmproxy.service pmfind.service pmcd.service
systemctl stop pmlogger.service >/dev/null 2>&1
systemctl stop pmie.service >/dev/null 2>&1
systemctl stop pmproxy.service >/dev/null 2>&1
systemctl stop pmcd.service >/dev/null 2>&1
systemctl stop pmlogger.service pmie.service pmproxy.service pmfind.service pmcd.service >/dev/null 2>&1
%else
/sbin/service pmlogger stop >/dev/null 2>&1
/sbin/service pmie stop >/dev/null 2>&1
@ -2933,18 +2997,12 @@ for PMDA in dm nfsclient openmetrics ; do
%{install_file "$PCP_PMDAS_DIR/$PMDA" .NeedInstall}
fi
done
# increase default pmlogger recording frequency
sed -i 's/^\#\ PMLOGGER_INTERVAL.*/PMLOGGER_INTERVAL=10/g' "$PCP_SYSCONFIG_DIR/pmlogger"
# auto-enable these usually optional pmie rules
pmieconf -c enable dmthin
%if 0%{?rhel}
%if !%{disable_systemd}
systemctl restart pmcd >/dev/null 2>&1
systemctl restart pmlogger >/dev/null 2>&1
systemctl restart pmie >/dev/null 2>&1
systemctl enable pmcd >/dev/null 2>&1
systemctl enable pmlogger >/dev/null 2>&1
systemctl enable pmie >/dev/null 2>&1
systemctl restart pmcd pmlogger pmie >/dev/null 2>&1
systemctl enable pmcd pmlogger pmie >/dev/null 2>&1
%else
/sbin/chkconfig --add pmcd >/dev/null 2>&1
/sbin/chkconfig --add pmlogger >/dev/null 2>&1
@ -2972,13 +3030,18 @@ PCP_LOG_DIR=%{_logsdir}
%{install_file "$PCP_PMNS_DIR" .NeedRebuild}
%{install_file "$PCP_LOG_DIR/pmlogger" .NeedRewrite}
%if !%{disable_systemd}
# clean up any stale symlinks for deprecated pm*-poll services
rm -f %{_sysconfdir}/systemd/system/pm*.requires/pm*-poll.* >/dev/null 2>&1 || true
%systemd_postun_with_restart pmcd.service
%systemd_post pmcd.service
%systemd_postun_with_restart pmlogger.service
%systemd_post pmlogger.service
%systemd_postun_with_restart pmie.service
%systemd_post pmie.service
systemctl condrestart pmproxy.service >/dev/null 2>&1
%systemd_postun_with_restart pmproxy.service
%systemd_post pmproxy.service
%systemd_post pmfind.service
%else
/sbin/chkconfig --add pmcd >/dev/null 2>&1
/sbin/service pmcd condrestart
@ -3098,11 +3161,11 @@ PCP_LOG_DIR=%{_logsdir}
%files pmda-slurm -f pcp-pmda-slurm-files.rpm
%files pmda-vmware -f pcp-pmda-vmware-files.rpm
%files pmda-zimbra -f pcp-pmda-zimbra-files.rpm
%endif
%files pmda-denki -f pcp-pmda-denki-files.rpm
%files pmda-docker -f pcp-pmda-docker-files.rpm
%files pmda-lustrecomm -f pcp-pmda-lustrecomm-files.rpm
@ -3135,6 +3198,10 @@ PCP_LOG_DIR=%{_logsdir}
%files pmda-bcc -f pcp-pmda-bcc-files.rpm
%endif
%if !%{disable_bpf}
%files pmda-bpf -f pcp-pmda-bpf-files.rpm
%endif
%if !%{disable_bpftrace}
%files pmda-bpftrace -f pcp-pmda-bpftrace-files.rpm
%endif
@ -3152,6 +3219,8 @@ PCP_LOG_DIR=%{_logsdir}
%files pmda-lmsensors -f pcp-pmda-lmsensors-files.rpm
%files pmda-mongodb -f pcp-pmda-mongodb-files.rpm
%if !%{disable_mssql}
%files pmda-mssql -f pcp-pmda-mssql-files.rpm
%endif
@ -3274,13 +3343,69 @@ PCP_LOG_DIR=%{_logsdir}
%files -n python3-pcp -f python3-pcp.list.rpm
%endif
%if !%{disable_python2} || !%{disable_python3}
%files system-tools -f pcp-system-tools-files.rpm
%endif
%files zeroconf -f pcp-zeroconf-files.rpm
%changelog
* Wed Feb 02 2022 Nathan Scott <nathans@redhat.com> - 5.3.5-8
- Fix pcp-zeroconf logger interval override regression (BZ 1991763)
- Remove warnings from spec setup of PCP systemd units (BZ 2048024)
* Thu Dec 16 2021 Andreas Gerstmayr <agerstmayr@redhat.com> - 5.3.5-6
- pmdabcc: update qa/1118 testcase to match new output (BZ 2003956)
* Wed Dec 15 2021 Nathan Scott <nathans@redhat.com> - 5.3.5-4
- pmdabcc: resolve compilation issues of some bcc PMDA modules on
aarch64, ppc64le and s390x (BZ 2003956)
- Further improve pmlogger service startup latency (BZ 1973833)
- Additional improvements to farm systemd services (BZ 2027753)
* Thu Dec 09 2021 Nathan Scott <nathans@redhat.com> - 5.3.5-3
- Resolve failure in the Nvidia metrics agent (BZ 2029301)
- PMDA indom cache loading performance improvements (BZ 2030121)
- Consistent user experience for new farm services (BZ 2027753)
- Resilience improvements for the pmproxy service (BZ 2030140)
* Fri Nov 26 2021 Nathan Scott <nathans@redhat.com> - 5.3.5-2
- Updates to pmlogconf persistence changes (BZ 2017632)
* Wed Nov 10 2021 Nathan Scott <nathans@redhat.com> - 5.3.5-1
- Extend pmlogger(1) man page --interval option (BZ 2018083)
- Disable Avahi service advertisement by default (BZ 1899625)
- Use separate localhost and farm service cgroups (BZ 1991896)
- Update and extend the Nvidia GPU metric coverage (BZ 1690590)
- Assessment API and pmdamssql(1) share credentials (BZ 1951342)
- Improve remote pmlogger over slow network connections (BZ 1973833)
- Implement pcp-atop(1) support for Nvidia GPU reports (BZ 1984273)
- Drop DMA[32] zones when the zone is not in node 0 (BZ 1985519)
- Support kernel changes to /proc/zoneinfo metrics (BZ 1985523)
- Ensure pmlogconf persists configuration changes (BZ 2017632)
- Auto-reconnect lost redis connection in pmproxy (BZ 1989287)
- Rebase to a more recent upstream version of PCP (BZ 1991763)
* Fri Oct 15 2021 Mark Goodwin <mgoodwin@redhat.com> - 5.3.4-2
- Fix pmlogger manual start with service disabled (BZ 2018011)
* Fri Oct 08 2021 Nathan Scott <nathans@redhat.com> - 5.3.4-1
- Add pcp-atop(1) support for 'curscal' values (BZ 1984271)
- Fix pcp-atop(1) reporting perfevent 'ipc' values (BZ 1986264)
- Fix pmlogger(1) exiting on receipt of SIGALRM (BZ 2004771)
- Fix values of some hacluster metrics on s390x (BZ 2008298)
- Add new pmdads389(1) metrics for replication (BZ 1966122)
- Rebase to a more recent upstream version of PCP (BZ 1991763)
* Wed Sep 15 2021 Nathan Scott <nathans@redhat.com> - 5.3.3-1
- Add new pmdads389(1) metrics for replication (BZ 1966122)
- Add label support to pmdahacluster(1) metrics (BZ 1972277)
- Improve pmieconf rules for saturated processors (BZ 1994680)
- Auto-upgrade pmcd.conf for python2 to python3 (BZ 1988403)
- Document unloading pmdakvm(1) for kernel module (BZ 1977740)
- Add option to pmdalinux(1) for slabinfo metrics (BZ 1962902)
- Fix for OpenMetrics scripting generating AVCs (BZ 1985818)
- Resolve some pcp-testsuite failures (BZ 1980459, 1981686)
- Rebase to a more recent upstream version of PCP (BZ 1991763)
* Sat Aug 28 2021 Nathan Scott <nathans@redhat.com> - 5.3.1-5
- Fix pmdapodman initialization and selinux policy (BZ 1962019)