rpms/glusterfs
5
0
Fork 0
Code Issues Pull Requests Releases Activity

autobuild v6.0-33

Browse Source 
Resolves: bz#1812789 bz#1813917 bz#1823703 bz#1823706 bz#1825195
Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
This commit is contained in:
Rinku Kothiya 2020-04-29 05:33:35 -04:00
parent 44ca233177
commit 7aeaa66127
6 changed files with 388 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
0363-common-ha-cluster-status-shows-FAILOVER-when-actuall.patch Normal file
View File

@ -0,0 +1,47 @@
From d7c0dc7107a024d28196a4582bacf28ddcfbeb69 Mon Sep 17 00:00:00 2001
From: "Kaleb S. KEITHLEY" <kkeithle@redhat.com>
Date: Tue, 14 Apr 2020 07:59:22 -0400
Subject: [PATCH 363/367] common-ha: cluster status shows "FAILOVER" when
actually HEALTHY
pacemaker devs change the format of the ouput of `pcs status`
Expected to find a line in the format:
Online: ....
but now it's
* Online: ...
And the `grep -E "^Online:" no longer finds the list of nodes that
are online.
https://review.gluster.org/#/c/glusterfs/+/24333/
Change-Id: If2aa1e7b53c766c625d7b4cc222a83ea2c0bd72d
BUG: 1823706
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/197367
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
extras/ganesha/scripts/ganesha-ha.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/extras/ganesha/scripts/ganesha-ha.sh b/extras/ganesha/scripts/ganesha-ha.sh
index 0b0050a..df333a1 100644
--- a/extras/ganesha/scripts/ganesha-ha.sh
+++ b/extras/ganesha/scripts/ganesha-ha.sh
@@ -935,7 +935,7 @@ status()
done
# print the nodes that are expected to be online
- grep -E "^Online:" ${scratch}
+ grep -E "Online:" ${scratch}
echo
--
1.8.3.1

97
0364-dht-fixing-rebalance-failures-for-files-with-holes.patch Normal file
View File

@ -0,0 +1,97 @@
From 5b1bfebacac649e6f5051316e4075309caf93901 Mon Sep 17 00:00:00 2001
From: Barak Sason Rofman <bsasonro@redhat.com>
Date: Tue, 21 Apr 2020 19:13:41 +0300
Subject: [PATCH 364/367] dht - fixing rebalance failures for files with holes
Rebalance process handling of files which contains holes casued
rebalance to fail with "No space left on device" errors.
This patch modifies the code-flow in such a way that files with holes
will be rebalanced correctly.
backport of https://review.gluster.org/#/c/glusterfs/+/24357/
>fixes: #1187
>Change-Id: I89bc3d4ea7f074db7213d759c49307f379543932
>Signed-off-by: Barak Sason Rofman <bsasonro@redhat.com>
BUG: 1823703
Change-Id: I89bc3d4ea7f074db7213d759c49307f379543932
Signed-off-by: Barak Sason Rofman <bsasonro@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/198579
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/dht/src/dht-rebalance.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
diff --git a/xlators/cluster/dht/src/dht-rebalance.c b/xlators/cluster/dht/src/dht-rebalance.c
index f4c62b8..7d9df02 100644
--- a/xlators/cluster/dht/src/dht-rebalance.c
+++ b/xlators/cluster/dht/src/dht-rebalance.c
@@ -650,7 +650,7 @@ out:
static int
__dht_rebalance_create_dst_file(xlator_t *this, xlator_t *to, xlator_t *from,
loc_t *loc, struct iatt *stbuf, fd_t **dst_fd,
- int *fop_errno)
+ int *fop_errno, int file_has_holes)
{
int ret = -1;
int ret2 = -1;
@@ -819,7 +819,7 @@ __dht_rebalance_create_dst_file(xlator_t *this, xlator_t *to, xlator_t *from,
/* No need to bother about 0 byte size files */
if (stbuf->ia_size > 0) {
- if (conf->use_fallocate) {
+ if (conf->use_fallocate && !file_has_holes) {
ret = syncop_fallocate(to, fd, 0, 0, stbuf->ia_size, NULL, NULL);
if (ret < 0) {
if (ret == -EOPNOTSUPP || ret == -EINVAL || ret == -ENOSYS) {
@@ -846,9 +846,7 @@ __dht_rebalance_create_dst_file(xlator_t *this, xlator_t *to, xlator_t *from,
goto out;
}
}
- }
-
- if (!conf->use_fallocate) {
+ } else {
ret = syncop_ftruncate(to, fd, stbuf->ia_size, NULL, NULL, NULL,
NULL);
if (ret < 0) {
@@ -1728,9 +1726,13 @@ dht_migrate_file(xlator_t *this, loc_t *loc, xlator_t *from, xlator_t *to,
goto out;
}
+ /* Try to preserve 'holes' while migrating data */
+ if (stbuf.ia_size > (stbuf.ia_blocks * GF_DISK_SECTOR_SIZE))
+ file_has_holes = 1;
+
/* create the destination, with required modes/xattr */
ret = __dht_rebalance_create_dst_file(this, to, from, loc, &stbuf, &dst_fd,
- fop_errno);
+ fop_errno, file_has_holes);
if (ret) {
gf_msg(this->name, GF_LOG_ERROR, 0, 0,
"Create dst failed"
@@ -1774,8 +1776,8 @@ dht_migrate_file(xlator_t *this, loc_t *loc, xlator_t *from, xlator_t *to,
* destination. We need to do update this only post migration
* as in case of failure the linkto needs to point to the source
* subvol */
- ret = __dht_rebalance_create_dst_file(this, to, from, loc, &stbuf,
- &dst_fd, fop_errno);
+ ret = __dht_rebalance_create_dst_file(
+ this, to, from, loc, &stbuf, &dst_fd, fop_errno, file_has_holes);
if (ret) {
gf_log(this->name, GF_LOG_ERROR,
"Create dst failed"
@@ -1862,9 +1864,6 @@ dht_migrate_file(xlator_t *this, loc_t *loc, xlator_t *from, xlator_t *to,
ret = 0;
goto out;
}
- /* Try to preserve 'holes' while migrating data */
- if (stbuf.ia_size > (stbuf.ia_blocks * GF_DISK_SECTOR_SIZE))
- file_has_holes = 1;
ret = __dht_rebalance_migrate_data(this, defrag, from, to, src_fd, dst_fd,
stbuf.ia_size, file_has_holes,
--
1.8.3.1

70
0365-build-geo-rep-requires-relevant-selinux-permission-f.patch Normal file
View File

@ -0,0 +1,70 @@
From 36180d21dc4b16619b75d65d51eaf37df4e0e2d3 Mon Sep 17 00:00:00 2001
From: Sunny Kumar <sunkumar@redhat.com>
Date: Mon, 20 Apr 2020 12:15:42 +0100
Subject: [PATCH 365/367] build: geo-rep requires relevant selinux permission
for rsync
If selinux is set in enforcing mode geo-rep goes into faulty state.
To avoid this from happening some relevant selinux booleans need to be set
in 'on' state to allow rsync operation.
Backport of:
>Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/24348.
>Change-Id: Ia8ce530d6548c2a545f4c99c600f5aac2bbb3363
>Fixes: #1182
>Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
BUG: 1813917
Change-Id: Ia8ce530d6548c2a545f4c99c600f5aac2bbb3363
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/198599
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
glusterfs.spec.in | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/glusterfs.spec.in b/glusterfs.spec.in
index 7c8a751..5ed07e7 100644
--- a/glusterfs.spec.in
+++ b/glusterfs.spec.in
@@ -130,6 +130,12 @@
## All %%global definitions should be placed here and keep them sorted
##
+# selinux booleans whose defalut value needs modification
+# these booleans will be consumed by "%%selinux_set_booleans" macro.
+%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
+%global selinuxbooleans rsync_full_access=1 rsync_client=1
+%endif
+
%if ( 0%{?fedora} ) || ( 0%{?rhel} && 0%{?rhel} > 6 )
%global _with_systemd true
%endif
@@ -515,6 +521,12 @@ Requires: python%{_pythonver}-gluster = %{version}-%{release}
Requires: rsync
Requires: util-linux
Requires: %{name}-libs%{?_isa} = %{version}-%{release}
+# required for setting selinux bools
+%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
+Requires: selinux-policy-targeted
+Requires(post): selinux-policy-targeted
+BuildRequires: selinux-policy-devel
+%endif
%description geo-replication
GlusterFS is a distributed file-system capable of scaling to several
@@ -941,6 +953,9 @@ exit 0
%if ( 0%{!?_without_georeplication:1} )
%post geo-replication
+%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
+%selinux_set_booleans %{selinuxbooleans}
+%endif
if [ $1 -ge 1 ]; then
%systemd_postun_with_restart glusterd
fi
--
1.8.3.1

55
0366-snapshot-fix-python3-issue-in-gcron.patch Normal file
View File

@ -0,0 +1,55 @@
From d7b84014cbb19e65dfae6248af47cc23fabc64e5 Mon Sep 17 00:00:00 2001
From: Sunny Kumar <sunkumar@redhat.com>
Date: Wed, 22 Apr 2020 15:09:16 +0100
Subject: [PATCH 366/367] snapshot: fix python3 issue in gcron
`$gcron.py test_vol Job`
Traceback:
File "/usr/sbin/gcron.py", line 189, in <module>
main()
File "/usr/sbin/gcron.py", line 121, in main
initLogger(script_name)
File "/usr/sbin/gcron.py", line 44, in initLogger
logfile = os.path.join(out.strip(), script_name[:-3]+".log")
File "/usr/lib64/python3.6/posixpath.py", line 94, in join
genericpath._check_arg_types('join', a, *p)
File "/usr/lib64/python3.6/genericpath.py", line 151, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
Solution: Added the 'universal_newlines' flag to Popen.
Backport of:
>Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/24364/
>Change-Id: I4c7a0e5bce605e4c134f6786c9dd8162b89fc77f
>Fixes: #1193
>Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
BUG: 1825195
Change-Id: I4c7a0e5bce605e4c134f6786c9dd8162b89fc77f
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/198641
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
extras/snap_scheduler/gcron.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/extras/snap_scheduler/gcron.py b/extras/snap_scheduler/gcron.py
index 1127be0..cc16310 100755
--- a/extras/snap_scheduler/gcron.py
+++ b/extras/snap_scheduler/gcron.py
@@ -38,7 +38,8 @@ def initLogger(script_name):
sh.setFormatter(formatter)
process = subprocess.Popen(["gluster", "--print-logdir"],
- stdout=subprocess.PIPE)
+ stdout=subprocess.PIPE,
+ universal_newlines=True)
out, err = process.communicate()
if process.returncode == 0:
logfile = os.path.join(out.strip(), script_name[:-3]+".log")
--
1.8.3.1

95
0367-dht-Handle-setxattr-and-rm-race-for-directory-in-reb.patch Normal file
View File

@ -0,0 +1,95 @@
From aef8e51b9974603d397cc8f5301b24451d012e46 Mon Sep 17 00:00:00 2001
From: Susant Palai <spalai@redhat.com>
Date: Fri, 24 Apr 2020 13:32:51 +0530
Subject: [PATCH 367/367] dht: Handle setxattr and rm race for directory in
rebalance
Problem: Selfheal as part of directory does not return an error if
the layout setxattr fails. This is because the actual lookup fop
must have been successful to proceed for layout heal. Hence, we could
not tell if fix-layout failed in rebalance.
Solution: We can check this information in the layout structure that
whether all the xlators have returned error.
> fixes: #1200
> hange-Id: I3e5f2a36c0d934c21476a73a9a5473d8e490cde7
> Signed-off-by: Susant Palai <spalai@redhat.com>
(backport of https://review.gluster.org/#/c/glusterfs/+/24375/)
BUG: 1812789
Change-Id: I897826c4c2e883b3085c9314deff32d649b4588e
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/198726
Reviewed-by: Mohit Agrawal <moagrawa@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/dht/src/dht-common.c | 19 +++++++++++++++++++
xlators/cluster/dht/src/dht-common.h | 3 +++
xlators/cluster/dht/src/dht-rebalance.c | 11 +++++++++++
3 files changed, 33 insertions(+)
diff --git a/xlators/cluster/dht/src/dht-common.c b/xlators/cluster/dht/src/dht-common.c
index d0b5287..7890e7a 100644
--- a/xlators/cluster/dht/src/dht-common.c
+++ b/xlators/cluster/dht/src/dht-common.c
@@ -11286,3 +11286,22 @@ dht_pt_fgetxattr(call_frame_t *frame, xlator_t *this, fd_t *fd, const char *key,
FIRST_CHILD(this)->fops->fgetxattr, fd, key, xdata);
return 0;
}
+
+/* The job of this function is to check if all the xlators have updated
+ * error in the layout. */
+int
+dht_dir_layout_error_check(xlator_t *this, inode_t *inode)
+{
+ dht_layout_t *layout = NULL;
+ int i = 0;
+
+ layout = dht_layout_get(this, inode);
+ for (i = 0; i < layout->cnt; i++) {
+ if (layout->list[i].err == 0) {
+ return 0;
+ }
+ }
+
+ /* Returning the first xlator error as all xlators have errors */
+ return layout->list[0].err;
+}
diff --git a/xlators/cluster/dht/src/dht-common.h b/xlators/cluster/dht/src/dht-common.h
index ce11f02..4d2aae6 100644
--- a/xlators/cluster/dht/src/dht-common.h
+++ b/xlators/cluster/dht/src/dht-common.h
@@ -1544,4 +1544,7 @@ dht_pt_rename(call_frame_t *frame, xlator_t *this, loc_t *oldloc, loc_t *newloc,
int32_t
dht_check_remote_fd_failed_error(dht_local_t *local, int op_ret, int op_errno);
+int
+dht_dir_layout_error_check(xlator_t *this, inode_t *inode);
+
#endif /* _DHT_H */
diff --git a/xlators/cluster/dht/src/dht-rebalance.c b/xlators/cluster/dht/src/dht-rebalance.c
index 7d9df02..33cacfe 100644
--- a/xlators/cluster/dht/src/dht-rebalance.c
+++ b/xlators/cluster/dht/src/dht-rebalance.c
@@ -3928,6 +3928,17 @@ gf_defrag_fix_layout(xlator_t *this, gf_defrag_info_t *defrag, loc_t *loc,
}
ret = syncop_setxattr(this, loc, fix_layout, 0, NULL, NULL);
+
+ /* In case of a race where the directory is deleted just before
+ * layout setxattr, the errors are updated in the layout structure.
+ * We can use this information to make a decision whether the directory
+ * is deleted entirely.
+ */
+ if (ret == 0) {
+ ret = dht_dir_layout_error_check(this, loc->inode);
+ ret = -ret;
+ }
+
if (ret) {
if (-ret == ENOENT || -ret == ESTALE) {
gf_msg(this->name, GF_LOG_INFO, -ret, DHT_MSG_LAYOUT_FIX_FAILED,
--
1.8.3.1

25
glusterfs.spec
View File

@ -130,6 +130,12 @@
## All %%global definitions should be placed here and keep them sorted
##
# selinux booleans whose defalut value needs modification
# these booleans will be consumed by "%%selinux_set_booleans" macro.
%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
%global selinuxbooleans rsync_full_access=1 rsync_client=1
%endif
%if ( 0%{?fedora} ) || ( 0%{?rhel} && 0%{?rhel} > 6 )
%global _with_systemd true
%endif
@ -231,7 +237,7 @@ Release: 0.1%{?prereltag:.%{prereltag}}%{?dist}
%else
Name: glusterfs
Version: 6.0
Release: 32%{?dist}
Release: 33%{?dist}
ExcludeArch: i686
%endif
License: GPLv2 or LGPLv3+
@ -671,6 +677,11 @@ Patch0359: 0359-dht-gf_defrag_process_dir-is-called-even-if-gf_defra.patch
Patch0360: 0360-rpc-Make-ssl-log-more-useful.patch
Patch0361: 0361-snap_scheduler-python3-compatibility-and-new-test-ca.patch
Patch0362: 0362-write-behind-fix-data-corruption.patch
Patch0363: 0363-common-ha-cluster-status-shows-FAILOVER-when-actuall.patch
Patch0364: 0364-dht-fixing-rebalance-failures-for-files-with-holes.patch
Patch0365: 0365-build-geo-rep-requires-relevant-selinux-permission-f.patch
Patch0366: 0366-snapshot-fix-python3-issue-in-gcron.patch
Patch0367: 0367-dht-Handle-setxattr-and-rm-race-for-directory-in-reb.patch
%description
GlusterFS is a distributed file-system capable of scaling to several
@ -879,6 +890,12 @@ Requires: python%{_pythonver}-gluster = %{version}-%{release}
Requires: rsync
Requires: util-linux
Requires: %{name}-libs%{?_isa} = %{version}-%{release}
# required for setting selinux bools
%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
Requires: selinux-policy-targeted
Requires(post): selinux-policy-targeted
BuildRequires: selinux-policy-devel
%endif
%description geo-replication
GlusterFS is a distributed file-system capable of scaling to several
@ -1378,6 +1395,9 @@ exit 0
%if ( 0%{!?_without_georeplication:1} )
%post geo-replication
%if ( 0%{?rhel} && 0%{?rhel} >= 8 )
%selinux_set_booleans %{selinuxbooleans}
%endif
if [ $1 -ge 1 ]; then
%systemd_postun_with_restart glusterd
fi
@ -2400,6 +2420,9 @@ fi
%endif
%changelog
* Wed Apr 29 2020 Rinku Kothiya <rkothiya@redhat.com> - 6.0-33
- fixes bugs bz#1812789 bz#1813917 bz#1823703 bz#1823706 bz#1825195
* Sat Apr 04 2020 Rinku Kothiya <rkothiya@redhat.com> - 6.0-32
- fixes bugs bz#1781543 bz#1812789 bz#1812824 bz#1817369 bz#1819059