rpms/glusterfs
5
0
Fork 0
Code Issues Pull Requests Releases Activity

autobuild v6.0-12

Browse Source 
Resolves: bz#1730914 bz#1731448 bz#1732770 bz#1732792 bz#1733531
Resolves: bz#1734305 bz#1734534 bz#1734734 bz#1735514 bz#1737705
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
This commit is contained in:
Sunil Kumar Acharya 2019-08-09 10:45:54 -04:00
parent 4797c9e01f
commit 9bd22fde48
12 changed files with 858 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
0266-geo-rep-Fix-mount-broker-setup-issue.patch Normal file
View File

@ -0,0 +1,63 @@
From cb9d0fa4bd2664556f0564406037f9fb7fb781a6 Mon Sep 17 00:00:00 2001
From: Kotresh HR <khiremat@redhat.com>
Date: Wed, 31 Jul 2019 15:40:55 +0530
Subject: [PATCH 266/276] geo-rep: Fix mount broker setup issue
Even the use builtin 'type' command as in patch [1]
causes issues if argument in question is not part of PATH
environment variable for that user. This patch fixes the
same by doing source /etc/profile. This was already being
used in another part of script.
[1] https://review.gluster.org/23089
Backport of:
> Patch: https://review.gluster.org/23136
> Change-Id: Iceb78835967ec6a4350983eec9af28398410c002
> fixes: bz#1734738
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: Iceb78835967ec6a4350983eec9af28398410c002
BUG: 1734734
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177867
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
geo-replication/src/gverify.sh | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/geo-replication/src/gverify.sh b/geo-replication/src/gverify.sh
index 692c1d6..f5f70d2 100755
--- a/geo-replication/src/gverify.sh
+++ b/geo-replication/src/gverify.sh
@@ -180,6 +180,8 @@ function main()
> $log_file
inet6=$7
+ local cmd_line
+ local ver
# Use FORCE_BLOCKER flag in the error message to differentiate
# between the errors which the force command should bypass
@@ -206,13 +208,14 @@ function main()
exit 1;
fi;
+ cmd_line=$(cmd_slave);
if [[ -z "${GR_SSH_IDENTITY_KEY}" ]]; then
- ssh -p ${SSH_PORT} -oNumberOfPasswordPrompts=0 -oStrictHostKeyChecking=no $2@$3 "type -p gluster"
+ ver=$(ssh -p ${SSH_PORT} -oNumberOfPasswordPrompts=0 -oStrictHostKeyChecking=no $2@$3 bash -c "'$cmd_line'")
else
- ssh -p ${SSH_PORT} -i ${GR_SSH_IDENTITY_KEY} -oNumberOfPasswordPrompts=0 -oStrictHostKeyChecking=no $2@$3 "type -p gluster"
+ ver=$(ssh -p ${SSH_PORT} -i ${GR_SSH_IDENTITY_KEY} -oNumberOfPasswordPrompts=0 -oStrictHostKeyChecking=no $2@$3 bash -c "'$cmd_line'")
fi
- if [ $? -ne 0 ]; then
+ if [ -z "$ver" ]; then
echo "FORCE_BLOCKER|gluster command not found on $3 for user $2." > $log_file
exit 1;
fi;
--
1.8.3.1

143
0267-posix-ctime-Fix-race-during-lookup-ctime-xattr-heal.patch Normal file
View File

@ -0,0 +1,143 @@
From cf13847a6341b7519ed0dc51e3b9ecf12444a3e4 Mon Sep 17 00:00:00 2001
From: Kotresh HR <khiremat@redhat.com>
Date: Mon, 29 Jul 2019 16:22:10 +0530
Subject: [PATCH 267/276] posix/ctime: Fix race during lookup ctime xattr heal
Problem:
Ctime heals the ctime xattr ("trusted.glusterfs.mdata") in lookup
if it's not present. In a multi client scenario, there is a race
which results in updating the ctime xattr to older value.
e.g. Let c1 and c2 be two clients and file1 be the file which
doesn't have the ctime xattr. Let the ctime of file1 be t1.
(from backend, ctime heals time attributes from backend when not present).
Now following operations are done on mount
c1 -> ls -l /mnt/file1 | c2 -> ls -l /mnt/file1;echo "append" >> /mnt/file1;
The race is that the both c1 and c2 didn't fetch the ctime xattr in lookup,
so both of them tries to heal ctime to time 't1'. If c2 wins the race and
appends the file before c1 heals it, it sets the time to 't1' and updates
it to 't2' (because of append). Now c1 proceeds to heal and sets it to 't1'
which is incorrect.
Solution:
Compare the times during heal and only update the larger time. This is the
general approach used in ctime feature but got missed with healing legacy
files.
> upstream patch : https://review.gluster.org/#/c/glusterfs/+/23131/
>fixes: bz#1734299
>Change-Id: I930bda192c64c3d49d0aed431ce23d3bc57e51b7
>Signed-off-by: Kotresh HR <khiremat@redhat.com>
BUG: 1734305
Change-Id: I930bda192c64c3d49d0aed431ce23d3bc57e51b7
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177866
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/storage/posix/src/posix-metadata.c | 76 +++++++++++++++++++++++-------
1 file changed, 58 insertions(+), 18 deletions(-)
diff --git a/xlators/storage/posix/src/posix-metadata.c b/xlators/storage/posix/src/posix-metadata.c
index 647c0bb..57791fa 100644
--- a/xlators/storage/posix/src/posix-metadata.c
+++ b/xlators/storage/posix/src/posix-metadata.c
@@ -344,33 +344,73 @@ posix_set_mdata_xattr_legacy_files(xlator_t *this, inode_t *inode,
struct mdata_iatt *mdata_iatt, int *op_errno)
{
posix_mdata_t *mdata = NULL;
+ posix_mdata_t imdata = {
+ 0,
+ };
int ret = 0;
+ gf_boolean_t mdata_already_set = _gf_false;
GF_VALIDATE_OR_GOTO("posix", this, out);
GF_VALIDATE_OR_GOTO(this->name, inode, out);
LOCK(&inode->lock);
{
- mdata = GF_CALLOC(1, sizeof(posix_mdata_t), gf_posix_mt_mdata_attr);
- if (!mdata) {
- gf_msg(this->name, GF_LOG_ERROR, ENOMEM, P_MSG_NOMEM,
- "Could not allocate mdata. gfid: %s",
- uuid_utoa(inode->gfid));
- ret = -1;
- *op_errno = ENOMEM;
- goto unlock;
- }
+ ret = __inode_ctx_get1(inode, this, (uint64_t *)&mdata);
+ if (ret == 0 && mdata) {
+ mdata_already_set = _gf_true;
+ } else if (ret == -1 || !mdata) {
+ mdata = GF_CALLOC(1, sizeof(posix_mdata_t), gf_posix_mt_mdata_attr);
+ if (!mdata) {
+ gf_msg(this->name, GF_LOG_ERROR, ENOMEM, P_MSG_NOMEM,
+ "Could not allocate mdata. gfid: %s",
+ uuid_utoa(inode->gfid));
+ ret = -1;
+ *op_errno = ENOMEM;
+ goto unlock;
+ }
+
+ ret = posix_fetch_mdata_xattr(this, NULL, -1, inode, (void *)mdata,
+ op_errno);
+ if (ret == 0) {
+ /* Got mdata from disk. This is a race, another client
+ * has healed the xattr during lookup. So set it in inode
+ * ctx */
+ __inode_ctx_set1(inode, this, (uint64_t *)&mdata);
+ mdata_already_set = _gf_true;
+ } else {
+ *op_errno = 0;
+ mdata->version = 1;
+ mdata->flags = 0;
+ mdata->ctime.tv_sec = mdata_iatt->ia_ctime;
+ mdata->ctime.tv_nsec = mdata_iatt->ia_ctime_nsec;
+ mdata->atime.tv_sec = mdata_iatt->ia_atime;
+ mdata->atime.tv_nsec = mdata_iatt->ia_atime_nsec;
+ mdata->mtime.tv_sec = mdata_iatt->ia_mtime;
+ mdata->mtime.tv_nsec = mdata_iatt->ia_mtime_nsec;
- mdata->version = 1;
- mdata->flags = 0;
- mdata->ctime.tv_sec = mdata_iatt->ia_ctime;
- mdata->ctime.tv_nsec = mdata_iatt->ia_ctime_nsec;
- mdata->atime.tv_sec = mdata_iatt->ia_atime;
- mdata->atime.tv_nsec = mdata_iatt->ia_atime_nsec;
- mdata->mtime.tv_sec = mdata_iatt->ia_mtime;
- mdata->mtime.tv_nsec = mdata_iatt->ia_mtime_nsec;
+ __inode_ctx_set1(inode, this, (uint64_t *)&mdata);
+ }
+ }
- __inode_ctx_set1(inode, this, (uint64_t *)&mdata);
+ if (mdata_already_set) {
+ /* Compare and update the larger time */
+ imdata.ctime.tv_sec = mdata_iatt->ia_ctime;
+ imdata.ctime.tv_nsec = mdata_iatt->ia_ctime_nsec;
+ imdata.atime.tv_sec = mdata_iatt->ia_atime;
+ imdata.atime.tv_nsec = mdata_iatt->ia_atime_nsec;
+ imdata.mtime.tv_sec = mdata_iatt->ia_mtime;
+ imdata.mtime.tv_nsec = mdata_iatt->ia_mtime_nsec;
+
+ if (posix_compare_timespec(&imdata.ctime, &mdata->ctime) > 0) {
+ mdata->ctime = imdata.ctime;
+ }
+ if (posix_compare_timespec(&imdata.mtime, &mdata->mtime) > 0) {
+ mdata->mtime = imdata.mtime;
+ }
+ if (posix_compare_timespec(&imdata.atime, &mdata->atime) > 0) {
+ mdata->atime = imdata.atime;
+ }
+ }
ret = posix_store_mdata_xattr(this, NULL, -1, inode, mdata);
if (ret) {
--
1.8.3.1

46
0268-rpc-transport-have-default-listen-port.patch Normal file
View File

@ -0,0 +1,46 @@
From 872e344c0ab40c37b1872c32f5d5fddc097a1460 Mon Sep 17 00:00:00 2001
From: Atin Mukherjee <amukherj@redhat.com>
Date: Mon, 5 Aug 2019 21:16:35 +0530
Subject: [PATCH 268/276] rpc/transport: have default listen-port
With release-6, we now can have transport.socket.listen-port parameter
configurable in glusterd.vol. However the default value wasn't defined
in the code and this breaks the backward compatibility where if one has
a modified glusterd.vol file, then post upgrade the same file will be
retained and the new changes introduced as part of the release wouldn't
be available in the glusterd.vol. So it's important that for each new
options introduced in glusterd.vol file backward compatibility is
guaranteed.
> upstream patch : https://review.gluster.org/#/c/glusterfs/+/23160/
>Fixes: bz#1737676
>Change-Id: I776b28bff786320cda299fe673d824024dc9803e
>Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
BUG: 1734534
Change-Id: I776b28bff786320cda299fe673d824024dc9803e
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177862
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
rpc/rpc-transport/socket/src/name.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/rpc/rpc-transport/socket/src/name.c b/rpc/rpc-transport/socket/src/name.c
index ca14402..7f18cc4 100644
--- a/rpc/rpc-transport/socket/src/name.c
+++ b/rpc/rpc-transport/socket/src/name.c
@@ -367,6 +367,8 @@ af_inet_server_get_local_sockaddr(rpc_transport_t *this, struct sockaddr *addr,
listen_port_data = dict_get(options, "transport.socket.listen-port");
if (listen_port_data) {
listen_port = data_to_uint16(listen_port_data);
+ } else {
+ listen_port = GF_DEFAULT_SOCKET_LISTEN_PORT;
}
listen_host_data = dict_get(options, "transport.socket.bind-address");
--
1.8.3.1

58
0269-ec-fix-truncate-lock-to-cover-the-write-in-tuncate-c.patch Normal file
View File

@ -0,0 +1,58 @@
From 7c2d6e82d7d3430ad8a557b6ae726765f7e874e9 Mon Sep 17 00:00:00 2001
From: Kinglong Mee <kinglongmee@gmail.com>
Date: Fri, 12 Apr 2019 11:35:55 +0800
Subject: [PATCH 269/276] ec: fix truncate lock to cover the write in tuncate
clean
ec_truncate_clean does writing under the lock granted for truncate,
but the lock is calculated by ec_adjust_offset_up, so that,
the write in ec_truncate_clean is out of lock.
fixes: bz#1732770
Upstream-patch: https://review.gluster.org/c/glusterfs/+/22552
Change-Id: Idbe1fd48d26afe49c36b77db9f12e0907f5a4134
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177973
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/ec/src/ec-inode-write.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/xlators/cluster/ec/src/ec-inode-write.c b/xlators/cluster/ec/src/ec-inode-write.c
index a903664..ea55140 100644
--- a/xlators/cluster/ec/src/ec-inode-write.c
+++ b/xlators/cluster/ec/src/ec-inode-write.c
@@ -1405,6 +1405,7 @@ int32_t
ec_manager_truncate(ec_fop_data_t *fop, int32_t state)
{
ec_cbk_data_t *cbk;
+ off_t offset_down;
switch (state) {
case EC_STATE_INIT:
@@ -1416,16 +1417,19 @@ ec_manager_truncate(ec_fop_data_t *fop, int32_t state)
/* Fall through */
case EC_STATE_LOCK:
+ offset_down = fop->user_size;
+ ec_adjust_offset_down(fop->xl->private, &offset_down, _gf_true);
+
if (fop->id == GF_FOP_TRUNCATE) {
ec_lock_prepare_inode(
fop, &fop->loc[0],
EC_UPDATE_DATA | EC_UPDATE_META | EC_QUERY_INFO,
- fop->offset, EC_RANGE_FULL);
+ offset_down, EC_RANGE_FULL);
} else {
ec_lock_prepare_fd(
fop, fop->fd,
EC_UPDATE_DATA | EC_UPDATE_META | EC_QUERY_INFO,
- fop->offset, EC_RANGE_FULL);
+ offset_down, EC_RANGE_FULL);
}
ec_lock(fop);
--
1.8.3.1

42
0270-cluster-ec-inherit-healing-from-lock-when-it-has-inf.patch Normal file
View File

@ -0,0 +1,42 @@
From 84d8a0ca5b521b9d87679ffebe420fe69869961d Mon Sep 17 00:00:00 2001
From: Kinglong Mee <kinglongmee@gmail.com>
Date: Mon, 8 Jul 2019 21:13:28 +0800
Subject: [PATCH 270/276] cluster/ec: inherit healing from lock when it has
info
If lock has info, fop should inherit healing mask from it.
Otherwise, fop cannot inherit right healing when changed_flags is zero.
Upstream-patch: https://review.gluster.org/c/glusterfs/+/23010
Change-Id: Ife80c9169d2c555024347a20300b0583f7e8a87f
fixes: bz#1732792
Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177974
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/ec/src/ec-common.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/xlators/cluster/ec/src/ec-common.c b/xlators/cluster/ec/src/ec-common.c
index e2e582f..db1ff5b 100644
--- a/xlators/cluster/ec/src/ec-common.c
+++ b/xlators/cluster/ec/src/ec-common.c
@@ -1412,11 +1412,12 @@ ec_get_size_version(ec_lock_link_t *link)
set_dirty = ec_set_dirty_flag(link, ctx, dirty);
/* If ec metadata has already been retrieved, do not try again. */
- if (ctx->have_info && (!set_dirty)) {
+ if (ctx->have_info) {
if (ec_is_data_fop(fop->id)) {
fop->healing |= lock->healing;
}
- goto unlock;
+ if (!set_dirty)
+ goto unlock;
}
/* Determine if there's something we need to retrieve for the current
--
1.8.3.1

116
0271-cluster-ec-fix-EIO-error-for-concurrent-writes-on-sp.patch Normal file
View File

@ -0,0 +1,116 @@
From 52d71ad0e5c27808e7d8eea8a0920298837e408c Mon Sep 17 00:00:00 2001
From: Xavi Hernandez <xhernandez@redhat.com>
Date: Wed, 17 Jul 2019 14:50:22 +0200
Subject: [PATCH 271/276] cluster/ec: fix EIO error for concurrent writes on
sparse files
EC doesn't allow concurrent writes on overlapping areas, they are
serialized. However non-overlapping writes are serviced in parallel.
When a write is not aligned, EC first needs to read the entire chunk
from disk, apply the modified fragment and write it again.
The problem appears on sparse files because a write to an offset
implicitly creates data on offsets below it (so, in some way, they
are overlapping). For example, if a file is empty and we read 10 bytes
from offset 10, read() will return 0 bytes. Now, if we write one byte
at offset 1M and retry the same read, the system call will return 10
bytes (all containing 0's).
So if we have two writes, the first one at offset 10 and the second one
at offset 1M, EC will send both in parallel because they do not overlap.
However, the first one will try to read missing data from the first chunk
(i.e. offsets 0 to 9) to recombine the entire chunk and do the final write.
This read will happen in parallel with the write to 1M. What could happen
is that half of the bricks process the write before the read, and the
half do the read before the write. Some bricks will return 10 bytes of
data while the otherw will return 0 bytes (because the file on the brick
has not been expanded yet).
When EC tries to recombine the answers from the bricks, it can't, because
it needs more than half consistent answers to recover the data. So this
read fails with EIO error. This error is propagated to the parent write,
which is aborted and EIO is returned to the application.
The issue happened because EC assumed that a write to a given offset
implies that offsets below it exist.
This fix prevents the read of the chunk from bricks if the current size
of the file is smaller than the read chunk offset. This size is
correctly tracked, so this fixes the issue.
Also modifying ec-stripe.t file for Test #13 within it.
In this patch, if a file size is less than the offset we are writing, we
fill zeros in head and tail and do not consider it strip cache miss.
That actually make sense as we know what data that part holds and there is
no need of reading it from bricks.
Upstream-patch: https://review.gluster.org/c/glusterfs/+/23066
Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6
fixes: bz#1731448
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177975
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
tests/basic/ec/ec-stripe.t | 2 +-
xlators/cluster/ec/src/ec-inode-write.c | 26 +++++++++++++++++---------
2 files changed, 18 insertions(+), 10 deletions(-)
diff --git a/tests/basic/ec/ec-stripe.t b/tests/basic/ec/ec-stripe.t
index 1e940eb..98b9229 100644
--- a/tests/basic/ec/ec-stripe.t
+++ b/tests/basic/ec/ec-stripe.t
@@ -202,7 +202,7 @@ TEST truncate -s 0 $B0/test_file
TEST truncate -s 0 $M0/test_file
TEST dd if=$B0/misc_file of=$B0/test_file bs=1022 count=5 oflag=seek_bytes,sync seek=400 conv=notrunc
TEST dd if=$B0/misc_file of=$M0/test_file bs=1022 count=5 oflag=seek_bytes,sync seek=400 conv=notrunc
-check_statedump_md5sum 4 5
+check_statedump_md5sum 4 4
clean_file_unmount
### 14 - Truncate to invalidate all but one the stripe in cache ####
diff --git a/xlators/cluster/ec/src/ec-inode-write.c b/xlators/cluster/ec/src/ec-inode-write.c
index ea55140..a45e6d6 100644
--- a/xlators/cluster/ec/src/ec-inode-write.c
+++ b/xlators/cluster/ec/src/ec-inode-write.c
@@ -2013,20 +2013,28 @@ ec_writev_start(ec_fop_data_t *fop)
if (err != 0) {
goto failed_fd;
}
+ tail = fop->size - fop->user_size - fop->head;
if (fop->head > 0) {
- found_stripe = ec_get_and_merge_stripe(ec, fop, EC_STRIPE_HEAD);
- if (!found_stripe) {
- if (ec_make_internal_fop_xdata(&xdata)) {
- err = -ENOMEM;
- goto failed_xdata;
+ if (current > fop->offset) {
+ found_stripe = ec_get_and_merge_stripe(ec, fop, EC_STRIPE_HEAD);
+ if (!found_stripe) {
+ if (ec_make_internal_fop_xdata(&xdata)) {
+ err = -ENOMEM;
+ goto failed_xdata;
+ }
+ ec_readv(fop->frame, fop->xl, -1, EC_MINIMUM_MIN,
+ ec_writev_merge_head, NULL, fd, ec->stripe_size,
+ fop->offset, 0, xdata);
+ }
+ } else {
+ memset(fop->vector[0].iov_base, 0, fop->head);
+ memset(fop->vector[0].iov_base + fop->size - tail, 0, tail);
+ if (ec->stripe_cache && (fop->size <= ec->stripe_size)) {
+ ec_add_stripe_in_cache(ec, fop);
}
- ec_readv(fop->frame, fop->xl, -1, EC_MINIMUM_MIN,
- ec_writev_merge_head, NULL, fd, ec->stripe_size,
- fop->offset, 0, xdata);
}
}
- tail = fop->size - fop->user_size - fop->head;
if ((tail > 0) && ((fop->head == 0) || (fop->size > ec->stripe_size))) {
/* Current locking scheme will make sure the 'current' below will
* never decrease while the fop is in progress, so the checks will
--
1.8.3.1

90
0272-cluster-ec-Always-read-from-good-mask.patch Normal file
View File

@ -0,0 +1,90 @@
From 220b95085847b5f6a9e5bee7a9519efe72600e6a Mon Sep 17 00:00:00 2001
From: Pranith Kumar K <pkarampu@redhat.com>
Date: Thu, 18 Jul 2019 11:25:31 +0530
Subject: [PATCH 272/276] cluster/ec: Always read from good-mask
There are cases where fop->mask may have fop->healing added
and readv shouldn't be wound on fop->healing. To avoid this
always wind readv to lock->good_mask
Upstream-patch: https://review.gluster.org/c/glusterfs/+/23069
fixes: bz#1730914
Change-Id: I2226ef0229daf5ff315d51e868b980ee48060b87
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177976
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/ec/src/ec-common.c | 3 +++
xlators/cluster/ec/src/ec-inode-write.c | 27 ++++++++++++++++++++++-----
2 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/xlators/cluster/ec/src/ec-common.c b/xlators/cluster/ec/src/ec-common.c
index db1ff5b..28b31c9 100644
--- a/xlators/cluster/ec/src/ec-common.c
+++ b/xlators/cluster/ec/src/ec-common.c
@@ -654,6 +654,9 @@ ec_child_select(ec_fop_data_t *fop)
* unlock should go on all subvols where lock is performed*/
if (fop->parent && !ec_internal_op(fop)) {
fop->mask &= (fop->parent->mask & ~fop->parent->healing);
+ if (ec_is_data_fop(fop->id)) {
+ fop->healing |= fop->parent->healing;
+ }
}
if ((fop->mask & ~ec->xl_up) != 0) {
diff --git a/xlators/cluster/ec/src/ec-inode-write.c b/xlators/cluster/ec/src/ec-inode-write.c
index a45e6d6..4f35b6d 100644
--- a/xlators/cluster/ec/src/ec-inode-write.c
+++ b/xlators/cluster/ec/src/ec-inode-write.c
@@ -1977,6 +1977,20 @@ ec_get_and_merge_stripe(ec_t *ec, ec_fop_data_t *fop, ec_stripe_part_t which)
return found;
}
+static uintptr_t
+ec_get_lock_good_mask(inode_t *inode, xlator_t *xl)
+{
+ ec_lock_t *lock = NULL;
+ ec_inode_t *ictx = NULL;
+ LOCK(&inode->lock);
+ {
+ ictx = __ec_inode_get(inode, xl);
+ lock = ictx->inode_lock;
+ }
+ UNLOCK(&inode->lock);
+ return lock->good_mask;
+}
+
void
ec_writev_start(ec_fop_data_t *fop)
{
@@ -2022,9 +2036,10 @@ ec_writev_start(ec_fop_data_t *fop)
err = -ENOMEM;
goto failed_xdata;
}
- ec_readv(fop->frame, fop->xl, -1, EC_MINIMUM_MIN,
- ec_writev_merge_head, NULL, fd, ec->stripe_size,
- fop->offset, 0, xdata);
+ ec_readv(fop->frame, fop->xl,
+ ec_get_lock_good_mask(fop->fd->inode, fop->xl),
+ EC_MINIMUM_MIN, ec_writev_merge_head, NULL, fd,
+ ec->stripe_size, fop->offset, 0, xdata);
}
} else {
memset(fop->vector[0].iov_base, 0, fop->head);
@@ -2047,8 +2062,10 @@ ec_writev_start(ec_fop_data_t *fop)
err = -ENOMEM;
goto failed_xdata;
}
- ec_readv(fop->frame, fop->xl, -1, EC_MINIMUM_MIN,
- ec_writev_merge_tail, NULL, fd, ec->stripe_size,
+ ec_readv(fop->frame, fop->xl,
+ ec_get_lock_good_mask(fop->fd->inode, fop->xl),
+ EC_MINIMUM_MIN, ec_writev_merge_tail, NULL, fd,
+ ec->stripe_size,
fop->offset + fop->size - ec->stripe_size, 0, xdata);
}
} else {
--
1.8.3.1

86
0273-cluster-ec-Fix-reopen-flags-to-avoid-misbehavior.patch Normal file
View File

@ -0,0 +1,86 @@
From d5f931b334ac7abccaf30d277ce3ca9cfae0da5b Mon Sep 17 00:00:00 2001
From: Pranith Kumar K <pkarampu@redhat.com>
Date: Mon, 29 Jul 2019 14:08:37 +0530
Subject: [PATCH 273/276] cluster/ec: Fix reopen flags to avoid misbehavior
Problem:
when a file needs to be re-opened O_APPEND and O_EXCL
flags are not filtered in EC.
- O_APPEND should be filtered because EC doesn't send O_APPEND below EC for
open to make sure writes happen on the individual fragments instead of at the
end of the file.
- O_EXCL should be filtered because shd could have created the file so even
when file exists open should succeed
- O_CREAT should be filtered because open happens with gfid as parameter. So
open fop will create just the gfid which will lead to problems.
Fix:
Filter out these two flags in reopen.
Upstream-patch:https://review.gluster.org/#/c/glusterfs/+/23121/
Change-Id: Ia280470fcb5188a09caa07bf665a2a94bce23bc4
fixes: bz#1735514
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177977
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/ec/src/ec-common.c | 4 +++-
xlators/cluster/ec/src/ec-inode-write.c | 7 +++++--
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/xlators/cluster/ec/src/ec-common.c b/xlators/cluster/ec/src/ec-common.c
index 28b31c9..5fb4610 100644
--- a/xlators/cluster/ec/src/ec-common.c
+++ b/xlators/cluster/ec/src/ec-common.c
@@ -101,6 +101,7 @@ ec_fix_open(ec_fop_data_t *fop, uintptr_t mask)
{
uintptr_t need_open = 0;
int ret = 0;
+ int32_t flags = 0;
loc_t loc = {
0,
};
@@ -121,6 +122,7 @@ ec_fix_open(ec_fop_data_t *fop, uintptr_t mask)
goto out;
}
+ flags = fop->fd->flags & (~(O_TRUNC | O_APPEND | O_CREAT | O_EXCL));
if (IA_IFDIR == fop->fd->inode->ia_type) {
ec_opendir(fop->frame, fop->xl, need_open,
EC_MINIMUM_ONE | EC_FOP_NO_PROPAGATE_ERROR, NULL, NULL,
@@ -128,7 +130,7 @@ ec_fix_open(ec_fop_data_t *fop, uintptr_t mask)
} else {
ec_open(fop->frame, fop->xl, need_open,
EC_MINIMUM_ONE | EC_FOP_NO_PROPAGATE_ERROR, NULL, NULL, &loc,
- fop->fd->flags & (~O_TRUNC), fop->fd, NULL);
+ flags, fop->fd, NULL);
}
out:
diff --git a/xlators/cluster/ec/src/ec-inode-write.c b/xlators/cluster/ec/src/ec-inode-write.c
index 4f35b6d..2f28e11 100644
--- a/xlators/cluster/ec/src/ec-inode-write.c
+++ b/xlators/cluster/ec/src/ec-inode-write.c
@@ -1985,10 +1985,13 @@ ec_get_lock_good_mask(inode_t *inode, xlator_t *xl)
LOCK(&inode->lock);
{
ictx = __ec_inode_get(inode, xl);
- lock = ictx->inode_lock;
+ if (ictx)
+ lock = ictx->inode_lock;
}
UNLOCK(&inode->lock);
- return lock->good_mask;
+ if (lock)
+ return lock->good_mask;
+ return 0;
}
void
--
1.8.3.1

49
0274-cluster-ec-Update-lock-good_mask-on-parent-fop-failu.patch Normal file
View File

@ -0,0 +1,49 @@
From 4c2aa7adef3df500043dd45614d20c9987e6c0d9 Mon Sep 17 00:00:00 2001
From: Pranith Kumar K <pkarampu@redhat.com>
Date: Fri, 2 Aug 2019 12:05:09 +0530
Subject: [PATCH 274/276] cluster/ec: Update lock->good_mask on parent fop
failure
When discard/truncate performs write fop, it should do so
after updating lock->good_mask to make sure readv happens
on the correct mask
Upstream-patch: https://review.gluster.org/c/glusterfs/+/23147
fixes: bz#1730914
Change-Id: Idfef0bbcca8860d53707094722e6ba3f81c583b7
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/177978
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Sunil Kumar Heggodu Gopala Acharya <sheggodu@redhat.com>
---
xlators/cluster/ec/src/ec-common.h | 2 ++
xlators/cluster/ec/src/ec-inode-write.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/xlators/cluster/ec/src/ec-common.h b/xlators/cluster/ec/src/ec-common.h
index e948342..3c69471 100644
--- a/xlators/cluster/ec/src/ec-common.h
+++ b/xlators/cluster/ec/src/ec-common.h
@@ -204,4 +204,6 @@ void
ec_reset_entry_healing(ec_fop_data_t *fop);
char *
ec_msg_str(ec_fop_data_t *fop);
+void
+ec_lock_update_good(ec_lock_t *lock, ec_fop_data_t *fop);
#endif /* __EC_COMMON_H__ */
diff --git a/xlators/cluster/ec/src/ec-inode-write.c b/xlators/cluster/ec/src/ec-inode-write.c
index 2f28e11..8bfa3b4 100644
--- a/xlators/cluster/ec/src/ec-inode-write.c
+++ b/xlators/cluster/ec/src/ec-inode-write.c
@@ -89,6 +89,8 @@ ec_update_write(ec_fop_data_t *fop, uintptr_t mask, off_t offset, uint64_t size)
goto out;
}
+ if (fop->locks[0].lock)
+ ec_lock_update_good(fop->locks[0].lock, fop);
vector.iov_base = iobuf->ptr;
vector.iov_len = size;
memset(vector.iov_base, 0, vector.iov_len);
--
1.8.3.1

74
0275-cluster-ec-Create-heal-task-with-heal-process-id.patch Normal file
View File

@ -0,0 +1,74 @@
From 0864f1ad12394a5748d92aa0ed5b455135426bc3 Mon Sep 17 00:00:00 2001
From: Ashish Pandey <aspandey@redhat.com>
Date: Tue, 30 Jul 2019 10:32:39 +0530
Subject: [PATCH 275/276] cluster/ec: Create heal task with heal process id
Problem:
ec_data_undo_pending calls syncop_fxattrop->SYNCOP without
a frame. In this case SYNCOP gets the frame of the task.
However, when we create a synctask for heal we provide
frame as NULL.
Now, if the read-only feature is ON, it will receive the
process ID of the shd as 0 and will consider that it as
not an internal process. This will prevent healing of a
file with "Read-only file system" error message log.
Solution:
While launching heal, create a synctask using frame and set
process id of the SHD which is -6.
> upstream patch : https://review.gluster.org/#/c/glusterfs/+/23129/
>Change-Id: I37195399c85de322cbcac75633888922c4e3db4a
>Fixes: bz#1734252
BUG: 1733531
Change-Id: I37195399c85de322cbcac75633888922c4e3db4a
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/178038
Tested-by: RHGS Build Bot <nigelb@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
---
xlators/cluster/ec/src/ec-heal.c | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/xlators/cluster/ec/src/ec-heal.c b/xlators/cluster/ec/src/ec-heal.c
index 2fa1f11..0f0f398 100644
--- a/xlators/cluster/ec/src/ec-heal.c
+++ b/xlators/cluster/ec/src/ec-heal.c
@@ -2647,13 +2647,31 @@ void
ec_launch_heal(ec_t *ec, ec_fop_data_t *fop)
{
int ret = 0;
+ call_frame_t *frame = NULL;
+
+ frame = create_frame(ec->xl, ec->xl->ctx->pool);
+ if (!frame) {
+ goto out;
+ ret = -1;
+ }
+
+ ec_owner_set(frame, frame->root);
+ /*Do heal as root*/
+ frame->root->uid = 0;
+ frame->root->gid = 0;
+ /*Mark the fops as internal*/
+ frame->root->pid = GF_CLIENT_PID_SELF_HEALD;
ret = synctask_new(ec->xl->ctx->env, ec_synctask_heal_wrap, ec_heal_done,
- NULL, fop);
+ frame, fop);
+out:
if (ret < 0) {
ec_fop_set_error(fop, ENOMEM);
ec_heal_fail(ec, fop);
}
+
+ if (frame)
+ STACK_DESTROY(frame->root);
}
void
--
1.8.3.1

74
0276-features-utime-always-update-ctime-at-setattr.patch Normal file
View File

@ -0,0 +1,74 @@
From 7f5658a299081cec4c77d3cca4e70099cd59b1fc Mon Sep 17 00:00:00 2001
From: Kinglong Mee <kinglongmee@gmail.com>
Date: Mon, 5 Aug 2019 11:08:02 +0800
Subject: [PATCH 276/276] features/utime: always update ctime at setattr
For the nfs EXCLUSIVE mode create may sets a later time
to mtime (at verifier), it should not set to ctime for
storage.ctime does not allowed set ctime to a earlier time.
/* Earlier, mdata was updated only if the existing time is less
* than the time to be updated. This would fail the scenarios
* where mtime can be set to any time using the syscall. Hence
* just updating without comparison. But the ctime is not
* allowed to changed to older date.
*/
According to kernel's setattr, always set ctime at setattr,
and doesnot set ctime from mtime at storage.ctime.
>Change-Id: I5cfde6cb7f8939da9617506e3dc80bd840e0d749
>fixes: bz#1737288
>Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/23154/
BUG: 1737705
Change-Id: I5cfde6cb7f8939da9617506e3dc80bd840e0d749
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://code.engineering.redhat.com/gerrit/178225
Tested-by: RHGS Build Bot <nigelb@redhat.com>
---
xlators/features/utime/src/utime-gen-fops-c.py | 13 +------------
xlators/storage/posix/src/posix-metadata.c | 2 +-
2 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/xlators/features/utime/src/utime-gen-fops-c.py b/xlators/features/utime/src/utime-gen-fops-c.py
index 8730a51..a8637ff 100755
--- a/xlators/features/utime/src/utime-gen-fops-c.py
+++ b/xlators/features/utime/src/utime-gen-fops-c.py
@@ -82,18 +82,7 @@ gf_utime_@NAME@ (call_frame_t *frame, xlator_t *this,
@LONG_ARGS@)
{
gl_timespec_get(&frame->root->ctime);
-
- if (!valid) {
- frame->root->flags |= MDATA_CTIME;
- }
-
- if (valid & (GF_SET_ATTR_UID | GF_SET_ATTR_GID)) {
- frame->root->flags |= MDATA_CTIME;
- }
-
- if (valid & GF_SET_ATTR_MODE) {
- frame->root->flags |= MDATA_CTIME;
- }
+ frame->root->flags |= MDATA_CTIME;
STACK_WIND (frame, gf_utime_@NAME@_cbk, FIRST_CHILD(this),
FIRST_CHILD(this)->fops->@NAME@, @SHORT_ARGS@);
diff --git a/xlators/storage/posix/src/posix-metadata.c b/xlators/storage/posix/src/posix-metadata.c
index 57791fa..5cbdc98 100644
--- a/xlators/storage/posix/src/posix-metadata.c
+++ b/xlators/storage/posix/src/posix-metadata.c
@@ -631,7 +631,7 @@ posix_update_utime_in_mdata(xlator_t *this, const char *real_path, int fd,
tv.tv_sec = stbuf->ia_mtime;
SET_TIMESPEC_NSEC_OR_TIMEVAL_USEC(tv, stbuf->ia_mtime_nsec);
- flag.ctime = 1;
+ flag.ctime = 0;
flag.mtime = 1;
flag.atime = 0;
--
1.8.3.1

18
glusterfs.spec
View File

@ -231,7 +231,7 @@ Release: 0.1%{?prereltag:.%{prereltag}}%{?dist}
%else
Name: glusterfs
Version: 6.0
Release: 11%{?dist}
Release: 12%{?dist}
ExcludeArch: i686
%endif
License: GPLv2 or LGPLv3+
@ -574,6 +574,17 @@ Patch0262: 0262-glusterd-ctime-Disable-ctime-by-default.patch
Patch0263: 0263-tests-fix-ctime-related-tests.patch
Patch0264: 0264-gfapi-Fix-deadlock-while-processing-upcall.patch
Patch0265: 0265-fuse-add-missing-GF_FREE-to-fuse_interrupt.patch
Patch0266: 0266-geo-rep-Fix-mount-broker-setup-issue.patch
Patch0267: 0267-posix-ctime-Fix-race-during-lookup-ctime-xattr-heal.patch
Patch0268: 0268-rpc-transport-have-default-listen-port.patch
Patch0269: 0269-ec-fix-truncate-lock-to-cover-the-write-in-tuncate-c.patch
Patch0270: 0270-cluster-ec-inherit-healing-from-lock-when-it-has-inf.patch
Patch0271: 0271-cluster-ec-fix-EIO-error-for-concurrent-writes-on-sp.patch
Patch0272: 0272-cluster-ec-Always-read-from-good-mask.patch
Patch0273: 0273-cluster-ec-Fix-reopen-flags-to-avoid-misbehavior.patch
Patch0274: 0274-cluster-ec-Update-lock-good_mask-on-parent-fop-failu.patch
Patch0275: 0275-cluster-ec-Create-heal-task-with-heal-process-id.patch
Patch0276: 0276-features-utime-always-update-ctime-at-setattr.patch
%description
GlusterFS is a distributed file-system capable of scaling to several
@ -2282,6 +2293,11 @@ fi
%endif
%changelog
* Fri Aug 09 2019 Sunil Kumar Acharya <sheggodu@redhat.com> - 6.0-12
- fixes bugs bz#1730914 bz#1731448 bz#1732770 bz#1732792 bz#1733531
bz#1734305 bz#1734534 bz#1734734 bz#1735514 bz#1737705 bz#1732774
bz#1732793
* Tue Aug 06 2019 Sunil Kumar Acharya <sheggodu@redhat.com> - 6.0-11
- fixes bugs bz#1733520 bz#1734423