Compare commits

...

No commits in common. "imports/c8s-stream-10.5/mariadb-10.5.8-4.module+el8.4.0+10414+1abe6987" and "c8-stream-10.3" have entirely different histories.

21 changed files with 1294 additions and 2987 deletions

2
.gitignore vendored
View File

@ -1 +1 @@
SOURCES/mariadb-10.5.8.tar.gz
SOURCES/mariadb-10.3.39.tar.gz

View File

@ -1 +1 @@
6442a3c9d9d316086ce6822f18e2e026bd8422d0 SOURCES/mariadb-10.5.8.tar.gz
25972d22ed05249782141392f0893e71c7d549a9 SOURCES/mariadb-10.3.39.tar.gz

View File

@ -0,0 +1,132 @@
socat tunnel for encrypted rsync SST
====================================
`wsrep_sst_rsync_tunnel` is an extension of the rsync-based [SST](http://galeracluster.com/documentation-webpages/glossary.html#term-state-snapshot-transfer)
implementation that ships with mariadb. Its purpose is to encrypt
communication between the donor and the joiner during an SST.
Encryption is implemented by means of a socat tunnel, using OPENSSL
addresses. It can be configured via the regular openssl flags exposed
by socat.
## How to configure the script
This SST script can configured by setting a few keys in your favorite
mariadb option file in addition to the usual galera settings.
[mysqld]
...
bind_address=<node-name>
wsrep_sst_method=rsync_tunnel
...
[sst]
tca=/path/to/your/ca-file.crt
tcert=/path/to/node/certificate.crt
tkey=/path/to/node/key.key
sockopt=<openssl-address-options-as-per-socat-manual>
When a joiner node requests an SST, `wsrep_sst_rsync_tunnel` uses
socat to listen to incoming SSL connections on port 4444 in lieu of
the original rsync daemon. Received data will be forwarded to the
rscynd daemon started locally to replicate the database.
When a donor node serves the SST, `wsrep_sst_rsync_tunnel` makes
a series of rsync calls that target a locally started socat daemon.
The daemon tunnels all rsync traffic into an encrypted SSL connection
that targets the joiner's end of the socat tunnel.
Encryption parameters are specified under the `[sst]` group in the
mariadb option file, where `tkey` and `tcert` are respectively the key
and the certificate that are used by both sides of the socat tunnel.
Each node typically has a different key and cert. Both key and
certificate can be combined into a single PEM file and referenced by
`tcert`. Option `tca` holds a list of the trusted signing
certificates.
In case you need to tweak the creation of the SSL connection, you can
pass valid socat options (as per socat manual) via the `sockopt` key.
For debugging purpose, the exact socat command that is being executed
shows up in the mariadb log file.
Note that socat verifies that the certificate's commonName matches
that of the host that is being targeted. The target name comes from
the value configured in `bind_address`, so it's important that it
matches the certificate's commonName. An IP address can be used for
`bind_address`, but you may get into trouble in case different
hostnames resolve to the same IP (e.g. multiple networks per host).
## Examples of use
Suppose you're running a 3-node galera cluster
`node1.my.cluster`, `node2.my.cluster`, `node3.my.cluster`.
### Scenario: using self-signed certificates
On each node, create a key and a certificate, and bundle them into a
single PEM file. For instance on `node1.my.cluster`:
openssl genrsa -out /tls/mysql-$(hostname -f).key 2048
openssl req -new -key /tls/mysql-$(hostname -f).key -x509 -days 365000 -subj "/CN=$(hostname -f)" -out /tls/mysql-$(hostname -f).crt -batch
cat /tls/mysql-$(hostname -f).key /tls/mysql-$(hostname -f).crt > /tls/mysql.pem
Then, on each node, create a cafile that will contain all the certs to
trust:
for n in node1.my.cluster node2.my.cluster node3.my.cluster; do
ssh $n 'cat /tls/mysql-$(hostname -f).crt' >> /tls/all-mysql.crt
done
Once you have those two files on each host, you can configure the SST
appropriately. For instance from `/etc/my.cnf.d/galera.cnf`:
[mysqld]
...
[sst]
tca=/tls/all-mysql.crt
tcert=/tls/mysql.pem
### Scenario: using self-signed certificates, without verification
By default, when socat tries to establish a SSL connection to a peer,
it also verifies that it can trust the peer's certificate. If for some
reason you need to disable that feature, you can amend the previous
configuration with a sockopt option:
[mysqld]
...
[sst]
tca=/tls/all-mysql.crt
tcert=/tls/mysql.pem
sockopt="verify=0"
The associated sockopt value is passed to socat when
the donor or the joiner configures his part of the tunnel.
Note: please do not do so in production, this is inherently insecure
as you will not verify the identity of the peer you're connecting to!
### Scenario: using certificates from a CA
Suppose you have a FreeIPA service which generated a key file and a
certificate file for the three galera nodes, respectively located at
/tls/mysql.key and /tls/mysql.crt.
Assuming that the certificate for the FreeIPA server is available at
/etc/ipa/ca.crt, you can configure you galera servers as follows:
[sst]
tca=/etc/ipa/ca.crt
tcert=/tls/mysql.crt
tkey=/tls/mysql.key
## License
Copyright © 2017 [Damien Ciabrini](https://github.com/dciabrin).
This work is derived from the original `wsrep_rsync_sst`, copyright
© 2010-2014 [Codership Oy](https://github.com/codership).
Released under the GNU GPLv2.

View File

@ -0,0 +1,101 @@
--- mariadb-10.3.39/mysql-test/include/default_mysqld.cnf 2023-05-03 06:32:44.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/include/default_mysqld.cnf 2023-07-07 13:58:40.255283041 +0200
@@ -127,3 +127,8 @@ local-infile
# tables. Let's enable it in the [server] group, because this group
# is read after [mysqld] and [embedded]
loose-aria
+
+[mysqltest]
+loose-ssl-ca=@ENV.MYSQL_TEST_DIR/std_data/cacert.pem
+loose-ssl-cert=@ENV.MYSQL_TEST_DIR/std_data/server-cert.pem
+loose-ssl-key=@ENV.MYSQL_TEST_DIR/std_data/server-key.pem
--- mariadb-10.3.39/mysql-test/include/wait_until_connected_again.inc 2023-05-03 06:32:44.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/include/wait_until_connected_again.inc 2023-07-07 13:55:30.424368106 +0200
@@ -11,7 +11,7 @@ let $counter= 5000;
let $mysql_errno= 9999;
while ($mysql_errno)
{
- --error 0,ER_SERVER_SHUTDOWN,ER_CONNECTION_KILLED,ER_LOCK_WAIT_TIMEOUT,2002,2006,2013
+ --error 0,ER_SERVER_SHUTDOWN,ER_CONNECTION_KILLED,ER_LOCK_WAIT_TIMEOUT,2002,2006,2013,2026
show status;
dec $counter;
--- mariadb-10.3.39/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2023-05-03 06:32:45.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/suite/rpl/t/rpl_err_ignoredtable.test 2023-07-07 13:54:31.152082427 +0200
@@ -53,7 +53,7 @@ insert into t4 values (3),(4);
connection master;
# The get_lock function causes warning for unsafe statement.
--disable_warnings
---error 0,1317,2013
+--error 0,1317,2013,2026
reap;
--enable_warnings
connection master1;
--- mariadb-10.3.39/mysql-test/suite/innodb/t/innodb_bug51920.test 2023-05-03 06:32:44.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/suite/innodb/t/innodb_bug51920.test 2023-07-07 15:11:39.000404508 +0200
@@ -36,7 +36,7 @@ let $wait_condition =
# depending on platform.
#
connection con1;
--- error 1317, 2006, 2013, ER_CONNECTION_KILLED
+-- error 1317, 2006, 2013, 2026, ER_CONNECTION_KILLED
reap;
connection default;
DROP TABLE bug51920;
--- mariadb-10.3.39/mysql-test/main/lock_kill.test 2023-05-03 06:32:44.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/main/lock_kill.test 2023-07-07 15:13:54.335086789 +0200
@@ -17,7 +17,7 @@ LOCK TABLE t1 WRITE;
eval KILL $conid;
--enable_query_log
--connection con1
---error 0,2006,2013,ER_CONNECTION_KILLED
+--error 0,2006,2013,2026,ER_CONNECTION_KILLED
reap;
--connection default
--disconnect con1
@@ -35,7 +35,7 @@ LOCK TABLE t1 WRITE, t2 WRITE;
eval KILL $conid;
--enable_query_log
--connection con1
---error 0,2006,2013,ER_CONNECTION_KILLED
+--error 0,2006,2013,2026,ER_CONNECTION_KILLED
reap;
--connection default
--disconnect con1
--- mariadb-10.3.39/mysql-test/main/loadxml.test 2023-05-03 06:32:44.000000000 +0200
+++ ../../mariadb-10.3.39/mysql-test/main/loadxml.test 2023-07-07 15:15:14.862492763 +0200
@@ -83,7 +83,7 @@ connection default;
connection addconroot;
# Read response from connection to avoid packets out-of-order when disconnecting
# Note, that connection can already be dead due to previously issued kill
---error 0,2013
+--error 0,2013,2026
--reap
disconnect addconroot;
connection default;
--- mariadb-10.3.39/plugin/disks/mysql-test/disks/disks.test 2023-05-03 06:32:45.000000000 +0200
+++ ../../mariadb-10.3.39/plugin/disks/mysql-test/disks/disks.test 2023-07-10 11:48:28.859497746 +0200
@@ -1,3 +1,3 @@
--replace_regex /varchar\([0-9]+\)/varchar(pathlen)/
show create table information_schema.disks;
-select sum(Total) > sum(Available), sum(Total)>sum(Used) from information_schema.disks;
+select sum(Total) >= sum(Available), sum(Total) >= sum(Used) from information_schema.disks;
--- mariadb-10.3.39/plugin/disks/mysql-test/disks/disks.result 2023-05-03 06:32:45.000000000 +0200
+++ ../../mariadb-10.3.39/plugin/disks/mysql-test/disks/disks.result 2023-07-10 12:47:10.460233056 +0200
@@ -7,6 +7,6 @@ DISKS CREATE TEMPORARY TABLE `DISKS` (
`Used` bigint(32) NOT NULL,
`Available` bigint(32) NOT NULL
) ENGINE=MEMORY DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci
-select sum(Total) > sum(Available), sum(Total)>sum(Used) from information_schema.disks;
-sum(Total) > sum(Available) sum(Total)>sum(Used)
+select sum(Total) >= sum(Available), sum(Total) >= sum(Used) from information_schema.disks;
+sum(Total) >= sum(Available) sum(Total) >= sum(Used)
1 1

View File

@ -0,0 +1,13 @@
Harden the "hardened" flags even more to comply with RHEL8 security rules
--- mariadb-10.3.25/CMakeLists.txt 2020-10-05 18:19:45.000000000 +0200
+++ mariadb-10.3.25/CMakeLists.txt_patched 2020-11-03 01:29:52.369426705 +0100
@@ -247,7 +247,7 @@ IF(NOT WITH_TSAN)
# security-enhancing flags
MY_CHECK_AND_SET_COMPILER_FLAG("-pie -fPIC")
MY_CHECK_AND_SET_LINKER_FLAG("-Wl,-z,relro,-z,now")
- MY_CHECK_AND_SET_COMPILER_FLAG("-fstack-protector --param=ssp-buffer-size=4")
+ MY_CHECK_AND_SET_COMPILER_FLAG("-fstack-protector-strong --param=ssp-buffer-size=4")
MY_CHECK_AND_SET_COMPILER_FLAG("-D_FORTIFY_SOURCE=2" RELEASE RELWITHDEBINFO)
ENDIF()
ENDIF()

View File

@ -1,29 +0,0 @@
This scirpt is ran by the systemd service.
In Fedora the service has priviledges dropped to the mysql user.
Thus "chown 0" will always fail
Never parse 'ls' output!
http://mywiki.wooledge.org/BashFAQ/087
--- mariadb-10.4.12/scripts/mysql_install_db.sh 2020-01-26 21:43:53.000000000 +0100
+++ mariadb-10.4.12/scripts/mysql_install_db.sh_patched 2020-01-29 11:11:09.448812331 +0100
@@ -482,13 +482,16 @@ if test -n "$user"
then
if test -z "$srcdir" -a "$in_rpm" -eq 0
then
- chown 0 "$pamtooldir/auth_pam_tool_dir/auth_pam_tool" && \
- chmod 04755 "$pamtooldir/auth_pam_tool_dir/auth_pam_tool"
- if test $? -ne 0
+ if [ `stat "$pamtooldir/auth_pam_tool_dir/auth_pam_tool" -c %u` -ne 0 ]
then
+ chown 0 "$pamtooldir/auth_pam_tool_dir/auth_pam_tool" && \
+ chmod 04755 "$pamtooldir/auth_pam_tool_dir/auth_pam_tool"
+ if test $? -ne 0
+ then
echo "Couldn't set an owner to '$pamtooldir/auth_pam_tool_dir/auth_pam_tool'."
echo "It must be root, the PAM authentication plugin doesn't work otherwise.."
echo
+ fi
fi
chown $user "$pamtooldir/auth_pam_tool_dir" && \
chmod 0700 "$pamtooldir/auth_pam_tool_dir"

View File

@ -1,30 +0,0 @@
# Fixing conflict with groonga package
# https://bugzilla.redhat.com/show_bug.cgi?id=1763287
--- mariadb-10.3.18/storage/mroonga/vendor/groonga/CMakeLists.txt.withoutoption 2019-11-11 14:01:07.762595716 +0100
+++ mariadb-10.3.18/storage/mroonga/vendor/groonga/CMakeLists.txt 2019-11-11 14:33:05.224012458 +0100
@@ -86,7 +86,9 @@
set(INCLUDE_DIR "include")
set(GRN_INCLUDE_DIR "include/groonga")
set(DATA_DIR "share")
-set(GRN_DATA_DIR "${DATA_DIR}/${GRN_PROJECT_NAME}")
+if(NOT DEFINED GRN_DATA_DIR)
+ set(GRN_DATA_DIR "${DATA_DIR}/${GRN_PROJECT_NAME}")
+endif()
set(CONFIG_DIR "etc")
set(GRN_CONFIG_DIR "${CONFIG_DIR}/${GRN_PROJECT_NAME}")
set(GRN_CONFIG_PATH "${CMAKE_INSTALL_PREFIX}/${GRN_CONFIG_DIR}/groonga.conf")
--- mariadb-10.3.18/storage/mroonga/vendor/groonga/vendor/plugins/groonga-normalizer-mysql/CMakeLists.txt.withoutoption 2019-11-11 14:34:22.661005715 +0100
+++ mariadb-10.3.18/storage/mroonga/vendor/groonga/vendor/plugins/groonga-normalizer-mysql/CMakeLists.txt 2019-11-11 14:35:59.962244120 +0100
@@ -16,7 +16,9 @@
# MA 02110-1335 USA
cmake_minimum_required(VERSION 2.6)
-set(GROONGA_NORMALIZER_MYSQL_PROJECT_NAME "groonga-normalizer-mysql")
+if (NOT DEFINED GROONGA_NORMALIZER_MYSQL_PROJECT_NAME)
+ set(GROONGA_NORMALIZER_MYSQL_PROJECT_NAME "groonga-normalizer-mysql")
+endif()
project("${GROONGA_NORMALIZER_MYSQL_PROJECT_NAME}")
if(DEFINED GROONGA_NORMALIZER_MYSQL_EMBED)

View File

@ -26,15 +26,13 @@ Update 6/2018
beeing able to send the SIGHUP to the process and read the mysqld pid file, which root can.
* Submited as PR: https://github.com/MariaDB/server/pull/807
Update 02/2021
* Enhance the script as proposed in:
https://mariadb.com/kb/en/rotating-logs-on-unix-and-linux/
* Discussion continues in:
https://jira.mariadb.org/browse/MDEV-16621
Update 01/2022
* added delaycompress option
* see https://mariadb.com/kb/en/rotating-logs-on-unix-and-linux
--- mariadb-10.5.8/support-files/mysql-log-rotate.sh 2021-02-12 08:37:47.857289694 +0100
+++ mariadb-10.5.8/support-files/mysql-log-rotate.sh_pacthed 2021-02-12 08:40:26.420372325 +0100
@@ -3,35 +3,22 @@
--- mariadb-10.3.32/support-files/mysql-log-rotate.sh 2022-01-14 17:03:27.000000000 +0100
+++ mariadb-10.3.32/support-files/mysql-log-rotate.sh_patched 2022-01-17 15:07:54.205379672 +0100
@@ -3,36 +3,22 @@
# in the [mysqld] section as follows:
#
# [mysqld]
@ -58,15 +56,15 @@ Update 02/2021
- # create 600 mysql mysql
+@LOG_LOCATION@ {
+ create 600 mysql mysql
su mysql mysql
notifempty
daily
rotate 3
missingok
compress
+ delaycompress
+ sharedscripts
postrotate
# just if mariadbd is really running
# just if mysqld is really running
- if test -x @bindir@/mysqladmin && \
- @bindir@/mysqladmin ping &>/dev/null
- then

View File

@ -0,0 +1,12 @@
--- mariadb-10.3.39/scripts/wsrep_sst_mariabackup.sh 2023-08-11 11:31:40.415022889 +0200
+++ ../../mariadb-10.3.39/scripts/wsrep_sst_mariabackup.sh 2023-08-11 11:32:01.924161077 +0200
@@ -340,6 +340,9 @@ get_transfer()
"Use workaround for socat $SOCAT_VERSION bug"
fi
fi
+ if check_for_version "$SOCAT_VERSION" '1.7.4'; then
+ tcmd="$tcmd,no-sni=1"
+ fi
fi
if [ "${sockopt#*,dhparam=}" = "$sockopt" ]; then

View File

@ -1,14 +1,16 @@
--- mariadb-10.4.14/support-files/CMakeLists.txt 2020-08-06 17:28:28.000000000 +0200
+++ mariadb-10.4.14/support-files/CMakeLists.txt_patched 2020-09-03 13:21:07.826658279 +0200
@@ -187,6 +187,7 @@ IF(UNIX)
COMPONENT SharedLibraries)
INSTALL(FILES rpm/mysql-clients.cnf DESTINATION ${INSTALL_SYSCONF2DIR}
COMPONENT Client)
+ CONFIGURE_FILE(rpm/server.cnf ${CMAKE_CURRENT_SOURCE_DIR}/rpm/server.cnf @ONLY)
INSTALL(FILES rpm/server.cnf DESTINATION ${INSTALL_SYSCONF2DIR}
COMPONENT IniFiles)
INSTALL(FILES rpm/enable_encryption.preset DESTINATION ${INSTALL_SYSCONF2DIR}
diff -up mariadb-10.1.8/support-files/CMakeLists.txt.p9 mariadb-10.1.8/support-files/CMakeLists.txt
--- mariadb-10.2.32/support-files/CMakeLists.txt 2020-05-08 13:45:27.000000000 +0200
+++ mariadb-10.2.32/support-files/CMakeLists.txt_pacthed 2020-05-13 10:11:30.884190396 +0200
@@ -100,7 +100,8 @@ IF(UNIX)
ENDIF()
CONFIGURE_FILE(mariadb.pc.in ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc @ONLY)
- INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc DESTINATION ${INSTALL_LIBDIR}/pkgconfig COMPONENT Development)
+ CONFIGURE_FILE(rpm/server.cnf ${CMAKE_CURRENT_BINARY_DIR}/rpm/server.cnf @ONLY)
+ INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc DESTINATION ${INSTALL_SHAREDIR}/pkgconfig COMPONENT Development)
INSTALL(FILES mysql.m4 DESTINATION ${INSTALL_SHAREDIR}/aclocal COMPONENT Development)
diff -up mariadb-10.0.15/support-files/rpm/server.cnf.ownsetup mariadb-10.0.15/support-files/rpm/server.cnf
--- mariadb-10.0.15/support-files/rpm/server.cnf.ownsetup 2015-01-24 23:55:55.110063592 +0100
+++ mariadb-10.0.15/support-files/rpm/server.cnf 2015-01-24 23:57:42.308114387 +0100

View File

@ -3,17 +3,18 @@ Use PCDIR CMake option, if configured
Upstream install the server pkgconfig file into arch-independent directory
Reported to upstream as: https://jira.mariadb.org/browse/MDEV-14340
--- mariadb-10.5.5/support-files/CMakeLists.txt.old 2020-09-30 10:36:08.582490318 +0200
+++ mariadb-10.5.5/support-files/CMakeLists.txt 2020-09-30 10:38:58.079710848 +0200
@@ -91,7 +91,11 @@
ENDIF()
--- mariadb-10.3.12/support-files/CMakeLists.txt 2019-03-20 15:25:53.423283135 +0100
+++ mariadb-10.3.12/support-files/CMakeLists.txt_patched 2019-03-20 15:38:56.372819958 +0100
@@ -82,7 +82,12 @@ IF(UNIX)
CONFIGURE_FILE(mariadb.pc.in ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc @ONLY)
CONFIGURE_FILE(rpm/server.cnf ${CMAKE_CURRENT_BINARY_DIR}/rpm/server.cnf @ONLY)
+IF(INSTALL_PCDIR)
+ INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc DESTINATION ${INSTALL_PCDIR} COMPONENT Development)
+ELSE()
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc DESTINATION ${INSTALL_LIBDIR}/pkgconfig COMPONENT Development)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/mariadb.pc DESTINATION ${INSTALL_SHAREDIR}/pkgconfig COMPONENT Development)
+ENDIF()
+
INSTALL(FILES mysql.m4 DESTINATION ${INSTALL_SHAREDIR}/aclocal COMPONENT Development)

View File

@ -1,12 +1,11 @@
We have some downstream patches and other scripts that include variables to
be expanded by cmake. Cmake needs to know about them, so adding them manually.
# Install libgcc as mylibgcc.a
--- mariadb-10.5.5/scripts/CMakeLists.txt.old 2020-09-24 10:13:35.272589689 +0200
+++ mariadb-10.5.5/scripts/CMakeLists.txt 2020-09-24 10:17:31.428985798 +0200
@@ -377,6 +377,34 @@
INSTALL_LINK(${file} ${binname} ${INSTALL_BINDIR} ${${file}_COMPONENT})
ENDIF()
--- mariadb-10.3.8/scripts/CMakeLists.txt 2018-07-02 09:34:11.000000000 +0200
+++ mariadb-10.3.8/scripts/CMakeLists.txt_patched 2018-07-03 10:58:15.954670153 +0200
@@ -361,6 +361,34 @@ ELSE()
COMPONENT ${${file}_COMPONENT}
)
ENDFOREACH()
+
+ # files for systemd

File diff suppressed because it is too large Load Diff

View File

@ -1,13 +0,0 @@
diff -up mariadb-10.3.9/mysql-test/main/ssl_cipher.test.fixtest mariadb-10.3.9/mysql-test/main/ssl_cipher.test
--- mariadb-10.3.13/mysql-test/main/ssl_cipher.test 2019-02-20 08:59:09.000000000 +0100
+++ mariadb-10.3.13/mysql-test/main/ssl_cipher.test_patched 2019-02-22 11:22:01.250256060 +0100
@@ -97,7 +97,9 @@ drop user mysqltest_1@localhost;
let $restart_parameters=--ssl-cipher=AES128-SHA;
source include/restart_mysqld.inc;
connect (ssl_con,localhost,root,,,,,SSL);
+--replace_regex /TLS_AES_.*/AES128-SHA/
SHOW STATUS LIKE 'Ssl_cipher';
+--replace_regex /TLS_AES_.*/AES128-SHA/
SHOW STATUS LIKE 'Ssl_cipher_list';
disconnect ssl_con;
connection default;

View File

@ -1,2 +1,3 @@
# Fails since 10.3.17, only on armv7hl
versioning.partition :
# Fails on aarch64
innodb.innodb_buffer_pool_resize :
innodb.innodb_buffer_pool_resize_with_chunks :

View File

@ -1,60 +1,27 @@
# The SSL test are failing correctly. Fro more explanation, see:
# https://jira.mariadb.org/browse/MDEV-8404?focusedCommentId=84275&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-84275
main.ssl_7937 : #1399847
main.ssl_8k_key :
main.ssl_crl : #1399847
# Fails everywhere
innodb.innodb_defrag_binlog :
# ------------------------------
# Tests that fails because of 'Self Signed Certificate in the Certificate Chain'
perfschema.cnf_option :
main.ssl_7937 :
main.ssl_8k_key :
main.ssl_crl :
main.ssl_system_ca :
main.userstat :
rpl.rpl_row_img_blobs :
rpl.rpl_row_img_eng_min :
rpl.rpl_row_img_eng_noblob :
sys_vars.slave_parallel_threads_basic :
# ------------------------------
# Expected to fail, the plugin is not build with server, but 'mariadb-connector-c' instead
plugins.auth_ed25519 :
plugins.multiauth :
# ------------------------------
perfschema.nesting : #1399847
perfschema.socket_summary_by_instance_func : #1399847
perfschema.cnf_option :
perfschema.nesting :
perfschema.socket_summary_by_event_name_func :
perfschema.socket_summary_by_instance_func :
# ------------------------------
# Fails since 10.1.12
innodb.innodb_defrag_binlog :
plugins.feedback_plugin_load :
# Fails everywhere since 10.2.15
main.userstat :
rpl.rpl_row_img_blobs :
rpl.rpl_row_img_eng_min :
rpl.rpl_row_img_eng_noblob :
# Fails everywhere since 10.4.11
main.events_bugs :
sys_vars.tcp_nodelay :
# Fails on i686
encryption.innodb-redo-badkey :
# Fails since 10.5.2
main.mysqld--help2 :
disks.disks :
disks.disks_notembedded :
# Fails since 10.5.3
main.mysqld--help-aria :
# Fails since 10.5.4
main.ssl_system_ca :
# Fails since 10.5.7
innodb.innodb_wl6326_big :
plugins.feedback_plugin_load :
sys_vars.slave_parallel_threads_basic :
# Fails only on i686
main.myisampack :
main.myisampack :
# Fails everywhere in 10.5.8
rpl.rpl_innodb_mixed_dml :
# We don't build the plugin in server; we build it in mariadb-connector-c instead
plugins.auth_ed25519 :

View File

@ -1,2 +1,30 @@
# Fails on ppc64le since 10.4.12
oqgraph.social :
# Fails on ppc64le
parts.partition_alter1_1_innodb :
parts.partition_alter1_2_innodb :
parts.partition_alter1_1_2_innodb :
parts.partition_alter1_2_1_innodb :
parts.partition_alter2_1_1_innodb :
parts.partition_alter1_2_2_innodb :
parts.partition_alter2_1_2_innodb :
parts.partition_alter2_2_1_innodb :
parts.partition_alter2_2_2_innodb :
parts.partition_alter4_innodb :
parts.partition_basic_innodb :
parts.part_supported_sql_func_innodb :
rpl.rpl_loaddata_m :
#
stress.ddl_innodb :
innodb.innodb_buffer_pool_resize :
innodb.innodb_buffer_pool_resize_with_chunks :
innodb.innodb_bulk_create_index :
innodb.innodb_defrag_binlog :
innodb.innodb_defrag_concurrent :
innodb_gis.kill_server :
gcol.innodb_virtual_basic :
# Unstable (randomly failing) tests
innodb_gis.rtree_search :
main.type_ranges :

View File

@ -1,3 +1,8 @@
# Fails since 10.5.2
perfschema.memory_aggregate_32bit :
period.overlaps :
# Fails on s390x
disks.disks :
disks.disks_notembedded :
# related to MDEV-20194
# first check of `undefined` table causes warning,
# instead INSERT and ALTER should cause it
innodb.row_size_error_log_warnings_3 :

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,492 @@
#!/bin/bash -ue
# Copyright (C) 2010-2014 Codership Oy
# Copyright (C) 2017-2020 Damien Ciabrini <damien.ciabrini@gmail.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING. If not, write to the
# Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston
# MA 02110-1301 USA.
# This is a reference script for rsync-based state snapshot tansfer
# over an encrypted communication channel, managed by socat
RSYNC_PID= # rsync pid file
RSYNC_CONF= # rsync configuration file
RSYNC_REAL_PID= # rsync process id
SOCAT_PID= # socat pid file
SOCAT_REAL_PID= # socat process id
SOCAT_OPTS= # openssl connection args
MODULE="rsync_tunnel_sst"
OS=$(uname)
[ "$OS" == "Darwin" ] && export -n LD_LIBRARY_PATH
# Setting the path for lsof on CentOS
export PATH="/usr/sbin:/sbin:$PATH"
. $(dirname $0)/wsrep_sst_common
wsrep_check_programs rsync socat
cleanup_pid()
{
local real_pid=$1
[ "0" != "$real_pid" ] && \
kill $real_pid && \
sleep 0.5 && \
kill -9 $real_pid >/dev/null 2>&1 || \
:
}
cleanup_tunnel()
{
if [ -n "$SOCAT_REAL_PID" ] && ps -p "$SOCAT_REAL_PID" >/dev/null 2>&1; then
wsrep_log_info "cleanup socat PID: $SOCAT_REAL_PID"
cleanup_pid $SOCAT_REAL_PID
fi
rm -rf "$SOCAT_PID"
}
cleanup_joiner()
{
wsrep_log_info "Joiner cleanup. rsync PID: $RSYNC_REAL_PID"
[ -n "$RSYNC_REAL_PID" ] && cleanup_pid $RSYNC_REAL_PID
rm -rf "$RSYNC_CONF"
rm -rf "$MAGIC_FILE"
rm -rf "$RSYNC_PID"
cleanup_tunnel
wsrep_log_info "Joiner cleanup done."
if [ "${WSREP_SST_OPT_ROLE}" = "joiner" ];then
wsrep_cleanup_progress_file
fi
}
# Check whether process is still running.
check_pid()
{
local pid_file=$1
[ -r "$pid_file" ] && ps -p $(cat $pid_file) >/dev/null 2>&1
}
check_pid_and_port()
{
local pid_file=$1
local service_pid=$2
local service_port=$3
local service_host=$4
local service_name=$5
if ! which lsof > /dev/null; then
wsrep_log_error "lsof tool not found in PATH! Make sure you have it installed."
exit 2 # ENOENT
fi
local port_info=$(lsof -i "@"$service_host:$service_port -Pn 2>/dev/null | \
grep "(LISTEN)")
local is_service=$(echo $port_info | \
grep -w '^'"$service_name"'[[:space:]]\+'"$service_pid" 2>/dev/null)
if [ -n "$port_info" -a -z "$is_service" ]; then
wsrep_log_error "$service_name daemon port '$service_port' has been taken"
exit 16 # EBUSY
fi
if ! check_pid $pid_file; then
wsrep_log_error "$service_name process terminated unexpectedly"
exit 10 # ECHILD
fi
[ -n "$port_info" ] && [ -n "$is_service" ] && \
[ $(cat $pid_file) -eq $service_pid ]
}
config_from_cnf()
{
local group=$1
local key=$2
echo $($MY_PRINT_DEFAULTS $group | grep -- "--$key=" | cut -d= -f2- | tail -1)
}
setup_tunnel_args()
{
tca=$(config_from_cnf sst tca)
tkey=$(config_from_cnf sst tkey)
tcert=$(config_from_cnf sst tcert)
sockopt=$(config_from_cnf sst sockopt)
if [ -z "$tcert" ]; then
wsrep_log_error "Encryption certificate not found in my.cnf"
exit 3
else
SOCAT_OPTS="cert=$tcert"
fi
[ -n "$tkey" ] && SOCAT_OPTS="$SOCAT_OPTS,key=$tkey"
[ -n "$tca" ] && SOCAT_OPTS="$SOCAT_OPTS,cafile=$tca"
wsrep_log_info "Encryption setting to be used for socat tunnel: $SOCAT_OPTS"
[ -n "$sockopt" ] && SOCAT_OPTS="$SOCAT_OPTS,$sockopt"
}
MAGIC_FILE="$WSREP_SST_OPT_DATA/rsync_tunnel_sst_complete"
rm -rf "$MAGIC_FILE"
BINLOG_TAR_FILE="$WSREP_SST_OPT_DATA/wsrep_sst_binlog.tar"
BINLOG_N_FILES=1
rm -f "$BINLOG_TAR_FILE" || :
if ! [ -z $WSREP_SST_OPT_BINLOG ]
then
BINLOG_DIRNAME=$(dirname $WSREP_SST_OPT_BINLOG)
BINLOG_FILENAME=$(basename $WSREP_SST_OPT_BINLOG)
fi
WSREP_LOG_DIR=${WSREP_LOG_DIR:-""}
# if WSREP_LOG_DIR env. variable is not set, try to get it from my.cnf
if [ -z "$WSREP_LOG_DIR" ]; then
WSREP_LOG_DIR=$($MY_PRINT_DEFAULTS --mysqld \
| grep -- '--innodb[-_]log[-_]group[-_]home[-_]dir=' \
| cut -b 29- )
fi
if [ -n "$WSREP_LOG_DIR" ]; then
# handle both relative and absolute paths
WSREP_LOG_DIR=$(cd $WSREP_SST_OPT_DATA; mkdir -p "$WSREP_LOG_DIR"; cd $WSREP_LOG_DIR; pwd -P)
else
# default to datadir
WSREP_LOG_DIR=$(cd $WSREP_SST_OPT_DATA; pwd -P)
fi
# Old filter - include everything except selected
# FILTER=(--exclude '*.err' --exclude '*.pid' --exclude '*.sock' \
# --exclude '*.conf' --exclude core --exclude 'galera.*' \
# --exclude grastate.txt --exclude '*.pem' \
# --exclude '*.[0-9][0-9][0-9][0-9][0-9][0-9]' --exclude '*.index')
# New filter - exclude everything except dirs (schemas) and innodb files
FILTER=(-f '- /lost+found' -f '- /.fseventsd' -f '- /.Trashes'
-f '+ /wsrep_sst_binlog.tar' -f '+ /ib_lru_dump' -f '+ /ibdata*' -f '+ /*/' -f '- /*')
SOCAT_PID="$WSREP_SST_OPT_DATA/$MODULE-socat.pid"
if check_pid $SOCAT_PID
then
wsrep_log_error "socat tunnel already running."
exit 114 # EALREADY
fi
rm -rf "$SOCAT_PID"
setup_tunnel_args
if [ "$WSREP_SST_OPT_ROLE" = "donor" ]
then
SOCAT_JOINER_ADDR=$(echo $WSREP_SST_OPT_ADDR | awk -F'/' '{print $1}')
# map to name in case we received an IP
SOCAT_JOINER_HOST=$(getent hosts $SOCAT_JOINER_ADDR | awk '{ print $2 }')
if [ -z "$SOCAT_JOINER_HOST" ]; then
SOCAT_JOINER_HOST=$SOCAT_JOINER_ADDR
fi
SOCAT_PORT=$(echo $SOCAT_JOINER_ADDR | awk -F ':' '{ print $2 }')
if [ -z "$SOCAT_PORT" ]
then
SOCAT_PORT=4444
fi
TARGET_ADDR=localhost:$SOCAT_PORT/$MODULE
trap cleanup_tunnel EXIT
# Socat forwards rsync connections to the joiner
SOCAT_SRC=tcp-listen:$SOCAT_PORT,bind=localhost,reuseaddr,fork
SOCAT_DST=openssl:$SOCAT_JOINER_HOST,$SOCAT_OPTS
wsrep_log_info "Setting up tunnel for donor: socat $SOCAT_SRC $SOCAT_DST"
socat $SOCAT_SRC $SOCAT_DST &
SOCAT_REAL_PID=$!
# This is ok because a local galera node doesn't run SST concurrently
echo $SOCAT_REAL_PID >"$SOCAT_PID"
until check_pid_and_port $SOCAT_PID $SOCAT_REAL_PID $SOCAT_PORT localhost "socat"
do
sleep 0.2
done
if [ $WSREP_SST_OPT_BYPASS -eq 0 ]
then
FLUSHED="$WSREP_SST_OPT_DATA/tables_flushed"
ERROR="$WSREP_SST_OPT_DATA/sst_error"
rm -rf "$FLUSHED"
rm -rf "$ERROR"
# Use deltaxfer only for WAN
inv=$(basename $0)
[ "$inv" = "wsrep_sst_rsync_wan" ] && WHOLE_FILE_OPT="" \
|| WHOLE_FILE_OPT="--whole-file"
echo "flush tables"
# Wait for :
# (a) Tables to be flushed, AND
# (b) Cluster state ID & wsrep_gtid_domain_id to be written to the file, OR
# (c) ERROR file, in case flush tables operation failed.
while [ ! -r "$FLUSHED" ] && ! grep -q ':' "$FLUSHED" >/dev/null 2>&1
do
# Check whether ERROR file exists.
if [ -f "$ERROR" ]
then
# Flush tables operation failed.
rm -rf "$ERROR"
exit 255
fi
sleep 0.2
done
STATE="$(cat $FLUSHED)"
rm -rf "$FLUSHED"
sync
if ! [ -z $WSREP_SST_OPT_BINLOG ]
then
# Prepare binlog files
pushd $BINLOG_DIRNAME &> /dev/null
binlog_files_full=$(tail -n $BINLOG_N_FILES ${BINLOG_FILENAME}.index)
binlog_files=""
for ii in $binlog_files_full
do
binlog_files="$binlog_files $(basename $ii)"
done
if ! [ -z "$binlog_files" ]
then
wsrep_log_info "Preparing binlog files for transfer:"
tar -cvf $BINLOG_TAR_FILE $binlog_files >&2
fi
popd &> /dev/null
fi
# first, the normal directories, so that we can detect incompatible protocol
RC=0
rsync --owner --group --perms --links --specials \
--ignore-times --inplace --dirs --delete --quiet \
$WHOLE_FILE_OPT "${FILTER[@]}" "$WSREP_SST_OPT_DATA/" \
rsync://$TARGET_ADDR >&2 || RC=$?
if [ "$RC" -ne 0 ]; then
wsrep_log_error "rsync returned code $RC:"
case $RC in
12) RC=71 # EPROTO
wsrep_log_error \
"rsync server on the other end has incompatible protocol. " \
"Make sure you have the same version of rsync on all nodes."
;;
22) RC=12 # ENOMEM
;;
*) RC=255 # unknown error
;;
esac
exit $RC
fi
# second, we transfer InnoDB log files
rsync --owner --group --perms --links --specials \
--ignore-times --inplace --dirs --delete --quiet \
$WHOLE_FILE_OPT -f '+ /ib_logfile[0-9]*' -f '- **' "$WSREP_LOG_DIR/" \
rsync://$TARGET_ADDR-log_dir >&2 || RC=$?
if [ $RC -ne 0 ]; then
wsrep_log_error "rsync innodb_log_group_home_dir returned code $RC:"
exit 255 # unknown error
fi
# then, we parallelize the transfer of database directories, use . so that pathconcatenation works
pushd "$WSREP_SST_OPT_DATA" >/dev/null
count=1
[ "$OS" == "Linux" ] && count=$(grep -c processor /proc/cpuinfo)
[ "$OS" == "Darwin" -o "$OS" == "FreeBSD" ] && count=$(sysctl -n hw.ncpu)
find . -maxdepth 1 -mindepth 1 -type d -not -name "lost+found" -print0 | \
xargs -I{} -0 -P $count \
rsync --owner --group --perms --links --specials \
--ignore-times --inplace --recursive --delete --quiet \
$WHOLE_FILE_OPT --exclude '*/ib_logfile*' "$WSREP_SST_OPT_DATA"/{}/ \
rsync://$TARGET_ADDR/{} >&2 || RC=$?
popd >/dev/null
if [ $RC -ne 0 ]; then
wsrep_log_error "find/rsync returned code $RC:"
exit 255 # unknown error
fi
else # BYPASS
wsrep_log_info "Bypassing state dump."
# Store donor's wsrep GTID (state ID) and wsrep_gtid_domain_id
# (separated by a space).
STATE="$WSREP_SST_OPT_GTID $WSREP_SST_OPT_GTID_DOMAIN_ID"
fi
echo "continue" # now server can resume updating data
echo "$STATE" > "$MAGIC_FILE"
rsync --archive --quiet --checksum "$MAGIC_FILE" rsync://$TARGET_ADDR
# to avoid cleanup race, stop tunnel before declaring the SST finished.
# This ensures galera won't start a new SST locally before we exit.
cleanup_tunnel
echo "done $STATE"
elif [ "$WSREP_SST_OPT_ROLE" = "joiner" ]
then
wsrep_check_programs lsof socat
touch $SST_PROGRESS_FILE
MYSQLD_PID=$WSREP_SST_OPT_PARENT
RSYNC_PID="$WSREP_SST_OPT_DATA/$MODULE.pid"
if check_pid $RSYNC_PID
then
wsrep_log_error "rsync daemon already running."
exit 114 # EALREADY
fi
rm -rf "$RSYNC_PID"
ADDR=$WSREP_SST_OPT_ADDR
RSYNC_PORT=$(echo $ADDR | awk -F ':' '{ print $2 }')
if [ -z "$RSYNC_PORT" ]
then
RSYNC_PORT=4444
ADDR="$(echo $ADDR | awk -F ':' '{ print $1 }'):$RSYNC_PORT"
fi
SOCAT_ADDR=$(echo $ADDR | awk -F ':' '{ print $1 }')
# map to name in case we received an IP
SOCAT_HOST=$(getent hosts $SOCAT_ADDR | awk '{ print $2 }')
if [ -z "$SOCAT_HOST" ]; then
SOCAT_HOST=$SOCAT_ADDR
fi
SOCAT_PORT=$RSYNC_PORT
trap "exit 32" HUP PIPE
trap "exit 3" INT TERM ABRT
trap cleanup_joiner EXIT
RSYNC_CONF="$WSREP_SST_OPT_DATA/$MODULE.conf"
if [ -n "${MYSQL_TMP_DIR:-}" ] ; then
SILENT="log file = $MYSQL_TMP_DIR/rsynd.log"
else
SILENT=""
fi
cat << EOF > "$RSYNC_CONF"
pid file = $RSYNC_PID
use chroot = no
read only = no
timeout = 300
$SILENT
[$MODULE]
path = $WSREP_SST_OPT_DATA
[$MODULE-log_dir]
path = $WSREP_LOG_DIR
EOF
# rm -rf "$DATA"/ib_logfile* # we don't want old logs around
# Socat receives rsync connections from the donor
SOCAT_SRC=openssl-listen:$SOCAT_PORT,bind=$SOCAT_HOST,reuseaddr,fork,$SOCAT_OPTS
SOCAT_DST=tcp:localhost:$RSYNC_PORT
wsrep_log_info "Setting up tunnel for joiner: socat $SOCAT_SRC $SOCAT_DST"
socat $SOCAT_SRC $SOCAT_DST &
SOCAT_REAL_PID=$!
# This is ok because a local galera node doesn't run SST concurrently
echo $SOCAT_REAL_PID >"$SOCAT_PID"
until check_pid_and_port $SOCAT_PID $SOCAT_REAL_PID $SOCAT_PORT $SOCAT_HOST "socat"
do
sleep 0.2
done
wsrep_log_info "rsync --daemon --no-detach --address localhost --port $RSYNC_PORT --config \"$RSYNC_CONF\""
rsync --daemon --no-detach --address localhost --port $RSYNC_PORT --config "$RSYNC_CONF" &
RSYNC_REAL_PID=$!
until check_pid_and_port $RSYNC_PID $RSYNC_REAL_PID $RSYNC_PORT localhost "rsync"
do
sleep 0.2
done
echo "ready $ADDR/$MODULE"
# wait for SST to complete by monitoring magic file
while [ ! -r "$MAGIC_FILE" ] && check_pid "$RSYNC_PID" && \
check_pid "$SOCAT_PID" && ps -p $MYSQLD_PID >/dev/null
do
sleep 1
done
# to avoid cleanup race, we can tear down the socat tunnel now
# before signaling the end of the SST to galera.
cleanup_tunnel
if ! ps -p $MYSQLD_PID >/dev/null
then
wsrep_log_error \
"Parent mysqld process (PID:$MYSQLD_PID) terminated unexpectedly."
exit 32
fi
if ! [ -z $WSREP_SST_OPT_BINLOG ]
then
pushd $BINLOG_DIRNAME &> /dev/null
if [ -f $BINLOG_TAR_FILE ]
then
# Clean up old binlog files first
rm -f ${BINLOG_FILENAME}.*
wsrep_log_info "Extracting binlog files:"
tar -xvf $BINLOG_TAR_FILE >&2
for ii in $(ls -1 ${BINLOG_FILENAME}.*)
do
echo ${BINLOG_DIRNAME}/${ii} >> ${BINLOG_FILENAME}.index
done
fi
popd &> /dev/null
fi
if [ -r "$MAGIC_FILE" ]
then
# UUID:seqno & wsrep_gtid_domain_id is received here.
cat "$MAGIC_FILE" # Output : UUID:seqno wsrep_gtid_domain_id
else
# this message should cause joiner to abort
echo "rsync process ended without creating '$MAGIC_FILE'"
fi
wsrep_cleanup_progress_file
# cleanup_joiner
else
wsrep_log_error "Unrecognized role: '$WSREP_SST_OPT_ROLE'"
exit 22 # EINVAL
fi
rm -f $BINLOG_TAR_FILE || :
exit 0

File diff suppressed because it is too large Load Diff