--
Testsuite results for the 8.4.7 release verified
There were some failing tests:
| auth_sec.admin_channel_tls
| auth_sec.admin_channel_tls_startup
| auth_sec.cert_verify
| auth_sec.cert_verify_openssl
| x.mysqlxtest_mode_ssl
| x.mysqlxtest_mode_ssl_unixsocket
and all of them fail because the test certificates in the 'std_data' testsuite directory have expired.
Some of those tests fail gracefuly, some not. But all of them produce variant of this error message:
"Server SSL certificate doesn't verify: certificate has expired"
--
I believe this is a recuring timebomb bug in MySQL.
I found this upstream commit dealing with the issue few years ago:
8c22b5f6df
but this time different certificates needs refreshing, e.g.:
| std_data/server-cert-verify-fail.pem
| std_data/server-cert-verify-pass.pem
and maybe some more.
--
Resolves: RHEL-128143
We've received a bug report, that the 'rpm -qc' incorrectly returns the MySQL
server default log file '/var/log/mysql/mysqld.log' as a configuration file.
https://issues.redhat.com/browse/RHEL-57601
This was caused by us packing it in the SPECfile '%files' section with the
'%config %ghost' directives. The reason was historical. We tried to mark
the file as owned by the package, but at the same time to not pack it,
and also prevent its deleteion on package removal.
The correct way is to not list the file at all.
The parent directory is properly packed and owned instead (already), so it should be
clear that the files inside are associated with that package. This is how we already
do it with the DB datadir '/var/lib/mysql'.
Resolves: RHEL-112386
While investigating this issue, we also reviewed the log file creation.
The log file is created automatically by the MySQL server when needed.
This can even happen in the middle of the server runtime, when the file
is (re)moved, and the DB server is tasked to flush the logs.
I checked the MySQL server documentation, and tested it, and I found, that the logfile
is indeed created by the server. The only possible reason why the file might be needed
to be created beforehand, is when the file lies on a location to which the MySQL server
won't have permissions to write to.
This is likely the case of MySQL upstream RPMs, which place it by default to the '/var/log':
96d58aa2b9/packaging/rpm-fedora/mysql.spec.in (L703)
However Fedora and RHEL uses a designated directory '/var/log/mysql', which is owened by 'mysql:mysql',
so the issue does not exists here.
The project contains tens of thousands mentions in various formats of
various licenses in various formats.
I did what I could, but there certainly is a space for improvement.
I hope the following description of my steps may serve as a solid
justification of my findings.
=========== =========== ===========
=========== =========== ===========
GPL & LGPL
There is a difference between "GPL-2.0-only" and "GPL-2.0-or-later"
https://spdx.org/licenses/GPL-2.0-only.htmlhttps://spdx.org/licenses/GPL-2.0-or-later.html
I understood that the license text is identical (I've ran 'diff' on top of them to verify)
and the difference is decided by the license header present in the files containing the code itself.
This "*-only" and "*-or-later" differenciation simmilar in other versions of GPL and LGLP
I like GREP. SO here we go:
\# This command is supposed to:
\# - find all occurrences of the string "Public License"
\# - prefixed by "GNU", "Lesser", "Library", "General", each word is optional (to match both GPL and LGPL and all variants of how people write them)
\# - while every word can have any number of any white characters - including newlines - between them (to match text wrapped between lines)
\# - all of that followed by string "version" (to only match text mentioning the specific version)
\# - with any characters between the first part and the "version" string
\# - followed by a digit (to exclude any result not talking about a concrete version specifically, e.g. text around string "from time to time" in the GPL licenses)
\# - match 3 more characters (so we catch the whole version number)
\# - but the "version" string and number must occur no later than any dot "." or double newline (to only match text inside the license texts or license headers, but not code)
\# Once found,
\# - and match everything until dot or double newline (for further parsing of the strings "or later" etc)
\# - replace any newline with space (" "), so the whole above match (as well s everything else) is put on a single line
\# - replace string in the variable DIR_NAME with the same string prepended by newline, so each result is on a separate line
❯ export DIR_NAME="mysql-8.4.6"
❯ export TMPFILE=$(mktemp)
❯ grep -Pzoi '(?s)((GNU\s*)?(Lesser\s*)?(Library\s*)?(General\s*)?Public\s*License)(?:(?!\.\n|\n\n).)*?version\s*\d[\s\S]{0,3}(?:(?!\.\n|\n\n).)*' -r "${DIR_NAME}" | sed ':a;N;$!ba;s/\n/ /g' | sed "s|${DIR_NAME}/|\n${DIR_NAME}/|g" > ${TMPFILE}
The resulting file contains 17894 results.
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 3" | tee >(wc -l)
retuns 10 results, from which:
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 3" | grep -i --binary-files=text -e " lesser" -e " library" | tee >(wc -l)
0 is LGPL
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 3" | grep -v -i --binary-files=text -e " lesser" -e " library" | tee >(wc -l)
10 is GPL
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 3" | grep -i --binary-files=text -e "later" -e " or" | tee >(wc -l)
10 results "GPL-3.0-or-later"
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 3" | grep -i --binary-files=text -e "later" -e " or" | tee >(wc -l)
0 results "GPL-3.0-only"
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | tee >(wc -l)
retuns 17879 results, from which:
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | tee >(wc -l)
15 is LGPL
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -v -i --binary-files=text -e " lesser" -e " library" | tee >(wc -l)
17864 is GPL
and
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i --binary-files=text -e "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.0" | tee >(wc -l)
12 is LGPL 2.0
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.0" | grep -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
0 is "LGPL-2.0-or-later"
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.0" | grep -v -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
12 is "LGPL-2.0-only"
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2\.[^0]" | tee >(wc -l)
3 is LGPL 2.1
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2\.[^0]" | grep -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
2 is "LGPL-2.1-or-later"
both findings are in the 'LICENSE' file.
However that doesn't necessiraly interests me, as the 'LICENSE' file is a list of just potential licenses.
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2\.[^0]" | grep -v -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
1 is "LGPL-2.1-only"
the finding is in the 'LICENSE' file.
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -v -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.[0\s]" -e "version 2.\s" | tee >(wc -l)
17862 is GPL 2.0
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -v -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.[0\s]" -e "version 2.\s" | grep -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
33 is "GPL-2.0-or-later"
found in:
extra/gperftools/
extra/libbacktrace/
router/
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -v -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2[ ,;\<]" -e "version 2.[0\s]" -e "version 2.\s" | grep -v -i --binary-files=text -e "later" -e " or " | tee >(wc -l)
17829 is "GPL-2.0-only"
~190 unique files with the FOSS 1.0 exception
~9500 unique files without it
and
❯ cat ${TMPFILE} | grep -i --binary-files=text "version 2" | grep -v -i --binary-files=text -e " lesser" -e " library" | grep -i --binary-files=text -e "version 2\.[123456789]" | tee >(wc -l)
2 GPL 2.1
weird ... GPL 2.1 doesn't exists, only LGPL 2.1 does
and by examining the files, both are false positives, as the text matched was: "The OpenLDAP Public License Version 2.8, 17 August 2003"
=========== =========== ===========
These were the GPL-family licenses.
=========== =========== ===========
=========== =========== ===========
Now I tired a search that omits any GPL-family license to see what remained.
❯ grep -i licens -r "${DIR_NAME}/" | \
grep -v \
-e "// This source code is licensed under both the GPLv2 (found in the" \
-e "// COPYING file in the root directory) and Apache 2.0 License" \
-e "// (found in the LICENSE.Apache file in the root directory)." \
-e "// Use of this source code is governed by a BSD-style license that can be" \
-e "// found in the LICENSE file. See the AUTHORS file for names of contributors." \
-e "it under the terms of the GNU General Public License as published by" \
-e "the Free Software Foundation; version 2 of the License." \
-e "GNU General Public License for more details." \
-e "You should have received a copy of the GNU General Public License" \
-e "it under the terms of the GNU General Public License, version 2.0," \
-e "but not limited to OpenSSL) that is licensed under separate terms," \
-e "as designated in a particular file or component or in included license" \
-e "separately licensed software that they have included with MySQL." \
-e "GNU General Public License, version 2.0, for more details." \
-e "License as published by the Free Software Foundation; either" \
-e "version 2.1 of the License, or (at your option) any later version." \
-e "Lesser General Public License for more details." \
-e "License along with this library; if not, write to the Free Software" \
-e "License version 2.1 as published by the Free Software Foundation." \
-e "the terms of the GNU General Public License as published by the Free Software" \
-e "License along with this library; if not, write to the Free" \
-e "version 2 of the License, or (at your option) any later version." \
-e "Library General Public License for more details." \
-e "License along with this library; if not see <http://www.gnu.org/licenses>" \
-e "GNU General Public License, version 2.0" \
| grep -i "licens" > ${TMPFILE}
Which yelds 33132 results to go through.
This is tremendously attention demanding, and likely place for oversights.
I've managed to discover the following:
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i -e "separately licensed software that they have either included with" | wc -l
9347
This is the Universal-FOSS-exception-1.0. I had to check which licenses are these exceptions connected to.
I found it connected to "GPL-2.0-only".
Around 190 "GPL-2.0-only" licensed files have the exception, while around 9500 "LGPL-2.0-only" licensed files does not have it.
I found that all "LGPL-2.0-only" license files also have the exception.
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i -e "www.boost.org/LICENSE_" -e "See accompanying file LICENSE_1_0.txt" -e "Version 1.0. (See accompanying file" -e "accompanying file LICENSE_1_0.txt" -e "Boost Software License" -e "http://www.boost.org/users/license.html" | wc -l
21337
The "BSL-1.0" license
=========== =========== ===========
❯ cat ${TMPFILE} | grep -i -e "apache license"
mysql-8.4.6/storage/ndb/test/crund/src/META-INF/persistence.xml: to you under the Apache License, Version 2.0 (the
mysql-8.4.6/storage/ndb/test/crund/src/META-INF/persistence.xml: http://www.apache.org/licenses/LICENSE-2.0
The "Apache-2.0" licnese
=========== =========== ===========
❯ cat ${TMPFILE} | grep -v -i -e "www.boost.org/LICENSE_" -e "See accompanying file LICENSE_1_0.txt" -e "Version 1.0. (See accompanying file" -e "accompanying file LICENSE_1_0.txt" -e "Boost Software License" -e "http://www.boost.org/users/license.html" | grep -v -i -e "apache" | grep -e "MIT"
The "MIT" license
=========== =========== ===========
❯ cat ${TMPFILE} | grep -v -i -e "www.boost.org/LICENSE_" -e "See accompanying file LICENSE_1_0.txt" -e "Version 1.0. (See accompanying file" -e "accompanying file LICENSE_1_0.txt" -e "Boost Software License" -e "http://www.boost.org/users/license.html" | grep -v -i -e "apache" | grep -v -e "MIT" | grep -e "BSD"
The "BSD-2-Clause" License
=========== =========== ===========
❯ cat ${TMPFILE} | grep -v -i -e "www.boost.org/LICENSE_" -e "See accompanying file LICENSE_1_0.txt" -e "Version 1.0. (See accompanying file" -e "accompanying file LICENSE_1_0.txt" -e "Boost Software License" -e "http://www.boost.org/users/license.html" | grep -v -i -e "apache" | grep -v -e "MIT" | grep -i -e "dual license"
mysql-8.4.6/storage/innobase/ut/crc32.cc: Dual licensed under BSD license and GPLv2.
The "( GPL-2.0-only OR BSD-2-Clause )" license
=========== =========== ===========
I hope I caught all of the licenses.
If that's not the case, please contribute a fix or at least report a bug against this package so we can correct it.
Related: RHEL-104250
The package conflicts between MariaDB and MySQL packages are only set against the un-versioned names.
(the un-versioned names are only provided by the distribution default version)
The conflicts has to be extended to cover the '-any' virtually provided names,
which are provided by all versions of the versioned packages.
Related: RHEL-104250
as it is required by the server subpackage thus removing it when the
client subpackage is not being built will result in a build failure.
Related: RHEL-104250
-client subpackage as mysql_plugin is deprecated from version 5.7.11
and has been removed in version 8.0 therefore it is no longer provided
by upstream and it caused build failures when rm could not find it.
Related: RHEL-104250
Upstream release notes:
https://dev.mysql.com/doc/relnotes/mysql/8.4/en/news-8-4-5.html
I have ran the extended testsuite during builds on f40-f43 on all
architectures in 2 runs (that is 32 individual build logs parsed for the
upcoming statistics)
These are the statistics of the test results containing the number of
times a test has failed and its name, the tests that fail because of a
timeout are marked with `-- timed out`. Tests not marked with timeout
failed on asserts.
These tests are not added to the rh-skipped-test-lists-*.list.
The results are split into all architectures together:
12 rpl.rpl_eventlog_psi_memory
12 rpl.rpl_channel_creation_under_replica_load
12 innodb_fts.optimize_big
11 innodb_fts.bug_34846823 -- timed out
10 perfschema.system_events_plugin
8 rpl_gtid.mysqldump_bug33630199 -- timed out
8 perfschema.system_events_component
7 sys_vars.myisam_data_pointer_size_func -- timed out
7 rpl.rpl_parallel_ddl_myisam
7 rpl.rpl_parallel_ddl_innodb
7 parts.partition_reverse_scan_icp -- timed out
7 innodb.multi_value_basic -- timed out
7 innodb.lob_recovery -- timed out
7 federated.federated_server -- timed out
5 rpl_gtid.rpl_perfschema_applier_status_by_worker_gtid_skipped_transaction_mts
3 perfschema.transaction_nested_events
3 innodb_undo.undo_settings
2 x.connection_not_alive
2 perfschema.relaylog
The ppc64le architecture (since it had the most amount of failing tests
by far and the same tests were failing there almost every time):
8 rpl.rpl_eventlog_psi_memory
8 rpl_gtid.mysqldump_bug33630199 -- timed out
7 sys_vars.myisam_data_pointer_size_func -- timed out
7 rpl.rpl_parallel_ddl_myisam
7 rpl.rpl_parallel_ddl_innodb
7 rpl.rpl_channel_creation_under_replica_load
7 perfschema.system_events_plugin
7 parts.partition_reverse_scan_icp -- timed out
7 innodb.multi_value_basic -- timed out
7 innodb.lob_recovery -- timed out
7 innodb_fts.optimize_big
7 innodb_fts.bug_34846823 -- timed out
7 federated.federated_server -- timed out
6 perfschema.system_events_component
4 rpl_gtid.rpl_perfschema_applier_status_by_worker_gtid_skipped_transaction_mts
2 innodb_undo.undo_settings
And other architectures excluding ppc64le:
5 rpl.rpl_channel_creation_under_replica_load
5 innodb_fts.optimize_big
4 rpl.rpl_eventlog_psi_memory
4 innodb_fts.bug_34846823 -- timed out
3 perfschema.transaction_nested_events
3 perfschema.system_events_plugin
2 x.connection_not_alive
2 perfschema.system_events_component
2 perfschema.relaylog
1 rpl_gtid.rpl_perfschema_applier_status_by_worker_gtid_skipped_transaction_mts
1 innodb_undo.undo_settings
Related: RHEL-87738
The other occurences are functions that are called with various arguments through the SPECfile.
I tried to replace them too, but the resulting logic did not expanded correcly and resulted in
various FTI issues.
Related: RHEL-87738
Add "--mysqld=--skip-innodb-use-native-aio" option for running the testsuite in CentOS Stream / RHEL build system.
Remove the skipped auth_sec tests from the skipped tests list, they no longer exist.
Dependency 'perl(File::Compare)' added in order for the 'binlog.binlog_mysqlbinlog_linux' test to run.
Update the skipped tests lists regarding the 'main' suite.
Related: RHEL-63025
mysql8.4-test.x86_64: E: file-parent-ownership-mismatch Path "/usr/share/mysql-test/platform-specific-tests.list" owned by "root" is stored in directory owned by "mysql"
mysql8.4-server.x86_64: E: missing-dependency-to-logrotate for logrotate script /etc/logrotate.d/mysqld
"macro-in-comment"
Related: RHEL-63025
When
%attr(-,mysql,mysql) %{_datadir}/mysql-test
is used,the path, *and everything on it* is added
Only when prepended with '%dir', it won't added anything else than the directory itself
The mistake resulted in all files under
%attr(-,mysql,mysql) %{_datadir}/mysql-test
to be present in both subpackages 'mysql-test' and 'mysql-test-data'.
Explanation of the options per:
https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_MYSQL_TEST_RUN_PL.html
The MTR_MAX_PARALLEL environment variable, if set, specifies the maximum number
of parallel workers that can be spawned when the --parallel=auto option is specified.
If --parallel=auto is not specified, MTR_MAX_PARALLEL variable has no effect.
Explanation of the options per:
https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_MYSQL_TEST_RUN_PL.html
--port-base=P
The value may also be given with the environment variable MTR_PORT_BASE.
The variable MTR_PORT_BASE is a more logical replacement for the original variable MTR_BUILD_THREAD.
It gives the actual port number directly (will be rounded down to a multiple of 10).
If you use MTR_BUILD_THREAD, the port number is found by multiplying this by 10 and adding 10000.
Specify base of port numbers to be used; a block of 10 will be allocated. P should be divisible by 10;
if it is not, it will be rounded down. If running with more than one parallel test thread, thread 2
will use the next block of 10 and so on.
If the port number is given as auto, which is also the default,
mysql-test-run.pl will allocate a number unique to this host.
--port-base was added as a more logical alternative to --build-thread.
If both are used, --port-base takes precedence.
Explanation of the options per:
https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_MYSQL_TEST_RUN_PL.html
--max-test-fail=N
Stop execution after the specified number of tests have failed, to avoid
using up resources (and time) in case of massive failures. retries are not
counted. Defaults to 10, set to 0 for no limit.
--retry=N
If a test fails, it is retried up to a maximum of N runs, but will terminate
after 2 failures. Default is 3, set to 1 or 0 for no retries. This option has
no effect unless --force is also used; without it, test execution will terminate
after the first failure.
Small sub-package 'mysql-test' will remain architecture dependent
and will continue to carry all architecture dependent parts of the testsuite.
New huge sub-package 'mysql-test-data' will hold all of the
achitecture independent data and will be built as 'noarch'.
This will save over 1GB of redundant data per each package build.
Two reasons:
1) This license is included due to the Perl module 'Expect'. However I wasn't able to actually
locate it's sources in the MySQL source tarball. I believe it's a remnant of a code that was
removed in the past.
I've asked the MySQL upstream to verify my claim and either disprove it or fix the LICENSE
file in wihch they mention it.
For this reason I believe the 'GPL-1.0-or-later' also does not apply, so I'm removing it too.
I'll put the 'GPL-1.0-or-later' should upstream disprove my findings.
2) As per clarification on the Fedora License mailing list:
https://lists.fedoraproject.org/archives/list/legal@lists.fedoraproject.org/message/LBAZXYBAA4YHJ2GEWEP5S7QXVOB76MUR/
"
If your package is licensed under a known choice of two licenses and
one is a "good" license and one is a "bad" license, then the License:
field must reflect the "good" license only. This is highly uncommon in
Fedora packages apart from the case of Perl modules dual licensed
under the GPL and the Artistic License 1.0. In that case you must pick
the appropriate identifier for the GPL side (which in Perl modules
will typically map to SPDX "GPL-1.0-or-later"). You are encouraged to
include a comment memorializing this, for example:
\# Upstream project is dual licensed GPL | Artistic 1.0
"
After discussion with change owner, I've prepared the correct solution.
There is a difference between Fedora and CentOS Stream 10 / RHEL 10:
- in CentOS Stream 10 / RHEL 10, the change is self-contained by the
'openssl-devel' package itself, which now defines OPENSSL_NO_ENGINE
in the headers so new application builds don't use engine at all
- in Fedora, the headers containing the OPENSSL_NO_ENGINE definition
are put into a separate sub-package 'openssl-devel-engine', instead
of being put into the 'openssl-devel'
mysql-c99.patch
- directory "plugin/innodb_memcached" no longer exists
boost-1.76.0-fix_multiprecision_issue_419-ppc64le.patch
- upstreamed
ExcludeArch: %{ix86}
- MySQL 8.4 is not supported on 32-bit architectures
%ifnarch aarch64 s390x
BuildRequires: libquadmath-devel
%endif
- sub-package of gcc
- Only available on some architectures
%{_bindir}/mysql_upgrade
- was deprecated in 8.0
- now no longer available
%global runselftest 0
- disable testsuite during development time,
takes ages to complete on s390x and produces random failures on ppc64le
- I've checked that the result is sane
