Implement running of the minimal testsuite

Once the full testsuite has been run for the particular MySQL release; execute just the "main" suite on subsequent runs for that release.

The implementation default to running of the whole testsuite, as until now, if the maintainer won't set the flag that running of the minimal testsuite is enough for this release.

This feature aims to help with faster and more stable package builds when adding small fixes on top of already tested release;
and to ease the life of Fedora Linux Release Engineers during (mass) rebuilds.

Related: #1969430

mysql-mtr.patch

@@ -0,0 +1,102 @@
+Avoid errors like:
+
+|  Path length (109) is longer than maximum supported length (108) and will be truncated at /usr/lib64/perl5/vendor_perl/Socket.pm line 880, <GEN16094> line 1.
+|  worker[2] mysql-test-run: *** ERROR: Socket path '/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.s390x/usr/share/mysql-test/var/tmp/2' too long,
+|  it would be  truncated and thus not possible to use for connection to  MySQL Server. Set a shorter with --tmpdir=<path> option
+
+===
+
+On Fedora 32:
+
+|  $ grep -e "PATH" /usr/include/linux/limits.h
+|  #define PATH_MAX        4096	/* # chars in a path name including nul */
+
+===
+
+Thus setting the maximum path length on Fedora to 108 characters is just too short.
+
+BTW on the modern filesystems you can easily create path longer than PATH_MAX.
+The PATH_MAX constant is unsafe.
+Interesting article: https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html
+
+===
+
+The question is why haven't I encountered this issue until recently ?
+
+After the recent tweaks to the testsuite (between 8.0.24 and 8.0.25 release), the generated --tmpdir path changed from
+|  --tmpdir=/builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/mysql-test/var/tmp/
+to
+|  --tmpdir=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/tmp/
+
+===
+
+The whole setup changed as follows:
+BEFORE:
+
+|  Installing system database
+|  ### safe_path: /builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/runtime_output_directory///mysqltest_safe_process --verbose -- /builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/runtime_output_directory/mysqld --no-defaults --initialize-insecure --loose-skip-ndbcluster --tmpdir=/builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/mysql-test/var/tmp/ --core-file --datadir=/builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/mysql-test/var/data/ --secure-file-priv=/builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/mysql-test/var --innodb_buffer_pool_size=24M --innodb-log-file-size=5M --innodb_autoextend_increment=8 --character-sets-dir=/builddir/build/BUILD/mysql-8.0.24/share/charsets --loose-auto_generate_certs=OFF --loose-sha256_password_auto_generate_rsa_keys=OFF --loose-caching_sha2_password_auto_generate_rsa_keys=OFF --init-file=/builddir/build/BUILD/mysql-8.0.24/x86_64-redhat-linux-gnu/mysql-test/var/tmp/bootstrap.sql
+
+AFTER:
+
+|  Installing system database
+|  ### safe_path: /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/bin//mysqltest_safe_process --verbose -- /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/libexec/mysqld --no-defaults --initialize-insecure --loose-skip-ndbcluster --tmpdir=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/tmp/ --core-file --datadir=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/data/ --secure-file-priv=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var --innodb_buffer_pool_size=24M --innodb-log-file-size=5M --innodb_autoextend_increment=8 --character-sets-dir=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql/charsets --loose-auto_generate_certs=OFF --loose-sha256_password_auto_generate_rsa_keys=OFF --loose-caching_sha2_password_auto_generate_rsa_keys=OFF --init-file=/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/tmp/bootstrap.sql
+
+===
+
+The likely cause is the added
+
+|  cd %{buildroot}%{_datadir}/mysql-test
+
+which was not originally present in the SPECfile.
+
+However the MariaDB implementation does not have this issue, even though it has the same SPECfile %check phase code.
+
+
+===
+
+In the extended log, you can see '/tmp/XfTFAis2Jl' being created and deleted short after.
+Even though the script warns about the path length, tries to workaround it; it destroyes that workaround short after and use the too-long path instead.
+
+I'm not sure whether the output is synchronous, but even if it wasn'tit still looks weird.
+
+| + cd /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test
+| Logging: ./mysql-test-run.pl  --verbose --parallel=auto --force --retry=2 --suite-timeout=900 --testcase-timeout=30 --mysqld=--binlog-format=mixed --max-test-fail=5 --report-unstable-tests --clean-vardir --suite=main --mem --skip-test-list=platform-specific-tests.list
+| Path length (109) is longer than maximum supported length (108) and will be truncated at /usr/lib64/perl5/vendor_perl/Socket.pm line 880.
+| Too long tmpdir path '/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/tmp'  creating a shorter one
+|  - Using tmpdir: '/tmp/XfTFAis2Jl'
+| > Collecting: main
+| > testdir: /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/t
+| > resdir: /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/r
+| > Collecting: i_main
+| Removing old var directory
+| > opt_vardir: /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var
+| > Removing /builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var/
+| > Removing /dev/shm/var_933_jfTb
+| > Removing /tmp/XfTFAis2Jl/
+| Creating var directory '/builddir/build/BUILDROOT/mysql-8.0.24-1.fc35.x86_64/usr/share/mysql-test/var'
+| > Creating /dev/shm/var_933_jfTb
+|  - symlinking 'var' to '/dev/shm/var_933_jfTb'
+
+
+
+
+--- mysql-8.0.24/mysql-test/mysql-test-run.pl	2021-05-10 04:29:44.391897891 +0200
++++ mysql-8.0.24/mysql-test/mysql-test-run.pl_patched	2021-05-10 04:38:51.031702753 +0200
+@@ -3374,17 +3374,6 @@ sub setup_vardir() {
+   mkpath("$opt_vardir/tmp");
+   mkpath($opt_tmpdir) if ($opt_tmpdir ne "$opt_vardir/tmp");
+ 
+-  # On some operating systems, there is a limit to the length of a
+-  # UNIX domain socket's path far below PATH_MAX. Don't allow that
+-  # to happen.
+-  my $res =
+-    check_socket_path_length("$opt_tmpdir/mysqld.NN.sock", $opt_parallel);
+-  if ($res) {
+-    mtr_error("Socket path '$opt_tmpdir' too long, it would be ",
+-              "truncated and thus not possible to use for connection to ",
+-              "MySQL Server. Set a shorter with --tmpdir=<path> option");
+-  }
+-
+   # Copy all files from std_data into var/std_data
+   # and make them world readable
+   copytree("$glob_mysql_test_dir/std_data", "$opt_vardir/std_data", "0022");

mysql.spec

@@ -11,9 +11,15 @@
 # --nocheck is not possible (e.g. in koji build)
 %{!?runselftest:%global runselftest 1}
 
-# Set this to 1 to see which tests fail
+# Set this to 1 to see which tests fail, but 0 on production ready build
-%global check_testsuite 1
+%global ignore_testsuite_result 0
 
+# The last version on which the full testsuite has been run
+# In case of further rebuilds of that version, don't require full testsuite to be run
+# run only "main" suite
+%global last_tested_version 8.0.24
+# Set to 1 to force run the testsuite even if it was already tested in current version
+%global force_run_testsuite 0
 # Aditional SELinux rules
 %global require_mysql_selinux 1
 
@@ -115,6 +121,7 @@ Patch5:           %{pkgnamepatch}-paths.patch
 Patch51:          %{pkgnamepatch}-chain-certs.patch
 Patch52:          %{pkgnamepatch}-sharedir.patch
 Patch55:          %{pkgnamepatch}-rpath.patch
+Patch56:          %{pkgnamepatch}-mtr.patch
 Patch75:          %{pkgnamepatch}-arm32-timer.patch
 Patch79:          %{pkgnamepatch}-openssl3.patch
 Patch80:          %{pkgnamepatch}-fix-includes-robin-hood.patch
@@ -385,6 +392,7 @@ the MySQL sources.
 %patch51 -p1
 %patch52 -p1
 %patch55 -p1
+%patch56 -p1
 %patch75 -p1
 %patch79 -p1
 %patch80 -p1
@@ -604,6 +612,8 @@ rm %{buildroot}%{_bindir}/{mysql_client_test,mysqlxtest,mysqltest_safe_process,z
 rm -r %{buildroot}%{_datadir}/mysql-test
 %endif
 
+
+
 %check
 %if %{with test}
 %if %runselftest
@@ -617,22 +627,44 @@ cp ../../mysql-test/%{skiplist} .
 #   The resulting port must be between 5000 and 32767
 export MTR_BUILD_THREAD=$(( $(date +%s) % 2200 ))
 
-./mtr %{?with_debug:--debug-server} \
+(
-  --parallel=auto --force --retry=2 \
+  set -ex
-  --mysqld=--binlog-format=mixed \
+  cd %{buildroot}%{_datadir}/mysql-test
-  --suite-timeout=720 --testcase-timeout=30 \
+
-  --report-unstable-tests --clean-vardir \
+  export common_testsuite_arguments=" %{?with_debug:--debug-server} --parallel=auto --force --retry=2 --suite-timeout=900 --testcase-timeout=30 --mysqld=--binlog-format=mixed --max-test-fail=5 --report-unstable-tests --clean-vardir "
-%if %{check_testsuite}
+
-  --max-test-fail=200 || :
+  # If full testsuite has already been run on this version and we don't explicitly want the full testsuite to be run
-%else
+  if [[ "%{last_tested_version}" == "%{version}" ]] && [[ %{force_run_testsuite} -eq 0 ]]
-  --skip-test-list=%{skiplist}
+  then
-%endif
+    # in further rebuilds only run the basic "main" suite (~800 tests)
-rm -r var $(readlink var)
+    echo "running only base testsuite"
+    perl ./mysql-test-run.pl $common_testsuite_arguments --suite=main --mem --skip-test-list=%{skiplist}
+  fi
+
+ # If either this version wasn't marked as tested yet or I explicitly want to run the testsuite, run everything we have (~4000 test)
+  if [[ "%{last_tested_version}" != "%{version}" ]] || [[ %{force_run_testsuite} -ne 0 ]]
+  then
+    echo "running advanced testsuite"
+    perl ./mysql-test-run.pl $common_testsuite_arguments \
+    %if %{ignore_testsuite_result}
+      --max-test-fail=9999 || :
+    %else
+      --skip-test-list=%{skiplist}
+    %endif
+  fi
+
+  # There might be a dangling symlink left from the testing, remove it to not be installed
+  rm -r var $(readlink var)
+)
+
 popd
 popd
+
 %endif
 %endif
 
+
+
 %pre server
 /usr/sbin/groupadd -g 27 -o -r mysql >/dev/null 2>&1 || :
 /usr/sbin/useradd -M -N -g mysql -o -r -d %{mysqluserhome} -s /sbin/nologin \
@@ -650,6 +682,8 @@ fi
 %postun server
 %systemd_postun_with_restart %{daemon_name}.service
 
+
+
 %if %{with client}
 %files
 %{_bindir}/mysql

rh-skipped-tests-list-base.list

@@ -5,6 +5,7 @@ main.explain_tree                          : BUG#0 failing since 8.0.24 on F32+
 
 auth_sec.keyring_file_data_qa              : BUG#0 failing on F32+
 innodb.alter_kill                          : BUG#0 failing on F34+
+main.mtr_unit_tests                        : BUG#0 failing on F35+
 
 # Fails when -DENABLED_LOCAL_INFILE=ON
 main.mysql_load_data_local_dir             : BUG#0 failing on F32+