Test: rework execution of unit-tests

We have been running just a subset of osbuild unit tests and also not as
a root, so some of them would be skipped.

In addition, the downstream patches were never applied on sources which
were used for unit-testing. Thus if any issue with test was just
backported as a patch, this had no effect on the test result.

Rework how tests are executed by using a dedicated script which will
take care of the installation of test dependencies, preparation of
sources including application of downstream patches and execute all unit
tests.

Rework also the FMF test metadata and test discovery.

Related: rhbz#2174847

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This commit is contained in:
Tomáš Hozza 2023-04-05 15:59:50 +02:00
parent 52630621bc
commit 288b280073
No known key found for this signature in database
GPG Key ID: C5887AD51D9F3C2D
4 changed files with 109 additions and 9 deletions

View File

@ -1,14 +1,11 @@
summary: Run osbuild unit tests from source
summary: Run osbuild unit tests from dist-git sources
prepare:
how: install
package:
- git
- make
- python3-mako
- python3-pytest
- osbuild
discover:
how: fmf
dist-git-source: true
filter: tag:unit-test
execute:
how: tmt

View File

@ -0,0 +1,4 @@
summary: Run osbuild unit tests from dist-git sources
tag: unit-test
duration: 2h
test: ./run-unit-tests.sh

View File

@ -0,0 +1,101 @@
#!/usr/bin/bash
# Execute osbuild unit tests from a checked out dist-git repo
set -euxo pipefail
source /etc/os-release
case "${ID}-${VERSION_ID}" in
fedora-*)
PKG_MAINT_TOOL="fedpkg"
sudo dnf install -y "${PKG_MAINT_TOOL}"
;;
rhel-8.*)
PKG_MAINT_TOOL="rhpkg"
# rhpkg is not available in the default repos
sudo curl -L -o /etc/yum.repos.d/rcm-tools-rhel-8-baseos.repo http://download.devel.redhat.com/rel-eng/internal/rcm-tools-rhel-8-baseos.repo
;;
rhel-9.*)
PKG_MAINT_TOOL="rhpkg"
# rhpkg is not available in the default repos
sudo curl -L -o /etc/yum.repos.d/rcm-tools-rhel-9-baseos.repo http://download.devel.redhat.com/rel-eng/internal/rcm-tools-rhel-9-baseos.repo
;;
centos-*)
PKG_MAINT_TOOL="centpkg"
# centpkg is not available in the default repos
sudo dnf install -y epel-release
# CRB repos are available only for CentOS Stream 9
if [ "${VERSION_ID}" == "9" ]; then
dnf config-manager --set-enabled crb
fi
;;
*)
echo "Error: unsupported OS: ${ID}-${VERSION_ID}"
exit 1
;;
esac
# Move to the checked out git repo with the test plans
# this should be the root of the dist-git repo
cd "${TMT_TREE}"
# the content of the directory is copy of the dist-git repo, but the .git directory is missing
# so we need to create it to make all *pkg tools happy
git init
# install all test dependencies
sudo dnf install -y \
"${PKG_MAINT_TOOL}" \
rpmdevtools \
python3-mako \
python3-pytest \
dosfstools
sudo dnf builddep -y osbuild.spec
# Make sure that /usr/lib/systemd/boot/efi/linuxx64.efi.stub is available to enable pe32p tests
case "${ID}-${VERSION_ID}" in
rhel-8.* | centos-8)
sudo dnf install -y systemd-udev
;;
*)
sudo dnf install -y systemd-boot-unsigned
;;
esac
TARGET_BRANCH="${TARGET_BRANCH:-}"
PKG_MAINT_TOOL_EXTRA_ARGS=""
if [ -n "${TARGET_BRANCH}" ]; then
PKG_MAINT_TOOL_EXTRA_ARGS="--release ${TARGET_BRANCH}"
fi
# shellcheck disable=SC2086
${PKG_MAINT_TOOL} ${PKG_MAINT_TOOL_EXTRA_ARGS} prep
# Extract the Source0 basename without extension
SRC_DIR=$(rpmdev-spectool --source 0 osbuild.spec | awk '{print $2}' | sed 's/.tar.gz//')
SRC_DIR="${SRC_DIR##*/}"
cd "${SRC_DIR}"
# Some unit tests depend on the fact that the source code is in a git repo
git init .
git add *
git commit -m "Initial commit"
# Run the unit tests
# Note:
# - Ignore the boot test, because it requires qemu-system-x86_64 not available on all distributions.
# - Ignore source code tests, which run linters, since we can't ensure that all linters are available
# and of the same version as in upstream.
# - Explicitly mark btrfs as unsupported on CentOS / RHEL
if [ "${ID}" == "centos" ] || [ "${ID}" == "rhel" ]; then
UNSUPPORTED_FS="--unsupported-fs btrfs"
fi
sudo python3 -m pytest \
--rootdir "$(pwd)" \
--ignore "$(pwd)/test/src" \
${UNSUPPORTED_FS:-} \
-k "not test_boot" \
-v \
"$(pwd)/test/"

View File

@ -1,2 +0,0 @@
summary: Run unit tests using upstream source
test: cd ../osbuild-*/ && python3 -m pytest test/mod