2.37.2-2: rebase cleanup, add upstream tests to gating

Related: #1969348
This commit is contained in:
Karel Zak 2021-08-24 11:28:31 +02:00
parent d4e6dd63e6
commit 42def072d9
9 changed files with 230 additions and 401 deletions

View File

@ -0,0 +1,26 @@
From 331c5e0c54d9cb6f67dc3e825eec2d78c67d8ce6 Mon Sep 17 00:00:00 2001
From: Karel Zak <kzak@redhat.com>
Date: Mon, 23 Aug 2021 15:15:38 +0200
Subject: [PATCH 1/2] tests: make ./run.sh more robust
Signed-off-by: Karel Zak <kzak@redhat.com>
---
tests/run.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/run.sh b/tests/run.sh
index 9d26406c4..d020bfe88 100755
--- a/tests/run.sh
+++ b/tests/run.sh
@@ -165,7 +165,7 @@ OPTS="$OPTS --srcdir=$top_srcdir --builddir=$top_builddir"
if [ -z "$has_asan_opt" ]; then
if [ -e "$top_builddir/Makefile" ]; then
asan=$(awk '/^ASAN_LDFLAGS/ { print $3 }' $top_builddir/Makefile)
- else
+ elif [ -f "$top_builddir/meson.conf" ]; then
. "$top_builddir/meson.conf"
fi
if [ -n "$asan" ]; then
--
2.31.1

View File

@ -0,0 +1,65 @@
From 85ae61dd6d956e7c9fe2b22b8c46bb1d0bfd13da Mon Sep 17 00:00:00 2001
From: Karel Zak <kzak@redhat.com>
Date: Mon, 23 Aug 2021 16:28:52 +0200
Subject: [PATCH] tests: make mount/fstab-all more robust
Signed-off-by: Karel Zak <kzak@redhat.com>
---
tests/ts/mount/fstab-all | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/ts/mount/fstab-all b/tests/ts/mount/fstab-all
index acc64e462..6b7018823 100755
--- a/tests/ts/mount/fstab-all
+++ b/tests/ts/mount/fstab-all
@@ -79,6 +79,7 @@ echo "${TS_DEVICE}4 ${MOUNTPOINT}D ext4 rw,defaults 0 0" >> $MY_FSTAB
ts_init_subtest "basic"
$TS_CMD_MOUNT --all --fstab $MY_FSTAB >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT ${MOUNTPOINT}{A,B,C,D}
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
@@ -87,6 +88,7 @@ ts_finalize_subtest
ts_init_subtest "filter-type"
$TS_CMD_MOUNT --all --fstab $MY_FSTAB -t ext4 >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT ${MOUNTPOINT}D
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
@@ -95,6 +97,7 @@ ts_finalize_subtest
ts_init_subtest "filter-notype"
$TS_CMD_MOUNT --all --fstab $MY_FSTAB -t noext4 >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT ${MOUNTPOINT}{A,B,C}
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
@@ -103,6 +106,7 @@ ts_finalize_subtest
ts_init_subtest "filter-option"
$TS_CMD_MOUNT --all --fstab $MY_FSTAB -O ro >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT ${MOUNTPOINT}C
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
@@ -111,6 +115,7 @@ ts_finalize_subtest
ts_init_subtest "override-option"
$TS_CMD_MOUNT --all --fstab $MY_FSTAB -o ro >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT ${MOUNTPOINT}{A,B,C,D}
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
@@ -132,6 +137,7 @@ $TS_CMD_MOUNT --all \
--target-prefix $MY_ROOT \
-o X-mount.mkdir >> $TS_OUTPUT 2>> $TS_ERRLOG
[ $? == 0 ] || ts_log "mount failed"
+udevadm settle
$TS_CMD_UMOUNT $MY_ROOT/foo/{A,B,C,D}
[ $? == 0 ] || ts_log "umount failed"
ts_finalize_subtest
--
2.31.1

View File

@ -0,0 +1,41 @@
From abe16d0d34413555fbd621f90a0b93c2105116a2 Mon Sep 17 00:00:00 2001
From: Karel Zak <kzak@redhat.com>
Date: Tue, 24 Aug 2021 10:49:32 +0200
Subject: [PATCH] tests: make eject umount tests more robust
Signed-off-by: Karel Zak <kzak@redhat.com>
---
tests/ts/eject/umount | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tests/ts/eject/umount b/tests/ts/eject/umount
index 0a89b022e..a6d1f251d 100755
--- a/tests/ts/eject/umount
+++ b/tests/ts/eject/umount
@@ -84,6 +84,7 @@ mkfs.ext2 -q -F $TS_DEVICE
udevadm settle
mkdir -p $TS_MOUNTPOINT
$TS_CMD_MOUNT $TS_DEVICE $TS_MOUNTPOINT
+udevadm settle
$TS_CMD_EJECT --force $TS_DEVICE && ts_log "Success"
deinit_device
ts_finalize_subtest
@@ -96,6 +97,7 @@ mkdir -p ${TS_MOUNTPOINT}1
mkdir -p ${TS_MOUNTPOINT}2
$TS_CMD_MOUNT ${TS_DEVICE}1 ${TS_MOUNTPOINT}1
$TS_CMD_MOUNT ${TS_DEVICE}2 ${TS_MOUNTPOINT}2
+udevadm settle
$TS_CMD_EJECT --force $TS_DEVICE && ts_log "Success"
deinit_device
ts_finalize_subtest
@@ -116,6 +118,7 @@ mkdir -p ${TS_MOUNTPOINT}1
mkdir -p ${TS_MOUNTPOINT}2
$TS_CMD_MOUNT ${TS_DEVICE}1 ${TS_MOUNTPOINT}1
$TS_CMD_MOUNT ${TS_DEVICE}2 ${TS_MOUNTPOINT}2
+udevadm settle
$TS_CMD_EJECT --force ${TS_DEVICE}1 && ts_log "Success"
deinit_device
ts_finalize_subtest
--
2.31.1

View File

@ -1,396 +0,0 @@
From 57898c3a7ee8fc5933cddd4526bb3980bef85a02 Mon Sep 17 00:00:00 2001
From: Karel Zak <kzak@redhat.com>
Date: Tue, 1 Sep 2020 10:15:14 +0200
Subject: [PATCH] libmount: remove read-mountinfo workaround
This workaround has been introduced by
http://github.com/karelzak/util-linux/commit/e4925f591c1bfb83719418b56b952830d15b77eb
And originally requested by https://github.com/systemd/systemd/issues/10872
It seems we do not need it anymore as the problem should be fixed in kernel since 5.8
(kernel commit 9f6c61f96f2d97cbb5f7fa85607bc398f843ff0f).
Note that the libmount solution is very expensive as it repeats read()
many times (until we get consistent result) if kernel is busy with
mount table modification. This behaviour makes events management in
systemd (or other places) pretty difficult as read mountinfo takes
time on busy systems.
Addresses: https://github.com/systemd/systemd/pull/16537
Signed-off-by: Karel Zak <kzak@redhat.com>
---
configure.ac | 1 -
libmount/src/mountP.h | 2 -
libmount/src/tab_parse.c | 87 ++++----------------
libmount/src/utils.c | 166 ---------------------------------------
4 files changed, 15 insertions(+), 241 deletions(-)
diff --git a/configure.ac b/configure.ac
index 2d178f3af..1e31ca3e2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -504,7 +504,6 @@ AC_CHECK_FUNCS([ \
err \
errx \
explicit_bzero \
- fmemopen \
fsync \
utimensat \
getdomainname \
diff --git a/libmount/src/mountP.h b/libmount/src/mountP.h
index d8ba0abad..ee97c6b4a 100644
--- a/libmount/src/mountP.h
+++ b/libmount/src/mountP.h
@@ -98,7 +98,6 @@ extern int mnt_valid_tagname(const char *tagname);
extern int append_string(char **a, const char *b);
extern const char *mnt_statfs_get_fstype(struct statfs *vfs);
-extern int is_procfs_fd(int fd);
extern int is_file_empty(const char *name);
extern int mnt_is_readonly(const char *path)
@@ -124,7 +123,6 @@ extern void mnt_free_filesystems(char **filesystems);
extern char *mnt_get_kernel_cmdline_option(const char *name);
extern int mnt_stat_mountpoint(const char *target, struct stat *st);
extern int mnt_lstat_mountpoint(const char *target, struct stat *st);
-extern FILE *mnt_get_procfs_memstream(int fd, char **membuf);
/* tab.c */
extern int is_mountinfo(struct libmnt_table *tb);
diff --git a/libmount/src/tab_parse.c b/libmount/src/tab_parse.c
index 329987bcb..ac12dce54 100644
--- a/libmount/src/tab_parse.c
+++ b/libmount/src/tab_parse.c
@@ -694,7 +694,15 @@ static int kernel_fs_postparse(struct libmnt_table *tb,
return rc;
}
-static int __table_parse_stream(struct libmnt_table *tb, FILE *f, const char *filename)
+/**
+ * mnt_table_parse_stream:
+ * @tb: tab pointer
+ * @f: file stream
+ * @filename: filename used for debug and error messages
+ *
+ * Returns: 0 on success, negative number in case of error.
+ */
+int mnt_table_parse_stream(struct libmnt_table *tb, FILE *f, const char *filename)
{
int rc = -1;
int flags = 0;
@@ -773,40 +781,6 @@ err:
return rc;
}
-/**
- * mnt_table_parse_stream:
- * @tb: tab pointer
- * @f: file stream
- * @filename: filename used for debug and error messages
- *
- * Returns: 0 on success, negative number in case of error.
- */
-int mnt_table_parse_stream(struct libmnt_table *tb, FILE *f, const char *filename)
-{
- int fd, rc;
- FILE *memf = NULL;
- char *membuf = NULL;
-
- /*
- * For /proc/#/{mountinfo,mount} we read all file to memory and use it
- * as memory stream. For more details see mnt_read_procfs_file().
- */
- if ((fd = fileno(f)) >= 0
- && (tb->fmt == MNT_FMT_GUESS ||
- tb->fmt == MNT_FMT_MOUNTINFO ||
- tb->fmt == MNT_FMT_MTAB)
- && is_procfs_fd(fd)
- && (memf = mnt_get_procfs_memstream(fd, &membuf))) {
-
- rc = __table_parse_stream(tb, memf, filename);
- fclose(memf);
- free(membuf);
- } else
- rc = __table_parse_stream(tb, f, filename);
-
- return rc;
-}
-
/**
* mnt_table_parse_file:
* @tb: tab pointer
@@ -822,49 +796,18 @@ int mnt_table_parse_stream(struct libmnt_table *tb, FILE *f, const char *filenam
int mnt_table_parse_file(struct libmnt_table *tb, const char *filename)
{
FILE *f;
- int rc, fd = -1;
+ int rc;
if (!filename || !tb)
return -EINVAL;
- /*
- * Try to use read()+poll() to realiably read all
- * /proc/#/{mount,mountinfo} file to memory
- */
- if (tb->fmt != MNT_FMT_SWAPS
- && strncmp(filename, "/proc/", 6) == 0) {
-
- FILE *memf;
- char *membuf = NULL;
-
- fd = open(filename, O_RDONLY|O_CLOEXEC);
- if (fd < 0) {
- rc = -errno;
- goto done;
- }
- memf = mnt_get_procfs_memstream(fd, &membuf);
- if (memf) {
- rc = __table_parse_stream(tb, memf, filename);
-
- fclose(memf);
- free(membuf);
- close(fd);
- goto done;
- }
- /* else fallback to fopen/fdopen() */
- }
-
- if (fd >= 0)
- f = fdopen(fd, "r" UL_CLOEXECSTR);
- else
- f = fopen(filename, "r" UL_CLOEXECSTR);
-
+ f = fopen(filename, "r" UL_CLOEXECSTR);
if (f) {
- rc = __table_parse_stream(tb, f, filename);
+ rc = mnt_table_parse_stream(tb, f, filename);
fclose(f);
} else
rc = -errno;
-done:
+
DBG(TAB, ul_debugobj(tb, "parsing done [filename=%s, rc=%d]", filename, rc));
return rc;
}
@@ -921,7 +864,7 @@ static int __mnt_table_parse_dir(struct libmnt_table *tb, const char *dirname)
f = fopen_at(dd, d->d_name, O_RDONLY|O_CLOEXEC, "r" UL_CLOEXECSTR);
if (f) {
- __table_parse_stream(tb, f, d->d_name);
+ mnt_table_parse_stream(tb, f, d->d_name);
fclose(f);
}
}
@@ -962,7 +905,7 @@ static int __mnt_table_parse_dir(struct libmnt_table *tb, const char *dirname)
f = fopen_at(dirfd(dir), d->d_name,
O_RDONLY|O_CLOEXEC, "r" UL_CLOEXECSTR);
if (f) {
- __table_parse_stream(tb, f, d->d_name);
+ mnt_table_parse_stream(tb, f, d->d_name);
fclose(f);
}
}
diff --git a/libmount/src/utils.c b/libmount/src/utils.c
index 92829ebb0..40b45f11d 100644
--- a/libmount/src/utils.c
+++ b/libmount/src/utils.c
@@ -19,7 +19,6 @@
#include <fcntl.h>
#include <pwd.h>
#include <grp.h>
-#include <poll.h>
#include <blkid.h>
#include "strutils.h"
@@ -448,13 +447,6 @@ const char *mnt_statfs_get_fstype(struct statfs *vfs)
return NULL;
}
-int is_procfs_fd(int fd)
-{
- struct statfs sfs;
-
- return fstatfs(fd, &sfs) == 0 && sfs.f_type == STATFS_PROC_MAGIC;
-}
-
/**
* mnt_match_fstype:
* @type: filesystem type
@@ -1174,164 +1166,7 @@ done:
return 1;
}
-#if defined(HAVE_FMEMOPEN) || defined(TEST_PROGRAM)
-
-/*
- * This function tries to minimize possible races when we read
- * /proc/#/{mountinfo,mount} files.
- *
- * The idea is to minimize number of read()s and check by poll() that during
- * the read the mount table has not been modified. If yes, than re-read it
- * (with some limitations to avoid never ending loop).
- *
- * Returns: <0 error, 0 success, 1 too many attempts
- */
-static int read_procfs_file(int fd, char **buf, size_t *bufsiz)
-{
- size_t bufmax = 0;
- int rc = 0, tries = 0, ninters = 0;
- char *bufptr = NULL;
-
- assert(buf);
- assert(bufsiz);
-
- *bufsiz = 0;
- *buf = NULL;
-
- do {
- ssize_t ret;
-
- if (!bufptr || bufmax == *bufsiz) {
- char *tmp;
-
- bufmax = bufmax ? bufmax * 2 : (16 * 1024);
- tmp = realloc(*buf, bufmax);
- if (!tmp)
- break;
- *buf = tmp;
- bufptr = tmp + *bufsiz;
- }
-
- errno = 0;
- ret = read(fd, bufptr, bufmax - *bufsiz);
-
- if (ret < 0) {
- /* error */
- if ((errno == EAGAIN || errno == EINTR) && (ninters++ < 5)) {
- xusleep(200000);
- continue;
- }
- break;
-
- } if (ret > 0) {
- /* success -- verify no event during read */
- struct pollfd fds[] = {
- { .fd = fd, .events = POLLPRI }
- };
-
- rc = poll(fds, 1, 0);
- if (rc < 0)
- break; /* poll() error */
- if (rc > 0) {
- /* event -- read all again */
- if (lseek(fd, 0, SEEK_SET) != 0)
- break;
- *bufsiz = 0;
- bufptr = *buf;
- tries++;
-
- if (tries > 10)
- /* busy system? -- wait */
- xusleep(10000);
- continue;
- }
-
- /* successful read() without active poll() */
- (*bufsiz) += (size_t) ret;
- bufptr += ret;
- tries = ninters = 0;
- } else {
- /* end-of-file */
- goto success;
- }
- } while (tries <= 100);
-
- rc = errno ? -errno : 1;
- free(*buf);
- return rc;
-
-success:
- return 0;
-}
-
-/*
- * Create FILE stream for data from read_procfs_file()
- */
-FILE *mnt_get_procfs_memstream(int fd, char **membuf)
-{
- size_t sz = 0;
- off_t cur;
-
- *membuf = NULL;
-
- /* in case of error, rewind to the original position */
- cur = lseek(fd, 0, SEEK_CUR);
-
- if (read_procfs_file(fd, membuf, &sz) == 0 && sz > 0) {
- FILE *memf = fmemopen(*membuf, sz, "r");
- if (memf)
- return memf; /* success */
-
- free(*membuf);
- *membuf = NULL;
- }
-
- /* error */
- if (cur != (off_t) -1)
- lseek(fd, cur, SEEK_SET);
- return NULL;
-}
-#else
-FILE *mnt_get_procfs_memstream(int fd __attribute((__unused__)),
- char **membuf __attribute((__unused__)))
-{
- return NULL;
-}
-#endif /* HAVE_FMEMOPEN */
-
-
#ifdef TEST_PROGRAM
-static int test_proc_read(struct libmnt_test *ts, int argc, char *argv[])
-{
- char *buf = NULL;
- char *filename = argv[1];
- size_t bufsiz = 0;
- int rc = 0, fd = open(filename, O_RDONLY);
-
- if (fd <= 0) {
- warn("%s: cannot open", filename);
- return -errno;
- }
-
- rc = read_procfs_file(fd, &buf, &bufsiz);
- close(fd);
-
- switch (rc) {
- case 0:
- fwrite(buf, 1, bufsiz, stdout);
- free(buf);
- break;
- case 1:
- warnx("too many attempts");
- break;
- default:
- warn("%s: cannot read", filename);
- break;
- }
-
- return rc;
-}
-
static int test_match_fstype(struct libmnt_test *ts, int argc, char *argv[])
{
char *type = argv[1];
@@ -1513,7 +1348,6 @@ int main(int argc, char *argv[])
{ "--guess-root", test_guess_root, "[<maj:min>]" },
{ "--mkdir", test_mkdir, "<path>" },
{ "--statfs-type", test_statfs_type, "<path>" },
- { "--read-procfs", test_proc_read, "<path>" },
{ NULL }
};
--
2.25.4

View File

@ -4,6 +4,21 @@
- classic
- container
- atomic
vars:
# standard-test-basic directory for tests are relative to {{ tenv_workdir }}
tenv_workdir: /var/test
pre_tasks:
- import_role:
name: standard-test-source
vars:
fetch_only: True
- name: Copy files including source to test environment
synchronize:
src: "{{ playbook_dir }}/.."
dest: "{{ tenv_workdir }}"
mode: push
ssh_args: "-o UserKnownHostsFile=/dev/null"
roles:
- role: standard-test-beakerlib
tests:
@ -11,3 +26,32 @@
required_packages:
- util-linux
- util-linux-user
- role: standard-test-basic
required_packages:
- util-linux
- util-linux-user
- coreutils
- bzip2
- xz
- bc
- device-mapper
- iproute
- mdadm
- e2fsprogs
- binutils
- procps-ng
- sed
- gawk
- file
- git
- rpm-build
tests:
- prepare-source:
dir: ./
run: rpmbuild -bp {{tenv_workdir}}/*.spec --nodeps --define "_sourcedir {{tenv_workdir}}" --define "_builddir {{tenv_workdir}}/source"
- flatten-source:
dir: ./
run: shopt -s dotglob; mv {{tenv_workdir}}/source/*/* {{tenv_workdir}}/source
- upstream:
dir: upstream
run: ./runtest.sh

40
tests/upstream/runtest.sh Executable file
View File

@ -0,0 +1,40 @@
#!/bin/bash
# Copyright (C) 2019 Karel Zak <kzak@redhat.com>
srcdir="../source"
testdir="${srcdir}/tests"
echo "srcdir: $srcdir"
echo "testdir: $testdir"
if [ ! -x "${testdir}/run.sh" ]; then
echo "upstream tests not found"
exit 1
fi
# We don't compile the tests, but it still needs some hints.
#
echo > ${srcdir}/config.h
echo "#define HAVE_WIDECHAR 1" >> ${srcdir}/config.h
echo "#define HAVE_LINUX_NET_NAMESPACE_H 1" >> ${srcdir}/config.h
# Remove stuff we don't have in RHEL
#
rm -rf ${testdir}/ts/misc/line
version_tests=$(cat ${srcdir}/.version)
version_system=$(rpm -q util-linux)
echo
echo "Used versions: "
echo " upstream tests: $version_tests"
echo " installed util-linux: $version_system"
echo
# Run upstream tests
#
${testdir}/run.sh --use-system-commands --noskip-commands --show-diff
exit $?

View File

@ -2,7 +2,7 @@
Summary: A collection of basic system utilities
Name: util-linux
Version: 2.37.2
Release: 1%{?dist}
Release: 2%{?dist}
License: GPLv2 and GPLv2+ and LGPLv2+ and BSD with advertising and Public Domain
URL: http://en.wikipedia.org/wiki/Util-linux
@ -91,10 +91,15 @@ Requires: util-linux-core = %{version}-%{release}
### Ready for upstream?
###
# 151635 - makeing /var/log/lastlog
Patch0: login-lastlog-create.patch
Patch0: 0000-login-lastlog-create.patch
# Add `/run/motd.d` to the hardcoded MOTD_FILE
# https://github.com/coreos/console-login-helper-messages/issues/60
Patch1: login-default-motd-file.patch
Patch1: 0001-login-default-motd-file.patch
# Integrate upstream tests
Patch2: 0002-tests-make-.-run.sh-more-robust.patch
Patch3: 0003-tests-make-mount-fstab-all-more-robust.patch
Patch4: 0004-tests-make-eject-umount-tests-more-robust.patch
%description
The util-linux package contains a large variety of low-level system
@ -944,8 +949,12 @@ fi
%{_libdir}/python*/site-packages/libmount/
%changelog
* Mon Aug 23 2021 Karel Zak <kzak@redhat.com> - 2.37.2-2
- cleanup previous rebase (fix #1969348)
- integrate upstream tests to CI
* Fri Aug 20 2021 Karel Zak <kzak@redhat.com> - 2.37.2-1
- upgrade to v2.37.1 (fix #1969348)
- upgrade to v2.37.2 (fix #1969348)
* Tue Aug 10 2021 Mohan Boddu <mboddu@redhat.com>
- Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
@ -980,7 +989,7 @@ fi
* Mon Nov 16 2020 Karel Zak <kzak@redhat.com> - 2.36.1-1
- upgrade to stable upstream 2.36.1
* Tue Jul 14 2020 Tom Stellard <tstellar@redhat.com> - 2.36-4
* Thu Nov 12 2020 Tom Stellard <tstellar@redhat.com> - 2.36-4
- Use make macros
- https://fedoraproject.org/wiki/Changes/UseMakeBuildInstallMacro