470 lines
18 KiB
Diff
470 lines
18 KiB
Diff
From 6ce72aaa5963b28a2a01b47f332fa20c02eeb0db Mon Sep 17 00:00:00 2001
|
|
From: Jason Gunthorpe <jgg@mellanox.com>
|
|
Date: Thu, 31 Oct 2019 13:17:11 -0300
|
|
Subject: [PATCH rdma-core 10/13] buildlib: Remove travis CI
|
|
|
|
[ Upstream commit ae029ac74d62b12ea69127c91f83ad5944130764 ]
|
|
|
|
Azure Pipelines replaces it completely now, remove the word 'travis' from
|
|
the source tree.
|
|
|
|
v26 was supposed to transition to azp for releases and it lacks the
|
|
ability for travis to make the offical github tar.gz, so this needs to be
|
|
back ported.
|
|
|
|
Cc: stable@linux-rdma.org #v26
|
|
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
|
|
Signed-off-by: Nicolas Morey-Chaisemartin <nmoreychaisemartin@suse.com>
|
|
---
|
|
.travis.yml | 45 -----------
|
|
CMakeLists.txt | 2 +-
|
|
Documentation/stable.md | 8 +-
|
|
README.md | 12 +--
|
|
buildlib/cbuild | 173 ++--------------------------------------
|
|
buildlib/check-build | 6 --
|
|
buildlib/travis-build | 58 --------------
|
|
7 files changed, 17 insertions(+), 287 deletions(-)
|
|
delete mode 100644 .travis.yml
|
|
delete mode 100755 buildlib/travis-build
|
|
|
|
diff --git a/.travis.yml b/.travis.yml
|
|
deleted file mode 100644
|
|
index e00b0165..00000000
|
|
--- a/.travis.yml
|
|
+++ /dev/null
|
|
@@ -1,45 +0,0 @@
|
|
-language: c
|
|
-dist: xenial
|
|
-addons:
|
|
- # We run our builds sequentially in one VM rather than try and use the
|
|
- # matrix feature. This is because Travis is unreasonably inefficient
|
|
- # doing this APT setup pass.
|
|
- apt:
|
|
- sources:
|
|
- - ubuntu-toolchain-r-test
|
|
- packages:
|
|
- - debhelper
|
|
- - dh-systemd
|
|
- - fakeroot
|
|
- - gcc-8
|
|
- - git-core
|
|
- - libnl-3-dev
|
|
- - libnl-route-3-dev
|
|
- - libudev-dev
|
|
- - make
|
|
- - ninja-build
|
|
- - pandoc
|
|
- - python-docutils
|
|
- - pkg-config
|
|
- - python
|
|
- - valgrind
|
|
- - sparse
|
|
- - wget
|
|
- - abi-compliance-checker
|
|
- - abi-dumper
|
|
-
|
|
- # 32 bit support packages
|
|
- - gcc-multilib
|
|
- # xenial craziness, need to give specific version of multilib,
|
|
- # in addition to general multilib
|
|
- - gcc-8-multilib
|
|
- - lib32gcc-8-dev
|
|
-
|
|
- # pyverbs
|
|
- - python3-dev
|
|
- - python3-pip
|
|
-
|
|
-before_script:
|
|
- - http_proxy= pip3 install cython
|
|
-script:
|
|
- - buildlib/travis-build
|
|
diff --git a/CMakeLists.txt b/CMakeLists.txt
|
|
index 59ffdf83..b5da62b1 100644
|
|
--- a/CMakeLists.txt
|
|
+++ b/CMakeLists.txt
|
|
@@ -44,7 +44,7 @@
|
|
# prefers python3 if available.
|
|
# -DNO_PYVERBS=1 (default, build pyverbs)
|
|
# Invoke cython to build pyverbs. Usually you will run with this option
|
|
-# is set, but it will be disabled for travis runs.
|
|
+# set
|
|
# -DENABLE_IBDIAGS_COMPAT=True (default False)
|
|
# Include obsolete scripts. These scripts are replaced by C programs with
|
|
# a different interface now.
|
|
diff --git a/Documentation/stable.md b/Documentation/stable.md
|
|
index 76f5cf32..c12b2768 100644
|
|
--- a/Documentation/stable.md
|
|
+++ b/Documentation/stable.md
|
|
@@ -10,7 +10,7 @@ Branched stable releases, off a mainline release, are on as-needed basis and lim
|
|
|
|
All bug fixes are to be backported from mainline and applied by stable branch maintainer.
|
|
|
|
-Branched stable releases will append an additional release number (e.g. 15.1) and will ensure that Travis CI reports a successful build.
|
|
+Branched stable releases will append an additional release number (e.g. 15.1) and will ensure that Azure Pipelines CI reports a successful build.
|
|
|
|
Regular stable releases will be generated at the same time as mainline releases.
|
|
Additional stable releases can be generated if the need arise (Needed by distributions or OFED).
|
|
@@ -73,17 +73,17 @@ so that latters patches/fixes can be checked against this reference.
|
|
|
|
To do that, the creator of the branch should run
|
|
```
|
|
-./buildlib/cbuild build-images travis
|
|
+./buildlib/cbuild build-images azp
|
|
mkdir ABI
|
|
touch ABI/.gitignore
|
|
git add ABI/.gitignore
|
|
git commit -m "ABI Files"
|
|
-./buildlib/cbuild pkg travis
|
|
+./buildlib/cbuild pkg azp
|
|
git add ABI/*
|
|
git commit --amend
|
|
```
|
|
|
|
-'cbuild pkg travis' will fail as the ABI verification step files, but it will
|
|
+'cbuild pkg azp' will fail as the ABI verification step files, but it will
|
|
produce the ABI reference files.
|
|
|
|
Note that the ABI directory must NOT be committed at any point in the master branch.
|
|
diff --git a/README.md b/README.md
|
|
index 451ff7fc..36273ad9 100644
|
|
--- a/README.md
|
|
+++ b/README.md
|
|
@@ -1,4 +1,4 @@
|
|
-[](https://travis-ci.org/linux-rdma/rdma-core)
|
|
+[](https://dev.azure.com/ucfconsort/rdma-core/_build/latest?definitionId=2&branchName=master)
|
|
|
|
# RDMA Core Userspace Libraries and Daemons
|
|
|
|
@@ -137,13 +137,13 @@ Make sure that your contribution can be licensed under the same
|
|
license as the original code you are patching, and that you have all
|
|
necessary permissions to release your work.
|
|
|
|
-## TravisCI
|
|
+## Azure Pipelines CI
|
|
|
|
-Submitted patches must pass the TravisCI automatic builds without warnings.
|
|
-A build similar to TravisCI can be run locally using docker and the
|
|
+Submitted patches must pass the Azure Pipelines CI automatic builds without
|
|
+warnings. A build similar to AZP can be run locally using docker and the
|
|
'buildlib/cbuild' script.
|
|
|
|
```sh
|
|
-$ buildlib/cbuild build-images travis
|
|
-$ buildlib/cbuild pkg travis
|
|
+$ buildlib/cbuild build-images azp
|
|
+$ buildlib/cbuild pkg azp
|
|
```
|
|
diff --git a/buildlib/cbuild b/buildlib/cbuild
|
|
index 742a9e22..9825e099 100755
|
|
--- a/buildlib/cbuild
|
|
+++ b/buildlib/cbuild
|
|
@@ -290,100 +290,6 @@ class debian_experimental(APTEnvironment):
|
|
" ".join(sorted(self.pkgs))));
|
|
return res;
|
|
|
|
-class travis(APTEnvironment):
|
|
- """This parses the .travis.yml "apt" add on and converts it to a dockerfile,
|
|
- basically creating a container that is similar to what travis would
|
|
- use. Note this does not use the base travis image, nor does it install the
|
|
- typical travis packages."""
|
|
- docker_parent = "ubuntu:16.04";
|
|
- name = "travis";
|
|
- is_deb = True;
|
|
- _yaml = None;
|
|
-
|
|
- def get_yaml(self):
|
|
- if self._yaml:
|
|
- return self._yaml;
|
|
-
|
|
- # Load the commands from the travis file
|
|
- with open(".travis.yml") as F:
|
|
- self._yaml = yaml.safe_load(F);
|
|
- return self._yaml;
|
|
- yaml = property(get_yaml);
|
|
-
|
|
- def get_repos(self):
|
|
- """Return a list of things to add with apt-add-repository"""
|
|
- Source = collections.namedtuple("Source",["sourceline","key_url"]);
|
|
-
|
|
- # See https://github.com/travis-ci/apt-source-whitelist/blob/master/ubuntu.json
|
|
- pre_defined = {
|
|
- "ubuntu-toolchain-r-test": Source("ppa:ubuntu-toolchain-r/test",None),
|
|
- };
|
|
-
|
|
- # Unique the sources
|
|
- res = set();
|
|
- for src in self.yaml["addons"]["apt"]["sources"]:
|
|
- if isinstance(src,dict):
|
|
- res.add(Source(sourceline=src["sourceline"],
|
|
- key_url=src.get("key_url",None)));
|
|
- else:
|
|
- res.add(pre_defined[src]);
|
|
-
|
|
- # Add the sources
|
|
- scmds = [];
|
|
- scmds.extend("apt-key add /etc/apt/trusted.gpg.d/%s"%(os.path.basename(I.key_url))
|
|
- for I in res if I.key_url is not None);
|
|
- scmds.extend("http_proxy= apt-add-repository -y %s"%(pipes.quote(I.sourceline))
|
|
- for I in res);
|
|
-
|
|
- # Download the keys
|
|
- cmds = ["ADD %s /etc/apt/trusted.gpg.d/"%(I.key_url)
|
|
- for I in res if I.key_url is not None];
|
|
-
|
|
- cmds.append("RUN " + " && ".join(scmds));
|
|
- return cmds;
|
|
-
|
|
- def get_before_script(self):
|
|
- """Return a list of commands to run from before_script"""
|
|
- cmds = ["RUN useradd -ms /bin/bash travis && \\"
|
|
- "su -l -c %s"%(pipes.quote(" && ".join(self.yaml["before_script"]))) + " travis"];
|
|
- return cmds
|
|
-
|
|
- def get_clang(self):
|
|
- """We are using the clang that comes in travis, which is not part of our base
|
|
- docker container, install something similar by hand."""
|
|
- llvm_tar = "clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz"
|
|
- cmds = [
|
|
- """RUN wget -q http://releases.llvm.org/7.0.0/{0} -O /tmp/{0} && \\
|
|
- tar xf /tmp/{0} -C /usr/local/ && \\
|
|
- rm /tmp/{0} && \\
|
|
- (cd /usr/local/bin/ && ln -sf ../clang*/bin/clang-7 .)""".format(llvm_tar)];
|
|
- return cmds;
|
|
-
|
|
- def get_cython(self):
|
|
- return ["""RUN pip3 install cython"""]
|
|
-
|
|
- def get_docker_file(self,tmpdir):
|
|
- # First this to get apt-add-repository
|
|
- self.pkgs = {"software-properties-common"}
|
|
- res = APTEnvironment.get_docker_file(self,tmpdir);
|
|
-
|
|
- # Sources list from the travis.yml
|
|
- res.lines.extend(self.get_repos());
|
|
-
|
|
- # Package list from the travis.yml
|
|
- # Travis uses the new cmake built into the image, we need to get an
|
|
- # older version from ubuntu.
|
|
- res.lines.append("RUN apt-get update && apt-get install -y --no-install-recommends %s"%(
|
|
- " ".join(sorted(["cmake"] + self.yaml["addons"]["apt"]["packages"]))));
|
|
-
|
|
- # Adding before_script commands
|
|
- res.lines.extend(self.get_before_script())
|
|
-
|
|
- res.lines.extend(self.get_clang())
|
|
- res.lines.extend(self.get_cython())
|
|
-
|
|
- return res;
|
|
-
|
|
# -------------------------------------------------------------------------
|
|
|
|
class ZypperEnvironment(Environment):
|
|
@@ -521,7 +427,6 @@ deb [arch=arm64,ppc64el] http://ports.ubuntu.com/ bionic-updates main universe""
|
|
environments = [centos6(),
|
|
centos7(),
|
|
centos7_epel(),
|
|
- travis(),
|
|
xenial(),
|
|
bionic(),
|
|
jessie(),
|
|
@@ -572,7 +477,7 @@ def env_choices_pkg():
|
|
"""All the names that can be used with ToEnvAction"""
|
|
envs = set(("all",));
|
|
for I in environments:
|
|
- if I.name == "travis" or getattr(I,"is_deb",False) or getattr(I,"is_rpm",False):
|
|
+ if getattr(I,"is_deb",False) or getattr(I,"is_rpm",False):
|
|
envs.add(I.name);
|
|
envs.update(I.aliases);
|
|
return envs;
|
|
@@ -660,12 +565,12 @@ def get_tar_file(args,tarfn,pandoc_prebuilt=False):
|
|
return;
|
|
|
|
# When the OS does not support pandoc we got through the extra step to
|
|
- # build pandoc output in the travis container and include it in the
|
|
+ # build pandoc output in the azp container and include it in the
|
|
# tar.
|
|
if not args.use_prebuilt_pandoc:
|
|
- subprocess.check_call(["buildlib/cbuild","make","travis","docs"]);
|
|
+ subprocess.check_call(["buildlib/cbuild","make","azure_pipelines","docs"]);
|
|
|
|
- cmd_make_dist_tar(argparse.Namespace(BUILD="build-travis",tarfn=tarfn,
|
|
+ cmd_make_dist_tar(argparse.Namespace(BUILD="build-azure_pipelines",tarfn=tarfn,
|
|
script_pwd="",tag=None));
|
|
|
|
def run_rpm_build(args,spec_file,env):
|
|
@@ -839,70 +744,6 @@ def copy_abi_files(src):
|
|
print("Changed ABI File: ", ref_fn);
|
|
shutil.copy(cur_fn, ref_fn);
|
|
|
|
-def run_travis_build(args,env):
|
|
- with private_tmp(args) as tmpdir:
|
|
- os.mkdir(os.path.join(tmpdir,"src"));
|
|
- os.mkdir(os.path.join(tmpdir,"tmp"));
|
|
-
|
|
- opwd = os.getcwd();
|
|
- with inDirectory(os.path.join(tmpdir,"src")):
|
|
- subprocess.check_call(["git",
|
|
- "--git-dir",os.path.join(opwd,".git"),
|
|
- "reset","--hard","HEAD"]);
|
|
- subprocess.check_call(["git",
|
|
- "--git-dir",os.path.join(opwd,".git"),
|
|
- "fetch",
|
|
- "--no-tags",
|
|
- "https://github.com/linux-rdma/rdma-core.git","HEAD",
|
|
- "master"]);
|
|
- base = subprocess.check_output(["git",
|
|
- "--git-dir",os.path.join(opwd,".git"),
|
|
- "merge-base",
|
|
- "HEAD","FETCH_HEAD"]).decode().strip();
|
|
-
|
|
- home = os.path.join(os.path.sep,"home","travis");
|
|
- home_build = os.path.join(os.path.sep,home,"build");
|
|
-
|
|
- opts = [
|
|
- "run",
|
|
- "--read-only",
|
|
- "--rm=true",
|
|
- "-v","%s:%s"%(tmpdir, home_build),
|
|
- "-w",os.path.join(home_build,"src"),
|
|
- "-u",str(os.getuid()),
|
|
- "-e","TRAVIS_COMMIT_RANGE=%s..HEAD"%(base),
|
|
- "-e","TRAVIS_BRANCH=%s"%(base),
|
|
- "-e","TRAVIS_EVENT_TYPE=pull_request",
|
|
- "-e","HOME=%s"%(home),
|
|
- "-e","TMPDIR=%s"%(os.path.join(home_build,"tmp")),
|
|
- ] + map_git_args(opwd,os.path.join(home_build,"src"));
|
|
-
|
|
- # Load the commands from the travis file
|
|
- with open(os.path.join(opwd,".travis.yml")) as F:
|
|
- cmds = yaml.safe_load(F)["script"];
|
|
-
|
|
- with open(os.path.join(tmpdir,"go.sh"),"w") as F:
|
|
- print("#!/bin/bash", file=F);
|
|
- print("set -e", file=F);
|
|
- for I in cmds:
|
|
- print(I, file=F);
|
|
-
|
|
- if args.run_shell:
|
|
- opts.append("-ti");
|
|
- opts.append(env.image_name());
|
|
-
|
|
- if args.run_shell:
|
|
- opts.append("/bin/bash");
|
|
- else:
|
|
- opts.extend(["/bin/bash",os.path.join(home_build,"go.sh")]);
|
|
-
|
|
- try:
|
|
- docker_cmd(args,*opts);
|
|
- except subprocess.CalledProcessError as e:
|
|
- copy_abi_files(os.path.join(tmpdir, "src/ABI"));
|
|
- raise;
|
|
- copy_abi_files(os.path.join(tmpdir, "src/ABI"));
|
|
-
|
|
def run_azp_build(args,env):
|
|
# Load the commands from the pipelines file
|
|
with open("buildlib/azure-pipelines.yml") as F:
|
|
@@ -995,7 +836,7 @@ def args_pkg(parser):
|
|
parser.add_argument("--run-shell",default=False,action="store_true",
|
|
help="Instead of running the build, enter a shell");
|
|
parser.add_argument("--use-prebuilt-pandoc",default=False,action="store_true",
|
|
- help="Do not rebuild the pandoc cache in build-travis/pandoc-prebuilt/");
|
|
+ help="Do not rebuild the pandoc cache in build-azure_pipelines/pandoc-prebuilt/");
|
|
parser.add_argument("--with", default=[],action="append", dest="with_flags",
|
|
help="Enable specified feature in RPM builds");
|
|
parser.add_argument("--without", default=[],action="append", dest="without_flags",
|
|
@@ -1003,9 +844,7 @@ def args_pkg(parser):
|
|
def cmd_pkg(args):
|
|
"""Build a package in the given environment."""
|
|
for env in args.ENV:
|
|
- if env.name == "travis":
|
|
- run_travis_build(args,env);
|
|
- elif env.name == "azure_pipelines":
|
|
+ if env.name == "azure_pipelines":
|
|
run_azp_build(args,env);
|
|
elif getattr(env,"is_deb",False):
|
|
run_deb_build(args,env);
|
|
diff --git a/buildlib/check-build b/buildlib/check-build
|
|
index 46053527..ab8524e5 100755
|
|
--- a/buildlib/check-build
|
|
+++ b/buildlib/check-build
|
|
@@ -364,12 +364,6 @@ def get_cc_args_from_pkgconfig(args, name, static):
|
|
if not static:
|
|
return opts
|
|
|
|
- # The old pkg-config that travis uses incorrectly removes duplicated
|
|
- # flags, which breaks linking.
|
|
- if (name == "ibverbs" and
|
|
- subprocess.check_output(["pkg-config", "--version"]).decode().strip() == "0.26"):
|
|
- opts.insert(0, "-libverbs")
|
|
-
|
|
# Only static link the pkg-config stuff, otherwise we get warnings about
|
|
# static linking portions of glibc that need NSS.
|
|
opts.insert(0, "-Wl,-Bstatic")
|
|
diff --git a/buildlib/travis-build b/buildlib/travis-build
|
|
deleted file mode 100755
|
|
index 48c1c8f6..00000000
|
|
--- a/buildlib/travis-build
|
|
+++ /dev/null
|
|
@@ -1,58 +0,0 @@
|
|
-#!/bin/bash
|
|
-
|
|
-PATH=/home/`whoami`/.local/bin:$PATH
|
|
-
|
|
-# Stop on error
|
|
-set -e
|
|
-# Echo all commands to Travis log
|
|
-set -x
|
|
-
|
|
-mkdir build-travis build32 build-sparse
|
|
-
|
|
-# Build with latest clang first
|
|
-cd build-travis
|
|
-CC=clang-7 CFLAGS=-Werror cmake -GNinja .. -DIOCTL_MODE=both -DENABLE_STATIC=1
|
|
-ninja
|
|
-../buildlib/check-build --src .. --cc clang-7
|
|
-
|
|
-# 32 bit build to check format strings/etc
|
|
-cd ../build32
|
|
-# travis is not configured in a way that enables all 32 bit
|
|
-# packages. We could fix this with some sudo stuff.. For now turn off libnl
|
|
-CC=gcc-8 CFLAGS="-Werror -m32 -msse3" cmake -GNinja .. -DENABLE_RESOLVE_NEIGH=0 -DIOCTL_MODE=both -DNO_PYVERBS=1
|
|
-ninja
|
|
-
|
|
-# Run sparse on the subdirectories which are sparse clean
|
|
-cd ../build-sparse
|
|
-mv ../CMakeLists.txt ../CMakeLists-orig.txt
|
|
-grep -v "# NO SPARSE" ../CMakeLists-orig.txt > ../CMakeLists.txt
|
|
-CC=cgcc CFLAGS="-Werror" cmake -GNinja .. -DIOCTL_MODE=both -DNO_PYVERBS=1
|
|
-ninja | grep -v '^\[' | tee out
|
|
-# sparse does not fail gcc on messages
|
|
-if [ -s out ]; then
|
|
- false
|
|
-fi
|
|
-mv ../CMakeLists-orig.txt ../CMakeLists.txt
|
|
-
|
|
-# Test with coherent DMA mode disabled (ie as would be on ARM32, etc)
|
|
-cd ../build-travis
|
|
-cp ../util/udma_barrier.h ../util/udma_barrier.h.old
|
|
-echo "#error Fail" >> ../util/udma_barrier.h
|
|
-rm CMakeCache.txt
|
|
-CC=clang-7 CFLAGS=-Werror cmake -GNinja .. -DIOCTL_MODE=both
|
|
-ninja
|
|
-cp ../util/udma_barrier.h.old ../util/udma_barrier.h
|
|
-
|
|
-# Finally run through gcc-8 64 bit through the debian packaging This gives a
|
|
-# good clue if patches are changing packaging related things, the RPM stuff
|
|
-# will have to be audited by hand.
|
|
-
|
|
-# When running cmake through debian/rules it is hard to set -Werror,
|
|
-# instead force it on by changing the CMakeLists.txt
|
|
-cd ..
|
|
-echo 'set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")' >> buildlib/RDMA_EnableCStd.cmake
|
|
-sed -i -e 's/-DCMAKE_BUILD_TYPE=Release/-DCMAKE_BUILD_TYPE=Debug/g' debian/rules
|
|
-sed -i -e 's/ninja \(.*\)-v/ninja \1/g' debian/rules
|
|
-
|
|
-CC=gcc-8 debian/rules build
|
|
-fakeroot debian/rules binary
|
|
--
|
|
2.20.1
|
|
|