From patchwork Thu Aug 16 17:59:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 144441 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2377448ljj; Thu, 16 Aug 2018 11:03:40 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwCO98X4ybhguuuvamwxzBMDw9gZfBPv8geH14XiGW9soP3eyor3JflUZVHC4dH1Pv9cp3h X-Received: by 2002:a37:27ca:: with SMTP id n193-v6mr2376111qkn.444.1534442620008; Thu, 16 Aug 2018 11:03:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534442620; cv=none; d=google.com; s=arc-20160816; b=NoOiuc6DdB3cEWT/bsy55e98tQglZ1ZmQiJ3TfbmA7HJlgs+1+mmbnKMvdZMFzK8jK 3B1M+4QHbBevsD1RMkmZplb+W+AVrFV7ACPo2KuwRQuAhQSiDga919H2zxTZInXuoF1M SiFKHcsxH2dm/9VM6P10NDIOVotS7eTBUXk6hX0nEI8lmD6vGji0E1oh/cO9nYl1o9xc HPT3nXJtAx/q9l7g48l8e8JJkGBiJrIjllpL8jP8F9wJwjKj5MmADWkdLuhlCTtLa4xj r6m/lTOkqVzEgw1rHxGKYM0LW19eMZTZITJn+tRjchpDg3AcpgJGwaorBCrZP6/2RKVJ C0rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=S6wS7zNYtWEkyxO3XhNy5tsmXPJsov1rilYtlu/Dito=; b=e2U8TpW0bsniQ0qkT5ibnfJoUpYdcNEdClU26iG1IOX/mJW/xq5vHiEVtSQJYCIQwX ETVNPCC0b18DBlFigPeZ0UTFDtAxAXtPzFoVeicVN/0a+KbwZVv9eBx3f2J5hn/e0Cmy 4x1Lrox0VQuqFPbVVUtkd70R7C9u26kGzKMOAfMHbra840P35mpssRZoGMFdgFpBGVex DtWY0uFhiLHSdeRXQNnTQHoRpA7AG+gWOmttnouLq6RYYwaZxWbAoh/H0ql86Ly94cjq H/c2/Oi5GzRjXzGtJke+3q+kkRFTtFKmE3/WZDR91F5wq029xxExSqTE5Gm2M8ZJ9dZD lRxw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id r1-v6si15689007qkf.172.2018.08.16.11.03.39; Thu, 16 Aug 2018 11:03:39 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 803BF68615; Thu, 16 Aug 2018 18:03:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MAILING_LIST_MULTI autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id E367F68681; Thu, 16 Aug 2018 18:00:46 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 90F9368610; Thu, 16 Aug 2018 18:00:25 +0000 (UTC) Received: from forward103p.mail.yandex.net (forward103p.mail.yandex.net [77.88.28.106]) by lists.linaro.org (Postfix) with ESMTPS id 749B060F59 for ; Thu, 16 Aug 2018 18:00:16 +0000 (UTC) Received: from mxback14g.mail.yandex.net (mxback14g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:93]) by forward103p.mail.yandex.net (Yandex) with ESMTP id 8FC5F218100D for ; Thu, 16 Aug 2018 21:00:14 +0300 (MSK) Received: from smtp1o.mail.yandex.net (smtp1o.mail.yandex.net [2a02:6b8:0:1a2d::25]) by mxback14g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id cJA8O5rgOF-0EOutk7u; Thu, 16 Aug 2018 21:00:14 +0300 Received: by smtp1o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 161oJSJY57-0DfWJ5In; Thu, 16 Aug 2018 21:00:13 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 16 Aug 2018 17:59:59 +0000 Message-Id: <1534442411-9502-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1534442411-9502-1-git-send-email-odpbot@yandex.ru> References: <1534442411-9502-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 647 Subject: [lng-odp] [PATCH v6 1/13] travis: add docker tests X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Maxim Uvarov initial Travis docker runs Signed-off-by: Maxim Uvarov --- /** Email created from pull request 647 (muvarov:master_docker2) ** https://github.com/Linaro/odp/pull/647 ** Patch: https://github.com/Linaro/odp/pull/647.patch ** Base sha: 6886a2ca3d3dd1157302ddee424c36f632903bd0 ** Merge commit sha: f6c5d19f7fb9f83b8a2cbbed692a510146ca7df9 **/ .travis.yml | 362 +++++++------------------------------ scripts/ci/build_dpdk.sh | 89 +++++++++ scripts/ci/build_x86_64.sh | 45 +++++ scripts/ci/coverage.sh | 35 ++++ scripts/ci/distcheck.sh | 25 +++ 5 files changed, 261 insertions(+), 295 deletions(-) create mode 100755 scripts/ci/build_dpdk.sh create mode 100755 scripts/ci/build_x86_64.sh create mode 100755 scripts/ci/coverage.sh create mode 100755 scripts/ci/distcheck.sh diff --git a/.travis.yml b/.travis.yml index 26bbe1a32..0f0db2931 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,6 +7,12 @@ # pushing to github/coverity_scan will also launch a static analysis # See https://scan.coverity.com/travis_ci +# +# Travis uses Docker images which mainained here: +# https://github.com/Linaro/odp-docker-images +# CI scirpts are maintained under ./scripts/ci/ directory +# which passed into container during the test run. + language: c sudo: required dist: trusty @@ -49,6 +55,7 @@ env: # you need generated new one at https://codecov.io specific for your repo. - CODECOV_TOKEN=a733c34c-5f5c-4ff1-af4b-e9f5edb1ab5e - DPDK_VERS="17.11.3" + - DOCKER_NAMESPACE="lngodp" matrix: - CONF="" - CONF="--disable-abi-compat" @@ -63,278 +70,45 @@ env: compiler: - gcc - - clang-3.8 - -before_install: - - # Install cross toolchains, etc - # apt-get update may fail thanks to Ubuntu removing Packages/indices while not removing relevant parts of Release file - - if [ -n "$CROSS_ARCH" ] ; - then - BUILD_GNU_TYPE=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_BUILD_GNU_TYPE` ; - CROSS_GNU_TYPE=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_HOST_GNU_TYPE` ; - CROSS_MULTIARCH=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_HOST_MULTIARCH` ; - CROSS="--host="$CROSS_GNU_TYPE" --build="$BUILD_GNU_TYPE"" ; - sudo dpkg --add-architecture "$CROSS_ARCH" ; - PKGS="build-essential libc6-dev:$CROSS_ARCH libssl-dev:$CROSS_ARCH zlib1g-dev:$CROSS_ARCH libconfig-dev:$CROSS_ARCH libstdc++-4.8-dev:$CROSS_ARCH libpcap0.8-dev:$CROSS_ARCH" ; - if [ "$CROSS_ARCH" = "i386" ] ; - then - PKGS="$PKGS g++-multilib" ; - else - PKGS="$PKGS g++-$CROSS_GNU_TYPE" ; - sudo sed -e 's/^deb http/deb [arch=amd64] http/g' /etc/apt/sources.list -i ; - sudo -E apt-add-repository -y "deb [arch=$CROSS_ARCH] http://ports.ubuntu.com trusty main" ; - sudo -E apt-add-repository -y "deb [arch=$CROSS_ARCH] http://ports.ubuntu.com trusty-updates main" ; - fi ; - if [ "$CROSS_ARCH" != "armhf" ] ; - then - PKGS="$PKGS libnuma-dev:$CROSS_ARCH" ; - fi ; - sudo cat /etc/apt/sources.list ; - sudo -E apt-get -y update || true ; - sudo -E apt-get -y --no-install-suggests --no-install-recommends --force-yes install $PKGS || exit 1 ; - export PKG_CONFIG_PATH=/usr/lib/${CROSS_MULTIARCH}/pkgconfig:/usr/${CROSS_MULTIARCH}/lib/pkgconfig ; - fi - - if [ "${CC#clang}" != "${CC}" ] ; - then - if [ -n "$CROSS_ARCH" ] ; - then - export CC="${CC} --target=$CROSS_GNU_TYPE" ; - if [ "$CROSS_ARCH" = "i386" ] ; - then - DPDK_CFLAGS="-m32" ; - else - DPDK_CROSS="$CROSS_GNU_TYPE-" ; - DPDK_CFLAGS="--target=$CROSS_GNU_TYPE" ; - fi - fi ; - export CXX="${CC/clang/clang++}"; - elif [ "$CROSS_ARCH" = "i386" ] ; - then - export CC="gcc -m32" ; - export CXX="g++ -m32" ; - DPDK_CFLAGS="-m32" ; - elif [ -n "$CROSS_ARCH" ] ; - then - export CC="$CROSS_GNU_TYPE"-gcc ; - export CXX="$CROSS_GNU_TYPE"-g++ ; - DPDK_CROSS="$CROSS_GNU_TYPE-" ; - fi - - if test ! -L /usr/lib/ccache/${CC%% *} ; then sudo ln -s -t /usr/lib/ccache/ `which ${CC%% *}` ; fi - - ccache -s - # Install cunit for the validation tests because distro version is too old and fails C99 compile - - sudo apt-get remove libcunit1-dev libcunit1 - - export LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" - - | - if [ ! -f "$HOME/cunit-install/$CROSS_ARCH/lib/libcunit.a" ]; then - export CUNIT_VERSION=2.1-3 - curl -sSOL https://github.com/Linaro/libcunit/releases/download/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2 - tar -jxf *.bz2 - pushd CUnit* - libtoolize --force --copy - aclocal - autoheader - automake --add-missing --include-deps --copy - autoconf - ./configure --prefix=$HOME/cunit-install/$CROSS_ARCH --enable-debug --enable-automated --enable-basic --enable-console --enable-examples --enable-test $CROSS || cat config.log - make -j $(nproc) - sudo make install - popd - fi - - export PKG_CONFIG_PATH="$HOME/cunit-install/$CROSS_ARCH/lib/pkgconfig:${PKG_CONFIG_PATH}" - - find $HOME/cunit-install - - # workaround for tap driver issues on recent Travis images - # Allow forwaring on virtual interfaces used for testing - - sudo iptables --policy FORWARD ACCEPT - + - clang install: - - echo 1000 | sudo tee /proc/sys/vm/nr_hugepages - - sudo mkdir -p /mnt/huge - - sudo mount -t hugetlbfs nodev /mnt/huge - - - if [ -z "$CROSS_ARCH" ] ; - then - sudo apt-get -qq update ; - sudo apt-get install linux-headers-`uname -r` ; - fi - - gem install asciidoctor - - # DPDK pktio. Cache will be updated automatically to ${DPDK_VERS} - - | - case "$CROSS_ARCH" in - "arm64") - DPDK_TARGET="arm64-armv8a-linuxapp-" - ;; - "armhf") - DPDK_TARGET="arm-armv7a-linuxapp-" - ;; - "i386") - DPDK_TARGET="i686-native-linuxapp-" - ;; - "") - DPDK_TARGET="x86_64-native-linuxapp-" - DPDK_MACHINE=snb - ;; - esac - - | - if [ -n "$DPDK_TARGET" ] ; then - if [ "${CC#clang}" != "${CC}" ] ; then - DPDKCC=clang ; - else - DPDKCC=gcc ; - fi - if [ -n "$DPDK_SHARED" ] ; then - TARGET="${DPDK_TARGET}$DPDKCC"-shared - LIBDPDKEXT=so - export LD_LIBRARY_PATH="`pwd`/${TARGET}:$LD_LIBRARY_PATH" - echo $LD_LIBRARY_PATH - else - TARGET="${DPDK_TARGET}$DPDKCC" - LIBDPDKEXT=a - fi - DPDK_TARGET="${DPDK_TARGET}gcc" - CACHED_DPDK_VERS=`fgrep Version dpdk/pkg/dpdk.spec | cut -d " " -f 2` - if [ ! -d dpdk -o "${CACHED_DPDK_VERS}" != "${DPDK_VERS}" ]; then - rm -rf dpdk - mkdir dpdk - pushd dpdk - git init - git -c advice.detachedHead=false fetch -q --depth=1 http://dpdk.org/git/dpdk-stable v${DPDK_VERS} - git checkout -f FETCH_HEAD - popd - fi - if [ ! -f "dpdk/${TARGET}/usr/local/lib/libdpdk.$LIBDPDKEXT" ]; then - pushd dpdk - git log --oneline --decorate - # AArch64 && ARMv7 fixup - sed -i -e 's/40900/40800/g' lib/librte_eal/common/include/arch/arm/rte_vect.h - sed -i -e 's/!(/!(defined(__arm__) \&\& defined(__clang__) || /g' lib/librte_eal/common/include/arch/arm/rte_byteorder.h - sed -i -e 's/__GNUC__/defined(__arm__) \&\& defined(__clang__) || __GNUC__/' lib/librte_eal/common/include/generic/rte_byteorder.h - sed -i -e 's,\$(CC),\0 $(EXTRA_CFLAGS),g' lib/librte_acl/Makefile - make config T=${DPDK_TARGET} O=${TARGET} - pushd ${TARGET} - sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config - # OCTEON TX driver includes ARM v8.1 instructions - sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_PMD=).*,\1n,' .config - sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=).*,\1n,' .config - sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL=).*,\1n,' .config - if test -n "${DPDK_MACHINE}" ; then - sed -ri 's,(CONFIG_RTE_MACHINE=).*,\1"'${DPDK_MACHINE}'",' .config - fi - if test -n "${DPDK_SHARED}" ; then - sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config - fi - if test -n "$CROSS_ARCH" ; then - sed -ri -e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' .config - sed -ri -e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' .config - fi - sed -ri -e 's,(CONFIG_RTE_TOOLCHAIN=).*,\1"'${DPDKCC}'",' .config - sed -ri -e '/CONFIG_RTE_TOOLCHAIN_.*/d' .config - echo CONFIG_RTE_TOOLCHAIN_${DPDKCC^^}=y >> .config - popd - make build O=${TARGET} EXTRA_CFLAGS="-fPIC $DPDK_CFLAGS" CROSS="$DPDK_CROSS" CC="$CC" HOSTCC=gcc -j $(nproc) - make install O=${TARGET} DESTDIR=${TARGET} - rm -r ./doc ./${TARGET}/app ./${TARGET}/build - popd - fi - EXTRA_CONF="$EXTRA_CONF --with-dpdk-path=`pwd`/dpdk/${TARGET}/usr/local" - fi - -# Netmap pktio - - | - if [ -z "$CROSS_ARCH" ]; then - if [ ! -f "netmap/LINUX/netmap.ko" ]; then - git -c advice.detachedHead=false clone -q --depth=1 --single-branch --branch=v11.2 https://github.com/luigirizzo/netmap.git - pushd netmap/LINUX - ./configure - make -j $(nproc) - popd - fi - sudo insmod ./netmap/LINUX/netmap.ko - EXTRA_CONF="$EXTRA_CONF --with-netmap-path=`pwd`/netmap" - fi - + - sudo apt-get install linux-headers-`uname -r` script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - --enable-debug=full - --enable-helper-linux - $CROSS $EXTRA_CONF $CONF - - make -j $(nproc) - - mkdir /dev/shm/odp - # Run all tests only for default configuration - - if [ -z "$CROSS_ARCH" ] ; then - if [ -n "$CONF" ] ; then - sudo ODP_CONFIG_FILE="`pwd`/config/odp-linux-generic.conf" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - else - sudo ODP_SCHEDULER=basic LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=sp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=iquery LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=scalable LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - fi + - if [ "${CC#clang}" != "${CC}" ] ; then LD=ld CXX=clang++; fi + - if [ -n "${CROSS_ARCH}" ] ; then + docker run -i -t -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" -e DPDK_SHARED="${DPDK_SHARED}" + maximuvarov/odp_ubuntu_14.04.5 /root/build_${CROSS_ARCH}.sh ; + else + docker run --privileged -i -t + -v `pwd`:/odp -v/lib/modules:/lib/modules -v/usr/src:/usr/src --shm-size 8g + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" -e DPDK_SHARED="${DPDK_SHARED}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/build_x86_64.sh $CONF ; fi - - make install - - - echo "Checking linking and run from install..." - - pushd $HOME - - echo "Dynamic link.." - - ${CC} ${CFLAGS} ${OLDPWD}/example/hello/odp_hello.c -o odp_hello_inst_dynamic `PKG_CONFIG_PATH=${HOME}/odp-install/lib/pkgconfig:${PKG_CONFIG_PATH} pkg-config --cflags --libs libodp-linux` - - if [ -z "$CROSS_ARCH" ] ; then - LD_LIBRARY_PATH="${HOME}/odp-install/lib:$LD_LIBRARY_PATH" ./odp_hello_inst_dynamic ; - fi - - | - # it is not possible to do static linking if we only have shared DPDK library. Compiler complains about missing -ldpdk - if [ -z "$TARGET" -o -z "$DPDK_SHARED" ] ; then - echo "Static link.." - ${CC} ${CFLAGS} ${OLDPWD}/example/hello/odp_hello.c -o odp_hello_inst_static `PKG_CONFIG_PATH=${HOME}/odp-install/lib/pkgconfig:${PKG_CONFIG_PATH} pkg-config --cflags --libs libodp-linux --static` -static || exit 1 - if [ -z "$CROSS_ARCH" ] ; then - ./odp_hello_inst_static; - fi - fi - - popd - - ccache -s - jobs: include: - stage: test env: TEST=coverage compiler: gcc script: - - sudo pip install coverage - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - --with-dpdk-path=`pwd`/dpdk/${TARGET}/usr/local - --with-netmap-path=`pwd`/netmap CFLAGS="-O0 -coverage" - CXXFLAGS="-O0 -coverage" LDFLAGS="--coverage" - --enable-debug=full - --enable-helper-linux - - CCACHE_DISABLE=1 make -j $(nproc) - - mkdir -p /dev/shm/odp - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=basic ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=sp ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=iquery ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=scalable ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - bash <(curl -s https://codecov.io/bash) -X coveragepy + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src --shm-size 8g + -e CODECOV_TOKEN="${CODECOV_TOKEN}" + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/coverage.sh - stage: test env: TEST=distcheck compiler: gcc script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - - sudo PATH="$PATH" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make distcheck - - stage: test - env: TEST=distcheck-non-abi - compiler: gcc - script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - - sudo PATH="$PATH" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make distcheck DISTCHECK__CONFIGURE_FLAGS=--disable-abi-compat + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src --shm-size 8g + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/distcheck.sh - stage: "build only" env: TEST=doxygen compiler: gcc @@ -364,85 +138,83 @@ jobs: true fi - stage: "build only" - env: CONF="" - compiler: gcc - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux - - make -j $(nproc) - - stage: "build only" - env: CONF="" - compiler: clang-3.8 - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux - - make -j $(nproc) - - stage: "build only" - env: CROSS_ARCH="i386" - compiler: gcc - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux $CROSS - - make -j $(nproc) - - stage: "build only" - env: CROSS_ARCH="arm64" + env: Ubuntu14.04_arm64 compiler: gcc - install: true script: - - ./bootstrap - - ./configure --enable-helper-linux $CROSS - - make -j $(nproc) + - docker run -i -t -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + maximuvarov/odp_ubuntu_14.04.5 /root/build_arm64.sh - stage: test compiler: gcc env: CROSS_ARCH="arm64" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="arm64" - stage: test compiler: gcc env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" - stage: test compiler: gcc env: CROSS_ARCH="powerpc" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="powerpc" - stage: test compiler: gcc env: CROSS_ARCH="i386" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="i386" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test compiler: gcc env: CROSS_ARCH="arm64" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="arm64" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="powerpc" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="powerpc" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="i386" CONF="--disable-abi-compat" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="i386" CONF="--disable-abi-compat" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test canfail: yes env: TEST=checkpatch diff --git a/scripts/ci/build_dpdk.sh b/scripts/ci/build_dpdk.sh new file mode 100755 index 000000000..9ceac4529 --- /dev/null +++ b/scripts/ci/build_dpdk.sh @@ -0,0 +1,89 @@ +#!/bin/bash -x + +set -e + +DPDK_VERS="17.11.2" +CROSS= + + +case "$CROSS_ARCH" in + "arm64") + DPDK_TARGET="arm64-armv8a-linuxapp-" + ;; + "armhf") + DPDK_TARGET="arm-armv7a-linuxapp-" + ;; + "i386") + DPDK_TARGET="i686-native-linuxapp-" + ;; + "") + DPDK_TARGET="x86_64-native-linuxapp-" + DPDK_MACHINE=snb + ;; +esac + + +if [ -n "$DPDK_TARGET" ] ; then + if [ "${CC#clang}" != "${CC}" ] ; then + DPDKCC=clang ; + else + DPDKCC=gcc ; + fi + if [ -n "$DPDK_SHARED" ] ; then + TARGET="${DPDK_TARGET}$DPDKCC"-shared + LIBDPDKEXT=so + export LD_LIBRARY_PATH="`pwd`/${TARGET}:$LD_LIBRARY_PATH" + echo $LD_LIBRARY_PATH + else + TARGET="${DPDK_TARGET}$DPDKCC" + LIBDPDKEXT=a + fi + DPDK_TARGET="${DPDK_TARGET}gcc" + CACHED_DPDK_VERS=`fgrep Version dpdk/pkg/dpdk.spec | cut -d " " -f 2` + if [ ! -d dpdk -o "${CACHED_DPDK_VERS}" != "${DPDK_VERS}" ]; then + rm -rf dpdk + mkdir dpdk + pushd dpdk + git init + git -c advice.detachedHead=false fetch -q --depth=1 http://dpdk.org/git/dpdk-stable v${DPDK_VERS} + git checkout -f FETCH_HEAD + popd + fi + if [ ! -f "dpdk/${TARGET}/usr/local/lib/libdpdk.$LIBDPDKEXT" ]; then + pushd dpdk + git log --oneline --decorate + # AArch64 && ARMv7 fixup + sed -i -e 's/40900/40800/g' lib/librte_eal/common/include/arch/arm/rte_vect.h + sed -i -e 's/!(/!(defined(__arm__) \&\& defined(__clang__) || /g' lib/librte_eal/common/include/arch/arm/rte_byteorder.h + sed -i -e 's/__GNUC__/defined(__arm__) \&\& defined(__clang__) || __GNUC__/' lib/librte_eal/common/include/generic/rte_byteorder.h + sed -i -e 's,\$(CC),\0 $(EXTRA_CFLAGS),g' lib/librte_acl/Makefile + make config T=${DPDK_TARGET} O=${TARGET} + pushd ${TARGET} + sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config + # OCTEON TX driver includes ARM v8.1 instructions + sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_PMD=).*,\1n,' .config + sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=).*,\1n,' .config + sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL=).*,\1n,' .config + if test -n "${DPDK_MACHINE}" ; then + sed -ri 's,(CONFIG_RTE_MACHINE=).*,\1"'${DPDK_MACHINE}'",' .config + fi + if test -n "${DPDK_SHARED}" ; then + sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config + fi + if test -n "$CROSS_ARCH" ; then + sed -ri -e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' .config + sed -ri -e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' .config + fi + sed -ri -e 's,(CONFIG_RTE_TOOLCHAIN=).*,\1"'${DPDKCC}'",' .config + sed -ri -e '/CONFIG_RTE_TOOLCHAIN_.*/d' .config + echo CONFIG_RTE_TOOLCHAIN_${DPDKCC^^}=y >> .config + popd + make build O=${TARGET} EXTRA_CFLAGS="-fPIC $DPDK_CFLAGS" CROSS="$DPDK_CROSS" CC="${CC}" HOSTCC="gcc" -j $(nproc) + make install O=${TARGET} DESTDIR=${TARGET} + pwd + rm -r ./doc ./${TARGET}/app ./${TARGET}/build + popd + fi +fi + +echo "`pwd`/dpdk/${TARGET}/usr/local" > /tmp/dpdk_install_dir diff --git a/scripts/ci/build_x86_64.sh b/scripts/ci/build_x86_64.sh new file mode 100755 index 000000000..1dc039e96 --- /dev/null +++ b/scripts/ci/build_x86_64.sh @@ -0,0 +1,45 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +export CROSS_ARCH="" + +export TARGET="x86_64$DPDKCC" + +$CWD/build_dpdk.sh +DPDKPATH=`cat /tmp/dpdk_install_dir` + +echo 1000 | tee /proc/sys/vm/nr_hugepages +mkdir -p /mnt/huge +mount -t hugetlbfs nodev /mnt/huge + +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure ${CONF} \ + --with-dpdk-path=${DPDKPATH} + +make clean +make -j 8 +# Tell some time sensative ODP test that they can be skipped due to not +# isolated environment. +export CI="true" +make check + +if [ $? -ne 0 ]; then + find . -name "*.trs" | xargs grep -l '^.test-result. FAIL' | while read trs ; do echo FAILURE detected at $trs; cat ${trs%%.trs}.log ; done +fi + +cd ~ +rm -rf ${TDIR} + +umount /mnt/huge + diff --git a/scripts/ci/coverage.sh b/scripts/ci/coverage.sh new file mode 100755 index 000000000..4326cb7dd --- /dev/null +++ b/scripts/ci/coverage.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +export CROSS_ARCH="" +export TARGET="x86_64$DPDKCC" + +$CWD/build_dpdk.sh + +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure \ + CFLAGS="-O0 -coverage" CXXFLAGS="-O0 -coverage" LDFLAGS="--coverage" \ + --enable-debug=full --enable-helper-linux +make clean +export CCACHE_DISABLE=1 +make -j $(nproc) + +ODP_SCHEDULER=basic make check +ODP_SCHEDULER=sp make check +ODP_SCHEDULER=iquery make check +ODP_SCHEDULER=scalable make check +bash <(curl -s https://codecov.io/bash) -X coveragepy + +cd ~ +rm -rf ${TDIR} diff --git a/scripts/ci/distcheck.sh b/scripts/ci/distcheck.sh new file mode 100755 index 000000000..22013473b --- /dev/null +++ b/scripts/ci/distcheck.sh @@ -0,0 +1,25 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure --enable-user-guides + +make clean +make distcheck + +make clean +make distcheck DISTCHECK__CONFIGURE_FLAGS=--disable-abi-compat + +cd ~ +rm -rf ${TDIR}