From patchwork Thu Mar 27 12:56:22 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 27187 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oa0-f70.google.com (mail-oa0-f70.google.com [209.85.219.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 48A8E20062 for ; Thu, 27 Mar 2014 12:58:29 +0000 (UTC) Received: by mail-oa0-f70.google.com with SMTP id n16sf12737824oag.5 for ; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id :mime-version:cc:subject:precedence:list-id:list-unsubscribe :list-post:list-help:list-subscribe:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :list-archive:content-type:content-transfer-encoding; bh=JyrbHHnZk61WkS6Dmg32hb6jpSG6EppPHoytRlzxIok=; b=Qlr/gcNZ+gC5JhmHZK5OHdj1dcyQ3J8NSXRakF8SqJYAIJ8/Fcw/MOyHqNF8wqU+Ty k/V2c7daJob3SE8F94RctVpfdCoIby4xBNhJmk/qCEIDeiATta/bLAfAXoQcfy1LhPli P18laPZ0ogISQsoyRqirg4Fj4zg7+Y1U/iS//HQcjYo2ryMt20BThszDiPMaaabqIFKh 6fAxYTGZElDm7xahNPwawOAmIM4VEAEirA1bRwSBEdhMGtCR1kU+/wMJM+XHbRk91vJb 2yXUlf1wxrBOofUVaZeWPFY1X0iXeMaVwaH6bakMHNG/TwV2xPz77G3i9gsjWla8o0kw l/OQ== X-Gm-Message-State: ALoCoQmDDG4BkqIw9T7gsi0S3EgeHfkSsf4BhCkO6jiHm26GT1o0cHt5Rrwjl2h02/a6JBTaXl/b X-Received: by 10.182.109.200 with SMTP id hu8mr656384obb.20.1395925108715; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.48.202 with SMTP id o68ls1088433qga.53.gmail; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-Received: by 10.52.171.68 with SMTP id as4mr1133956vdc.0.1395925108611; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id la9si504710veb.200.2014.03.27.05.58.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 05:58:28 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id im17so4124544vcb.37 for ; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-Received: by 10.52.99.168 with SMTP id er8mr1129741vdb.26.1395925108465; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp25509vck; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) X-Received: by 10.221.40.10 with SMTP id to10mr1349761vcb.22.1395925108077; Thu, 27 Mar 2014 05:58:28 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id od9si512664vcb.65.2014.03.27.05.58.27 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 27 Mar 2014 05:58:28 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WT9rM-0004W5-9x; Thu, 27 Mar 2014 12:56:40 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WT9rK-0004Tc-4G for xen-devel@lists.xen.org; Thu, 27 Mar 2014 12:56:38 +0000 Received: from [85.158.143.35:58698] by server-1.bemta-4.messagelabs.com id C9/C0-09853-50024335; Thu, 27 Mar 2014 12:56:37 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-6.tower-21.messagelabs.com!1395924994!1291783!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15610 invoked from network); 27 Mar 2014 12:56:36 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-6.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 27 Mar 2014 12:56:36 -0000 X-IronPort-AV: E=Sophos;i="4.97,742,1389744000"; d="scan'208";a="114128494" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 27 Mar 2014 12:56:23 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Thu, 27 Mar 2014 08:56:22 -0400 Received: from spare.cam.xci-test.com ([10.80.2.80] helo=kazak.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WT9r4-0005st-IV; Thu, 27 Mar 2014 12:56:22 +0000 From: Ian Campbell To: Date: Thu, 27 Mar 2014 12:56:22 +0000 Message-ID: <1395924982-22346-1-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.9.0 MIME-Version: 1.0 X-DLP: MIA1 Cc: Dario Faggioli , Jim Fehlig , Ian Campbell , xen-devel@lists.xen.org Subject: [Xen-devel] [PATCH OSSTEST v2] Add libvirt build and test jobs. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: The ts-libvirt-build, sg-run-job and mfi-common changes have been tested in standalone mode. The ts-xen-install change has been tested in standalone mode, it is expected that the rest of test-*-*-libvirt will fail horribly at this time. The remaining changes are intended to introduce a libvirt push gate and have been tested as far as my limited ability to invoke cr-daily-branch by hand is able, but I am sure they are incomplete in some way or other. The libvirt.git initscript is horribly redhat specific, and the debian package one is large and complex, so ts-libvirt-build inserts its own simple initscript. The standalone wrapper is updated to handle path_libvirt in set-paths. libvirt.git's autogen.sh involves initialising/cloning a git submodule. AIUI the way git submodules work they are effectively versioned by the repo which references them and so there are no push gate implications here. Signed-off-by: Ian Campbell Cc: Jim Fehlig Cc: Dario Faggioli --- v2: Add a test job, which will clearly fail. Make ts-xen-install DTRT though --- Osstest/TestSupport.pm | 7 ++- ap-common | 5 ++ ap-fetch-version | 4 ++ ap-fetch-version-old | 5 ++ ap-print-url | 3 + ap-push | 5 ++ cr-daily-branch | 8 +++ cri-common | 1 + make-flight | 4 ++ mfi-common | 15 ++++- sg-run-job | 5 ++ standalone | 2 +- ts-libvirt-build | 158 +++++++++++++++++++++++++++++++++++++++++++++++++ ts-xen-install | 5 +- 14 files changed, 223 insertions(+), 4 deletions(-) create mode 100755 ts-libvirt-build diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index a957a69..c4a9bf2 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1747,7 +1747,12 @@ our %toolstacks= Command => 'xl', CfgPathVar => 'cfgpath', RestoreNeedsConfig => 1, - } + }, + 'libvirt' => { + NewDaemons => [qw(libvirtd)], + Dom0MemFixed => 1, + Command => 'virsh', + }, ); sub toolstack () { diff --git a/ap-common b/ap-common index cb0cfff..73bb90c 100644 --- a/ap-common +++ b/ap-common @@ -33,6 +33,10 @@ : ${TREEVCS_LINUX:=git} +: ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git} +: ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git} +: ${TREEVCS_LIBVIRT:=git} + : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git} : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git} : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git} @@ -56,6 +60,7 @@ fi : ${LOCALREV_XEN:=daily-cron.$branch} : ${LOCALREV_LINUX:=daily-cron.$branch} +: ${LOCALREV_LIBVIRT:=daily-cron.$branch} : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27} diff --git a/ap-fetch-version b/ap-fetch-version index aaf884d..2990462 100755 --- a/ap-fetch-version +++ b/ap-fetch-version @@ -69,6 +69,10 @@ linuxfirmware) repo_tree_rev_fetch_git linuxfirmware \ $UPSTREAM_TREE_LINUXFIRMWARE master daily-cron.$branch ;; +libvirt) + repo_tree_rev_fetch_git libvirt \ + $TREE_LIBVIRT master $LOCALREV_LIBVIRT + ;; osstest) if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then git fetch $HOME/testing.git pretest:ap-fetch >&2 diff --git a/ap-fetch-version-old b/ap-fetch-version-old index 21ff786..a32de0e 100755 --- a/ap-fetch-version-old +++ b/ap-fetch-version-old @@ -27,6 +27,7 @@ select_xenbranch : ${BASE_TAG_LINUX2639:=tested/2.6.39.x} : ${BASE_LOCALREV_LINUX:=daily-cron.$branch.old} +: ${BASE_LOCALREV_LIBVIRT:=daily-cron.$branch.old} : ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}} @@ -73,6 +74,10 @@ linuxfirmware) repo_tree_rev_fetch_git linuxfirmware \ $TREE_LINUXFIRMWARE master daily-cron-old.$branch ;; +libvirt) + repo_tree_rev_fetch_git libvirt \ + $TREE_LIBVIRT master $BASE_LOCALREV_LIBVIRT + ;; osstest) if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then git fetch -f $HOME/testing.git incoming:ap-fetch diff --git a/ap-print-url b/ap-print-url index f2812d0..2b55fce 100755 --- a/ap-print-url +++ b/ap-print-url @@ -46,6 +46,9 @@ linux-2.6.39) linuxfirmware) echo $UPSTREAM_TREE_LINUXFIRMWARE ;; +libvirt) + echo $TREE_LIBVIRT + ;; osstest) echo none:; ;; diff --git a/ap-push b/ap-push index dc0a166..af76ecc 100755 --- a/ap-push +++ b/ap-push @@ -32,6 +32,7 @@ select_xenbranch TREE_LINUX=$PUSH_TREE_LINUX TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-upstream-${xenbranch#xen-}.git TREE_XEN=$PUSH_TREE_XEN +TREE_LIBVIRT=$PUSH_TREE_LIBVIRT if info_linux_tree "$branch"; then cd $repos/linux @@ -72,6 +73,10 @@ linux-xen-*) cd $repos/linux git push $TREE_LINUX $revision:refs/heads/$branch ;; +libvirt) + cd $repos/libvirt + git push $TREE_LIBVIRT $revision:master + ;; osstest) git push $HOME/testing.git $revision:incoming git push $XENBITS:/home/xen/git/osstest.git $revision:master diff --git a/cr-daily-branch b/cr-daily-branch index 02fef15..a13d75a 100755 --- a/cr-daily-branch +++ b/cr-daily-branch @@ -133,6 +133,10 @@ if [ "x$REVISION_LINUX_ARM" = x ]; then export REVISION_LINUX_ARM fi fi +if [ "x$REVISION_LIBVIRT" = x ]; then + determine_version REVISION_LIBVIRT libvirt + export REVISION_LIBVIRT +fi if [ "x$REVISION_LINUXFIRMWARE" = x ]; then determine_version REVISION_LINUXFIRMWARE linuxfirmware export REVISION_LINUXFIRMWARE @@ -165,6 +169,10 @@ linuxfirmware) realtree=linuxfirmware NEW_REVISION=$REVISION_LINUXFIRMWARE ;; +libvirt) + realtree=libvirt + NEW_REVISION=$REVISION_LIBVIRT + ;; *) NEW_REVISION='' wantpush=false diff --git a/cri-common b/cri-common index 497d4e3..f472927 100644 --- a/cri-common +++ b/cri-common @@ -42,6 +42,7 @@ select_xenbranch () { linux) tree=linux; xenbranch=xen-unstable ;; linux-*) tree=linux; xenbranch=xen-unstable ;; linuxfirmware) tree=linuxfirmware; xenbranch=xen-unstable ;; + libvirt) tree=libvirt; xenbranch=xen-unstable ;; osstest) tree=osstest; xenbranch=xen-unstable ;; esac if [ "x$tree" = xlinux ]; then diff --git a/make-flight b/make-flight index 39fa7ba..efe10b5 100755 --- a/make-flight +++ b/make-flight @@ -244,6 +244,10 @@ test_matrix_do_one () { $debian_runvars all_hostflags=$most_hostflags fi + job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \ + $xenarch $dom0arch \ + $debian_runvars all_hostflags=$most_hostflags + do_freebsd_tests for qemuu_suffix in '' -qemut -qemuu; do diff --git a/mfi-common b/mfi-common index 598aa02..a8e528a 100644 --- a/mfi-common +++ b/mfi-common @@ -162,6 +162,16 @@ create_build_jobs () { tree_linuxfirmware=$TREE_LINUXFIRMWARE \ revision_linuxfirmware=$REVISION_LINUXFIRMWARE + ./cs-job-create $flight build-$arch-libvirt build-libvirt \ + arch=$arch \ + tree_xen=$TREE_XEN \ + $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \ + $suite_runvars \ + host_hostflags=$build_hostflags \ + buildjob=${bfi}build-$arch \ + tree_libvirt=$TREE_LIBVIRT revision_libvirt=$REVISION_LIBVIRT\ + ${TREEVCS_LIBVIRT:+treevcs_libvirt=}${TREEVCS_LIBVIRT} + case "$arch" in armhf) continue;; # don't do any other kernel builds esac @@ -193,6 +203,7 @@ job_create_test () { xenbuildjob="${bfi}build-$xenarch" buildjob="${bfi}build-$dom0arch" + tsbuildjob= case "$xenbranch:$toolstack" in xen-3.*-testing:*) ;; @@ -203,11 +214,13 @@ job_create_test () { *:xend) xenbuildjob="$xenbuildjob-xend" buildjob="${bfi}build-$dom0arch-xend" ;; + *:libvirt) tsbuildjob="libvirtbuildjob=$buildjob-libvirt" + ;; esac ./cs-job-create $flight $job $recipe toolstack=$toolstack \ $RUNVARS $TEST_RUNVARS $most_runvars \ - xenbuildjob=$xenbuildjob buildjob=$buildjob "$@" + xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@" } # Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one diff --git a/sg-run-job b/sg-run-job index d894711..e3c6b00 100755 --- a/sg-run-job +++ b/sg-run-job @@ -306,6 +306,7 @@ proc test-guest-nomigr {g} { proc need-hosts/build {} { return BUILD } proc need-hosts/build-kern {} { return BUILD } +proc need-hosts/build-libvirt {} { return BUILD } proc run-job/build {} { run-ts . = ts-xen-build @@ -315,6 +316,10 @@ proc run-job/build-kern {} { run-ts . = ts-kernel-build } +proc run-job/build-libvirt {} { + run-ts . = ts-libvirt-build +} + proc prepare-build-host {} { global jobinfo run-ts broken = ts-hosts-allocate + host diff --git a/standalone b/standalone index 37c0294..d10a223 100755 --- a/standalone +++ b/standalone @@ -166,7 +166,7 @@ case $op in job=$1; shift - for d in '' xen kern ; do + for d in '' xen kern libvirt ; do runvar="path_${d}dist" path="build/${d}dist.tar.gz" if [ -f "logs/$flight/$job/$path" ] ; then diff --git a/ts-libvirt-build b/ts-libvirt-build new file mode 100755 index 0000000..2e6552d --- /dev/null +++ b/ts-libvirt-build @@ -0,0 +1,158 @@ +#!/usr/bin/perl +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2014 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +use strict qw(vars); +use DBI; +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho= selecthost($whhost); + +my $leaf= "build.$flight.$job"; +my $homedir = get_host_property($ho, 'homedir', '/home/osstest'); +my $builddir= "$homedir/$leaf"; +my $xendist= "$builddir/xendist"; + +sub libvirtd_init (); + +my $makeflags= get_host_property($ho, 'build make flags', '-j4'); + +sub checkout () { + target_cmd($ho, "rm -rf $builddir && mkdir $builddir && mkdir $xendist", 600); + + my $path = get_stashed("path_dist", $r{"buildjob"}); + my $distcopy= "$builddir/dist.tar.gz"; + target_putfile_root($ho, 300, $path, $distcopy); + target_cmd_root($ho, "tar -C $xendist -hzxf $distcopy", 300); + + build_clone($ho, 'libvirt', $builddir, 'libvirt'); +} + +sub prep() { + target_install_packages($ho, qw(autoconf automake libtool xsltproc + libxml2-utils libxml2-dev libnl-dev + libdevmapper-dev w3c-dtd-xhtml)); +} + +sub config() { + target_cmd_build($ho, 1000, $builddir, <&1 && touch ../build-ok-stamp) |tee ../log + test -f ../build-ok-stamp #/ + echo ok. +END +} + +sub install() { + target_cmd_build($ho, 300, $builddir, < +# based on the skeletons that comes with dh_make +# +### BEGIN INIT INFO +# Provides: libvirt-bin libvirtd +# Required-Start: $network $local_fs $remote_fs $syslog +# Required-Stop: $local_fs $remote_fs $syslog +# Should-Start: avahi-daemon cgconfig +# Should-Stop: avahi-daemon cgconfig +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: libvirt management daemon +### END INIT INFO + +PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin +export PATH +DAEMON=/usr/local/sbin/libvirtd +NAME=libvirtd +DESC="libvirt management daemon" + +test -x $DAEMON || exit 0 +. /lib/lsb/init-functions + +PIDFILE=/var/run/$NAME.pid + +case "$1" in + start) + [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" + start-stop-daemon --start --pidfile $PIDFILE \ + --exec $DAEMON -- -d $libvirtd_opts + [ "$VERBOSE" != no ] && log_end_msg $? + ;; + stop) + [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" + start-stop-daemon --stop --pidfile $PIDFILE \ + --exec $DAEMON + [ "$VERBOSE" != no ] && log_end_msg $? + ;; + restart) + stop + start + ;; + reload|force-reload) + start-stop-daemon --stop --signal 1 --quiet --pidfile \ + /var/run/$NAME.pid --exec $DAEMON + ;; + *) + N=/etc/init.d/libvirtd + echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2 + exit 1 + ;; +esac + +exit 0 +END +} diff --git a/ts-xen-install b/ts-xen-install index 4a810e4..435fc3e 100755 --- a/ts-xen-install +++ b/ts-xen-install @@ -63,7 +63,10 @@ sub packages () { } sub extract () { - foreach my $part ('', 'kern', 'xen') { + my @parts = ('', 'kern', 'xen'); + push @parts, 'libvirt' if $r{toolstack} eq "libvirt"; + + foreach my $part (@parts) { target_extract_jobdistpath($ho, $part, "path_${part}dist", $r{"${part}buildjob"}, \%distpath); }