From patchwork Mon Apr 28 10:41:30 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 29198 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oa0-f72.google.com (mail-oa0-f72.google.com [209.85.219.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 96BE3202FE for ; Mon, 28 Apr 2014 10:42:40 +0000 (UTC) Received: by mail-oa0-f72.google.com with SMTP id eb12sf39998704oac.7 for ; Mon, 28 Apr 2014 03:42:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=bDyzkgXs3TPIVVHLVoRCFUOVF8Bscp8W036vjAAzotc=; b=TxBu1xyXln9/vdv/CWDnkvMueJ7bx5ib4g/OInzP8isythJ3Brb/vxU4tmgHiv2fgS za91Qbl/iCBJij9odbpei6PPL3jVoJzmnFR3ZEv21bFIAAZwgCDTBIzNeT+3ichxKp8Y I+91Dqn3p3ONCjToaJ4eC1NhP7EvQrfsBR0ga05fk2nrjZejyFvfma/zzMuknh9ibr7t 2oxUPIJNFR7Xk+pCZkjZ5r2YNGczAqzVueCMnyN9kArfEeGs2N0W8lGtvJPqqObJ9ESe imrzaKfcg8GYUWdYOga6YQCiqJEX/qbcf6FtZHsNjuUV+Ciy2CI+rsZO9AS4MiAHuONS ig1w== X-Gm-Message-State: ALoCoQnwb2bJrOChaqkI215Ue2bxgtYjKlSP95STt8yUguFSrzsPGFDDk8L3IhaCCFNAWLkKuxR5 X-Received: by 10.182.28.36 with SMTP id y4mr3080627obg.46.1398681760065; Mon, 28 Apr 2014 03:42:40 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.95.84 with SMTP id h78ls2541340qge.54.gmail; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) X-Received: by 10.58.220.161 with SMTP id px1mr23767313vec.13.1398681759917; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) Received: from mail-ve0-f172.google.com (mail-ve0-f172.google.com [209.85.128.172]) by mx.google.com with ESMTPS id dm2si3524109vec.111.2014.04.28.03.42.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 28 Apr 2014 03:42:39 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.128.172; Received: by mail-ve0-f172.google.com with SMTP id jx11so7757139veb.17 for ; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) X-Received: by 10.52.249.105 with SMTP id yt9mr967783vdc.34.1398681759804; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp107038vcb; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) X-Received: by 10.140.29.226 with SMTP id b89mr30478086qgb.48.1398681759410; Mon, 28 Apr 2014 03:42:39 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id c4si7535709qad.141.2014.04.28.03.42.39 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 28 Apr 2014 03:42:39 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Wej0R-0007wy-Il; Mon, 28 Apr 2014 10:41:51 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Wej0P-0007tb-64 for xen-devel@lists.xen.org; Mon, 28 Apr 2014 10:41:49 +0000 Received: from [85.158.139.211:38538] by server-10.bemta-5.messagelabs.com id DC/2C-27081-C603E535; Mon, 28 Apr 2014 10:41:48 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-15.tower-206.messagelabs.com!1398681700!979240!4 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 24308 invoked from network); 28 Apr 2014 10:41:47 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-15.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 28 Apr 2014 10:41:47 -0000 X-IronPort-AV: E=Sophos;i="4.97,943,1389744000"; d="scan'208";a="125320486" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 28 Apr 2014 10:41:46 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Mon, 28 Apr 2014 06:41:46 -0400 Received: from kazak.uk.xensource.com ([10.80.2.80] helo=zakaz.uk.xensource.com) by ukmail1.uk.xensource.com with smtp (Exim 4.69) (envelope-from ) id 1Wej0K-0007Zq-NL; Mon, 28 Apr 2014 11:41:45 +0100 Received: by zakaz.uk.xensource.com (sSMTP sendmail emulation); Mon, 28 Apr 2014 11:41:44 +0100 From: Ian Campbell To: Date: Mon, 28 Apr 2014 11:41:30 +0100 Message-ID: <1398681696-2773-9-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.9.0 In-Reply-To: <1398681672.29700.55.camel@kazak.uk.xensource.com> References: <1398681672.29700.55.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: Ian Campbell , xen-devel@lists.xen.org Subject: [Xen-devel] [PATCH OSSTEST v2 09/15] distros: add support for installing Debian PV guests via d-i, flight and jobs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: This patch introduces ts-debian-di-install which can install Debian from a netboot (PXE) debian installer image. By default it installs from the d-i image used by osstest (using the special Xen PV guest enabled flavour where necessary) but it can also install the current release versions from Squeeze onwards (up to and including Sid) and the d-i daily builds. This is controlled by runvars {Guest}_diver = osstest|current and {Guest}_dist = squeeze|...|sid. The resulting guests boot the distro kernel using pygrub (pvgrub will follow). The distros flights differ substantially from the existing flights. Introduce make-distros-flight using the functionality previously refactored into mfi-common. The new flight tests all versions of Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf from Jessie onwards only) using the usual smoke tests. Add the new cases to sg-run-job Signed-off-by: Ian Campbell --- make-distros-flight | 101 +++++++++++++++++++++++++++++++ sg-run-job | 11 ++++ ts-debian-di-install | 167 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 279 insertions(+) create mode 100755 make-distros-flight create mode 100755 ts-debian-di-install diff --git a/make-distros-flight b/make-distros-flight new file mode 100755 index 0000000..abc24ff --- /dev/null +++ b/make-distros-flight @@ -0,0 +1,101 @@ +#!/bin/bash + +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2013 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + + +set -e + +branch=$1 +xenbranch=$2 +blessing=$3 +buildflight=$4 + +flight=`./cs-flight-create $blessing $branch` + +. ap-common +. cri-common +. mfi-common + +defsuite=`getconfig DebianSuite` +defguestsuite=`getconfig GuestDebianSuite` + +if [ x$buildflight = x ]; then + + if [ "x$BUILD_LVEXTEND_MAX" != x ]; then + BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX " + fi + + WANT_XEND=false REVISION_LINUX_OLD=disable + + create_build_jobs + +else + + bfi=$buildflight. + +fi + +job_create_test_filter_callback () { + if [ "$xenarch" = "i386" ]; then return 1; fi + return 0 +} + +test_matrix_branch_filter_callback () { + : +} + +test_do_one_netboot () { + job_create_test test-$xenarch$kern-$dom0arch-$domU-$dist-netboot \ + test-debian-di xl $xenarch $dom0arch \ + kernbuildjob=${bfi}build-$dom0arch-$kernbuild \ + debian_arch=$domU \ + debian_dist=$dist \ + debian_method=netboot \ + debian_diver=current \ + all_hostflags=$most_hostflags +} + +test_matrix_do_one () { + case ${xenarch} in + amd64) domUarches="amd64 i386";; + armhf) domUarches="armhf";; + esac + + for domU in $domUarches ; do + for dist in squeeze wheezy jessie sid daily ; do + case ${domU}_${dist} in + armhf_squeeze) continue;; # No armhf in Squeeze + armhf_wheezy) continue;; # No armhf guest support in Wheezy + *) ;; + esac + + test_do_one_netboot + + done + + done +} + +test_matrix_iterate + +echo $flight + +# Local variables: +# mode: sh +# sh-basic-offset: 2 +# indent-tabs-mode: nil +# End: diff --git a/sg-run-job b/sg-run-job index 3d44331..789315a 100755 --- a/sg-run-job +++ b/sg-run-job @@ -246,6 +246,17 @@ proc run-job/test-debian {} { test-guest debian } +proc install-guest-debian-di {} { + run-ts . = ts-debian-di-install + run-ts . = ts-guest-start + debian +} + +proc need-hosts/test-debian-di {} { return host } +proc run-job/test-debian-di {} { + install-guest-debian-di + test-guest debian +} + proc need-hosts/test-freebsd {} { return host } proc run-job/test-freebsd {} { run-ts . = ts-freebsd-install diff --git a/ts-debian-di-install b/ts-debian-di-install new file mode 100755 index 0000000..a71a336 --- /dev/null +++ b/ts-debian-di-install @@ -0,0 +1,167 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2013 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +use strict qw(vars); +use DBI; +use Osstest; +use Osstest::Debian; +use Osstest::TestSupport; + +tsreadconfig(); + +our ($whhost,$gn) = @ARGV; +$whhost ||= 'host'; +$gn ||= 'debian'; + +our $ho= selecthost($whhost); + +our $ram_mb= 512; +our $disk_mb= 10000; + +our $guesthost= "$gn.guest.osstest"; +our $gho; + +sub prep () { + target_install_packages_norec($ho, qw(lvm2)); + + $gho= prepareguest($ho, $gn, $guesthost, 22, + $disk_mb, 40); + + prepareguest_part_lvmdisk($ho, $gho, $disk_mb); + + target_cmd_root($ho, "umount $gho->{Lvdev} ||:"); +} + +sub setup_netboot($$$) +{ + my ($didir, $arch, $suite) = @_; + + my $di_ver= $r{"$gho->{Guest}_diver"} || "osstest"; + + if ( $di_ver eq "osstest" ) { + my $di_path = $c{TftpPath}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.$c{TftpDiVersion}.'-'.$ho->{Suite}; + + my $netboot_kernel = "$di_path/vmlinuz-xen"; + my $netboot_initrd = "$di_path/initrd.gz-xen"; + + target_putfile_root($ho, 60, $netboot_kernel, "$didir/kernel_${suite}_${arch}"); + target_putfile_root($ho, 60, $netboot_initrd, "$didir/initrd_${suite}_${arch}"); + + store_runvar("$gho->{Guest}_netboot_kernel", $netboot_kernel); + store_runvar("$gho->{Guest}_netboot_initrd", $netboot_initrd); + } else { + my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}"; + + my $di_url = $suite eq "daily" ? + "http://d-i.debian.org/daily-images/$arch/daily/netboot/xen" : + "$mirror/dists/$suite/main/installer-$arch/$di_ver/images/netboot/xen"; + + my $netboot_kernel = "$di_url/vmlinuz"; + my $netboot_initrd = "$di_url/initrd.gz"; + + target_fetchurl($ho, $netboot_kernel, "$didir/kernel_${suite}_${arch}"); + target_fetchurl($ho, $netboot_initrd, "$didir/initrd_${suite}_${arch}"); + + store_runvar("$gho->{Guest}_netboot_kernel", $netboot_kernel); + store_runvar("$gho->{Guest}_netboot_initrd", $netboot_initrd); + } + + return <{Guest}_arch"}; + my $method= $r{"$gho->{Guest}_method"}; + + target_cmd_root($ho, <{Guest}_dist"}; + logm("$method $suite/$arch"); + + $method_cfg = setup_netboot("/root/di", $arch, $suite); + + $suite = "sid" if $suite eq "daily"; + + $ps_url = preseed_create_guest($gho, '', Suite=>$suite); + + $extra_disk = ""; + } + else + { + die "$method"; + } + + my $cmdline = join(" ", ( + "debian-installer/exit/always_halt=true", + "--", + "console=hvc0", + "auto-install/enable=true", + "hostname=$gho->{Name}", + "domain=$c{TestHostDomain}", + "url=$ps_url", + "DEBIAN_FRONTEND=text", + "netcfg/dhcp_timeout=150", + "netcfg/choose_interface=eth0", + )); + + my %install_xopts = ( + OnPowerOff => "preserve" + ); + + prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <{Lvdev},xvda,w' + ] +END + + my $cmd= toolstack()->{Command}." create ". + $r{ $gho->{Guest}.'_'. toolstack()->{CfgPathVar} }; + target_cmd_root($ho, $cmd, 300); + + guest_checkrunning($ho, $gho) or die "$gho->{Name} not running"; + + guest_await_shutdown($ho,$gho,2000); + guest_destroy($ho,$gho); + + my $blcfg = <{Lvdev},xvda,w' + ] +END + return; +} + +prep(); +ginstall();