From patchwork Fri Jun 7 09:05:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 166142 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp475028ili; Fri, 7 Jun 2019 03:52:30 -0700 (PDT) X-Google-Smtp-Source: APXvYqzXNorRa9+2u9ON8s1rtz3EpMJXevCBt7MKRXc8oEA7tyBpVSZiq57KFmraoJxyQpv6f2Hr X-Received: by 2002:a05:620a:12f8:: with SMTP id f24mr23543587qkl.202.1559904750053; Fri, 07 Jun 2019 03:52:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559904750; cv=none; d=google.com; s=arc-20160816; b=MHy015E6qQiF2tSQ1DKOXLwc3KkGeru8A+J1lf6h5zh9/zU+/gYV8Br2q9emzGB2Rl t+229r93vOA9YZ8ft6gjqUAx8v5ivGdVznQAod3UC3ANjlsb1mtzPke7drLiuN1j966K 0UvfaNcsg9BFpcE8OgrFKnnJM4OMyHolVkumOXD4cgrtZiJzj/UOJbTaWTRgAZ8OZVFG RF+wuywRHv/0cSv6l1DWLWau4klMY4XaKpogz5qNyFZncIVgQInSb2J/5j/GoNG78JAn yKH5Taygu7BT35JbWvuxsYAgmo0EljsH0unugle6w27cHBT6Zr8MwwkMovKCohQdKPvF f4XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=oJrbPHU6QMM17Z/P//VMEjEn9UEe0UK4q9allOg3/go=; b=drUObn2SxQSKskpyOR/reADEjr54NIwUJG2PQUbjwZQy9euaOq4e1MjgYpcXvjK1Uu GH/n09i6+lXLzRnGvgNTR/rdHdFwfnjOH+yVvRuH1MdJnKdevlIAYZPqhbkPtfaANNZA QZf1MV9a5OaNjtixJEUb6ECUm3OCBMviZ8L8v6WFT8zIgFLOHcmtGOwz7F6OzgSQ4Lz5 fwURHCFZ2yho7LoIKi0oXQv2hDKW25CZOJOqy33vOQjZ4wNwhgYghePK1qn70VKpE8Y5 VtgUetlOyq0JS2b49EPSqWY3bl3/BOHh0RXxzUO4VIw/P0LJxRmM2ZVmShsWQjsaGrPA WDrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=aEuG+GzN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.47 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.47]) by mx.google.com with ESMTPS id g17si1001272qto.189.2019.06.07.03.52.29 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 07 Jun 2019 03:52:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.47 as permitted sender) client-ip=209.51.188.47; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=aEuG+GzN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.47 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48272 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hZCU1-0006To-KV for patch@linaro.org; Fri, 07 Jun 2019 06:52:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50991) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hZBOM-0007e4-Ai for qemu-devel@nongnu.org; Fri, 07 Jun 2019 05:42:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hZBOK-0000pO-G7 for qemu-devel@nongnu.org; Fri, 07 Jun 2019 05:42:34 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:36569) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hZBOK-0000lE-8j for qemu-devel@nongnu.org; Fri, 07 Jun 2019 05:42:32 -0400 Received: by mail-wr1-x42c.google.com with SMTP id n4so1476650wrs.3 for ; Fri, 07 Jun 2019 02:42:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oJrbPHU6QMM17Z/P//VMEjEn9UEe0UK4q9allOg3/go=; b=aEuG+GzNMS4QTFB6KF60Z6oNGIGC5YokL7OlScAPDZ1vFIpphysthLUMfyt3CUp9ha 72G3QpoEfhmjpxwxyvGU8zkZNOpBscbf4yI5vAHBXE8CgKPhJlGAh0e1MNV3nNYvVhR9 jfNMmTwf3QRKLyDWaSWWEbSjFIEF6zmKSZq8nkikJLCwURHbiPTHX+8JYfvFL8thLwE3 MmyAwbcLwWl8iRQEt1t57BC5eB/eKgkCSAoSDqU+EKHcqf5I2p1H7ZYmMyFhLIcXQbM1 jn7HNakEdNSu2RMABuJxvR+ydO00izMXZ5r1613ALvYMacptQbXX4H9GbmpS/lFOAxxc qiTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oJrbPHU6QMM17Z/P//VMEjEn9UEe0UK4q9allOg3/go=; b=iC15ZwkUY6kwL5WY2RmmfsAVcM6x47v7y02lAIHZEx8CTLevPIMwPdqp6/ZK3mOz/l 4Wc7xcaSfr7RIGuVQVbjTdvmT6Qv3ZmbTdu395TrjdmZt+W8N03Uanzel099WO1CrDfd MxMkIAT2n76MDQbt/TyCxFEVbkd/Vzj7Hg8676b1vFmOMicCUBP3K+zPaSyYa9a/z/e5 4xGP8AAUr9vpP3Nfw7guGzPUlULSPHIz8fKMiHoP2elUDspPnmLYH1MKccMMrvZRdXUg eJTcKv54ekIbhF/TY9j+1kX0zhwsFHbj0N5vEAed5fleoI0Ty19ycr+XqIIiy8ZTfscC QovQ== X-Gm-Message-State: APjAAAUK/xj98VNQYyptqKw7V9uF0NZmv9J5PI9cDwUGGluDNMxunjlE 4PbTjySXuD2+Cd2sUKt25rzzvA== X-Received: by 2002:a5d:6583:: with SMTP id q3mr35571326wru.184.1559900549537; Fri, 07 Jun 2019 02:42:29 -0700 (PDT) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id k184sm2078868wmk.0.2019.06.07.02.42.28 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 07 Jun 2019 02:42:28 -0700 (PDT) Received: from zen.linaroharston. (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id EEA631FF9C; Fri, 7 Jun 2019 10:05:52 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Date: Fri, 7 Jun 2019 10:05:11 +0100 Message-Id: <20190607090552.12434-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190607090552.12434-1-alex.bennee@linaro.org> References: <20190607090552.12434-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::42c Subject: [Qemu-devel] [PULL 12/52] scripts: use git archive in archive-source X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Gerd Hoffmann , =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Gerd Hoffmann Use git archive to create tarballs of qemu and submodules instead of cloning the repository and the submodules. This is a order of magnitude faster because it doesn't fetch the submodules from the internet each time the script runs. Signed-off-by: Gerd Hoffmann Tested-by: Thomas Huth Tested-by: Philippe Mathieu-Daudé Message-Id: <20190520124716.30472-2-kraxel@redhat.com> [AJB: fixed up tabs] Signed-off-by: Alex Bennée -- 2.20.1 diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh index 8b89948260..ca94e49978 100755 --- a/scripts/archive-source.sh +++ b/scripts/archive-source.sh @@ -19,16 +19,25 @@ if test $# -lt 1; then fi tar_file=$(realpath "$1") -list_file="${tar_file}.list" -vroot_dir="${tar_file}.vroot" +sub_tdir=$(mktemp -d "${tar_file%.tar}.sub.XXXXXXXX") +sub_file="${sub_tdir}/submodule.tar" # We want a predictable list of submodules for builds, that is # independent of what the developer currently has initialized # in their checkout, because the build environment is completely # different to the host OS. submodules="dtc slirp ui/keycodemapdb tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3" +sub_deinit="" -trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15 +function cleanup() { + local status=$? + rm -rf "$sub_tdir" + if test "$sub_deinit" != ""; then + git submodule deinit $sub_deinit + fi + exit $status +} +trap "cleanup" 0 1 2 3 15 if git diff-index --quiet HEAD -- &>/dev/null then @@ -36,45 +45,26 @@ then else HEAD=$(git stash create) fi -git clone --shared . "$vroot_dir" -test $? -ne 0 && error "failed to clone into '$vroot_dir'" -for sm in $submodules; do - if test -d "$sm/.git" - then - git clone --shared "$sm" "$vroot_dir/$sm" - test $? -ne 0 && error "failed to clone submodule $sm" - fi -done - -cd "$vroot_dir" -test $? -ne 0 && error "failed to change into '$vroot_dir'" - -git checkout $HEAD -test $? -ne 0 && error "failed to checkout $HEAD revision" +git archive --format tar $HEAD > "$tar_file" +test $? -ne 0 && error "failed to archive qemu" for sm in $submodules; do - git submodule update --init $sm - test $? -ne 0 && error "failed to init submodule $sm" + status="$(git submodule status "$sm")" + smhash="${status#[ +-]}" + smhash="${smhash%% *}" + case "$status" in + -*) + sub_deinit="$sub_deinit $sm" + git submodule update --init "$sm" + test $? -ne 0 && error "failed to update submodule $sm" + ;; + +*) + echo "WARNING: submodule $sm is out of sync" + ;; + esac + (cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_file" + test $? -ne 0 && error "failed to archive submodule $sm ($smhash)" + tar --concatenate --file "$tar_file" "$sub_file" + test $? -ne 0 && error "failed append submodule $sm to $tar_file" done - -if test -n "$submodules"; then - { - git ls-files || error "git ls-files failed" - for sm in $submodules; do - (cd $sm; git ls-files) | sed "s:^:$sm/:" - if test "${PIPESTATUS[*]}" != "0 0"; then - error "git ls-files in submodule $sm failed" - fi - done - } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file" -else - git ls-files > "$list_file" -fi - -if test $? -ne 0; then - error "failed to generate list file" -fi - -tar -cf "$tar_file" -T "$list_file" || error "failed to create tar file" - exit 0