From patchwork Thu May 30 10:15:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 165430 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp781561ili; Thu, 30 May 2019 03:34:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqwNF8oMPW0HxQfi11juqjfuLc60FQwMSaRgRt3UcUS303zOQQ/KuI1UT5EunU2ZsTh3uMc3 X-Received: by 2002:a0c:8a8d:: with SMTP id 13mr2638037qvv.88.1559212443981; Thu, 30 May 2019 03:34:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559212443; cv=none; d=google.com; s=arc-20160816; b=q91qaWWvnyMpyW9yYZM4ki/wMQ1HahZAhhBUHAWekv+ykpxEvJ0gdPcSx/S7pj60sg jX78n4UMU52ryb8aVWuNx+PQbQLDd9gx3i9KTyFeZTbv+CLnzjUmrTqdjWBo0JUssbQB KDUL0hAWR7qU/d18ifarkvIELvCZ+rtAQ2Rp9xs7MWjwrclDio2PU7ExIFCkatUAzaEX 0sdiXnruI4MFZub+1TgZ1oWcNbiA231IWg0odt0N8TnZfqmyLuy1CryxEZQUKp+TYAdw OeKLb653woovSJZ3Z/dq+zZioS8eHM2wNxd675g1Mbs8eghKR3DM4/1sjLEJwg3vI4UT Ya5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=y2B2y1XQyJxhlxlLqIvvLBXxQCmx4xrmH/BIjlRjWcs=; b=esbD/GUXZZ6AGldq49xm6r4J7bhJExcPRp2UsbNHh9JXwPb0uGPpK7/1Jyy6awiuMV 6Pt1NBSd69UbyZfEPJVgBwufLpGFz+xvwtQg5vRQwXVwAXzDSZWlRdBT+UXrxyVrOcxB aATYLwQ31WfvGFRgP41epBZCL+7lMOVdUmwQYrGTGaliqjxJ2HnuK2wc9xNSMOz0cD91 QgqMxJggBHmSNJj4V1InSz0o0ZQ+FUQXatZt9ih1xc0bBlvM3qw/A7Hr7fOuUm4YCAWx xHuLgrHyudmnewaWLwpQKSpoa/Zz3J1kvJyGHYe/RWfFXmczUjYuHZ44WJnNTM/kcIiV WCnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=f0wR+s4P; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y203si817350qka.30.2019.05.30.03.34.03 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 30 May 2019 03:34:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=f0wR+s4P; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:50893 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hWINn-0000D4-Hh for patch@linaro.org; Thu, 30 May 2019 06:34:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54434) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hWI6f-0002iD-7u for qemu-devel@nongnu.org; Thu, 30 May 2019 06:16:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hWI6e-0006cZ-2r for qemu-devel@nongnu.org; Thu, 30 May 2019 06:16:21 -0400 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]:39187) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hWI6d-0006bp-Qi for qemu-devel@nongnu.org; Thu, 30 May 2019 06:16:20 -0400 Received: by mail-wr1-x444.google.com with SMTP id x4so3823233wrt.6 for ; Thu, 30 May 2019 03:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y2B2y1XQyJxhlxlLqIvvLBXxQCmx4xrmH/BIjlRjWcs=; b=f0wR+s4PFVjNDk4AMOB+0UFj5XJN5P3DQ+suNvlR1vlOZFnL/ZdlFwTdxwMLARF6Tr gpbCJkbLz95o0AgRFTltIfiu1vdPzisBn+wo6PNUqvmmm2IDVlZ6u4plFy3sxSzDDcRz Us7bfyFayZm58+kC7y40nKaa9aZ+wvtZe9cn53GsyUpLVMmcq0Dwa4kwI0SViCPy8fNy PwDRUw/YygeIiMlUo4GX13xiafwhWPlS6LljgFWIZNgVGArq5naOngOSj/5qyyRabV2X HGn3Y/+4XzA91bHjENRe9euUc9hHhVvM44ddQX9Jo8ZWJMkMkJz+CA+Q4UPo2cIB4vhm mA7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y2B2y1XQyJxhlxlLqIvvLBXxQCmx4xrmH/BIjlRjWcs=; b=sVtUe1WaUcxvxrXyujWC57rj/tfdpyM2J4iR0EJUn8c3gM6WYs3YfJ7pfLfhg6AxXo Gg2zU96/j2HBGTbeVdwmXmhz84PHXE8DizuvrjrtRFEmBor4L9JJ4LU5s6fNTYGvhW+a uVjeDKNWGecYDsRiCvQk/xY1UQ7cWwnnE86aDxybE8wQgaCcsrdaffiyyNjybT40KBjW LFNeTR2W5rL3qXVm+WLIXoZlDwtpFyfyI1PqCBRGSkcS73gKtfADMtAZLk6KuTgPNK+I zH5NkGlXaV9SorGQU+l6xAopXAKD3rIzgg95dhvyoG1yADwX3Ofoqn2B6W6kCwr0QF5r bIDw== X-Gm-Message-State: APjAAAXtMJRgb+A51GJUB6R7YLimQdUY3W1qej7nUN5ag+eDA3gXJcZ+ NYtll63b243Wf7+wlVn+4Cz7y49DxHA= X-Received: by 2002:a5d:51cf:: with SMTP id n15mr958810wrv.216.1559211378693; Thu, 30 May 2019 03:16:18 -0700 (PDT) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id o4sm2355995wmo.20.2019.05.30.03.16.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 30 May 2019 03:16:17 -0700 (PDT) Received: from zen.linaroharston. (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 20DA71FF9D; Thu, 30 May 2019 11:16:05 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Thu, 30 May 2019 11:15:50 +0100 Message-Id: <20190530101603.22254-14-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190530101603.22254-1-alex.bennee@linaro.org> References: <20190530101603.22254-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::444 Subject: [Qemu-devel] [PATCH v1 13/26] scripts: use git archive in archive-source X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Gerd Hoffmann Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Gerd Hoffmann Use git archive to create tarballs of qemu and submodules instead of cloning the repository and the submodules. This is a order of magnitude faster because it doesn't fetch the submodules from the internet each time the script runs. Signed-off-by: Gerd Hoffmann Tested-by: Thomas Huth Tested-by: Philippe Mathieu-Daudé Message-Id: <20190520124716.30472-2-kraxel@redhat.com> [AJB: fixed up tabs] Signed-off-by: Alex Bennée --- scripts/archive-source.sh | 72 +++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 41 deletions(-) -- 2.20.1 diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh index 8b899482600..ca94e49978f 100755 --- a/scripts/archive-source.sh +++ b/scripts/archive-source.sh @@ -19,16 +19,25 @@ if test $# -lt 1; then fi tar_file=$(realpath "$1") -list_file="${tar_file}.list" -vroot_dir="${tar_file}.vroot" +sub_tdir=$(mktemp -d "${tar_file%.tar}.sub.XXXXXXXX") +sub_file="${sub_tdir}/submodule.tar" # We want a predictable list of submodules for builds, that is # independent of what the developer currently has initialized # in their checkout, because the build environment is completely # different to the host OS. submodules="dtc slirp ui/keycodemapdb tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3" +sub_deinit="" -trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15 +function cleanup() { + local status=$? + rm -rf "$sub_tdir" + if test "$sub_deinit" != ""; then + git submodule deinit $sub_deinit + fi + exit $status +} +trap "cleanup" 0 1 2 3 15 if git diff-index --quiet HEAD -- &>/dev/null then @@ -36,45 +45,26 @@ then else HEAD=$(git stash create) fi -git clone --shared . "$vroot_dir" -test $? -ne 0 && error "failed to clone into '$vroot_dir'" -for sm in $submodules; do - if test -d "$sm/.git" - then - git clone --shared "$sm" "$vroot_dir/$sm" - test $? -ne 0 && error "failed to clone submodule $sm" - fi -done - -cd "$vroot_dir" -test $? -ne 0 && error "failed to change into '$vroot_dir'" - -git checkout $HEAD -test $? -ne 0 && error "failed to checkout $HEAD revision" +git archive --format tar $HEAD > "$tar_file" +test $? -ne 0 && error "failed to archive qemu" for sm in $submodules; do - git submodule update --init $sm - test $? -ne 0 && error "failed to init submodule $sm" + status="$(git submodule status "$sm")" + smhash="${status#[ +-]}" + smhash="${smhash%% *}" + case "$status" in + -*) + sub_deinit="$sub_deinit $sm" + git submodule update --init "$sm" + test $? -ne 0 && error "failed to update submodule $sm" + ;; + +*) + echo "WARNING: submodule $sm is out of sync" + ;; + esac + (cd $sm; git archive --format tar --prefix "$sm/" $smhash) > "$sub_file" + test $? -ne 0 && error "failed to archive submodule $sm ($smhash)" + tar --concatenate --file "$tar_file" "$sub_file" + test $? -ne 0 && error "failed append submodule $sm to $tar_file" done - -if test -n "$submodules"; then - { - git ls-files || error "git ls-files failed" - for sm in $submodules; do - (cd $sm; git ls-files) | sed "s:^:$sm/:" - if test "${PIPESTATUS[*]}" != "0 0"; then - error "git ls-files in submodule $sm failed" - fi - done - } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file" -else - git ls-files > "$list_file" -fi - -if test $? -ne 0; then - error "failed to generate list file" -fi - -tar -cf "$tar_file" -T "$list_file" || error "failed to create tar file" - exit 0