From patchwork Thu May 26 14:27:42 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 68680 Delivered-To: patch@linaro.org Received: by 10.140.92.199 with SMTP id b65csp436930qge; Thu, 26 May 2016 07:33:02 -0700 (PDT) X-Received: by 10.140.97.7 with SMTP id l7mr8977340qge.71.1464273182541; Thu, 26 May 2016 07:33:02 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id j23si13184674qhc.70.2016.05.26.07.33.02 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 26 May 2016 07:33:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38675 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wLO-0004ed-2c for patch@linaro.org; Thu, 26 May 2016 10:33:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45223) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wG5-0001q7-II for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5wG3-0006nl-2k for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:32 -0400 Received: from mail-wm0-x229.google.com ([2a00:1450:400c:c09::229]:36454) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wG2-0006nf-OY for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:31 -0400 Received: by mail-wm0-x229.google.com with SMTP id n129so229942915wmn.1 for ; Thu, 26 May 2016 07:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ReFRkTS6KFD4U3w7zTlypGG+jwGUNShFUPiWOC+7/Gc=; b=eV5M9aWRQfywaKH/H+/mIah7ZLggkvE/a2ZI/4KBj8NEwEwu4+GGl4BL07JSMno2mq wA55OEkwwhCWdXYrJIoMW+cODEa9YrVB9t1uAjWaZaMYDUJttdLBqMy8G5WmORTnzf6T +utjHYtNoI7Kd3cs+yXiU90FCTr2nJKXDIsdk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ReFRkTS6KFD4U3w7zTlypGG+jwGUNShFUPiWOC+7/Gc=; b=MEJVVcg+zI3N2WAtaNZvnwWIMKAYqM85b5Uxs4goWiKqYDKTaFn4VImHZg02r8sywV K+5bQeDZgAu8mCEHBIh+U6KIbxfzZ9ySYUBXZNX7rxg4C24Dup5bvuDDi15XkNl3/Cbq i5O3zKngWyJ2437BbmNtg94vJ7Hz+vtp+WHxhNqUXWFpi7Y6EfyZ2wioq7dbtz6weahG WLz1Il0oXJXFCanGUJQt6KO0gkn1/3r8SDGdsNi4FKNXPP5DnDcie3uc2TWCZbo2ALQB H33btDZHloFJUIBkMVI9Y+hl1Qo8GUe3tvccBO6GHa2b5kc7gZEfiXsN5PFP+rgHAdSP iXaA== X-Gm-Message-State: ALyK8tIbaq471bz8Mkf1SE5WRb/FX2cAX93bRUi7Bicpp0VuXjOo6O7wWznCF7PLowmQS9wF X-Received: by 10.28.4.79 with SMTP id 76mr4121962wme.41.1464272850031; Thu, 26 May 2016 07:27:30 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g129sm3898709wme.1.2016.05.26.07.27.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 May 2016 07:27:29 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id E28513E0275; Thu, 26 May 2016 15:28:03 +0100 (BST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Thu, 26 May 2016 15:27:42 +0100 Message-Id: <1464272863-2285-2-git-send-email-alex.bennee@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1464272863-2285-1-git-send-email-alex.bennee@linaro.org> References: <1464272863-2285-1-git-send-email-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::229 Subject: [Qemu-devel] [RFC PATCH 1/2] tests/docker/docker.py: support --qemu option X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, famz@redhat.com, riku.voipio@linaro.org, steve.mcintyre@linaro.org, neil.williams@linaro.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" When passed the name of a qemu-$arch binary we copy it and any linked libraries into the docker build context. These can then be included by a dockerfile with the line: # Copy all of context into container ADD . / Signed-off-by: Alex Bennée --- tests/docker/docker.py | 58 ++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 52 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/tests/docker/docker.py b/tests/docker/docker.py index fe73de7..e9242f3 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -20,6 +20,8 @@ import atexit import uuid import argparse import tempfile +import re +from shutil import copyfile def _text_checksum(text): """Calculate a digest string unique to the text content""" @@ -37,6 +39,27 @@ def _guess_docker_command(): raise Exception("Cannot find working docker command. Tried:\n%s" % \ commands_txt) +def _find_user_binary(binary_name): + """ Find a binary in the QEMU source tree. Used for finding qemu-$arch.""" + top = os.path.abspath("%s/../../.." % sys.argv[0]) + linux_user = [ x for x in os.listdir(top) if x.endswith("-linux-user") ] + for x in linux_user: + check_path = "%s/%s/%s" % (top, x, binary_name) + if os.path.isfile(check_path): + print ("found %s" % check_path) + return check_path + return None + +def _copy_with_mkdir(src, root_dir, sub_path): + """Copy src into root_dir, creating sub_path as needed.""" + full_path = "%s/%s" % (root_dir, sub_path) + try: + os.makedirs(full_path) + except OSError: + print "skipping %s" % (full_path) + + copyfile(src, "%s/%s" % (full_path, os.path.basename(src))) + class Docker(object): """ Running Docker commands """ def __init__(self): @@ -86,18 +109,36 @@ class Docker(object): labels = json.loads(resp)[0]["Config"].get("Labels", {}) return labels.get("com.qemu.dockerfile-checksum", "") - def build_image(self, tag, dockerfile, df_path, quiet=True, argv=None): + def build_image(self, tag, dockerfile, quiet=True, qemu=None, argv=None): if argv == None: argv = [] + + # Create a temporary docker context to build in + tmp_dir = tempfile.mkdtemp(prefix="docker_build") + + # Copy the dockerfile into our work space tmp = dockerfile + "\n" + \ "LABEL com.qemu.dockerfile-checksum=%s" % \ _text_checksum(dockerfile) - dirname = os.path.dirname(df_path) - tmp_df = tempfile.NamedTemporaryFile(dir=dirname) + tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker") tmp_df.write(tmp) tmp_df.flush() + + # Do we want to copy QEMU into here? + if qemu: + _copy_with_mkdir(qemu, tmp_dir, "/usr/bin") + # do ldd bit here + ldd_output = subprocess.check_output(["ldd", qemu]) + for l in ldd_output.split("\n"): + s = re.search("(/.*/)(\S*)", l) + if s and len(s.groups())==2: + so_path=s.groups()[0] + so_lib=s.groups()[1] + _copy_with_mkdir("%s/%s" % (so_path, so_lib), + tmp_dir, so_path) + self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \ - [dirname], + [tmp_dir], quiet=quiet) def image_matches_dockerfile(self, tag, dockerfile): @@ -148,6 +189,7 @@ class BuildCommand(SubCommand): """ Build docker image out of a dockerfile. Arguments: """ name = "build" def args(self, parser): + parser.add_argument("--qemu", help="Include qemu-user binaries") parser.add_argument("tag", help="Image Tag") parser.add_argument("dockerfile", @@ -157,14 +199,18 @@ class BuildCommand(SubCommand): dockerfile = open(args.dockerfile, "rb").read() tag = args.tag + # find qemu binary + qbin=None + if args.qemu: + qbin=_find_user_binary(args.qemu) + dkr = Docker() if dkr.image_matches_dockerfile(tag, dockerfile): if not args.quiet: print "Image is up to date." return 0 - dkr.build_image(tag, dockerfile, args.dockerfile, - quiet=args.quiet, argv=argv) + dkr.build_image(tag, dockerfile, quiet=args.quiet, qemu=qbin, argv=argv) return 0 class CleanCommand(SubCommand):