From patchwork Fri Jan 25 14:00:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 156576 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp391620jaa; Fri, 25 Jan 2019 06:01:01 -0800 (PST) X-Google-Smtp-Source: ALg8bN4dqXJs5+ZGB/VpnS0uuFiJgRhmZC3l2biu3YIelumO6HeGsJdtEZYGM0JuceAWGHWumc+4 X-Received: by 2002:adf:ae1a:: with SMTP id x26mr11329790wrc.0.1548424861880; Fri, 25 Jan 2019 06:01:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548424861; cv=none; d=google.com; s=arc-20160816; b=RDWFZ7mfikkg5fexoxr8v01fNUld9KJjNymYOPFjB6045X2yxs6GHDJz04/xkcvYt7 6ntf20gfsCi9xtrqcM/diLncHsL1dvPLNV6ElCWEm0NGuMyz9CS1jo72GR0v5AMNvvFx NLhLxfLrN8F5y+zWu4Gp0V8Ty9A/zAc3LtfhyubaymXAJ9tpubj5hdSSvX7jrFx39ba8 UmkD6PKpOunegGneS9txWlFujH4grS4/lxB7RTv/qdF2zVycKJVpz7kRQPiV0gXPD9yY vr3bNMDpEOelEzOMp+VU2FlBLFtxI8jFdLPbE5JgGwPOSQHT+mR/PS+8TZPRrzzpu2Il nrLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=kcHZidWMgjIgAiQ9s7xaZIRe+WZ1Zp2g7COR29S7kIU=; b=WfV8OM9e/E9uvFJJpaI3XwEqXhxu+qwuhs6C2wC8pf9fWtdarQoo/N0O1cHf0Fd+LS Oa+PhW7D/GPii54y3RPn5U1yz/cdI3goMo/qzfkeccgaQniGngWBv4r7+q0tpXcn4vGz kYFf3tHQj6zb5yoLWF23aG0AaOUHq7D+myEi6BjXTmAcVS0kiyKSmCVHFEm+9hotHOC7 UYQqG7PqDij8QW04omj7A7Vf6fz07J7vs9HUU+pmw6O46LV80cfXatnEuNn2UZe7CxCG B2G7a7GbLQBBEFJ9Qw5ewKZp88/NvU6ixEc7aW+oX3CuqJTjaHBBmhB1p3PLXO9aePVb Ziuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XFCsGOVI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b13si73911017wrw.102.2019.01.25.06.01.01 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 25 Jan 2019 06:01:01 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XFCsGOVI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:45155 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gn22W-0003Yq-NX for patch@linaro.org; Fri, 25 Jan 2019 09:01:00 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52117) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gn224-0003VP-70 for qemu-devel@nongnu.org; Fri, 25 Jan 2019 09:00:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gn222-0000hM-MY for qemu-devel@nongnu.org; Fri, 25 Jan 2019 09:00:31 -0500 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]:43173) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gn221-0000e5-SF for qemu-devel@nongnu.org; Fri, 25 Jan 2019 09:00:30 -0500 Received: by mail-wr1-x435.google.com with SMTP id r10so10385210wrs.10 for ; Fri, 25 Jan 2019 06:00:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kcHZidWMgjIgAiQ9s7xaZIRe+WZ1Zp2g7COR29S7kIU=; b=XFCsGOVISSKrTT0Px6aEMoT3hQCKccKFNg2iR93V2F8p2aYKz6tspuo+e0A86VEWum ygz+8wihVtrTgs06eSB3Y6taLLWX7dtHFsE4BlgtZR3VoACgIi0abIBvhNkwuiRP0cvl 47FFEekQHCzGOx9JnMEW/GCegSbbfr7OB0rDI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kcHZidWMgjIgAiQ9s7xaZIRe+WZ1Zp2g7COR29S7kIU=; b=uCkReYQENn9gYcFUK9N9RD0WP445/qyCkak+li0/Va9xvotgp/dUxoXr9k5Xjo4Ywh iUhUfTgsKbyl9xt5zSRI1Q593TpuHDYttlxYJqfjFo3KVF54TtKsvOhSTKtghxm/WFK7 YNtcx222L5OON8dw+0w1m6+8ahkNhUTM3+7j123by+kdpYkhABonjsZHeVAJCfeK/1hl /hc9EhTSJxKuC5MOc2BbnXplJzvC03dMVX96XfktLrgsr+2NX3Q6nxYqFlw6TQRzJbnH B9Tda6D8S5ZOLiLx7lJu9mCvwvumHKXlXS623yJKzWkLvkzF7/PyeBCNIwY/F4SKQtTx PaQw== X-Gm-Message-State: AJcUukfcb2QA2KkwINMuKtrqCbOpdNs0oNe2Y9xU4C2KspWd1Sg4r3Nb wYnwgwSqdHfKfrSkgXclNwal1w== X-Received: by 2002:adf:db51:: with SMTP id f17mr11027589wrj.90.1548424826141; Fri, 25 Jan 2019 06:00:26 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id s1sm153704707wro.9.2019.01.25.06.00.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 25 Jan 2019 06:00:23 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 41C813E06F7; Fri, 25 Jan 2019 14:00:18 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Fri, 25 Jan 2019 14:00:11 +0000 Message-Id: <20190125140017.6092-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190125140017.6092-1-alex.bennee@linaro.org> References: <20190125140017.6092-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::435 Subject: [Qemu-devel] [PATCH v1 08/14] tests: docker.py be even smarter with persistent binfmt_misc X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" If we have a persistent mapping we don't need the QEMU binary copied into the container as the kernel has already opened the file and will pass the fd in. However the support libraries will still need to be there. Signed-off-by: Alex Bennée --- tests/docker/docker.py | 56 ++++++++++++++++++++++++++++-------------- 1 file changed, 38 insertions(+), 18 deletions(-) -- 2.17.1 diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 768728785f..a74338cb61 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -96,13 +96,22 @@ def _get_so_libs(executable): return libs -def _copy_binary_with_libs(src, dest_dir): - """Copy a binary executable and all its dependent libraries. +def _copy_binary_with_libs(src, bin_dest, dest_dir): + """Maybe copy a binary and all its dependent libraries. + + If bin_dest isn't set we only copy the support libraries because + we don't need qemu in the docker path to run (due to persistent + mapping). Indeed users may get confused if we aren't running what + is in the image. This does rely on the host file-system being fairly multi-arch - aware so the file don't clash with the guests layout.""" + aware so the file don't clash with the guests layout. + """ - _copy_with_mkdir(src, dest_dir, "/usr/bin") + if bin_dest: + _copy_with_mkdir(src, dest_dir, os.path.dirname(bin_dest)) + else: + print("only copying support libraries for %s" % (src)) libs = _get_so_libs(src) if libs: @@ -116,21 +125,26 @@ def _check_binfmt_misc(executable): The details of setting up binfmt_misc are outside the scope of this script but we should at least fail early with a useful - message if it won't work.""" + message if it won't work. + + Returns the configured binfmt path and a valid flag. For + persistent configurations we will still want to copy and dependent + libraries. + """ binary = os.path.basename(executable) binfmt_entry = "/proc/sys/fs/binfmt_misc/%s" % (binary) if not os.path.exists(binfmt_entry): print ("No binfmt_misc entry for %s" % (binary)) - return None + return None, False with open(binfmt_entry) as x: entry = x.read() if re.search("flags:.*F.*\n", entry): print("binfmt_misc for %s uses persistent(F) mapping to host binary\n" % (binary)) - return None + return None, True m = re.search("interpreter (\S+)\n", entry) interp = m.group(1) @@ -138,7 +152,8 @@ def _check_binfmt_misc(executable): print("binfmt_misc for %s does not point to %s, using %s" % (binary, executable, interp)) - return interp + return interp, True + def _read_qemu_dockerfile(img_name): # special case for Debian linux-user images @@ -345,7 +360,8 @@ class BuildCommand(SubCommand): # Validate binfmt_misc will work if args.include_executable: - if not _check_binfmt_misc(args.include_executable): + qpath, enabled = _check_binfmt_misc(args.include_executable) + if not enabled: return 1 # Is there a .pre file to run in the build context? @@ -368,7 +384,9 @@ class BuildCommand(SubCommand): # FIXME: there is no checksum of this executable and the linked # libraries, once the image built any change of this executable # or any library won't trigger another build. - _copy_binary_with_libs(args.include_executable, docker_dir) + _copy_binary_with_libs(args.include_executable, + qpath, docker_dir) + for filename in args.extra_files or []: _copy_with_mkdir(filename, docker_dir) cksum += [(filename, _file_checksum(filename))] @@ -400,14 +418,16 @@ class UpdateCommand(SubCommand): tmp_tar = TarFile(fileobj=tmp, mode='w') # Add the executable to the tarball, using the current - # configured binfmt_misc path. - ff = _check_binfmt_misc(args.executable) - if not ff: - bn = os.path.basename(args.executable) - ff = "/usr/bin/%s" % bn - print ("No binfmt_misc configured: copied to %s" % (ff)) - - tmp_tar.add(args.executable, arcname=ff) + # configured binfmt_misc path. If we don't get a path then we + # only need the support libraries copied + ff, enabled = _check_binfmt_misc(args.executable) + + if not enabled: + print("binfmt_misc not enabled, update disabled") + return 1 + + if ff: + tmp_tar.add(args.executable, arcname=ff) # Add any associated libraries libs = _get_so_libs(args.executable)