From patchwork Wed Jul 17 13:43:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 169146 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp999485ilk; Wed, 17 Jul 2019 06:53:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqzCyUJzTDiExqJuyM0kO7jFwJgju21DMacDqMELh+CtszR73QtMT2e9bX2zIH6WMkQyhNe1 X-Received: by 2002:a17:906:5ad0:: with SMTP id x16mr31644357ejs.23.1563371583855; Wed, 17 Jul 2019 06:53:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563371583; cv=none; d=google.com; s=arc-20160816; b=1DZD7hae5e0PNuddF6VmIaC3o6OPN/WFDSH+d9/eXurpB5u6OdewLis3abqgL629rt Sdy0YrdqRGitBhV6XdF8bqPygRJEdO1EtC6VP6F6htwGGuiM4O59XH2bh2BHpOJTHcRx oM0CHui731jSVUKNny0C1Ilzb7UgCdEtRuLsASQNnALI6AFf45sUKl+uXsyeDImm02+h X5ZA1VKvwmmF8M4XWubUH6u+5/SqoK9+S/KWPP6nd98txhdGiBGLbithMLuyLPP01LTe 6cNpotqF0rjPRYMm5PGJQRpqtTgKWtCy5tN+KBUVIrrxgH7neGvVex42Yk0dDrP3ZBOu hCgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=r0APmCoUIG/w2qLEz3oEEruxZ1VMqL8sI7RlOiUMs8k=; b=rV4S1ZzEmRKQirxwTIr2EhXlr1pqQq+oaBSnOuusUIXdMAq+uwgUyCzdQX5W4GZf9G xjk8LGFtqYWDDuckxQdjL7uc7PuD+3e5K0yxs9XS/UgZC0LrWUdDxgY6s1K2PLs3XIpV cU5g7ZanmkVzz5E/eJHHVIciH/zuc39unPqEsApImLcQpofuwMZyWa/fPyuH82n4QhRf bcPTByQTUz+wtLXBEwki2L4YY32oaprX3zLzmCxgmnDj/993LWwKEddsKDDKhhd/NZse QyTjThxsOtNiv02PfcnlCQR3V+m5ghLZshcf2VskOgdtSQQGu9XDXNFucXr3NT8m73bq tcww== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=vfgGfA+0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x18si12062082eji.250.2019.07.17.06.53.03 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 17 Jul 2019 06:53:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=vfgGfA+0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hnkMg-0004Uo-LN for patch@linaro.org; Wed, 17 Jul 2019 09:53:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57892) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hnkKl-0004wN-HE for qemu-devel@nongnu.org; Wed, 17 Jul 2019 09:51:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hnkKi-00016t-Jv for qemu-devel@nongnu.org; Wed, 17 Jul 2019 09:51:02 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:50859) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hnkKi-00015c-Bz for qemu-devel@nongnu.org; Wed, 17 Jul 2019 09:51:00 -0400 Received: by mail-wm1-x343.google.com with SMTP id v15so22273056wml.0 for ; Wed, 17 Jul 2019 06:51:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r0APmCoUIG/w2qLEz3oEEruxZ1VMqL8sI7RlOiUMs8k=; b=vfgGfA+0cUbAZ8WOL5VGfN7Yn+KIvSpudPq/bQyO0Y1foDZrOpJnmE24fG68US1ShK IW9uitMI/Bc0oQW9JltinHJgpNKQh9GRidGwXdZxe0dH8oSDDGpWSoNooEaiDl7tPWNx VqsP/BK0n2IvD+ijPxJzx/i1/1lA3VGTpBPkgKtecJp6EJXFu7+BPffnm9Ti2AMZs7CC ZGIplavUxCxnPXcznEANH3tJn1n86/CRrSlV6Kbq/DSlt6Bx7od+TNzPhJCQtyCSfV6s +1miIL2Zv7IQPv3VU6Emvo9XE81haDkgjok7Av48DrQCMxOaa4VsBFJ5OMY9enYDXXUx iyLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r0APmCoUIG/w2qLEz3oEEruxZ1VMqL8sI7RlOiUMs8k=; b=aup7bAYOi8Rsiy41ZAtmvi0ZhRfxfk0gWXk3yliTD70UKlcSvtkRSqpdICqKGpZARH Uquq3bxHXGvyoHjVpKEpX3lGaPByapWtlQ2KE6luHLlKybV1sbMYv9DjfRUYAJGz/g3g hY5+tU99pbq40dQYscrpGgjSig2GW+rt0GPKXVy35AWL/7TaL67NiqzUH74hBh5yYvoi 1UZ1eoCNtNdX4NNmvipzKvroir7CRk2xYptd7V6WNMga1q/VcEHX9GNpuuGFSQ8C38y/ rveDV+UqalSaZjmP+fQqMR9WZDnzF4/tNrkQdtqBybKQbKNgbdXAD+aoZ+75+JvOsjt4 jU7w== X-Gm-Message-State: APjAAAX8w2aG/TqH5vIqwB9K3e7BXen2MyCD72zYZmEi6tISpI2sW6Rx 8y6QIPUDf6u8rLbnhHdU5IgI6s6GOwA= X-Received: by 2002:a1c:f90f:: with SMTP id x15mr35821393wmh.69.1563371459136; Wed, 17 Jul 2019 06:50:59 -0700 (PDT) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 4sm53933619wro.78.2019.07.17.06.50.54 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 06:50:55 -0700 (PDT) Received: from zen.linaroharston. (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C11881FFAA; Wed, 17 Jul 2019 14:43:37 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Wed, 17 Jul 2019 14:43:34 +0100 Message-Id: <20190717134335.15351-23-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717134335.15351-1-alex.bennee@linaro.org> References: <20190717134335.15351-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::343 Subject: [Qemu-devel] [PATCH v2 22/23] tests: Run the iotests during "make check" again X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Thomas Huth People often forget to run the iotests before submitting patches or pull requests - this is likely due to the fact that we do not run the tests during our mandatory "make check" tests yet. Now that we've got a proper "auto" group of iotests that should be fine to run in every environment, we can enable the iotests during "make check" again by running the "auto" tests by default from the check-block.sh script. Some cases still need to be checked first, though: iotests need bash and GNU sed (otherwise they fail), and if gprof is enabled, it spoils the output of some test cases causing them to fail. So if we detect that one of the required programs is missing or that gprof is enabled, we still have to skip the iotests to avoid failures. And finally, since we are using check-block.sh now again, this patch also removes the qemu-iotests-quick.sh script since we do not need that anymore (and having two shell wrapper scripts around the block tests seems rather confusing than helpful). Signed-off-by: Thomas Huth [AJB: -makecheck to check-block.sh, move check-block to start and gate it] Signed-off-by: Alex Bennée Message-Id: <20190717111947.30356-4-thuth@redhat.com> Signed-off-by: Alex Bennée --- tests/Makefile.include | 10 +++++---- tests/check-block.sh | 44 ++++++++++++++++++++++++++++--------- tests/qemu-iotests-quick.sh | 8 ------- 3 files changed, 40 insertions(+), 22 deletions(-) delete mode 100755 tests/qemu-iotests-quick.sh -- 2.20.1 diff --git a/tests/Makefile.include b/tests/Makefile.include index fd7fdb86586..6f02dfcc019 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -142,7 +142,7 @@ check-unit-y += tests/test-uuid$(EXESUF) check-unit-y += tests/ptimer-test$(EXESUF) check-unit-y += tests/test-qapi-util$(EXESUF) -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh +check-block-$(call land,$(CONFIG_POSIX),$(CONFIG_SOFTMMU)) += tests/check-block.sh # All QTests for now are POSIX-only, but the dependencies are # really in libqtest, not in the testcases themselves. @@ -1092,8 +1092,10 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES) QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF) -.PHONY: check-tests/qemu-iotests-quick.sh -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) +.PHONY: check-tests/check-block.sh +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) \ + qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \ + $(patsubst %,%/all,$(filter %-softmmu,$(TARGET_DIRS))) $< .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) @@ -1167,7 +1169,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS)) check-block: $(patsubst %,check-%, $(check-block-y)) -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree +check: check-block check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-clean: rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) diff --git a/tests/check-block.sh b/tests/check-block.sh index f3d12fd602d..c8b6cec3f68 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -1,24 +1,48 @@ #!/bin/sh -FORMAT_LIST="raw qcow2 qed vmdk vpc" +# Honor the SPEED environment variable, just like we do it for the qtests. +if [ "$SPEED" = "slow" ]; then + format_list="raw qcow2" + group= +elif [ "$SPEED" = "thorough" ]; then + format_list="raw qcow2 qed vmdk vpc" + group= +else + format_list=qcow2 + group="-g auto" +fi + if [ "$#" -ne 0 ]; then - FORMAT_LIST="$@" + format_list="$@" +fi + +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then + echo "GPROF is enabled ==> Not running the qemu-iotests." + exit 0 fi -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64" -export QEMU_IMG_PROG="$PWD/qemu-img" -export QEMU_IO_PROG="$PWD/qemu-io" +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then + echo "No qemu-system binary available ==> Not running the qemu-iotests." + exit 0 +fi + +if ! command -v bash >/dev/null 2>&1 ; then + echo "bash not available ==> Not running the qemu-iotests." + exit 0 +fi -if [ ! -x $QEMU_PROG ]; then - echo "'make check-block' requires qemu-system-x86_64" - exit 1 +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then + if ! command -v gsed >/dev/null 2>&1; then + echo "GNU sed not available ==> Not running the qemu-iotests." + exit 0 + fi fi cd tests/qemu-iotests ret=0 -for FMT in $FORMAT_LIST ; do - ./check -T -nocache -$FMT || ret=1 +for fmt in $format_list ; do + ./check -makecheck -$fmt $group || ret=1 done exit $ret diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh deleted file mode 100755 index 0e554bb9724..00000000000 --- a/tests/qemu-iotests-quick.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/sh - -cd tests/qemu-iotests - -ret=0 -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1 - -exit $ret