From patchwork Fri Jun 8 12:33:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 137974 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp834242lji; Fri, 8 Jun 2018 05:52:41 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIKfbrgD/UJhakddE/bA9PEuj06fQq61o/5UM5sRhe6XgunhC/PZjsx7JxTLz0FptViQCOK X-Received: by 2002:a0c:9e5d:: with SMTP id z29-v6mr5504977qve.15.1528462361186; Fri, 08 Jun 2018 05:52:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528462361; cv=none; d=google.com; s=arc-20160816; b=SWi2+7tnotmicnYhKWZxIzdGCS+KIIGYQbbrohU5hP4WYZK3xjeQAYy3AgfMtE7Ivs 0j/BrOFZc5bR8cow/qjG9Z5rq7JDI4c1F7UyEQ7Trrv2LW0BB/eIvNvIxUp2cs1jGbEz IbXsS/CGvUZZbZbTOPDvSzjK/3wu3uYnDV8KSHbTuhPzlnIsPc9V9I7Y9uapCQBpY3Su MdlDHQy4YmlXupywjq7ar07HPQ25urGnY+ZuxFRT8koMF9QjaqCN9sCct9aEW5h2sVRY 7SxbASBPXZgFA/iWEdhjdRopDw6HjgxJfe3k5I89JhTfVpfM7qIK48WX+O4/FCgtbmDn heCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=fjynugjnAxDzGNNnwRtqNXN/KkRpgHkXOvtt7ZOkiOc=; b=R2IuhTNpGSQnrasF/q1DDg/Xf9fAvICXtRRB5MFpZDGfbuNb5iujDLuFLZrxLRC0OH VeiB/uS6vg1PhRk0EGCQPhiYWohsErsE5tHVVPi13Sxjt0xzkg3vt/LaqmiCywNi6uSI i6WZT72Ptd6kgh7e30cq9BZFN+8OJiizyI3FuktUAKJmOpypFOFu+1vxWtDusVJplhUP f7eX3vfUZ+oT/Fby7CwS+NPVqcDzTFYRjWoHRESiJ4sz6eP+eSzEq82JzeFne/SqNW6B a5zY+0Vt3LnSWu2JxIL4gUntP+FN47AIuGMM28bEVQff/W9B2sib2uByzl4R0ohDibSG 7Gtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XzaZvCDF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id a126-v6si8947072qkf.46.2018.06.08.05.52.40 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 08 Jun 2018 05:52:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XzaZvCDF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35462 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fRGsi-00005o-Ky for patch@linaro.org; Fri, 08 Jun 2018 08:52:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48258) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fRGfj-0005n2-R5 for qemu-devel@nongnu.org; Fri, 08 Jun 2018 08:39:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fRGfg-0000uU-Cc for qemu-devel@nongnu.org; Fri, 08 Jun 2018 08:39:15 -0400 Received: from mail-wm0-x232.google.com ([2a00:1450:400c:c09::232]:56221) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fRGfg-0000u3-34 for qemu-devel@nongnu.org; Fri, 08 Jun 2018 08:39:12 -0400 Received: by mail-wm0-x232.google.com with SMTP id v16-v6so3098682wmh.5 for ; Fri, 08 Jun 2018 05:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fjynugjnAxDzGNNnwRtqNXN/KkRpgHkXOvtt7ZOkiOc=; b=XzaZvCDF8w2vv6zSh1yVdEqqbtfuufEMcrOupb7uzScTLld8cJGVNwfaNQLFZEXeSj Ysd0LVvI+2MfJ0kUvYklVnm73nVERK5B1ugR+tGEdp/xI84ht4ImGM0NwjwrcSX1wxdM +i5eCPELv83QYT7nlHplKIyv4mdG4C0OKsk34= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fjynugjnAxDzGNNnwRtqNXN/KkRpgHkXOvtt7ZOkiOc=; b=guKGpAbvcjUUUTB+Rz4RYigrB0XZEv/XrvoUh48v6t6DM8Qn2HgSqb5m3H/Lifh6N3 AaveYtcKJ9XrXSMrgcKoX42r5mgGzz/YGrTBpq0kmKLVpGeuRVuvbglhfyGXE72RNL5e 7jwLBQeTrTSF8nHvvxh8lTMTxKTVNar+2FFBTFEk7QQVi6cvUm7mOsJ2UkaNZ5TqfC+u s5u9h1yNUlav0FI66BaZ2r4hcvC64nyc+1KfSrfXN2iKb5AbextT9q/KMSu1MwiyFBZz +ppZEVN8Fg8AH8/ZriHO7EqMh/pWUBvU/CyKJgpz6wmoRuUPBcsl69o1Ut1FWG8efmlN BnBg== X-Gm-Message-State: APt69E1mED4NZtwoVZiwaP8utCtRVD1FbqJig8gFkrB27ylzKcNmVgwv ip0st7ZSMW74g1UV90f3LPtnYQ== X-Received: by 2002:a1c:6fce:: with SMTP id c75-v6mr1418611wmi.83.1528461550963; Fri, 08 Jun 2018 05:39:10 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id b204-v6sm1641332wmh.22.2018.06.08.05.39.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 08 Jun 2018 05:39:08 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 017E63E0D8A; Fri, 8 Jun 2018 13:33:11 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de, pbonzini@redhat.com, stefanha@redhat.com, stefanb@linux.vnet.ibm.com, marcandre.lureau@redhat.com Date: Fri, 8 Jun 2018 13:33:02 +0100 Message-Id: <20180608123307.24773-45-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180608123307.24773-1-alex.bennee@linaro.org> References: <20180608123307.24773-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::232 Subject: [Qemu-devel] [PATCH v6 44/49] tests/tcg: add run, diff, and skip helper macros X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Eduardo Habkost , qemu-devel@nongnu.org, "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As we aren't using the default runners for all the test cases it is easy to miss out things like timeouts. To help with this we add some helpers and use them so we only need to make core changes in one place. Signed-off-by: Alex Bennée --- v5 - new for v5 v5.1 - diff-test->diff-out --- tests/tcg/Makefile | 13 ++++++++++--- tests/tcg/aarch64/Makefile.target | 6 ++---- tests/tcg/arm/Makefile.target | 6 ++---- tests/tcg/i386/Makefile.target | 10 ++++------ tests/tcg/multiarch/Makefile.target | 4 ++-- 5 files changed, 20 insertions(+), 19 deletions(-) -- 2.17.1 Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé diff --git a/tests/tcg/Makefile b/tests/tcg/Makefile index e7dbcdb5bf..d4c5140210 100644 --- a/tests/tcg/Makefile +++ b/tests/tcg/Makefile @@ -34,6 +34,15 @@ quiet-command = $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1, @$1)) +# $1 = test name, $2 = cmd, $3 = desc +run-test = $(call quiet-command, timeout $(TIMEOUT) $2 > $1.out,"TEST",$3) + +# $1 = test name, $2 = reference +diff-out = $(call quiet-command, diff -u $1.out $2 | head -n 10,"DIFF","$1.out with $2") + +# $1 = test name, $2 = reason +skip-test = @printf " SKIPPED %s because %s\n" $1 $2 + # Tests we are building TESTS= @@ -84,9 +93,7 @@ RUN_TESTS=$(patsubst %,run-%, $(TESTS)) RUN_TESTS+=$(EXTRA_RUNS) run-%: % - $(call quiet-command, \ - timeout $(TIMEOUT) $(QEMU) $< > $<.out, \ - "TEST", "$< on $(TARGET_NAME)") + $(call run-test, $<, $(QEMU) $<, "$< on $(TARGET_NAME)") .PHONY: run run: $(RUN_TESTS) diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index 7dba32138d..08c45b8470 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -13,7 +13,5 @@ TESTS:=$(AARCH64_TESTS) fcvt: LDFLAGS+=-lm run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(AARCH64_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) $<, "$< on $(TARGET_NAME)") + $(call diff-out,$<,$(AARCH64_SRC)/fcvt.ref) diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target index 14b5435fc6..aa4e4e3782 100644 --- a/tests/tcg/arm/Makefile.target +++ b/tests/tcg/arm/Makefile.target @@ -24,10 +24,8 @@ fcvt: LDFLAGS+=-lm # fcvt: CFLAGS+=-march=armv8.2-a+fp16 -mfpu=neon-fp-armv8 run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(ARM_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,fcvt,$(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,fcvt,$(ARM_SRC)/fcvt.ref) endif # On ARM Linux only supports 4k pages diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target index cd173363ee..97b7c23cf1 100644 --- a/tests/tcg/i386/Makefile.target +++ b/tests/tcg/i386/Makefile.target @@ -31,7 +31,7 @@ test-i386: test-i386.c test-i386-code16.S test-i386-vm86.S test-i386.h test-i386 # Specialist test runners run-runcom: runcom pi_10.com - $(call quiet-command, $(QEMU) ./runcom $(I386_SRC)/pi_10.com > runcom.out, "TEST", "$< on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) ./runcom $(I386_SRC)/pi_10.com,"$< on $(TARGET_NAME)") ifeq ($(SPEED), slow) @@ -40,13 +40,11 @@ test-i386-fprem.ref: test-i386-fprem run-test-i386-fprem: TIMEOUT=60 run-test-i386-fprem: test-i386-fprem - $(call quiet-command, \ - $(QEMU) $< > $<.out && \ - diff -u $(I386_SRC)/$<.ref $<.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref) else run-test-i386-fprem: test-i386-fprem - $(call quiet-command, /bin/true, "SLOW TEST", "$< SKIPPED on $(TARGET_NAME)") + $(call skip-test, $<, "SLOW") endif # On i386 and x86_64 Linux only supports 4k pages (large pages are a different hack) diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target index 90e45a881e..b77084c146 100644 --- a/tests/tcg/multiarch/Makefile.target +++ b/tests/tcg/multiarch/Makefile.target @@ -27,10 +27,10 @@ testthread: LDFLAGS+=-lpthread # default case (host page size) run-test-mmap: test-mmap - $(call quiet-command, $(QEMU) $< > test-mmap.out, "TEST", \ + $(call run-test, test-mmap, $(QEMU) $<, \ "$< (default) on $(TARGET_NAME)") # additional page sizes (defined by each architecture adding to EXTRA_RUNS) run-test-mmap-%: test-mmap - $(call quiet-command, $(QEMU) -p $* $< > test-mmap-$*.out, "TEST", \ + $(call run-test, test-mmap-$*, $(QEMU) -p $* $<,\ "$< ($* byte pages) on $(TARGET_NAME)")