From patchwork Thu Jun 21 06:25:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 139471 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp1685936lji; Wed, 20 Jun 2018 23:45:23 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKZxNRLRJ/Nm8QBIbPk27z2SnlPp1N/JT7HjzFmfX7zws5YmBR98CwyL2Fanb6fEWldeWhG X-Received: by 2002:ac8:3753:: with SMTP id p19-v6mr22628740qtb.43.1529563523449; Wed, 20 Jun 2018 23:45:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529563523; cv=none; d=google.com; s=arc-20160816; b=qulhuAEt7egTf7MF9BaAHZYDKKpZihrAMLIWANyYP/c3SxM3gKYEYaf1RI/SpBx7jg L8L47QtOrv+lN0oDNNunE7SOVMtOPXe9KlOb4goEQtJVX8GRiFyZM2c/Yg4yxEVissVc KERj8g+Fgs8Aye/aWPgXR+oTKkONYtNLDRqb5YDRiXsF4YhzK13gnzSS8XQB+OCFp+08 5N03mzBniLYTnQwKuhPK8bK5NocOxgMKYxmLwk5so0Vo/8irkm5SHXU0yzDIyd4urDip eFdoXyIfHKYjfxJz54W2Rq7vfbYuJk6FrjL3q8RehFyuumEcXEpIZkA1O+1dXL6FGr7B LuUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=SxZmE0/zGbyvZTTMQfRQY3KG2AmwxPyyIVlEFmj3hbw=; b=jomZNaLQKh/3x6Q8JPV/95dqaySQfW0rqc1pPgtl2KloufoCffQn/wmKzkAzQf5ptF MYMU8SEatcRVJKg/+1wM7zM6Nt/6U/ruAN9MO1imsNsntbo+Essk57VzTIrWri+/mpPB xj6MlIqWesnRdz9vyf1pAZEy52N0rWwOhmA4V+urDfhlMR3yJ2kB/ZpiM+cQ6y86ZcKD uKWdliOkv5p47LwW7VtyOOrVwhtHQnMjw9OZcinFnX0y9BmDU6Mnu8TARsefskEl8mdE pMP1+A26FP2T5a0p/yH3YH57tvLfqwGJ13ApNRbZXpcYrDny7WJtPNdsbnA/IQLwS/Q0 BcAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Mpl2WRle; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t67-v6si1245964qkb.321.2018.06.20.23.45.23 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 20 Jun 2018 23:45:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Mpl2WRle; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53333 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVtLO-0007Lp-Q4 for patch@linaro.org; Thu, 21 Jun 2018 02:45:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54465) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVt8V-0005ja-CJ for qemu-devel@nongnu.org; Thu, 21 Jun 2018 02:32:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVt8T-0003qX-Kj for qemu-devel@nongnu.org; Thu, 21 Jun 2018 02:32:03 -0400 Received: from mail-wm0-x233.google.com ([2a00:1450:400c:c09::233]:55309) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVt8T-0003oJ-C5 for qemu-devel@nongnu.org; Thu, 21 Jun 2018 02:32:01 -0400 Received: by mail-wm0-x233.google.com with SMTP id v16-v6so3107298wmh.5 for ; Wed, 20 Jun 2018 23:32:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SxZmE0/zGbyvZTTMQfRQY3KG2AmwxPyyIVlEFmj3hbw=; b=Mpl2WRleodEe61haecyMz9rD5tBNZw/p7F5TMJzQPU0fSZ59Q3b/6s4h28k84edHOj d98K6mhpb3o6wPO+QU9ZKkDB9OpEnBI2PmIaYnWCjbQuAHS0pThw7uxz+fH4TJdgSKP2 Dx06UnThpS4ZxPG2BYbs+Bxp/WukCNDVh9D4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SxZmE0/zGbyvZTTMQfRQY3KG2AmwxPyyIVlEFmj3hbw=; b=SF3wcVeaPKEs/rNftwy0YTlZ6yHd2OCIhjQ2PP0Sl3KK8Nn7W5ovmaPgbSWSDTLeTy tsn3z/BEnOH7wD9Bk1EJm0R/7A+pvv2yujretjo0cJxemk+vurimnSua5/pptti7QntP 61P5zPQ+hBu89Iq4bFcOxb4FG+BYpfdk2rAFKjDBmm7HDqWPY9N6Yit+X4P9Tw3Y0cwq knyVM4D/NHTiiwA7v0EsnWsKBeDpMNfAfsSAFq6/Xz4E/+RiBCizgYNzAPZ+WOIlv+Wg /yQTNSpXOqBkigv7lyi98JSNeFL/Kfuol+8hjZsZGEhUYqPVv8FVKeCWtEQMlZm673PV e49A== X-Gm-Message-State: APt69E0YRigP32PbMUVLOICij5FQHpgOQMFSQZy0LvfzIna8OT/f6E8T r8NWZvYQhDtZtRyjqdJQF+nPXw== X-Received: by 2002:a1c:3a91:: with SMTP id h139-v6mr4110204wma.94.1529562720249; Wed, 20 Jun 2018 23:32:00 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id l142-v6sm7599212wmd.16.2018.06.20.23.31.54 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 23:31:56 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 5D7CA3E1504; Thu, 21 Jun 2018 07:26:09 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Date: Thu, 21 Jun 2018 07:25:55 +0100 Message-Id: <20180621062605.941-48-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180621062605.941-1-alex.bennee@linaro.org> References: <20180621062605.941-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::233 Subject: [Qemu-devel] [PULL v2 47/57] tests/tcg: add run, diff, and skip helper macros X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: agraf@suse.de, Eduardo Habkost , =?utf-8?q?Alex_B?= =?utf-8?b?ZW5uw6ll?= , richard.henderson@linaro.org, qemu-devel@nongnu.org, f4bug@amsat.org, cota@braap.org, "open list:ARM" , Paolo Bonzini , famz@redhat.com, aurelien@aurel32.net, Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As we aren't using the default runners for all the test cases it is easy to miss out things like timeouts. To help with this we add some helpers and use them so we only need to make core changes in one place. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé -- 2.17.1 diff --git a/tests/tcg/Makefile b/tests/tcg/Makefile index e7dbcdb5bf..bf06415390 100644 --- a/tests/tcg/Makefile +++ b/tests/tcg/Makefile @@ -34,6 +34,15 @@ quiet-command = $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1, @$1)) +# $1 = test name, $2 = cmd, $3 = desc +run-test = $(call quiet-command, timeout $(TIMEOUT) $2 > $1.out,"TEST",$3) + +# $1 = test name, $2 = reference +diff-out = $(call quiet-command, diff -u $1.out $2 | head -n 10,"DIFF","$1.out with $2") + +# $1 = test name, $2 = reason +skip-test = @printf " SKIPPED %s on $(TARGET_NAME) because %s\n" $1 $2 + # Tests we are building TESTS= @@ -84,9 +93,7 @@ RUN_TESTS=$(patsubst %,run-%, $(TESTS)) RUN_TESTS+=$(EXTRA_RUNS) run-%: % - $(call quiet-command, \ - timeout $(TIMEOUT) $(QEMU) $< > $<.out, \ - "TEST", "$< on $(TARGET_NAME)") + $(call run-test, $<, $(QEMU) $<, "$< on $(TARGET_NAME)") .PHONY: run run: $(RUN_TESTS) diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index 7dba32138d..08c45b8470 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -13,7 +13,5 @@ TESTS:=$(AARCH64_TESTS) fcvt: LDFLAGS+=-lm run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(AARCH64_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) $<, "$< on $(TARGET_NAME)") + $(call diff-out,$<,$(AARCH64_SRC)/fcvt.ref) diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target index 14b5435fc6..aa4e4e3782 100644 --- a/tests/tcg/arm/Makefile.target +++ b/tests/tcg/arm/Makefile.target @@ -24,10 +24,8 @@ fcvt: LDFLAGS+=-lm # fcvt: CFLAGS+=-march=armv8.2-a+fp16 -mfpu=neon-fp-armv8 run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(ARM_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,fcvt,$(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,fcvt,$(ARM_SRC)/fcvt.ref) endif # On ARM Linux only supports 4k pages diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target index cd173363ee..97b7c23cf1 100644 --- a/tests/tcg/i386/Makefile.target +++ b/tests/tcg/i386/Makefile.target @@ -31,7 +31,7 @@ test-i386: test-i386.c test-i386-code16.S test-i386-vm86.S test-i386.h test-i386 # Specialist test runners run-runcom: runcom pi_10.com - $(call quiet-command, $(QEMU) ./runcom $(I386_SRC)/pi_10.com > runcom.out, "TEST", "$< on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) ./runcom $(I386_SRC)/pi_10.com,"$< on $(TARGET_NAME)") ifeq ($(SPEED), slow) @@ -40,13 +40,11 @@ test-i386-fprem.ref: test-i386-fprem run-test-i386-fprem: TIMEOUT=60 run-test-i386-fprem: test-i386-fprem - $(call quiet-command, \ - $(QEMU) $< > $<.out && \ - diff -u $(I386_SRC)/$<.ref $<.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref) else run-test-i386-fprem: test-i386-fprem - $(call quiet-command, /bin/true, "SLOW TEST", "$< SKIPPED on $(TARGET_NAME)") + $(call skip-test, $<, "SLOW") endif # On i386 and x86_64 Linux only supports 4k pages (large pages are a different hack) diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target index 90e45a881e..b77084c146 100644 --- a/tests/tcg/multiarch/Makefile.target +++ b/tests/tcg/multiarch/Makefile.target @@ -27,10 +27,10 @@ testthread: LDFLAGS+=-lpthread # default case (host page size) run-test-mmap: test-mmap - $(call quiet-command, $(QEMU) $< > test-mmap.out, "TEST", \ + $(call run-test, test-mmap, $(QEMU) $<, \ "$< (default) on $(TARGET_NAME)") # additional page sizes (defined by each architecture adding to EXTRA_RUNS) run-test-mmap-%: test-mmap - $(call quiet-command, $(QEMU) -p $* $< > test-mmap-$*.out, "TEST", \ + $(call run-test, test-mmap-$*, $(QEMU) -p $* $<,\ "$< ($* byte pages) on $(TARGET_NAME)")