From patchwork Thu Oct 27 18:36:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 619220 Delivered-To: patch@linaro.org Received: by 2002:a17:522:c983:b0:460:3032:e3c4 with SMTP id kr3csp509896pvb; Thu, 27 Oct 2022 12:17:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5C2g0G+IC0ZAVpkpE3LO0CeCFh6pWuI02vUyEtJIRUciCvae6+tKwEuexaZW3xpqTauOHq X-Received: by 2002:ad4:5b8b:0:b0:4bb:a86e:d7b4 with SMTP id 11-20020ad45b8b000000b004bba86ed7b4mr2390527qvp.108.1666898228380; Thu, 27 Oct 2022 12:17:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666898228; cv=none; d=google.com; s=arc-20160816; b=oq6glKxsKZ1TQdmTtfhihbdQEbesctxKga5RYD0fuCyUXeLZKmRcEj8U30tZZLipx9 HxXyhqIanVc1l19JlHRK1NtLB/frXypdPdnWVYeMx74Zh4kjj1sKBdHmuhTg5n11Ekh1 oZPuEHLzgCb+JrW1aw43+J/CshYiOt6MfecDxcDuwo+Sq0O2BHMyKQamhwP48SsqeFcM 2XTBjFgLG2hrA4+o0PwvQz21ahjDAREpl7eMZILuNd3wbvZ90mJg8FV1hAdDXRtglOvX 2mzPSOPPk9AsqFzqfkR79P0OEjRMoDcHsqLvnLw7/WjDieznSZ5rv32wZ9p3+yOTPl8h i8tQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ZL65v0q08abuMcd8/dJR1pUqsCkfFRqsKtdLqAeDaiw=; b=Im1SEkHvi0BLkB126E3q83XZv5HC3D/455JKKPDCT4Qpa2dnezkUVVy2YnDiO39WS/ gCyfEZrUy9pkjHhaPMAlYaATWD1EHT6mF11QBG/OyJ2SeuRaC4c8dZa+yXNpiM4dgwG2 Csp2GjUWkxvtHFXkVQWKYz9lOayX90S/3Azg3/l/zJ9IH9bBkOWqmt9oNaMArE/9W1yz rz3I3EiBGZMBiGfjlaupRGwGW4WmvGvgbTDXtF+dhgm76G++l2rDHOu2tBgyV6+lMS9T XaH8kEmIdiLC8wTu16JGYzH0zuCTyGEmxDjJm2IpOMDL/5YUbn6ZtOh4k461A/InPuUc 0wrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="qiJ+/yvc"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o12-20020ac872cc000000b0039cb84b2f4csi626171qtp.763.2022.10.27.12.17.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 27 Oct 2022 12:17:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="qiJ+/yvc"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oo7si-0005ao-AZ; Thu, 27 Oct 2022 14:45:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oo7sC-0005SG-1J for qemu-devel@nongnu.org; Thu, 27 Oct 2022 14:45:03 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oo7s6-0004jP-BZ for qemu-devel@nongnu.org; Thu, 27 Oct 2022 14:44:59 -0400 Received: by mail-wm1-x329.google.com with SMTP id n14so1581935wmq.3 for ; Thu, 27 Oct 2022 11:44:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZL65v0q08abuMcd8/dJR1pUqsCkfFRqsKtdLqAeDaiw=; b=qiJ+/yvctHt/wwB0cDMAsn79ysduxf0i4rVsh9GCRjFzl/RvNmvwBwIRZ97VrXY05t /ZCid/GvtUnub0pI73k6lLNpiqRUkt3ViwSkY1RjhuWfkdvgZ3+Ev+vZqtpvsgsi9nJl T592Iv083BoO3+p80NQwIPo/XaXFnDo8c1Tl+kZkR3prACfPZFGn7M6xxUOnIEbtdLhA v2pCGwfUBmBmW6DeKXGy+C626L4Qu7ZcBUPvJr+viP5y4UMZ1Pt0GgpXRlanVtp9LDHa FW4oRvzxyWwgXtjz4YI8hZi+cjfjLh7DGeURNqH+0SJ7hlkSj+02F+PPbQIHOcruomms cQEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZL65v0q08abuMcd8/dJR1pUqsCkfFRqsKtdLqAeDaiw=; b=rF7xivfgzX5/EFRD2bzsWLm+eXxlnzTN17enjAV+INqm41wrs3ecqr2TR7DObRqrgl Tsp4O5cGWdtR0tTeSNATQHiArs7gvrpOhsudBgWfjDOAG283h378+cLtzZGa9SpKG4K7 G7iOZ4rkx5UyAfTYlaEVtGQJ68jxiJOK5/5o6PU1cOcvxkkKLNl4Y5iGhkvgCx9iXK4s vkLKMqWTsD5JWSZBuKtmKRHU35WFge2iLi5pusEdOsmO7/dbFm2w+GChHzdRuA++SOo4 JxuixoVKYWU8ipoi6+dMbYLqPDa2+NK9FS+ahgV7q/Sujoq0m3QWOSg6pKec7hA23weF fMmw== X-Gm-Message-State: ACrzQf3W5LfXxbf7UdWO41RJEoOiNqjfE/F7DWSHTh/hRwztyfbnBzDa cvODRFNne459HKeAhmP85OIcqw== X-Received: by 2002:a05:600c:19ca:b0:3c6:ee5a:45fd with SMTP id u10-20020a05600c19ca00b003c6ee5a45fdmr6952156wmq.106.1666896292975; Thu, 27 Oct 2022 11:44:52 -0700 (PDT) Received: from zen.linaroharston ([185.81.254.11]) by smtp.gmail.com with ESMTPSA id e4-20020a5d5004000000b0023655e51c33sm1748613wrt.4.2022.10.27.11.44.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 11:44:51 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 3986D1FFCA; Thu, 27 Oct 2022 19:36:39 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: fam@euphon.net, berrange@redhat.com, f4bug@amsat.org, aurelien@aurel32.net, pbonzini@redhat.com, stefanha@redhat.com, crosa@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , Yoshinori Sato , Magnus Damm , Wainer dos Santos Moschetta , Beraldo Leal Subject: [PATCH v4 20/30] tests/avocado: disable sh4 rd2 tests on Gitlab Date: Thu, 27 Oct 2022 19:36:26 +0100 Message-Id: <20221027183637.2772968-21-alex.bennee@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221027183637.2772968-1-alex.bennee@linaro.org> References: <20221027183637.2772968-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Running repeated invocations on a number of test boxes show a fairly high error rate: $ retry.py -n 100 -c -- ./tests/venv/bin/avocado run tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d retry.py called with ['./tests/venv/bin/avocado', 'run', 'tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d'] Results: Run, Ret, Pass/Fail, Time, Total Pass, Total Run ... Results summary: 0: 94 times (94.00%), avg time 2.254 (0.00 varience/0.04 deviation) 1: 3 times (3.00%), avg time 1.837 (0.02 varience/0.14 deviation) 8: 3 times (3.00%), avg time 91.288 (0.02 varience/0.15 deviation) Examining the logs they fall into various categories of un-handled unaligned access by user space and unexpected FPU usage by the kernel which ultimately lead to the failure to reach the login prompt. This could be bugs in the translator that only get hit occasionally or just a flaky kernel - its hard to tell. To avoid these failures gating CI lets skip on GitLab. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Cc: Thomas Huth Cc: Yoshinori Sato Cc: Magnus Damm --- v2 - add import for skipIf --- tests/avocado/boot_linux_console.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py index eed4b49e6e..4c9d551f47 100644 --- a/tests/avocado/boot_linux_console.py +++ b/tests/avocado/boot_linux_console.py @@ -15,6 +15,7 @@ from avocado import skip from avocado import skipUnless +from avocado import skipIf from avocado_qemu import QemuSystemTest from avocado_qemu import exec_command from avocado_qemu import exec_command_and_wait_for_pattern @@ -1175,6 +1176,10 @@ def test_ppc_mac99(self): self.vm.add_args('-M', 'graphics=off') self.do_test_advcal_2018('15', tar_hash, 'invaders.elf') + # This test has a 6-10% failure rate on various hosts that look + # like issues with a buggy kernel. As a result we don't want it + # gating releases on Gitlab. + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_sh4_r2d(self): """ :avocado: tags=arch:sh4