diff mbox series

[02/25] tests/avocado: remove flaky test marking for test_sbsaref_edk2_firmware

Message ID 20231009164104.369749-3-alex.bennee@linaro.org
State Superseded
Headers show
Series October maintainer omnibus pre-PR (tests, gdbstub, plugins) | expand

Commit Message

Alex Bennée Oct. 9, 2023, 4:40 p.m. UTC 
After testing locally I decided to revert a5754847e0 (tests/avocado: Disable the
test_sbsaref_edk2_firmware by default) as the test seems pretty
stable:

   env QEMU_TEST_FLAKY_TESTS=1 retry.py -n 50 -c -- \
     ./tests/venv/bin/avocado run \
     ./tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_edk2_firmware

yields:

  Results summary:
  0: 50 times (100.00%), avg time 2.064 (0.04 varience/0.19 deviation)
  Ran command 50 times, 50 passes

Maybe f0ec14c78c (tests/avocado: Fix console data loss) has made it
more reliable?

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Thomas Huth <thuth@redhat.com>
---
 tests/avocado/machine_aarch64_sbsaref.py | 1 -
 1 file changed, 1 deletion(-)

Comments

Philippe Mathieu-Daudé Oct. 10, 2023, 11:56 a.m. UTC | #1 
On 9/10/23 18:40, Alex Bennée wrote:
> After testing locally I decided to revert a5754847e0 (tests/avocado: Disable the
> test_sbsaref_edk2_firmware by default) as the test seems pretty
> stable:
> 
>     env QEMU_TEST_FLAKY_TESTS=1 retry.py -n 50 -c -- \
>       ./tests/venv/bin/avocado run \
>       ./tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_edk2_firmware
> 
> yields:
> 
>    Results summary:
>    0: 50 times (100.00%), avg time 2.064 (0.04 varience/0.19 deviation)
>    Ran command 50 times, 50 passes
> 
> Maybe f0ec14c78c (tests/avocado: Fix console data loss) has made it
> more reliable?
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Thomas Huth <thuth@redhat.com>
> ---
>   tests/avocado/machine_aarch64_sbsaref.py | 1 -
>   1 file changed, 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
diff mbox series

Patch

diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
index b272053eaf..bdd1efc768 100644
--- a/tests/avocado/machine_aarch64_sbsaref.py
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -75,7 +75,6 @@  def fetch_firmware(self):
             "sbsa-ref",
         )
 
-    @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is not reliable')
     def test_sbsaref_edk2_firmware(self):
         """
         :avocado: tags=cpu:cortex-a57