| Message ID | 20240108151352.2199097-14-alex.bennee@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [PULL,01/13] tests/avocado: add a simple i386 replay kernel test | expand |
On 08/01/2024 16.13, Alex Bennée wrote: > With the latest fixes for #2010 and #2013 these tests look pretty > stable now. Of course the only way to be really sure is to run it in > the CI infrastructure and see what breaks. > > Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20231211091346.14616-14-alex.bennee@linaro.org> The replay tests seem still to be very flaky, I'm now getting: https://gitlab.com/thuth/qemu/-/jobs/5910241580#L227 https://gitlab.com/thuth/qemu/-/jobs/5910241593#L396 I'd suggest to revert this patch to disable them in the CI again. Thomas
Thomas Huth <thuth@redhat.com> writes: > On 08/01/2024 16.13, Alex Bennée wrote: >> With the latest fixes for #2010 and #2013 these tests look pretty >> stable now. Of course the only way to be really sure is to run it in >> the CI infrastructure and see what breaks. >> Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> Message-Id: <20231211091346.14616-14-alex.bennee@linaro.org> > > The replay tests seem still to be very flaky, I'm now getting: > > https://gitlab.com/thuth/qemu/-/jobs/5910241580#L227 > https://gitlab.com/thuth/qemu/-/jobs/5910241593#L396 > > I'd suggest to revert this patch to disable them in the CI again. Ok I'll move it back to flaky. I thought I'd eliminated the problem as I can do 100 runs without failure locally. Sadly there doesn't seem to be anything obvious in the logs as to whats going on. > > Thomas
Thomas Huth <thuth@redhat.com> writes: > On 08/01/2024 16.13, Alex Bennée wrote: >> With the latest fixes for #2010 and #2013 these tests look pretty >> stable now. Of course the only way to be really sure is to run it in >> the CI infrastructure and see what breaks. >> Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> Message-Id: <20231211091346.14616-14-alex.bennee@linaro.org> > > The replay tests seem still to be very flaky, I'm now getting: > > https://gitlab.com/thuth/qemu/-/jobs/5910241580#L227 > https://gitlab.com/thuth/qemu/-/jobs/5910241593#L396 > > I'd suggest to revert this patch to disable them in the CI again. I did but on reconsideration I'm only partially reverting for the x86_64 bits as everything else looks stable (unless you have more examples). I've raised a new bug to cover it: https://gitlab.com/qemu-project/qemu/-/issues/2094
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py index 1eaa36444cb..6fdcbd6ac3d 100644 --- a/tests/avocado/replay_kernel.py +++ b/tests/avocado/replay_kernel.py @@ -98,13 +98,10 @@ def test_i386_pc(self): self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) - # See https://gitlab.com/qemu-project/qemu/-/issues/2010 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck') def test_x86_64_pc(self): """ :avocado: tags=arch:x86_64 :avocado: tags=machine:pc - :avocado: tags=flaky """ kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' '/linux/releases/29/Everything/x86_64/os/images/pxeboot' @@ -135,8 +132,6 @@ def test_mips_malta(self): self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) - # See https://gitlab.com/qemu-project/qemu/-/issues/2013 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') def test_mips64el_malta(self): """ This test requires the ar tool to extract "data.tar.gz" from @@ -152,7 +147,6 @@ def test_mips64el_malta(self): :avocado: tags=arch:mips64el :avocado: tags=machine:malta - :avocado: tags=flaky """ deb_url = ('http://snapshot.debian.org/archive/debian/' '20130217T032700Z/pool/main/l/linux-2.6/' @@ -200,13 +194,10 @@ def test_arm_virt(self): self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1) - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test_arm_cubieboard_initrd(self): """ :avocado: tags=arch:arm :avocado: tags=machine:cubieboard - :avocado: tags=flaky """ deb_url = ('https://apt.armbian.com/pool/main/l/' 'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb') @@ -354,7 +345,6 @@ def test_m68k_mcf5208evb(self): file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) self.do_test_advcal_2018(file_path, 'sanity-clause.elf') - @skip("Test currently broken") # Console stuck as of 5.2-rc1 def test_microblaze_s3adsp1800(self): """ :avocado: tags=arch:microblaze @@ -389,7 +379,6 @@ def test_or1k_sim(self): file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) self.do_test_advcal_2018(file_path, 'vmlinux') - @skip("nios2 emulation is buggy under record/replay") def test_nios2_10m50(self): """ :avocado: tags=arch:nios2