diff mbox series

[RFC] tests/tcg: skip the vma-pthread test on CI

Message ID 20230116173233.2948746-1-alex.bennee@linaro.org
State Superseded
Headers show
Series [RFC] tests/tcg: skip the vma-pthread test on CI | expand

Commit Message

Alex Bennée Jan. 16, 2023, 5:32 p.m. UTC
We are getting a lot of failures that are not related to changes so
this could be a flaky test.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
---
 tests/tcg/multiarch/Makefile.target | 9 +++++++++
 1 file changed, 9 insertions(+)

Comments

Richard Henderson Jan. 16, 2023, 7:25 p.m. UTC | #1
On 1/16/23 07:32, Alex Bennée wrote:
> diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
> index e7213af492..ae8b3d7268 100644
> --- a/tests/tcg/multiarch/Makefile.target
> +++ b/tests/tcg/multiarch/Makefile.target
> @@ -42,6 +42,15 @@ munmap-pthread: LDFLAGS+=-pthread
>   vma-pthread: CFLAGS+=-pthread
>   vma-pthread: LDFLAGS+=-pthread
>   
> +# The vma-pthread seems very sensitive on gitlab and we currently
> +# don't know if its exposing a real bug or the test is flaky.
> +ifneq ($(GITLAB_CI),)
> +run-vma-pthread: vma-pthread
> +	$(call skip-test, $<, "flaky on CI?")
> +run-plugin-vma-pthread-with-%: vma-pthread
> +	$(call skip-test, $<, "flaky on CI?")
> +endif
> +

Ok I guess.  I'd have thought the ifdef around the entire mention of the test would be 
better -- no point in even building it.  But,

Acked-by: Richard Henderson <richard.henderson@linaro.org>


r~
Alex Bennée Jan. 16, 2023, 8:31 p.m. UTC | #2
I did consider it but it would involve messing about with filter to remove
the test from the wildcards. This way we don't forget about it when looking
through the logs.

I've not been able to get this to fail on any other machine though. It's
been rock solid over several thousand runs.

On Mon, 16 Jan 2023, 19:25 Richard Henderson, <richard.henderson@linaro.org>
wrote:

> On 1/16/23 07:32, Alex Bennée wrote:
> > diff --git a/tests/tcg/multiarch/Makefile.target
> b/tests/tcg/multiarch/Makefile.target
> > index e7213af492..ae8b3d7268 100644
> > --- a/tests/tcg/multiarch/Makefile.target
> > +++ b/tests/tcg/multiarch/Makefile.target
> > @@ -42,6 +42,15 @@ munmap-pthread: LDFLAGS+=-pthread
> >   vma-pthread: CFLAGS+=-pthread
> >   vma-pthread: LDFLAGS+=-pthread
> >
> > +# The vma-pthread seems very sensitive on gitlab and we currently
> > +# don't know if its exposing a real bug or the test is flaky.
> > +ifneq ($(GITLAB_CI),)
> > +run-vma-pthread: vma-pthread
> > +     $(call skip-test, $<, "flaky on CI?")
> > +run-plugin-vma-pthread-with-%: vma-pthread
> > +     $(call skip-test, $<, "flaky on CI?")
> > +endif
> > +
>
> Ok I guess.  I'd have thought the ifdef around the entire mention of the
> test would be
> better -- no point in even building it.  But,
>
> Acked-by: Richard Henderson <richard.henderson@linaro.org>
>
>
> r~
>
diff mbox series

Patch

diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index e7213af492..ae8b3d7268 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -42,6 +42,15 @@  munmap-pthread: LDFLAGS+=-pthread
 vma-pthread: CFLAGS+=-pthread
 vma-pthread: LDFLAGS+=-pthread
 
+# The vma-pthread seems very sensitive on gitlab and we currently
+# don't know if its exposing a real bug or the test is flaky.
+ifneq ($(GITLAB_CI),)
+run-vma-pthread: vma-pthread
+	$(call skip-test, $<, "flaky on CI?")
+run-plugin-vma-pthread-with-%: vma-pthread
+	$(call skip-test, $<, "flaky on CI?")
+endif
+
 # We define the runner for test-mmap after the individual
 # architectures have defined their supported pages sizes. If no
 # additional page sizes are defined we only run the default test.