Message ID | 20170331203939.4549-1-maxim.uvarov@linaro.org |
---|---|
State | Accepted |
Commit | 51e3b8776b78180741fa57a621f9d13b9ae8bbfb |
Headers | show |
Please review this patch. TM very often fails in Travis CI. This patch has to fix it. Maxim. On 31 March 2017 at 23:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > tm test fails time to time in Travis environment. Because > of we can not control that machine we can not do things like > taskset and core isolation there. So simple run test and ignore > it's result. Threat only segfault as actual error. Linaro CI > will still do full test. > https://bugs.linaro.org/show_bug.cgi?id=2881 > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> > --- > .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- > .../validation/api/traffic_mngr/traffic_mngr.sh | 25 > ++++++++++++++++++++++ > 2 files changed, 35 insertions(+), 2 deletions(-) > create mode 100755 test/common_plat/validation/ > api/traffic_mngr/traffic_mngr.sh > > diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am > b/test/common_plat/validation/api/traffic_mngr/Makefile.am > index 35e689a0..a012c1b3 100644 > --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am > +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am > @@ -1,10 +1,18 @@ > include ../Makefile.inc > > +TESTS_ENVIRONMENT += TEST_DIR=${builddir} > + > +TESTSCRIPTS = traffic_mngr.sh > +TEST_EXTENSIONS = .sh > + > +TESTS = $(TESTSCRIPTS) > + > noinst_LTLIBRARIES = libtesttraffic_mngr.la > libtesttraffic_mngr_la_SOURCES = traffic_mngr.c > > -test_PROGRAMS = traffic_mngr_main$(EXEEXT) > +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) > dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c > traffic_mngr_main_LDADD = libtesttraffic_mngr.la -lm $(LIBCUNIT_COMMON) > $(LIBODP) > > -EXTRA_DIST = traffic_mngr.h > +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) > +dist_check_SCRIPTS = $(TESTSCRIPTS) > diff --git a/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > new file mode 100755 > index 00000000..a7d54162 > --- /dev/null > +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > @@ -0,0 +1,25 @@ > +#!/bin/sh > +# > +# Copyright (c) 2017, Linaro Limited > +# All rights reserved. > +# > +# SPDX-License-Identifier: BSD-3-Clause > +# > + > +# directory where test binaries have been built > +TEST_DIR="${TEST_DIR:-$(dirname $0)}" > + > +# exit codes expected by automake for skipped tests > +TEST_SKIPPED=77 > + > +${TEST_DIR}/traffic_mngr_main${EXEEXT} > +ret=$? > + > +SIGSEGV=139 > + > +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne ${SIGSEGV} ]; > then > + echo "SKIP: skip due to not isolated environment" > + exit ${TEST_SKIPPED} > +fi > + > +exit $ret > -- > 2.11.0.295.gd7dffce > >
ping. If people do not want to see any entries for CI env in test suite code then I can make this patch be applied on each Travis run. If it's good I need a review. Some times this test fails. On 6 April 2017 at 17:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > Please review this patch. TM very often fails in Travis CI. This patch has > to fix it. > > Maxim. > > On 31 March 2017 at 23:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > >> tm test fails time to time in Travis environment. Because >> of we can not control that machine we can not do things like >> taskset and core isolation there. So simple run test and ignore >> it's result. Threat only segfault as actual error. Linaro CI >> will still do full test. >> https://bugs.linaro.org/show_bug.cgi?id=2881 >> >> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> >> --- >> .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- >> .../validation/api/traffic_mngr/traffic_mngr.sh | 25 >> ++++++++++++++++++++++ >> 2 files changed, 35 insertions(+), 2 deletions(-) >> create mode 100755 test/common_plat/validation/ap >> i/traffic_mngr/traffic_mngr.sh >> >> diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am >> b/test/common_plat/validation/api/traffic_mngr/Makefile.am >> index 35e689a0..a012c1b3 100644 >> --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am >> +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am >> @@ -1,10 +1,18 @@ >> include ../Makefile.inc >> >> +TESTS_ENVIRONMENT += TEST_DIR=${builddir} >> + >> +TESTSCRIPTS = traffic_mngr.sh >> +TEST_EXTENSIONS = .sh >> + >> +TESTS = $(TESTSCRIPTS) >> + >> noinst_LTLIBRARIES = libtesttraffic_mngr.la >> libtesttraffic_mngr_la_SOURCES = traffic_mngr.c >> >> -test_PROGRAMS = traffic_mngr_main$(EXEEXT) >> +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) >> dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c >> traffic_mngr_main_LDADD = libtesttraffic_mngr.la -lm $(LIBCUNIT_COMMON) >> $(LIBODP) >> >> -EXTRA_DIST = traffic_mngr.h >> +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) >> +dist_check_SCRIPTS = $(TESTSCRIPTS) >> diff --git a/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh >> b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh >> new file mode 100755 >> index 00000000..a7d54162 >> --- /dev/null >> +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh >> @@ -0,0 +1,25 @@ >> +#!/bin/sh >> +# >> +# Copyright (c) 2017, Linaro Limited >> +# All rights reserved. >> +# >> +# SPDX-License-Identifier: BSD-3-Clause >> +# >> + >> +# directory where test binaries have been built >> +TEST_DIR="${TEST_DIR:-$(dirname $0)}" >> + >> +# exit codes expected by automake for skipped tests >> +TEST_SKIPPED=77 >> + >> +${TEST_DIR}/traffic_mngr_main${EXEEXT} >> +ret=$? >> + >> +SIGSEGV=139 >> + >> +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne ${SIGSEGV} >> ]; then >> + echo "SKIP: skip due to not isolated environment" >> + exit ${TEST_SKIPPED} >> +fi >> + >> +exit $ret >> -- >> 2.11.0.295.gd7dffce >> >> >
Do I understand this patch correctly that it attempts to run the test and if it fails for any reason other than a segfault and we are running in Travis that it is then just ignored? That seems reasonable. On Tue, Apr 18, 2017 at 11:27 AM, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > ping. If people do not want to see any entries for CI env in test suite > code then I can make this patch be applied on each Travis run. If it's good > I need a review. Some times this test fails. > > On 6 April 2017 at 17:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > Please review this patch. TM very often fails in Travis CI. This patch > has > > to fix it. > > > > Maxim. > > > > On 31 March 2017 at 23:39, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > >> tm test fails time to time in Travis environment. Because > >> of we can not control that machine we can not do things like > >> taskset and core isolation there. So simple run test and ignore > >> it's result. Threat only segfault as actual error. Linaro CI > >> will still do full test. > >> https://bugs.linaro.org/show_bug.cgi?id=2881 > >> > >> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> > >> --- > >> .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- > >> .../validation/api/traffic_mngr/traffic_mngr.sh | 25 > >> ++++++++++++++++++++++ > >> 2 files changed, 35 insertions(+), 2 deletions(-) > >> create mode 100755 test/common_plat/validation/ap > >> i/traffic_mngr/traffic_mngr.sh > >> > >> diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> b/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> index 35e689a0..a012c1b3 100644 > >> --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> @@ -1,10 +1,18 @@ > >> include ../Makefile.inc > >> > >> +TESTS_ENVIRONMENT += TEST_DIR=${builddir} > >> + > >> +TESTSCRIPTS = traffic_mngr.sh > >> +TEST_EXTENSIONS = .sh > >> + > >> +TESTS = $(TESTSCRIPTS) > >> + > >> noinst_LTLIBRARIES = libtesttraffic_mngr.la > >> libtesttraffic_mngr_la_SOURCES = traffic_mngr.c > >> > >> -test_PROGRAMS = traffic_mngr_main$(EXEEXT) > >> +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) > >> dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c > >> traffic_mngr_main_LDADD = libtesttraffic_mngr.la -lm > $(LIBCUNIT_COMMON) > >> $(LIBODP) > >> > >> -EXTRA_DIST = traffic_mngr.h > >> +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) > >> +dist_check_SCRIPTS = $(TESTSCRIPTS) > >> diff --git a/test/common_plat/validation/api/traffic_mngr/traffic_mngr. > sh > >> b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > >> new file mode 100755 > >> index 00000000..a7d54162 > >> --- /dev/null > >> +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > >> @@ -0,0 +1,25 @@ > >> +#!/bin/sh > >> +# > >> +# Copyright (c) 2017, Linaro Limited > >> +# All rights reserved. > >> +# > >> +# SPDX-License-Identifier: BSD-3-Clause > >> +# > >> + > >> +# directory where test binaries have been built > >> +TEST_DIR="${TEST_DIR:-$(dirname $0)}" > >> + > >> +# exit codes expected by automake for skipped tests > >> +TEST_SKIPPED=77 > >> + > >> +${TEST_DIR}/traffic_mngr_main${EXEEXT} > >> +ret=$? > >> + > >> +SIGSEGV=139 > >> + > >> +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne ${SIGSEGV} > >> ]; then > >> + echo "SKIP: skip due to not isolated environment" > >> + exit ${TEST_SKIPPED} > >> +fi > >> + > >> +exit $ret > >> -- > >> 2.11.0.295.gd7dffce > >> > >> > > >
On 04/18/17 22:26, Bill Fischofer wrote: > Do I understand this patch correctly that it attempts to run the test > and if it fails for any reason other than a segfault and we are running > in Travis that it is then just ignored? That seems reasonable. > yes. that is the goal. Run test to make code coverage happy and check that there is no critical error i.e. segfault. If test does not fit in low/high acceptable borders for tm time slots then mark test as skipped due to not isolated environment Travis CI. Maxim. > On Tue, Apr 18, 2017 at 11:27 AM, Maxim Uvarov <maxim.uvarov@linaro.org > <mailto:maxim.uvarov@linaro.org>> wrote: > > ping. If people do not want to see any entries for CI env in test suite > code then I can make this patch be applied on each Travis run. If > it's good > I need a review. Some times this test fails. > > On 6 April 2017 at 17:39, Maxim Uvarov <maxim.uvarov@linaro.org > <mailto:maxim.uvarov@linaro.org>> wrote: > > > Please review this patch. TM very often fails in Travis CI. This > patch has > > to fix it. > > > > Maxim. > > > > On 31 March 2017 at 23:39, Maxim Uvarov <maxim.uvarov@linaro.org > <mailto:maxim.uvarov@linaro.org>> wrote: > > > >> tm test fails time to time in Travis environment. Because > >> of we can not control that machine we can not do things like > >> taskset and core isolation there. So simple run test and ignore > >> it's result. Threat only segfault as actual error. Linaro CI > >> will still do full test. > >> https://bugs.linaro.org/show_bug.cgi?id=2881 > <https://bugs.linaro.org/show_bug.cgi?id=2881> > >> > >> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org > <mailto:maxim.uvarov@linaro.org>> > >> --- > >> .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- > >> .../validation/api/traffic_mngr/traffic_mngr.sh | 25 > >> ++++++++++++++++++++++ > >> 2 files changed, 35 insertions(+), 2 deletions(-) > >> create mode 100755 test/common_plat/validation/ap > >> i/traffic_mngr/traffic_mngr.sh > >> > >> diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> b/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> index 35e689a0..a012c1b3 100644 > >> --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am > >> @@ -1,10 +1,18 @@ > >> include ../Makefile.inc > >> > >> +TESTS_ENVIRONMENT += TEST_DIR=${builddir} > >> + > >> +TESTSCRIPTS = traffic_mngr.sh > >> +TEST_EXTENSIONS = .sh > >> + > >> +TESTS = $(TESTSCRIPTS) > >> + > >> noinst_LTLIBRARIES = libtesttraffic_mngr.la > <http://libtesttraffic_mngr.la> > >> libtesttraffic_mngr_la_SOURCES = traffic_mngr.c > >> > >> -test_PROGRAMS = traffic_mngr_main$(EXEEXT) > >> +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) > >> dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c > >> traffic_mngr_main_LDADD = libtesttraffic_mngr.la > <http://libtesttraffic_mngr.la> -lm $(LIBCUNIT_COMMON) > >> $(LIBODP) > >> > >> -EXTRA_DIST = traffic_mngr.h > >> +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) > >> +dist_check_SCRIPTS = $(TESTSCRIPTS) > >> diff --git > a/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > >> b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > >> new file mode 100755 > >> index 00000000..a7d54162 > >> --- /dev/null > >> +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > >> @@ -0,0 +1,25 @@ > >> +#!/bin/sh > >> +# > >> +# Copyright (c) 2017, Linaro Limited > >> +# All rights reserved. > >> +# > >> +# SPDX-License-Identifier: BSD-3-Clause > >> +# > >> + > >> +# directory where test binaries have been built > >> +TEST_DIR="${TEST_DIR:-$(dirname $0)}" > >> + > >> +# exit codes expected by automake for skipped tests > >> +TEST_SKIPPED=77 > >> + > >> +${TEST_DIR}/traffic_mngr_main${EXEEXT} > >> +ret=$? > >> + > >> +SIGSEGV=139 > >> + > >> +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne > ${SIGSEGV} > >> ]; then > >> + echo "SKIP: skip due to not isolated environment" > >> + exit ${TEST_SKIPPED} > >> +fi > >> + > >> +exit $ret > >> -- > >> 2.11.0.295.gd7dffce > >> > >> > > > >
On Fri, Mar 31, 2017 at 3:39 PM, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > tm test fails time to time in Travis environment. Because > of we can not control that machine we can not do things like > taskset and core isolation there. So simple run test and ignore > it's result. Threat only segfault as actual error. Linaro CI > will still do full test. > https://bugs.linaro.org/show_bug.cgi?id=2881 > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> > Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- > .../validation/api/traffic_mngr/traffic_mngr.sh | 25 > ++++++++++++++++++++++ > 2 files changed, 35 insertions(+), 2 deletions(-) > create mode 100755 test/common_plat/validation/ > api/traffic_mngr/traffic_mngr.sh > > diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am > b/test/common_plat/validation/api/traffic_mngr/Makefile.am > index 35e689a0..a012c1b3 100644 > --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am > +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am > @@ -1,10 +1,18 @@ > include ../Makefile.inc > > +TESTS_ENVIRONMENT += TEST_DIR=${builddir} > + > +TESTSCRIPTS = traffic_mngr.sh > +TEST_EXTENSIONS = .sh > + > +TESTS = $(TESTSCRIPTS) > + > noinst_LTLIBRARIES = libtesttraffic_mngr.la > libtesttraffic_mngr_la_SOURCES = traffic_mngr.c > > -test_PROGRAMS = traffic_mngr_main$(EXEEXT) > +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) > dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c > traffic_mngr_main_LDADD = libtesttraffic_mngr.la -lm $(LIBCUNIT_COMMON) > $(LIBODP) > > -EXTRA_DIST = traffic_mngr.h > +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) > +dist_check_SCRIPTS = $(TESTSCRIPTS) > diff --git a/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > new file mode 100755 > index 00000000..a7d54162 > --- /dev/null > +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh > @@ -0,0 +1,25 @@ > +#!/bin/sh > +# > +# Copyright (c) 2017, Linaro Limited > +# All rights reserved. > +# > +# SPDX-License-Identifier: BSD-3-Clause > +# > + > +# directory where test binaries have been built > +TEST_DIR="${TEST_DIR:-$(dirname $0)}" > + > +# exit codes expected by automake for skipped tests > +TEST_SKIPPED=77 > + > +${TEST_DIR}/traffic_mngr_main${EXEEXT} > +ret=$? > + > +SIGSEGV=139 > + > +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne ${SIGSEGV} ]; > then > + echo "SKIP: skip due to not isolated environment" > + exit ${TEST_SKIPPED} > +fi > + > +exit $ret > -- > 2.11.0.295.gd7dffce > >
diff --git a/test/common_plat/validation/api/traffic_mngr/Makefile.am b/test/common_plat/validation/api/traffic_mngr/Makefile.am index 35e689a0..a012c1b3 100644 --- a/test/common_plat/validation/api/traffic_mngr/Makefile.am +++ b/test/common_plat/validation/api/traffic_mngr/Makefile.am @@ -1,10 +1,18 @@ include ../Makefile.inc +TESTS_ENVIRONMENT += TEST_DIR=${builddir} + +TESTSCRIPTS = traffic_mngr.sh +TEST_EXTENSIONS = .sh + +TESTS = $(TESTSCRIPTS) + noinst_LTLIBRARIES = libtesttraffic_mngr.la libtesttraffic_mngr_la_SOURCES = traffic_mngr.c -test_PROGRAMS = traffic_mngr_main$(EXEEXT) +bin_PROGRAMS = traffic_mngr_main$(EXEEXT) dist_traffic_mngr_main_SOURCES = traffic_mngr_main.c traffic_mngr_main_LDADD = libtesttraffic_mngr.la -lm $(LIBCUNIT_COMMON) $(LIBODP) -EXTRA_DIST = traffic_mngr.h +EXTRA_DIST = traffic_mngr.h $(TESTSCRIPTS) +dist_check_SCRIPTS = $(TESTSCRIPTS) diff --git a/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh new file mode 100755 index 00000000..a7d54162 --- /dev/null +++ b/test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh @@ -0,0 +1,25 @@ +#!/bin/sh +# +# Copyright (c) 2017, Linaro Limited +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause +# + +# directory where test binaries have been built +TEST_DIR="${TEST_DIR:-$(dirname $0)}" + +# exit codes expected by automake for skipped tests +TEST_SKIPPED=77 + +${TEST_DIR}/traffic_mngr_main${EXEEXT} +ret=$? + +SIGSEGV=139 + +if [ "${TRAVIS}" = "true" ] && [ $ret -ne 0 ] && [ $ret -ne ${SIGSEGV} ]; then + echo "SKIP: skip due to not isolated environment" + exit ${TEST_SKIPPED} +fi + +exit $ret
tm test fails time to time in Travis environment. Because of we can not control that machine we can not do things like taskset and core isolation there. So simple run test and ignore it's result. Threat only segfault as actual error. Linaro CI will still do full test. https://bugs.linaro.org/show_bug.cgi?id=2881 Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> --- .../validation/api/traffic_mngr/Makefile.am | 12 +++++++++-- .../validation/api/traffic_mngr/traffic_mngr.sh | 25 ++++++++++++++++++++++ 2 files changed, 35 insertions(+), 2 deletions(-) create mode 100755 test/common_plat/validation/api/traffic_mngr/traffic_mngr.sh -- 2.11.0.295.gd7dffce