cunit: fix exit status so failures are caught by make check

Running "make check" from the top level currently does not detect test
failures as the individual test applications report an exit status of 0
even when failures occur.

Stop using CU_get_error() as it reports framework errors rather than
test / suite failures, and the CU_cleanup_registry() clears the error
anyway. Instead exit with a failure code on framework errors, and only
exit with 0 if no tests or test suite init/cleanup functions failed.

Signed-off-by: Stuart Haslam <stuart.haslam@arm.com>
---
 test/cunit/odp_init.c  | 23 +++++++++++++----------
 test/cunit/odp_queue.c | 19 +++++++++----------
 2 files changed, 22 insertions(+), 20 deletions(-)

On 20 November 2014 11:26, Stuart Haslam <stuart.haslam@arm.com> wrote:

> Running "make check" from the top level currently does not detect test
> failures as the individual test applications report an exit status of 0
> even when failures occur.
>
> Stop using CU_get_error() as it reports framework errors rather than
> test / suite failures, and the CU_cleanup_registry() clears the error
> anyway. Instead exit with a failure code on framework errors, and only
> exit with 0 if no tests or test suite init/cleanup functions failed.
>
> Signed-off-by: Stuart Haslam <stuart.haslam@arm.com>
> ---
>  test/cunit/odp_init.c  | 23 +++++++++++++----------
>  test/cunit/odp_queue.c | 19 +++++++++----------
>  2 files changed, 22 insertions(+), 20 deletions(-)
>
> diff --git a/test/cunit/odp_init.c b/test/cunit/odp_init.c
> index 88e6235..dcd71ea 100644
> --- a/test/cunit/odp_init.c
> +++ b/test/cunit/odp_init.c
> @@ -33,24 +33,27 @@ static int finalise(void)
>
>  int main(void)
>  {
> +       int ret;
>         CU_pSuite ptr_suite = NULL;
> +
>         /* initialize the CUnit test registry */
>         if (CUE_SUCCESS != CU_initialize_registry())
> -               return CU_get_error();
> +               exit(EXIT_FAILURE);
> +
>

Without a print for the errors failure case, running the test standalone
gives no indication of the problem, the old code had the same issue.
Maybe add fprintf(stderr,"Test framework error\n");



>         /* add a suite to the registry */
>         ptr_suite = CU_add_suite(__FILE__, init, finalise);
> -       if (NULL == ptr_suite) {
> -               CU_cleanup_registry();
> -               return CU_get_error();
> -       }
> +       if (NULL == ptr_suite)
> +               exit(EXIT_FAILURE);
> +
>         /* add the tests to the suite */
> -       if (NULL == CU_ADD_TEST(ptr_suite, test_odp_init_global)) {
> -               CU_cleanup_registry();
> -               return CU_get_error();
> -       }
> +       if (NULL == CU_ADD_TEST(ptr_suite, test_odp_init_global))
> +               exit(EXIT_FAILURE);
> +
>         /* Run all tests using the CUnit Basic interface */
>         CU_basic_set_mode(CU_BRM_VERBOSE);
>         CU_basic_run_tests();
> +       ret = (CU_get_number_of_failure_records() != 0);
>         CU_cleanup_registry();
> -       return CU_get_error();
> +
> +       return ret;
>  }
> diff --git a/test/cunit/odp_queue.c b/test/cunit/odp_queue.c
> index 4d233e0..caabd3b 100644
> --- a/test/cunit/odp_queue.c
> +++ b/test/cunit/odp_queue.c
> @@ -131,27 +131,26 @@ static int finalize(void)
>
>  int main(void)
>  {
> +       int ret;
>         CU_pSuite ptr_suite = NULL;
>
>         /* initialize the CUnit test registry */
>         if (CUE_SUCCESS != CU_initialize_registry())
> -               return CU_get_error();
> +               exit(EXIT_FAILURE);
>
>         /* add the tests to the queue suite */
>         ptr_suite = CU_add_suite(__FILE__, init, finalize);
> -       if (NULL == ptr_suite) {
> -               CU_cleanup_registry();
> -               return CU_get_error();
> -       }
> +       if (NULL == ptr_suite)
> +               exit(EXIT_FAILURE);
>
> -       if (NULL == CU_ADD_TEST(ptr_suite, test_odp_queue_sunnyday)) {
> -               CU_cleanup_registry();
> -               return CU_get_error();
> -       }
> +       if (NULL == CU_ADD_TEST(ptr_suite, test_odp_queue_sunnyday))
> +               exit(EXIT_FAILURE);
>
>         /* Run all tests using the CUnit Basic interface */
>         CU_basic_set_mode(CU_BRM_VERBOSE);
>         CU_basic_run_tests();
> +       ret = (CU_get_number_of_failure_records() != 0);
>         CU_cleanup_registry();
> -       return CU_get_error();
> +
> +       return ret;
>  }
> --
> 2.1.1
>
>
>
> _______________________________________________
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>

On Thu, Nov 20, 2014 at 04:58:35PM +0000, Mike Holmes wrote:
> 
> On 20 November 2014 11:26, Stuart Haslam <stuart.haslam@arm.com> wrote:
> Running "make check" from the top level currently does not detect test
> failures as the individual test applications report an exit status of 0
> even when failures occur.
> 
> Stop using CU_get_error() as it reports framework errors rather than
> test / suite failures, and the CU_cleanup_registry() clears the error
> anyway. Instead exit with a failure code on framework errors, and only
> exit with 0 if no tests or test suite init/cleanup functions failed.
> 
> Signed-off-by: Stuart Haslam <stuart.haslam@arm.com>
> ---
>  test/cunit/odp_init.c  | 23 +++++++++++++----------
>  test/cunit/odp_queue.c | 19 +++++++++----------
>  2 files changed, 22 insertions(+), 20 deletions(-)
> 
> diff --git a/test/cunit/odp_init.c b/test/cunit/odp_init.c
> index 88e6235..dcd71ea 100644
> --- a/test/cunit/odp_init.c
> +++ b/test/cunit/odp_init.c
> @@ -33,24 +33,27 @@ static int finalise(void)
> 
>  int main(void)
>  {
> +       int ret;
>         CU_pSuite ptr_suite = NULL;
> +
>         /* initialize the CUnit test registry */
>         if (CUE_SUCCESS != CU_initialize_registry())
> -               return CU_get_error();
> +               exit(EXIT_FAILURE);
> +
> 
> Without a print for the errors failure case, running the test standalone gives no indication of the problem, the old code had the same issue.
> Maybe add fprintf(stderr,"Test framework error\n");
> 

OK - looks like there's a CU_get_error_msg() which could be printed.