[1/2] test: describe naming conventions for macro UNIT_TEST

Strict naming conventions have to be followed for Python function
generate_ut_subtest() to collect C unit tests to be executed via
command 'ut'.

Describe the requirements both on the C as well on the Python side.

Signed-off-by: Heinrich Schuchardt <xypron.glpk at gmx.de>
---
 include/test/test.h      | 24 +++++++++++++++++++++++-
 test/py/tests/test_ut.py | 17 ++++++++++++++++-
 2 files changed, 39 insertions(+), 2 deletions(-)

--
2.26.2

On 5/6/20 10:26 AM, Heinrich Schuchardt wrote:
> Strict naming conventions have to be followed for Python function
> generate_ut_subtest() to collect C unit tests to be executed via
> command 'ut'.
> 
> Describe the requirements both on the C as well on the Python side.

> +/**
> + * UNIT_TEST() - create linker generated list entry for unit a unit test
> + *
> + * The macro UNIT_TEST() is used to create a linker generated list entry. These
> + * list entries are enumerate tests that can be execute using the ut command.
> + * The list entries are used both by the implementation of the ut command as
> + * well as in a related Python test.
> + *
> + * For Python testing the subtests are collected in Python function
> + * generate_ut_subtest() by applying a regular expression to the lines of file
> + * u-boot.sym. The list entries have to follow strict naming conventions to be
> + * matched by the expression.
> + *
> + * Use UNIT_TEST(foo_test_bar, _flags, foo_test) for a test bar in test suite
> + * foo that can be executed via command 'ut foo bar' and is implemented in
> + * function foo_test_bar().
> + *
> + * @_name:	concatenation of name of the test suite, "_test_", and the name
> + *		of the test
> + * @_flags:	an integer field that can be evaluated by the test suite
> + *		implementation
> + * @_suite:	name of the test suite concatenated with "_test"
> + */

Perhaps the macro could simply take "foo" and "bar" as parameters, and
generate the function name foo_test_bar internally rather than having
the user pass it in. That way, compilation will actively fail if the
function isn't named correctly, since it won't match the reference
created by this macro.

To help make this easier, we could add another macro e.g.
UNIT_TEST_FUNC() that evaluates to just the expected function name, so
that people wouldn't have to know the naming convention when they
implement the function; they'd just write e.g.:

static int UNIT_TEST_FUNC(log, nolog_err)(struct unit_test_state *uts)

But certainly this series is a good first step, and fine even if we
don't implement this suggestion.

The series,
Reviewed-by: Stephen Warren <swarren at nvidia.com

On Wed, 6 May 2020 at 17:28, Stephen Warren <swarren at wwwdotorg.org> wrote:
>
> On 5/6/20 10:26 AM, Heinrich Schuchardt wrote:
> > Strict naming conventions have to be followed for Python function
> > generate_ut_subtest() to collect C unit tests to be executed via
> > command 'ut'.
> >
> > Describe the requirements both on the C as well on the Python side.
>
> > +/**
> > + * UNIT_TEST() - create linker generated list entry for unit a unit test
> > + *
> > + * The macro UNIT_TEST() is used to create a linker generated list entry. These
> > + * list entries are enumerate tests that can be execute using the ut command.
> > + * The list entries are used both by the implementation of the ut command as
> > + * well as in a related Python test.
> > + *
> > + * For Python testing the subtests are collected in Python function
> > + * generate_ut_subtest() by applying a regular expression to the lines of file
> > + * u-boot.sym. The list entries have to follow strict naming conventions to be
> > + * matched by the expression.
> > + *
> > + * Use UNIT_TEST(foo_test_bar, _flags, foo_test) for a test bar in test suite
> > + * foo that can be executed via command 'ut foo bar' and is implemented in
> > + * function foo_test_bar().
> > + *
> > + * @_name:   concatenation of name of the test suite, "_test_", and the name
> > + *           of the test
> > + * @_flags:  an integer field that can be evaluated by the test suite
> > + *           implementation
> > + * @_suite:  name of the test suite concatenated with "_test"
> > + */
>
> Perhaps the macro could simply take "foo" and "bar" as parameters, and
> generate the function name foo_test_bar internally rather than having
> the user pass it in. That way, compilation will actively fail if the
> function isn't named correctly, since it won't match the reference
> created by this macro.
>
> To help make this easier, we could add another macro e.g.
> UNIT_TEST_FUNC() that evaluates to just the expected function name, so
> that people wouldn't have to know the naming convention when they
> implement the function; they'd just write e.g.:
>
> static int UNIT_TEST_FUNC(log, nolog_err)(struct unit_test_state *uts)

I am not a huge fan of that. It looks weird to have the function name
auto-generated, and it defeats ctags, code search, etc.. Another
option might be to check for exported test/ functions that don't match
and print a warning?

>
> But certainly this series is a good first step, and fine even if we
> don't implement this suggestion.

Yes definitely.

>
> The series,
> Reviewed-by: Stephen Warren <swarren at nvidia.com

Reviewed-by: Simon Glass <sjg at chromium.org>

On Wed, May 06, 2020 at 06:26:07PM +0200, Heinrich Schuchardt wrote:

> Strict naming conventions have to be followed for Python function
> generate_ut_subtest() to collect C unit tests to be executed via
> command 'ut'.
> 
> Describe the requirements both on the C as well on the Python side.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk at gmx.de>
> Reviewed-by: Stephen Warren <swarren at nvidia.com
> Reviewed-by: Simon Glass <sjg at chromium.org>

Applied to u-boot/master, thanks!