| Message ID | 1438895298-31686-1-git-send-email-mike.holmes@linaro.org |
|---|---|
| State | Accepted |
| Commit | f3debf6776d308695b3dc8394154c7e639c71db8 |
| Headers | show |
On Thu, Aug 6, 2015 at 4:08 PM, Mike Holmes <mike.holmes@linaro.org> wrote: > From: Christophe Milard <christophe.milard@linaro.org> > > Signed-off-by: Christophe Milard <christophe.milard@linaro.org> > Signed-off-by: Mike Holmes <mike.holmes@linaro.org> > Reviewed-and-tested-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > v2 > Add DEPENDENCIES & README, whitespace clean up > > > DEPENDENCIES | 14 +- > README | 5 + > configure.ac | 25 +++ > doc/Makefile.am | 6 +- > doc/implementers-guide/Makefile.am | 10 ++ > doc/implementers-guide/implementers-guide.adoc | 211 > +++++++++++++++++++++++++ > 6 files changed, 267 insertions(+), 4 deletions(-) > create mode 100644 doc/implementers-guide/Makefile.am > create mode 100644 doc/implementers-guide/implementers-guide.adoc > > diff --git a/DEPENDENCIES b/DEPENDENCIES > index e2052bb..852beeb 100644 > --- a/DEPENDENCIES > +++ b/DEPENDENCIES > @@ -111,14 +111,24 @@ Prerequisites for building the OpenDataPlane (ODP) > API > > 5.0 Documentation & Doxygen > > +5.1 API Guide > See http://www.stack.nl/~dimitri/doxygen/manual/install.html > > The tested version of doxygen is 1.8.8 > > -5.1 HTML > +5.1.1 HTML > # Debian/Ubuntu > $ apt-get install doxygen graphviz > > -5.2 PDF > +5.1.2 PDF > # Debian/Ubuntu > $ apt-get install texlive-latex-recommended texlive-latex-extra > texlive-fonts-recommended > + > +5.2 User guides > + > +5.2.1 HTML > + # Ubuntu > + $ apt-get install asciidoc > + > + # Debian > + $ apt-get install asciidoc source-highlight > diff --git a/README b/README > index ccdbed6..cad5138 100644 > --- a/README > +++ b/README > @@ -29,6 +29,11 @@ Main git repository: > ./configure --enable-test-perf --enable-test-vald > make check > > + To generate the users guides: > + ./bootstrap > + ./configure --enable-user-guide > + make > + > Patches tracking system: > http://patches.opendataplane.org/project/lng-odp/list/ > > diff --git a/configure.ac b/configure.ac > index 24a3fa1..57d145c 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -224,6 +224,30 @@ fi > AM_CONDITIONAL([HAVE_DOXYGEN], [test "x${DOXYGEN}" = "xdoxygen"]) > > ########################################################################## > +# Check for asciidoc availability > +########################################################################## > +AC_CHECK_PROGS([ASCIIDOC], [asciidoc]) > +if test -z "$ASCIIDOC"; > + then AC_MSG_WARN([asciidoc not found - continuing without asciidoc > support]) > +fi > + > +########################################################################## > +# Enable/disable user guide generation > +########################################################################## > +user_guides=no > +AC_ARG_ENABLE([user-guides], > + [ --enable-user-guides generate supplemental users guides], > + [if test "x$enableval" = "xyes"; then > + if test -z "$ASCIIDOC"; > + then AC_MSG_ERROR([cannot generate user guides without > asciidoc]) > + else > + user_guides=yes > + fi > + fi]) > + > +AM_CONDITIONAL([user_guide], [test "x${user_guides}" = "xyes" ]) > + > +########################################################################## > # Set optional OpenSSL path > ########################################################################## > AC_ARG_WITH([openssl-path], > @@ -288,6 +312,7 @@ AM_CXXFLAGS="-std=c++11" > > AC_CONFIG_FILES([Makefile > doc/Makefile > + doc/implementers-guide/Makefile > example/Makefile > example/classifier/Makefile > example/generator/Makefile > diff --git a/doc/Makefile.am b/doc/Makefile.am > index d2f123a..73bd8e2 100644 > --- a/doc/Makefile.am > +++ b/doc/Makefile.am > @@ -1,6 +1,8 @@ > if HAVE_DOXYGEN > - > - > clean-local: > rm -rf output > endif > + > +if user_guide > +SUBDIRS = implementers-guide > +endif > diff --git a/doc/implementers-guide/Makefile.am > b/doc/implementers-guide/Makefile.am > new file mode 100644 > index 0000000..25437d3 > --- /dev/null > +++ b/doc/implementers-guide/Makefile.am > @@ -0,0 +1,10 @@ > +TARGET = $(top_srcdir)/doc/output/implementers-guide.html > + > +all-local: $(TARGET) > + > +$(TARGET): implementers-guide.adoc > + @mkdir -p $(top_srcdir)/doc/output > + asciidoc --out-file=$@ $< > + > +clean-local: > + rm -f $(TARGET) > diff --git a/doc/implementers-guide/implementers-guide.adoc > b/doc/implementers-guide/implementers-guide.adoc > new file mode 100644 > index 0000000..56d6e05 > --- /dev/null > +++ b/doc/implementers-guide/implementers-guide.adoc > @@ -0,0 +1,211 @@ > +OpenDataPlane (ODP) Implementers-Guide > +======================================= > +:toc: > + > +:numbered!: > +[abstract] > +Abstract > +-------- > +This document is intended to guide a new ODP developer. > +Further details about ODP may be found at http://opendataplane.org[ODP > homepage] > + > + > +:numbered: > +The validation Suite > +-------------------- > +ODP provides a comprehensive set of API validation tests that are > intended to be used by implementers during development and by application > developers to verify that a particular implementation meets their > requirements. > + > +The list of these tests is expected to grow as ODP grows. > + > +The list of test executables is run by the automake test harness, when > running "make check". > +Therefore, as required by this harness, each executable should return 0 > on success (tests passed), 77 on inconclusive, or any other values on > failure. > +The automake functionality shows a status line (PASSED/FAIL...) for each > of the ran test executables. > + > +It is expected that ODP developers will need to run tests as early as > possible in the development cycle, before all APIs have been implemented. > +Besides, although there are no APIs that are formally listed as optional, > it is also expected that there may be cases where a subset of APIs remain > unimplemented on a particular platform. > +Moreover, some platforms may require specific initialization/termination > code prior/after running the standard tests. > + > +To accommodate with these platform disparities, the ODP validation has > been divided in two distinct areas: > + > +* The platform agnostic area, > +* A platform dependent area (one per platform). > + > +Platform agnostic > +~~~~~~~~~~~~~~~~~ > +This grouping defines tests that are expected to be executable and > succeed on any platform, though possibly with very different performances, > depending on the underlying platform. > +They are written in plain C code, and may only use functions defined in > the standard libC library (besides the ODP functions being tested, of > course). > +No other languages (like scripting) are allowed as their usage would make > assumptions on the platform capability. > + > +This area is located at: '<ODP_ROOT>/test/validation/' > + > +The ODP API itself is ordered by module, where each module groups the set > of ODP API functions related to the same "topic". > +Examples of modules includes "classification" (API functions dealing with > ingres packets classification), time (functions dealing with time, > excluding timers which have their own module), timer,... > +The complete module list can be seen at: > http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP > Modules] + > +Within the platform agnostic area, the tests are also grouped by modules, > matching the ODP API modules: '<ODP_ROOT>/test/validation/' mainly contains > a list of directories matching each module name (as defined by the doxygen > "@defgroup" or "@ingroup" statement present in each API ".h" file). > + > +Within each of these directories, a library (called "lib<module>.la") and > its associated ".h" file (called "<module>.h") defines all the test > functions for this module as well as few other functions to initialize, > terminate, and group the tests. > +An executable called "<module>_main*", is also built. It is permissible > to generate more than one executable to cover the functionality in the test > library for the module. > +These executable(s) shall call all the tests for this module. + > +See <<anchor-1, Module test and naming convention>> for more details. > + > +It is important to be aware that the tests defined for a given module > (defined in 'test/validation/<module>') are focused to test the ODP > functions belonging to this module, but are not limited to use this > module's ODP functions only: many modules needs some interaction with some > other module to be tested. > +The obvious illustration of this is for module "init" whose functions are > required by all tests of all other modules (as ODP needs to be initialized > to test anything else). + > + > +There is a "Makefile.am" located at the top of the platform agnostic > area. Its role is limited to the construction of the different test > libraries and the "<module>_main*" executables. No tests are run from this > area when "make check" is performed. > + > +A convenience library '.../test/validation/libodptests.la' (and its > associated .h file, '.../test/validation/odptest.h') regrouping all tests > symbols of all modules may be built in the future. (The superlib) > + > +C_UNIT > +^^^^^^ > +Within a given test executable C_UNIT is used to run the different tests. > The usage of C_UNIT implies the following structure: > + > +* Tests are simple C functions. > +* Tests are grouped in arrays called test suites. Each test suite can be > associated with a suite initialization/termination function(s), called by > C_UNIT before and after the whole suite is ran. > +* An array of test suites (and associated init/term functions) defines > the test registry ran by the test executable. > + > +Moreover, two extra functions can be used to initialize/terminate the > test executable (these are not part of C_UNIT). + > +A test executable return success (0) if every tests of each suite succeed. > + > +More details about http://cunit.sourceforge.net/doc/index.html[C_Unit > users guide] > + > +[[anchor-1]] > +Module test and naming convention > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +* Tests, i.e. C functions which are used in CUNIT test suites are named: > + *<Module>_test_+++*+++* + > + where the suffix idendify the test. > + > +* Test arrays, i.e. arrays of CU_TestInfo, listing the test functions > belonging to a suite, are called: > + *<Module>_suite+++[_*]+++* + > + where the possible suffix can be used if many suites are declared. > + > +* CUNIT suite init and termination functions are called: > + *<Module>+++_suite[_*]_init()+++* and > *<Module>+++_suite[_*]_term()+++* respectively. + > + where the possible extra middle pattern can be used if many suites are > declared. > + > +* Suite arrays, i.e. arrays of CU_SuiteInfo used in executables (C_UNIT > registry) are called: > + *<Module>+++_suites[_*]+++* + > + where the possible suffix identifies the executable using it, if many. > + > +* Main executable function(s), are called: > + *<Module>+++_main[_*]+++* + > + where the possible suffix identifies the executable, if many, using it. > + > +* Init/term functions for the whole executable are called: > + *<Module>_init* > + *<Module>_term* > + > +All the above symbols are part of the generated lib<Module>.la libraries. > The generated main executable(s) (named <module>_+++main[_*]+++, where the > optional suffix is used to distinguish the executables belonging to the > same module, if many) simply call(s) the related <Module>_main+++[_*]+++ > from the library. > + > +Platform specific > +~~~~~~~~~~~~~~~~~ > +These tests are located under '<ODP_ROOT>/platform/<platform>/test'. > There is one such area for each platform implementing ODP. > +This location will be referred as <PLATFORM_SPECIFIC> in the rest of this > document. > + > +The normal case > +^^^^^^^^^^^^^^^ > +If the considered platform needs nothing specific to be tested this > directory will contain a single "Makefile.am". > +This "Makefile.am" then only lists the executables to be run on "make > check" (in the automake TEST variable): when the platform has nothing > specific to it, this just list the "<module>_main+++[_*]+++" executables, > picked from the platform agnostic area. > +For the linux-generic platform, most tested modules fall into this > category: currently, the > '<ODP_ROOT>/platform/linux-generic/test/Makefile.am' looks as follows: > + > +[source,am] > +---- > +include $(top_srcdir)/test/Makefile.inc > +TESTS_ENVIRONMENT += TEST_DIR=${top_builddir}/test/validation > + > +ODP_MODULES = pktio > + > +if test_vald > +TESTS = pktio/pktio_run \ > + ${top_builddir}/test/validation/buffer/buffer_main$(EXEEXT) \ > + > ${top_builddir}/test/validation/classification/classification_main$(EXEEXT) > \ > + ${top_builddir}/test/validation/cpumask/cpumask_main$(EXEEXT) \ > + ${top_builddir}/test/validation/crypto/crypto_main$(EXEEXT) \ > + ${top_builddir}/test/validation/errno/errno_main$(EXEEXT) \ > + ${top_builddir}/test/validation/init/init_main_ok$(EXEEXT) \ > + ${top_builddir}/test/validation/init/init_main_abort$(EXEEXT) \ > + ${top_builddir}/test/validation/init/init_main_log$(EXEEXT) \ > + ${top_builddir}/test/validation/packet/packet_main$(EXEEXT) \ > + ${top_builddir}/test/validation/pool/pool_main$(EXEEXT) \ > + ${top_builddir}/test/validation/queue/queue_main$(EXEEXT) \ > + ${top_builddir}/test/validation/random/random_main$(EXEEXT) \ > + ${top_builddir}/test/validation/scheduler/scheduler_main$(EXEEXT) \ > + > ${top_builddir}/test/validation/synchronizers/synchronizers_main$(EXEEXT) \ > + ${top_builddir}/test/validation/thread/thread_main$(EXEEXT) \ > + ${top_builddir}/test/validation/time/time_main$(EXEEXT) \ > + ${top_builddir}/test/validation/timer/timer_main$(EXEEXT) \ > + ${top_builddir}/test/validation/shmem/shmem_main$(EXEEXT) \ > + ${top_builddir}/test/validation/system/system_main$(EXEEXT) > + > +SUBDIRS = $(ODP_MODULES) > +endif > + > +---- > + > +With the exception for module pktio, all other modules testing just > involves calling the platform agnostic <module>_main executables (in > test/validation). > + > +When no platform specific testing is required, the > '<PLATFORM_SPECIFIC>/Makefile.am' is used to list the tests executables to > be run only, as these tests are actually built from the platform agnostic > side by the 'test/validation/Makefile.am' (and subdirectories). > '<PLATFORM_SPECIFIC>/Makefile.am' is involved in building only when > platform specific tests exists, as discussed below. > + > +Using other languages > +^^^^^^^^^^^^^^^^^^^^^ > +The pktio module, above, is actually tested using a bash script. This > script is needed to set up the interfaces used by the tests. The pktio_run > script actually eventually calls the platform agnostic > 'test/validation/pktio/pktio_main' after setting up the interfaces needed > by the tests. > +Notice that the path to the script is > '<PLATFORM_SPECIFIC>/pktio/pktio_run', i.e. it is private to this platform. > Any languages supported by the tested platform can be used there, as it > will not impact other platforms. > +The platform "private" executables (such as this script), of course, must > also return one of the return code expected by the automake test harness (0 > for success, 77 for inconclusive, other values for errors). > + > +Defining test wrappers > +^^^^^^^^^^^^^^^^^^^^^^ > +The pktio case above is actually using a script as wrapper around the > "standard" (platform independent) test executable. Wrappers can also be > defined by using the LOG_COMPILER variable of automake. > +This is applicable in cases where the same wrapper should be used for > more then one test, as the test name is passed has parameter to the > wrapper. A wrapper is just a program expecting one argument: the test name. > + > +Automake also supports the usage different wrappers based of the > executable filename suffix. See > https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] > for more information. > + > +To add a wrapper around the executed test, just add the following > LOG_COMPILER definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': > + > +[source,am] > +---- > +... > +if test_vald > +LOG_COMPILER = $(top_srcdir)/platform/linux-generic/test/wrapper-script > +TESTS = pktio/pktio_run \ > +... > +---- > + > +Here follows a dummy example of what wrapper-script could be: > + > +[source,bash] > +---- > +#!/bin/bash > + > +# The parameter, $1, is the name of the test executable to run > +echo "WRAPPER!!!" > +echo "running $1!" > + > +# run the test: > +$1 > +# remember the test result: > +res=$? > + > +echo "Do something to clean up the mess here :-)" > +# return the test result. > +exit $res > +---- > + > +Note how the above script stores the return code of the test executable > to return it properly to the automake test harness. > + > +Defining platform specific tests > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > +Sometimes, it may be necessary to call platform specific system calls to > check some functionality: For instance, testing odp_cpumask_* could involve > checking the underlying system CPU mask. On linux, such a test would > require using the CPU_ISSET macro, which is linux specific. Such a test > would be written in '<PLATFORM_SPECIFIC>/cpumask/...' The contents of this > directory would be very similar to the contents of the platform agnostic > side cpu_mask tests (including a Makefile.am...), but platform specific > test would be written there. > +'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the > platform specific tests (by listing their module name in SUBDIRS and > therefore calling the appropriate Makefile.am) and then it would call both > the platform agnostic executable(s) and the platform specific test > executable. > + > +Skipping tests during development > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > +During ODP development, it may be useful to skip some test. This can be > achieved by creating a new test executable (still on the platform side), > picking up the required tests from the platform agnostic lib<module>.la. > + > +The top Makefile would then call only the platform specific executable, > hence skipping the tests which have been omitted. > + > +TIP: You probably want to copy the platform-agnostic module main function > and prune it from the undesired tests when writing your own platform > specific main, for a given module. > + > +Permanently skipping test > +^^^^^^^^^^^^^^^^^^^^^^^^^^ > +If a platform wants to permanently skip a test (i.e. a part of the ODP > API is and will not be supported on that platform), it is recommended to > use the function odp_cunit_TBD() to removed the tests or suite from the > list of tests. This gives a chance to the test environment to trace this > removal. > -- > 2.1.4 > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/lng-odp >
diff --git a/DEPENDENCIES b/DEPENDENCIES index e2052bb..852beeb 100644 --- a/DEPENDENCIES +++ b/DEPENDENCIES @@ -111,14 +111,24 @@ Prerequisites for building the OpenDataPlane (ODP) API 5.0 Documentation & Doxygen +5.1 API Guide See http://www.stack.nl/~dimitri/doxygen/manual/install.html The tested version of doxygen is 1.8.8 -5.1 HTML +5.1.1 HTML # Debian/Ubuntu $ apt-get install doxygen graphviz -5.2 PDF +5.1.2 PDF # Debian/Ubuntu $ apt-get install texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended + +5.2 User guides + +5.2.1 HTML + # Ubuntu + $ apt-get install asciidoc + + # Debian + $ apt-get install asciidoc source-highlight diff --git a/README b/README index ccdbed6..cad5138 100644 --- a/README +++ b/README @@ -29,6 +29,11 @@ Main git repository: ./configure --enable-test-perf --enable-test-vald make check + To generate the users guides: + ./bootstrap + ./configure --enable-user-guide + make + Patches tracking system: http://patches.opendataplane.org/project/lng-odp/list/ diff --git a/configure.ac b/configure.ac index 24a3fa1..57d145c 100644 --- a/configure.ac +++ b/configure.ac @@ -224,6 +224,30 @@ fi AM_CONDITIONAL([HAVE_DOXYGEN], [test "x${DOXYGEN}" = "xdoxygen"]) ########################################################################## +# Check for asciidoc availability +########################################################################## +AC_CHECK_PROGS([ASCIIDOC], [asciidoc]) +if test -z "$ASCIIDOC"; + then AC_MSG_WARN([asciidoc not found - continuing without asciidoc support]) +fi + +########################################################################## +# Enable/disable user guide generation +########################################################################## +user_guides=no +AC_ARG_ENABLE([user-guides], + [ --enable-user-guides generate supplemental users guides], + [if test "x$enableval" = "xyes"; then + if test -z "$ASCIIDOC"; + then AC_MSG_ERROR([cannot generate user guides without asciidoc]) + else + user_guides=yes + fi + fi]) + +AM_CONDITIONAL([user_guide], [test "x${user_guides}" = "xyes" ]) + +########################################################################## # Set optional OpenSSL path ########################################################################## AC_ARG_WITH([openssl-path], @@ -288,6 +312,7 @@ AM_CXXFLAGS="-std=c++11" AC_CONFIG_FILES([Makefile doc/Makefile + doc/implementers-guide/Makefile example/Makefile example/classifier/Makefile example/generator/Makefile diff --git a/doc/Makefile.am b/doc/Makefile.am index d2f123a..73bd8e2 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -1,6 +1,8 @@ if HAVE_DOXYGEN - - clean-local: rm -rf output endif + +if user_guide +SUBDIRS = implementers-guide +endif diff --git a/doc/implementers-guide/Makefile.am b/doc/implementers-guide/Makefile.am new file mode 100644 index 0000000..25437d3 --- /dev/null +++ b/doc/implementers-guide/Makefile.am @@ -0,0 +1,10 @@ +TARGET = $(top_srcdir)/doc/output/implementers-guide.html + +all-local: $(TARGET) + +$(TARGET): implementers-guide.adoc + @mkdir -p $(top_srcdir)/doc/output + asciidoc --out-file=$@ $< + +clean-local: + rm -f $(TARGET) diff --git a/doc/implementers-guide/implementers-guide.adoc b/doc/implementers-guide/implementers-guide.adoc new file mode 100644 index 0000000..56d6e05 --- /dev/null +++ b/doc/implementers-guide/implementers-guide.adoc @@ -0,0 +1,211 @@ +OpenDataPlane (ODP) Implementers-Guide +======================================= +:toc: + +:numbered!: +[abstract] +Abstract +-------- +This document is intended to guide a new ODP developer. +Further details about ODP may be found at http://opendataplane.org[ODP homepage] + + +:numbered: +The validation Suite +-------------------- +ODP provides a comprehensive set of API validation tests that are intended to be used by implementers during development and by application developers to verify that a particular implementation meets their requirements. + +The list of these tests is expected to grow as ODP grows. + +The list of test executables is run by the automake test harness, when running "make check". +Therefore, as required by this harness, each executable should return 0 on success (tests passed), 77 on inconclusive, or any other values on failure. +The automake functionality shows a status line (PASSED/FAIL...) for each of the ran test executables. + +It is expected that ODP developers will need to run tests as early as possible in the development cycle, before all APIs have been implemented. +Besides, although there are no APIs that are formally listed as optional, it is also expected that there may be cases where a subset of APIs remain unimplemented on a particular platform. +Moreover, some platforms may require specific initialization/termination code prior/after running the standard tests. + +To accommodate with these platform disparities, the ODP validation has been divided in two distinct areas: + +* The platform agnostic area, +* A platform dependent area (one per platform). + +Platform agnostic +~~~~~~~~~~~~~~~~~ +This grouping defines tests that are expected to be executable and succeed on any platform, though possibly with very different performances, depending on the underlying platform. +They are written in plain C code, and may only use functions defined in the standard libC library (besides the ODP functions being tested, of course). +No other languages (like scripting) are allowed as their usage would make assumptions on the platform capability. + +This area is located at: '<ODP_ROOT>/test/validation/' + +The ODP API itself is ordered by module, where each module groups the set of ODP API functions related to the same "topic". +Examples of modules includes "classification" (API functions dealing with ingres packets classification), time (functions dealing with time, excluding timers which have their own module), timer,... +The complete module list can be seen at: http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP Modules] + +Within the platform agnostic area, the tests are also grouped by modules, matching the ODP API modules: '<ODP_ROOT>/test/validation/' mainly contains a list of directories matching each module name (as defined by the doxygen "@defgroup" or "@ingroup" statement present in each API ".h" file). + +Within each of these directories, a library (called "lib<module>.la") and its associated ".h" file (called "<module>.h") defines all the test functions for this module as well as few other functions to initialize, terminate, and group the tests. +An executable called "<module>_main*", is also built. It is permissible to generate more than one executable to cover the functionality in the test library for the module. +These executable(s) shall call all the tests for this module. + +See <<anchor-1, Module test and naming convention>> for more details. + +It is important to be aware that the tests defined for a given module (defined in 'test/validation/<module>') are focused to test the ODP functions belonging to this module, but are not limited to use this module's ODP functions only: many modules needs some interaction with some other module to be tested. +The obvious illustration of this is for module "init" whose functions are required by all tests of all other modules (as ODP needs to be initialized to test anything else). + + +There is a "Makefile.am" located at the top of the platform agnostic area. Its role is limited to the construction of the different test libraries and the "<module>_main*" executables. No tests are run from this area when "make check" is performed. + +A convenience library '.../test/validation/libodptests.la' (and its associated .h file, '.../test/validation/odptest.h') regrouping all tests symbols of all modules may be built in the future. (The superlib) + +C_UNIT +^^^^^^ +Within a given test executable C_UNIT is used to run the different tests. The usage of C_UNIT implies the following structure: + +* Tests are simple C functions. +* Tests are grouped in arrays called test suites. Each test suite can be associated with a suite initialization/termination function(s), called by C_UNIT before and after the whole suite is ran. +* An array of test suites (and associated init/term functions) defines the test registry ran by the test executable. + +Moreover, two extra functions can be used to initialize/terminate the test executable (these are not part of C_UNIT). + +A test executable return success (0) if every tests of each suite succeed. + +More details about http://cunit.sourceforge.net/doc/index.html[C_Unit users guide] + +[[anchor-1]] +Module test and naming convention +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* Tests, i.e. C functions which are used in CUNIT test suites are named: + *<Module>_test_+++*+++* + + where the suffix idendify the test. + +* Test arrays, i.e. arrays of CU_TestInfo, listing the test functions belonging to a suite, are called: + *<Module>_suite+++[_*]+++* + + where the possible suffix can be used if many suites are declared. + +* CUNIT suite init and termination functions are called: + *<Module>+++_suite[_*]_init()+++* and *<Module>+++_suite[_*]_term()+++* respectively. + + where the possible extra middle pattern can be used if many suites are declared. + +* Suite arrays, i.e. arrays of CU_SuiteInfo used in executables (C_UNIT registry) are called: + *<Module>+++_suites[_*]+++* + + where the possible suffix identifies the executable using it, if many. + +* Main executable function(s), are called: + *<Module>+++_main[_*]+++* + + where the possible suffix identifies the executable, if many, using it. + +* Init/term functions for the whole executable are called: + *<Module>_init* + *<Module>_term* + +All the above symbols are part of the generated lib<Module>.la libraries. The generated main executable(s) (named <module>_+++main[_*]+++, where the optional suffix is used to distinguish the executables belonging to the same module, if many) simply call(s) the related <Module>_main+++[_*]+++ from the library. + +Platform specific +~~~~~~~~~~~~~~~~~ +These tests are located under '<ODP_ROOT>/platform/<platform>/test'. There is one such area for each platform implementing ODP. +This location will be referred as <PLATFORM_SPECIFIC> in the rest of this document. + +The normal case +^^^^^^^^^^^^^^^ +If the considered platform needs nothing specific to be tested this directory will contain a single "Makefile.am". +This "Makefile.am" then only lists the executables to be run on "make check" (in the automake TEST variable): when the platform has nothing specific to it, this just list the "<module>_main+++[_*]+++" executables, picked from the platform agnostic area. +For the linux-generic platform, most tested modules fall into this category: currently, the '<ODP_ROOT>/platform/linux-generic/test/Makefile.am' looks as follows: + +[source,am] +---- +include $(top_srcdir)/test/Makefile.inc +TESTS_ENVIRONMENT += TEST_DIR=${top_builddir}/test/validation + +ODP_MODULES = pktio + +if test_vald +TESTS = pktio/pktio_run \ + ${top_builddir}/test/validation/buffer/buffer_main$(EXEEXT) \ + ${top_builddir}/test/validation/classification/classification_main$(EXEEXT) \ + ${top_builddir}/test/validation/cpumask/cpumask_main$(EXEEXT) \ + ${top_builddir}/test/validation/crypto/crypto_main$(EXEEXT) \ + ${top_builddir}/test/validation/errno/errno_main$(EXEEXT) \ + ${top_builddir}/test/validation/init/init_main_ok$(EXEEXT) \ + ${top_builddir}/test/validation/init/init_main_abort$(EXEEXT) \ + ${top_builddir}/test/validation/init/init_main_log$(EXEEXT) \ + ${top_builddir}/test/validation/packet/packet_main$(EXEEXT) \ + ${top_builddir}/test/validation/pool/pool_main$(EXEEXT) \ + ${top_builddir}/test/validation/queue/queue_main$(EXEEXT) \ + ${top_builddir}/test/validation/random/random_main$(EXEEXT) \ + ${top_builddir}/test/validation/scheduler/scheduler_main$(EXEEXT) \ + ${top_builddir}/test/validation/synchronizers/synchronizers_main$(EXEEXT) \ + ${top_builddir}/test/validation/thread/thread_main$(EXEEXT) \ + ${top_builddir}/test/validation/time/time_main$(EXEEXT) \ + ${top_builddir}/test/validation/timer/timer_main$(EXEEXT) \ + ${top_builddir}/test/validation/shmem/shmem_main$(EXEEXT) \ + ${top_builddir}/test/validation/system/system_main$(EXEEXT) + +SUBDIRS = $(ODP_MODULES) +endif + +---- + +With the exception for module pktio, all other modules testing just involves calling the platform agnostic <module>_main executables (in test/validation). + +When no platform specific testing is required, the '<PLATFORM_SPECIFIC>/Makefile.am' is used to list the tests executables to be run only, as these tests are actually built from the platform agnostic side by the 'test/validation/Makefile.am' (and subdirectories). '<PLATFORM_SPECIFIC>/Makefile.am' is involved in building only when platform specific tests exists, as discussed below. + +Using other languages +^^^^^^^^^^^^^^^^^^^^^ +The pktio module, above, is actually tested using a bash script. This script is needed to set up the interfaces used by the tests. The pktio_run script actually eventually calls the platform agnostic 'test/validation/pktio/pktio_main' after setting up the interfaces needed by the tests. +Notice that the path to the script is '<PLATFORM_SPECIFIC>/pktio/pktio_run', i.e. it is private to this platform. Any languages supported by the tested platform can be used there, as it will not impact other platforms. +The platform "private" executables (such as this script), of course, must also return one of the return code expected by the automake test harness (0 for success, 77 for inconclusive, other values for errors). + +Defining test wrappers +^^^^^^^^^^^^^^^^^^^^^^ +The pktio case above is actually using a script as wrapper around the "standard" (platform independent) test executable. Wrappers can also be defined by using the LOG_COMPILER variable of automake. +This is applicable in cases where the same wrapper should be used for more then one test, as the test name is passed has parameter to the wrapper. A wrapper is just a program expecting one argument: the test name. + +Automake also supports the usage different wrappers based of the executable filename suffix. See https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] for more information. + +To add a wrapper around the executed test, just add the following LOG_COMPILER definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': + +[source,am] +---- +... +if test_vald +LOG_COMPILER = $(top_srcdir)/platform/linux-generic/test/wrapper-script +TESTS = pktio/pktio_run \ +... +---- + +Here follows a dummy example of what wrapper-script could be: + +[source,bash] +---- +#!/bin/bash + +# The parameter, $1, is the name of the test executable to run +echo "WRAPPER!!!" +echo "running $1!" + +# run the test: +$1 +# remember the test result: +res=$? + +echo "Do something to clean up the mess here :-)" +# return the test result. +exit $res +---- + +Note how the above script stores the return code of the test executable to return it properly to the automake test harness. + +Defining platform specific tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Sometimes, it may be necessary to call platform specific system calls to check some functionality: For instance, testing odp_cpumask_* could involve checking the underlying system CPU mask. On linux, such a test would require using the CPU_ISSET macro, which is linux specific. Such a test would be written in '<PLATFORM_SPECIFIC>/cpumask/...' The contents of this directory would be very similar to the contents of the platform agnostic side cpu_mask tests (including a Makefile.am...), but platform specific test would be written there. +'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the platform specific tests (by listing their module name in SUBDIRS and therefore calling the appropriate Makefile.am) and then it would call both the platform agnostic executable(s) and the platform specific test executable. + +Skipping tests during development +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +During ODP development, it may be useful to skip some test. This can be achieved by creating a new test executable (still on the platform side), picking up the required tests from the platform agnostic lib<module>.la. + +The top Makefile would then call only the platform specific executable, hence skipping the tests which have been omitted. + +TIP: You probably want to copy the platform-agnostic module main function and prune it from the undesired tests when writing your own platform specific main, for a given module. + +Permanently skipping test +^^^^^^^^^^^^^^^^^^^^^^^^^^ +If a platform wants to permanently skip a test (i.e. a part of the ODP API is and will not be supported on that platform), it is recommended to use the function odp_cunit_TBD() to removed the tests or suite from the list of tests. This gives a chance to the test environment to trace this removal.