| Message ID | 20200928202650.2530280-1-keescook@chromium.org |
|---|---|
| Headers | show |
| Series | Extract run_kselftest.sh and generate stand-alone test list | expand |
On Mon, Sep 28, 2020 at 01:26:47PM -0700, Kees Cook wrote: > v2: > - update documentation > - include SPDX line in extracted script > v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/ > I'm not sure if the doc update are all appropriate. Need others help review. The script part looks good to me. Thanks for your update. Regards Hangbin
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote: > > v2: > - update documentation > - include SPDX line in extracted script > v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/ > > > Hi! > > I really like Hangbin Liu's intent[1] but I think we need to be a little > more clean about the implementation. This extracts run_kselftest.sh from > the Makefile so it can actually be changed without embeds, etc. Instead, > generate the test list into a text file. Everything gets much simpler. > :) > > And in patch 2, I add back Hangbin Liu's new options (with some extra > added) with knowledge of "collections" (i.e. Makefile TARGETS) and > subtests. This should work really well with LAVA too, which needs to > manipulate the lists of tests being run. > > Thoughts? I have tested this patch set on LAVA with full run and it went well. > > -Kees > > [1] https://lore.kernel.org/lkml/20200914022227.437143-1-liuhangbin@gmail.com/ > > Kees Cook (3): > selftests: Extract run_kselftest.sh and generate stand-alone test list > selftests/run_kselftest.sh: Make each test individually selectable > doc: dev-tools: kselftest.rst: Update examples and paths > > Documentation/dev-tools/kselftest.rst | 35 +++++---- > tools/testing/selftests/Makefile | 26 ++----- > tools/testing/selftests/lib.mk | 5 +- > tools/testing/selftests/run_kselftest.sh | 93 ++++++++++++++++++++++++ > 4 files changed, 124 insertions(+), 35 deletions(-) > create mode 100755 tools/testing/selftests/run_kselftest.sh > > -- > 2.25.1 > - Naresh
On Wed, Sep 30, 2020 at 09:47:49PM +0530, Naresh Kamboju wrote: > On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote: > > > > v2: > > - update documentation > > - include SPDX line in extracted script > > v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/ > > > > > > Hi! > > > > I really like Hangbin Liu's intent[1] but I think we need to be a little > > more clean about the implementation. This extracts run_kselftest.sh from > > the Makefile so it can actually be changed without embeds, etc. Instead, > > generate the test list into a text file. Everything gets much simpler. > > :) > > > > And in patch 2, I add back Hangbin Liu's new options (with some extra > > added) with knowledge of "collections" (i.e. Makefile TARGETS) and > > subtests. This should work really well with LAVA too, which needs to > > manipulate the lists of tests being run. > > > > Thoughts? > > I have tested this patch set on LAVA with full run and it went well. Thank you! You can include this as a tag too, so a "b4 am" will pick it up: Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org> -Kees > > > > > -Kees > > > > [1] https://lore.kernel.org/lkml/20200914022227.437143-1-liuhangbin@gmail.com/ > > > > Kees Cook (3): > > selftests: Extract run_kselftest.sh and generate stand-alone test list > > selftests/run_kselftest.sh: Make each test individually selectable > > doc: dev-tools: kselftest.rst: Update examples and paths > > > > Documentation/dev-tools/kselftest.rst | 35 +++++---- > > tools/testing/selftests/Makefile | 26 ++----- > > tools/testing/selftests/lib.mk | 5 +- > > tools/testing/selftests/run_kselftest.sh | 93 ++++++++++++++++++++++++ > > 4 files changed, 124 insertions(+), 35 deletions(-) > > create mode 100755 tools/testing/selftests/run_kselftest.sh > > > > -- > > 2.25.1 > > > > - Naresh
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote: > > Instead of building a script on the fly (which just repeats the same > thing for each test collection), move the script out of the Makefile and > into run_kselftest.sh, which reads kselftest-list.txt. > > Adjust the emit_tests target to report each test on a separate line so > that test running tools (e.g. LAVA) can easily remove individual > tests (for example, as seen in [1]). > > [1] https://github.com/Linaro/test-definitions/pull/208/commits/2e7b62155e4998e54ac0587704932484d4ff84c8 > > Signed-off-by: Kees Cook <keescook@chromium.org> Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org> - Naresh
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote: > > Update the installation commands and path details, detail the new > options available in the run_kselftests.sh script. > > Signed-off-by: Kees Cook <keescook@chromium.org> Reviewed-by: Naresh Kamboju <naresh.kamboju@linaro.org> > --- > Documentation/dev-tools/kselftest.rst | 35 +++++++++++++++++---------- > 1 file changed, 22 insertions(+), 13 deletions(-) > > diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst > index 469d115a95f1..a901def730d9 100644 > --- a/Documentation/dev-tools/kselftest.rst > +++ b/Documentation/dev-tools/kselftest.rst > @@ -125,32 +125,41 @@ Note that some tests will require root privileges. > Install selftests > ================= > > -You can use the kselftest_install.sh tool to install selftests in the > -default location, which is tools/testing/selftests/kselftest, or in a > -user specified location. > +You can use the "install" target of "make" (which calls the `kselftest_install.sh` > +tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`), > +or in a user specified location via the `INSTALL_PATH` "make" variable. > > To install selftests in default location:: > > - $ cd tools/testing/selftests > - $ ./kselftest_install.sh > + $ make -C tools/testing/selftests install > > To install selftests in a user specified location:: > > - $ cd tools/testing/selftests > - $ ./kselftest_install.sh install_dir > + $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path > > Running installed selftests > =========================== > > -Kselftest install as well as the Kselftest tarball provide a script > -named "run_kselftest.sh" to run the tests. > +Found in the install directory, as well as in the Kselftest tarball, > +is a script named `run_kselftest.sh` to run the tests. > > You can simply do the following to run the installed Kselftests. Please > note some tests will require root privileges:: > > - $ cd kselftest > + $ cd kselftest_install > $ ./run_kselftest.sh > > +To see the list of available tests, the `-l` option can be used:: > + > + $ ./run_kselftest.sh -l > + > +The `-c` option can be used to run all the tests from a test collection, or > +the `-t` option for specific single tests. Either can be used multiple times:: > + > + $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep > + > +For other features see the script usage output, seen with the `-h` option. > + > Packaging selftests > =================== > > @@ -160,9 +169,9 @@ different system. To package selftests, run:: > $ make -C tools/testing/selftests gen_tar > > This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By > -default, `.gz` format is used. The tar format can be overridden by specifying > -a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ option > -is supported, such as:: > +default, `.gz` format is used. The tar compression format can be overridden by > +specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ > +option is supported, such as:: > > $ make -C tools/testing/selftests gen_tar FORMAT=.xz > > -- > 2.25.1 > - Naresh