diff mbox series

[2/2] selftests/run_kselftest.sh: Make each test individually selectable

Message ID 20200925234527.1885234-3-keescook@chromium.org
State Accepted
Commit 5da1918446a1d50d57f2f6062f7fdede0b052473
Headers show
Series [1/2] selftests: Extract run_kselftest.sh and generate stand-alone test list | expand

Commit Message

Kees Cook Sept. 25, 2020, 11:45 p.m. UTC 
Currently with run_kselftest.sh there is no way to choose which test
we could run. All the tests listed in kselftest-list.txt are all run
every time. This patch enhanced the run_kselftest.sh to make the test
collections (or tests) individually selectable. e.g.:

$ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep

Additionally adds a way to list all known tests with "-l", usage
with "-h", and perform a dry run without running tests with "-n".

Co-developed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
---
 tools/testing/selftests/run_kselftest.sh | 77 ++++++++++++++++++++++--
 1 file changed, 71 insertions(+), 6 deletions(-)

Comments

Hangbin Liu Sept. 27, 2020, 2:48 a.m. UTC | #1 
On Fri, Sep 25, 2020 at 04:45:27PM -0700, Kees Cook wrote:
> Currently with run_kselftest.sh there is no way to choose which test
> we could run. All the tests listed in kselftest-list.txt are all run
> every time. This patch enhanced the run_kselftest.sh to make the test
> collections (or tests) individually selectable. e.g.:
> 
> $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
> 
> Additionally adds a way to list all known tests with "-l", usage
> with "-h", and perform a dry run without running tests with "-n".

This is better than my previous patch and we can modify run_kselftest.sh
easily. The Documentation/dev-tools/kselftest.rst should also be update.

Thanks
Hangbin
Kees Cook Sept. 28, 2020, 8:09 p.m. UTC | #2 
On Sun, Sep 27, 2020 at 10:48:40AM +0800, Hangbin Liu wrote:
> On Fri, Sep 25, 2020 at 04:45:27PM -0700, Kees Cook wrote:

> > Currently with run_kselftest.sh there is no way to choose which test

> > we could run. All the tests listed in kselftest-list.txt are all run

> > every time. This patch enhanced the run_kselftest.sh to make the test

> > collections (or tests) individually selectable. e.g.:

> > 

> > $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep

> > 

> > Additionally adds a way to list all known tests with "-l", usage

> > with "-h", and perform a dry run without running tests with "-n".

> 

> This is better than my previous patch and we can modify run_kselftest.sh

> easily. The Documentation/dev-tools/kselftest.rst should also be update.


Thanks! I will send a v2.

-- 
Kees Cook
diff mbox series

Patch

diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
index e07344be28ae..c08089f80e28 100755
--- a/tools/testing/selftests/run_kselftest.sh
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -4,21 +4,86 @@  cd $BASE_DIR
 TESTS="$BASE_DIR"/kselftest-list.txt
 if [ ! -r "$TESTS" ] ; then
 	echo "$0: Could not find list of tests to run ($TESTS)" >&2
-	exit 1
+	available=""
+else
+	available="$(cat "$TESTS")"
 fi
-available="$(cat "$TESTS")"
 
 . ./kselftest/runner.sh
 ROOT=$PWD
 
-if [ "$1" = "--summary" ] ; then
-	logfile="$BASE_DIR"/output.log
-	cat /dev/null > $logfile
+usage()
+{
+	cat <<EOF
+Usage: $0 [OPTIONS]
+  -s | --summary		Print summary with detailed log in output.log
+  -t | --test COLLECTION:TEST	Run TEST from COLLECTION
+  -c | --collection COLLECTION	Run all tests from COLLECTION
+  -l | --list			List the available collection:test entries
+  -d | --dry-run		Don't actually run any tests
+  -h | --help			Show this usage info
+EOF
+	exit $1
+}
+
+COLLECTIONS=""
+TESTS=""
+dryrun=""
+while true; do
+	case "$1" in
+		-s | --summary)
+			logfile="$BASE_DIR"/output.log
+			cat /dev/null > $logfile
+			shift ;;
+		-t | --test)
+			TESTS="$TESTS $2"
+			shift 2 ;;
+		-c | --collection)
+			COLLECTIONS="$COLLECTIONS $2"
+			shift 2 ;;
+		-l | --list)
+			echo "$available"
+			exit 0 ;;
+		-n | --dry-run)
+			dryrun="echo"
+			shift ;;
+		-h | --help)
+			usage 0 ;;
+		"")
+			break ;;
+		*)
+			usage 1 ;;
+	esac
+done
+
+# Add all selected collections to the explicit test list.
+if [ -n "$COLLECTIONS" ]; then
+	for collection in $COLLECTIONS ; do
+		found="$(echo "$available" | grep "^$collection:")"
+		if [ -z "$found" ] ; then
+			echo "No such collection '$collection'" >&2
+			exit 1
+		fi
+		TESTS="$TESTS $found"
+	done
+fi
+# Replace available test list with explicitly selected tests.
+if [ -n "$TESTS" ]; then
+	valid=""
+	for test in $TESTS ; do
+		found="$(echo "$available" | grep "^${test}$")"
+		if [ -z "$found" ] ; then
+			echo "No such test '$test'" >&2
+			exit 1
+		fi
+		valid="$valid $found"
+	done
+	available="$(echo "$valid" | sed -e 's/ /\n/g')"
 fi
 
 collections=$(echo "$available" | cut -d: -f1 | uniq)
 for collection in $collections ; do
 	[ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg
 	tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2)
-	(cd "$collection" && run_many $tests)
+	($dryrun cd "$collection" && $dryrun run_many $tests)
 done