mbox series

[RFC,v3,00/19] Generate machine-readable output

Message ID 20210115190936.26148-1-dwagner@suse.de
Headers show
Series Generate machine-readable output | expand

Message

Daniel Wagner Jan. 15, 2021, 7:09 p.m. UTC
changes v3:
  - A number of bug fixes added at the beginning of
    the series.
  - Add --output option to all tests which have a
    numeric results, not just failed/passed

changes v2:
  - Moved the common JSON parts into rt-util.[ch]
  - Add --output option to signaltest

The current output of cyclictest is optimized for humans to read. This
is all good when working directly with the tools. But for CI
integration it's a bit of pain. Furthermore, not all rt-tests use the
same output format.

By using some easy to parse existing machine-readable format we can use
standard libraries to parse the data. For example in jitterdebug there
is a short Python program[1] to visualize either the histogram[2] or
all samples[3].

The implementation for JSON output for this is very simple. The last
patch adds a version of jitterdebugs's JSON output, which looks like

{
  "file_version": 1,
  "version:": "cyclictest V 1.90",
  "num_threads": 2,
  "resolution_in_ns": 0,
  "cmdline:": "./cyclictest --affinity=1-2 --duration=1s --output=dump.json -h 1000 -p 80",
  "sysinfo": {
    "sysname": "Linux",
    "nodename": "beryllium",
    "release": "5.9.14-1-default",
    "version": "#1 SMP Sat Dec 12 06:57:32 UTC 2020 (c648a46)",
    "machine": "x86_64"
  },
  "thread": {
    "0": {
      "histogram": {
        "0": 16,
        "1": 853,
        "2": 80,
        "3": 50,
        "4": 1
      },
      "cycles": 1000,
      "min": 0,
      "max": 4,
      "avg": 1.17,
      "cpu": 1,
      "node": 0
    },
    "1": {
      "histogram": {
        "0": 14,
        "1": 833,
        "2": 93,
        "3": 56,
        "4": 4
      },
      "cycles": 1000,
      "min": 0,
      "max": 4,
      "avg": 1.20,
      "cpu": 2,
      "node": 0
    }
  }
}

It's just a rough version. I didn't try to make it generic for the
other rt-tests or make it as plugin as John was suggesting. I'd think
we could make this feature as compile option, if you want to keep the
program small. Obviously, we could also make the terminal output a
compile option, to keep it small.

Anyway, what do you think about it?

Thanks,
Daniel

[1] https://github.com/igaw/jitterdebugger/blob/master/jitterplot
[2] https://www.monom.org/data/jitterdebug/jitterplot-hist.png
[3] https://www.monom.org/data/jitterdebug/jitterplot-samples.png

Daniel Wagner (19):
  pip_stress: De-constify prio_min
  ptsematest: Return correct exit code when showing help
  rt-tests: Rename error.h to rt-error.h
  cyclictest: Move thread data to struct thread_param
  cyclictest: Remove libnuma API version 1 support
  signaltest: Move thread data to struct thread_param
  cyclicdeadline: Add missing step command line argument
  cyclicdeadline: Remove unused struct thread_param
  cyclicdeadline: Remove unused members in struct thread_stat
  rt-utils: Add JSON common header output helper
  cyclictest: Add JSON output feature
  signaltest: Add JSON output feature
  cyclicdeadline: Add JSON output feature
  pmqtest: Add JSON output feature
  ptsematest: Add JSON output feature
  sigwaittest: Add JSON output feature
  svsematest: Add JSON output feature
  oslat: Add JSON output feature
  rt-migrate-test: Add JSON output feature

 src/cyclictest/cyclictest.c           |  87 +++++++++++----
 src/include/pip_stress.h              |   5 +-
 src/include/{error.h => rt-error.h}   |   0
 src/include/rt-utils.h                |   4 +
 src/lib/error.c                       |   2 +-
 src/lib/rt-numa.c                     |   2 +-
 src/lib/rt-utils.c                    |  73 +++++++++++-
 src/oslat/oslat.c                     |  99 +++++++++++++---
 src/pi_tests/pi_stress.c              |   3 +-
 src/pi_tests/pip_stress.c             |   2 +-
 src/pmqtest/pmqtest.c                 | 143 +++++++++++++++++++-----
 src/ptsematest/ptsematest.c           | 128 +++++++++++++++++----
 src/rt-migrate-test/rt-migrate-test.c | 118 ++++++++++++++++----
 src/sched_deadline/cyclicdeadline.c   | 102 ++++++++++-------
 src/signaltest/signaltest.c           | 155 ++++++++++++++++++--------
 src/sigwaittest/sigwaittest.c         | 119 ++++++++++++++++----
 src/svsematest/svsematest.c           | 127 +++++++++++++++++----
 17 files changed, 927 insertions(+), 242 deletions(-)
 rename src/include/{error.h => rt-error.h} (100%)

Comments

John Kacur Jan. 26, 2021, 6:06 a.m. UTC | #1
On Fri, 15 Jan 2021, Daniel Wagner wrote:

> This variables is updated by the result of

> sched_get_priority_min(). Hence, it can't be const.

> 

> Signed-off-by: Daniel Wagner <dwagner@suse.de>

> ---

>  src/pi_tests/pip_stress.c | 2 +-

>  1 file changed, 1 insertion(+), 1 deletion(-)

> 

> diff --git a/src/pi_tests/pip_stress.c b/src/pi_tests/pip_stress.c

> index 4583aa40a6b8..8b91578f9ab8 100644

> --- a/src/pi_tests/pip_stress.c

> +++ b/src/pi_tests/pip_stress.c

> @@ -65,7 +65,7 @@ struct State {

>  struct State *statep;

>  

>  const int policy = SCHED_FIFO;

> -const int prio_min;	/* Initialized for the minimum priority of policy */

> +static int prio_min;	/* Initialized for the minimum priority of policy */

>  

>  static void usage(int error)

>  {

> -- 

> 2.29.2

> 

> 

Signed-off-by: John Kacur <jkacur@redhat.com>
John Kacur Jan. 26, 2021, 6:09 a.m. UTC | #2
On Fri, 15 Jan 2021, Daniel Wagner wrote:

> Instead always return success, use the provided exist code.

> 

> Signed-off-by: Daniel Wagner <dwagner@suse.de>

> ---

>  src/ptsematest/ptsematest.c | 2 +-

>  1 file changed, 1 insertion(+), 1 deletion(-)

> 

> diff --git a/src/ptsematest/ptsematest.c b/src/ptsematest/ptsematest.c

> index 2e392299fdc4..f8f075591f4b 100644

> --- a/src/ptsematest/ptsematest.c

> +++ b/src/ptsematest/ptsematest.c

> @@ -164,7 +164,7 @@ static void display_help(int error)

>  	       "                           without NUM, threads = max_cpus\n"

>  	       "                           without -t default = 1\n"

>  	       );

> -	exit(0);

> +	exit(error);

>  }

>  

>  

> -- 

> 2.29.2

> 

> 

Signed-off-by: John Kacur <jkacur@redhat.com>
John Kacur Jan. 26, 2021, 6:19 a.m. UTC | #3
On Fri, 15 Jan 2021, Daniel Wagner wrote:

> When adding long command line options support the --step

> argument was missed out. Add it back.

> 

> Fixes: 9723329835b3 ("cyclicdeadline: Add long command line options")

> Signed-off-by: Daniel Wagner <dwagner@suse.de>

> ---

>  src/sched_deadline/cyclicdeadline.c | 3 ++-

>  1 file changed, 2 insertions(+), 1 deletion(-)

> 

> diff --git a/src/sched_deadline/cyclicdeadline.c b/src/sched_deadline/cyclicdeadline.c

> index 19a7fcb6c488..cd12c7f313c2 100644

> --- a/src/sched_deadline/cyclicdeadline.c

> +++ b/src/sched_deadline/cyclicdeadline.c

> @@ -1013,11 +1013,12 @@ int main(int argc, char **argv)

>  			{ "duration",	required_argument,	NULL,	'D' },

>  			{ "help",	no_argument,		NULL,	'h' },

>  			{ "interval",	required_argument,	NULL,	'i' },

> +			{ "step",	required_argument,	NULL,	's' },

>  			{ "threads",	required_argument,	NULL,	't' },

>  			{ "quiet",	no_argument,		NULL,	'q' },

>  			{ NULL,		0,			NULL,	0   },

>  		};

> -		c = getopt_long(argc, argv, "a::c:D:hi:t:q", options, NULL);

> +		c = getopt_long(argc, argv, "a::c:D:hi:s:t:q", options, NULL);

>  		if (c == -1)

>  			break;

>  		switch (c) {

> -- 

> 2.29.2

> 

> 

Signed-off-by: John Kacur <jkacur@redhat.com>
John Kacur Jan. 26, 2021, 6:22 a.m. UTC | #4
On Fri, 15 Jan 2021, Daniel Wagner wrote:

> There is no user for this data struct. Remove it.

> 

> Signed-off-by: Daniel Wagner <dwagner@suse.de>

> ---

>  src/sched_deadline/cyclicdeadline.c | 17 -----------------

>  1 file changed, 17 deletions(-)

> 

> diff --git a/src/sched_deadline/cyclicdeadline.c b/src/sched_deadline/cyclicdeadline.c

> index cd12c7f313c2..71cde5781499 100644

> --- a/src/sched_deadline/cyclicdeadline.c

> +++ b/src/sched_deadline/cyclicdeadline.c

> @@ -47,23 +47,6 @@ typedef unsigned long long u64;

>  typedef unsigned int u32;

>  typedef int s32;

>  

> -/* Struct to transfer parameters to the thread */

> -struct thread_param {

> -	u64 runtime_us;

> -	u64 deadline_us;

> -

> -	int mode;

> -	int timermode;

> -	int signal;

> -	int clock;

> -	unsigned long max_cycles;

> -	struct thread_stat *stats;

> -	unsigned long interval;

> -	int cpu;

> -	int node;

> -	int tnum;

> -};

> -

>  /* Struct for statistics */

>  struct thread_stat {

>  	unsigned long cycles;

> -- 

> 2.29.2

> 

> 

Signed-off-by: John Kacur <jkacur@redhat.com>