[v6,5/7] tests/tcg: allow to check output of plugins

Message ID	20240706191335.878142-6-pierrick.bouvier@linaro.org
State	Superseded
Headers	show Delivered-To: patch@linaro.org Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; From: Pierrick Bouvier <pierrick.bouvier@linaro.org> To: qemu-devel@nongnu.org Cc: Paolo Bonzini <pbonzini@redhat.com>, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= <philmd@linaro.org>, Richard Henderson <richard.henderson@linaro.org>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Mahmoud Mandour <ma.mandourr@gmail.com>, Alexandre Iooss <erdnaxe@crans.org>, =?utf-8?q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>, Eduardo Habkost <eduardo@habkost.net>, Yanan Wang <wangyanan55@huawei.com>, Pierrick Bouvier <pierrick.bouvier@linaro.org>, Xingtao Yao <yaoxt.fnst@fujitsu.com> Subject: [PATCH v6 5/7] tests/tcg: allow to check output of plugins Date: Sat, 6 Jul 2024 12:13:33 -0700 Message-Id: <20240706191335.878142-6-pierrick.bouvier@linaro.org> In-Reply-To: <20240706191335.878142-1-pierrick.bouvier@linaro.org> References: <20240706191335.878142-1-pierrick.bouvier@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=pierrick.bouvier@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action Precedence: list Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org
Series	plugins: access values during a memory read/write \| expand [v6,0/7] plugins: access values during a memory read/write [v6,1/7] plugins: fix mem callback array size [v6,2/7] plugins: save value during memory accesses [v6,3/7] plugins: extend API to get latest memory value accessed [v6,4/7] tests/tcg: add mechanism to run specific tests with plugins [v6,5/7] tests/tcg: allow to check output of plugins [v6,6/7] tests/plugin/mem: add option to print memory accesses [v6,7/7] tests/tcg/x86_64: add test for plugin memory access

Message ID

20240706191335.878142-6-pierrick.bouvier@linaro.org

State

Superseded

Headers

Received-SPF: pass (google.com: domain of
 qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as
 permitted sender) client-ip=209.51.188.17;
From: Pierrick Bouvier <pierrick.bouvier@linaro.org>
To: qemu-devel@nongnu.org
Cc: Paolo Bonzini <pbonzini@redhat.com>, =?utf-8?q?Philippe_Mathieu-Daud?=
	=?utf-8?q?=C3=A9?= <philmd@linaro.org>,
 Richard Henderson <richard.henderson@linaro.org>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Mahmoud Mandour <ma.mandourr@gmail.com>, Alexandre Iooss <erdnaxe@crans.org>,
	=?utf-8?q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>,
 Eduardo Habkost <eduardo@habkost.net>, Yanan Wang <wangyanan55@huawei.com>,
 Pierrick Bouvier <pierrick.bouvier@linaro.org>,
 Xingtao Yao <yaoxt.fnst@fujitsu.com>
Subject: [PATCH v6 5/7] tests/tcg: allow to check output of plugins
Date: Sat,  6 Jul 2024 12:13:33 -0700
Message-Id: <20240706191335.878142-6-pierrick.bouvier@linaro.org>
In-Reply-To: <20240706191335.878142-1-pierrick.bouvier@linaro.org>
References: <20240706191335.878142-1-pierrick.bouvier@linaro.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=2607:f8b0:4864:20::62a;
 envelope-from=pierrick.bouvier@linaro.org; helo=mail-pl1-x62a.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org
Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org

Series

plugins: access values during a memory read/write | expand

Commit Message

Pierrick Bouvier July 6, 2024, 7:13 p.m. UTC

A specific plugin test can now read and check a plugin output, to ensure
it contains expected values.

Tested-by: Xingtao Yao <yaoxt.fnst@fujitsu.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 tests/tcg/Makefile.target | 7 +++++++
 1 file changed, 7 insertions(+)

Comments

Alex Bennée July 8, 2024, 4:06 p.m. UTC | #1

Pierrick Bouvier <pierrick.bouvier@linaro.org> writes:

> A specific plugin test can now read and check a plugin output, to ensure
> it contains expected values.
>
> Tested-by: Xingtao Yao <yaoxt.fnst@fujitsu.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> ---
>  tests/tcg/Makefile.target | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
> index 52616544d52..6f50b0176ea 100644
> --- a/tests/tcg/Makefile.target
> +++ b/tests/tcg/Makefile.target
> @@ -90,6 +90,7 @@ CFLAGS=
>  LDFLAGS=
>  
>  QEMU_OPTS=
> +CHECK_PLUGIN_OUTPUT_COMMAND=true
>  
>  
>  # If TCG debugging, or TCI is enabled things are a lot slower
> @@ -180,6 +181,9 @@ run-plugin-%:
>  		-plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>  		-d plugin -D $*.pout \
>  		 $(call strip-plugin,$<))
> +	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> +	       TEST, check plugin $(call extract-plugin,$@) output \
> +	       with $(call strip-plugin,$<))
>  else
>  run-%: %
>  	$(call run-test, $<, \
> @@ -194,6 +198,9 @@ run-plugin-%:
>  	   	  -plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>  	    	  -d plugin -D $*.pout \
>  		  $(QEMU_OPTS) $(call strip-plugin,$<))
> +	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> +	       TEST, check plugin $(call extract-plugin,$@) output \
> +	       with $(call strip-plugin,$<))

No need to have a null test, we can wrap the call in an if:

--8<---------------cut here---------------start------------->8---
modified   tests/tcg/Makefile.target
@@ -90,7 +90,7 @@ CFLAGS=
 LDFLAGS=
 
 QEMU_OPTS=
-CHECK_PLUGIN_OUTPUT_COMMAND=true
+CHECK_PLUGIN_OUTPUT_COMMAND=
 
 
 # If TCG debugging, or TCI is enabled things are a lot slower
@@ -181,9 +181,10 @@ run-plugin-%:
 		-plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
 		-d plugin -D $*.pout \
 		 $(call strip-plugin,$<))
-	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
-	       TEST, check plugin $(call extract-plugin,$@) output \
-	       with $(call strip-plugin,$<))
+	$(if $(CHECK_PLUGIN_OUTPUT_COMMAND),		\
+		$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
+		       TEST, check plugin $(call extract-plugin,$@) output \
+	       		with $(call strip-plugin,$<)))
 else
 run-%: %
 	$(call run-test, $<, \
@@ -198,9 +199,10 @@ run-plugin-%:
 	   	  -plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
 	    	  -d plugin -D $*.pout \
 		  $(QEMU_OPTS) $(call strip-plugin,$<))
-	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
-	       TEST, check plugin $(call extract-plugin,$@) output \
-	       with $(call strip-plugin,$<))
+	$(if $(CHECK_PLUGIN_OUTPUT_COMMAND),		\
+		$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
+		       TEST, check plugin $(call extract-plugin,$@) output \
+	       	with $(call strip-plugin,$<)))
--8<---------------cut here---------------end--------------->8---

>  endif
>  
>  gdb-%: %

Pierrick Bouvier July 12, 2024, 12:36 a.m. UTC | #2

On 7/8/24 09:06, Alex Bennée wrote:
> Pierrick Bouvier <pierrick.bouvier@linaro.org> writes:
> 
>> A specific plugin test can now read and check a plugin output, to ensure
>> it contains expected values.
>>
>> Tested-by: Xingtao Yao <yaoxt.fnst@fujitsu.com>
>> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>> ---
>>   tests/tcg/Makefile.target | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
>> index 52616544d52..6f50b0176ea 100644
>> --- a/tests/tcg/Makefile.target
>> +++ b/tests/tcg/Makefile.target
>> @@ -90,6 +90,7 @@ CFLAGS=
>>   LDFLAGS=
>>   
>>   QEMU_OPTS=
>> +CHECK_PLUGIN_OUTPUT_COMMAND=true
>>   
>>   
>>   # If TCG debugging, or TCI is enabled things are a lot slower
>> @@ -180,6 +181,9 @@ run-plugin-%:
>>   		-plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>>   		-d plugin -D $*.pout \
>>   		 $(call strip-plugin,$<))
>> +	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
>> +	       TEST, check plugin $(call extract-plugin,$@) output \
>> +	       with $(call strip-plugin,$<))
>>   else
>>   run-%: %
>>   	$(call run-test, $<, \
>> @@ -194,6 +198,9 @@ run-plugin-%:
>>   	   	  -plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>>   	    	  -d plugin -D $*.pout \
>>   		  $(QEMU_OPTS) $(call strip-plugin,$<))
>> +	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
>> +	       TEST, check plugin $(call extract-plugin,$@) output \
>> +	       with $(call strip-plugin,$<))
> 
> No need to have a null test, we can wrap the call in an if:
> 
> --8<---------------cut here---------------start------------->8---
> modified   tests/tcg/Makefile.target
> @@ -90,7 +90,7 @@ CFLAGS=
>   LDFLAGS=
>   
>   QEMU_OPTS=
> -CHECK_PLUGIN_OUTPUT_COMMAND=true
> +CHECK_PLUGIN_OUTPUT_COMMAND=
>   
>   
>   # If TCG debugging, or TCI is enabled things are a lot slower
> @@ -181,9 +181,10 @@ run-plugin-%:
>   		-plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>   		-d plugin -D $*.pout \
>   		 $(call strip-plugin,$<))
> -	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> -	       TEST, check plugin $(call extract-plugin,$@) output \
> -	       with $(call strip-plugin,$<))
> +	$(if $(CHECK_PLUGIN_OUTPUT_COMMAND),		\
> +		$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> +		       TEST, check plugin $(call extract-plugin,$@) output \
> +	       		with $(call strip-plugin,$<)))
>   else
>   run-%: %
>   	$(call run-test, $<, \
> @@ -198,9 +199,10 @@ run-plugin-%:
>   	   	  -plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
>   	    	  -d plugin -D $*.pout \
>   		  $(QEMU_OPTS) $(call strip-plugin,$<))
> -	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> -	       TEST, check plugin $(call extract-plugin,$@) output \
> -	       with $(call strip-plugin,$<))
> +	$(if $(CHECK_PLUGIN_OUTPUT_COMMAND),		\
> +		$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
> +		       TEST, check plugin $(call extract-plugin,$@) output \
> +	       	with $(call strip-plugin,$<)))
> --8<---------------cut here---------------end--------------->8---
> 
>>   endif
>>   
>>   gdb-%: %
> 

Thanks, I was looking exactly for this construction initially! Will 
change to that.

diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
index 52616544d52..6f50b0176ea 100644
--- a/tests/tcg/Makefile.target
+++ b/tests/tcg/Makefile.target
@@ -90,6 +90,7 @@  CFLAGS=
 LDFLAGS=
 
 QEMU_OPTS=
+CHECK_PLUGIN_OUTPUT_COMMAND=true
 
 
 # If TCG debugging, or TCI is enabled things are a lot slower
@@ -180,6 +181,9 @@  run-plugin-%:
 		-plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
 		-d plugin -D $*.pout \
 		 $(call strip-plugin,$<))
+	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
+	       TEST, check plugin $(call extract-plugin,$@) output \
+	       with $(call strip-plugin,$<))
 else
 run-%: %
 	$(call run-test, $<, \
@@ -194,6 +198,9 @@  run-plugin-%:
 	   	  -plugin $(PLUGIN_LIB)/$(call extract-plugin,$@)$(PLUGIN_ARGS) \
 	    	  -d plugin -D $*.pout \
 		  $(QEMU_OPTS) $(call strip-plugin,$<))
+	$(call quiet-command, $(CHECK_PLUGIN_OUTPUT_COMMAND) $*.pout, \
+	       TEST, check plugin $(call extract-plugin,$@) output \
+	       with $(call strip-plugin,$<))
 endif
 
 gdb-%: %

[v6,5/7] tests/tcg: allow to check output of plugins

Commit Message

Comments

Patch