[0/2] kvm/selftests: Two rseq_test fixes

There are two issues in current rseq_test implementation and the
series intends to fix them:

- From glibc-2.35, rseq information is registered by TLS. It means
  rseq_test is unable to register its own rseq information. PATCH[01]
  fixes the issue by reuse TLS's rseq information if needed.

- sched_getcpu() relies on glibc's implementation and it can simply
  returns the CPU ID cached in the rseq information. In this case,
  it's pointless to compare the return value from sched_getcpu()
  and that fetched from rseq information. PATCH[02] fixes the issue
  by replacing sched_getcpu() with getcpu().

Gavin Shan (2):
  KVM: selftests: Make rseq compatible with glibc-2.35
  KVM: selftests: Use getcpu() instead of sched_getcpu() in rseq_test

 tools/testing/selftests/kvm/rseq_test.c | 62 ++++++++++++++++++-------
 1 file changed, 44 insertions(+), 18 deletions(-)

Hi Paolo,

On 8/10/22 7:14 PM, Paolo Bonzini wrote:
> On 8/9/22 14:21, Mathieu Desnoyers wrote:
>>> For kvm/selftests, there are 3 architectures involved actually. So we
>>> just need consider 4 cases: aarch64, x86, s390 and other. For other
>>> case, we just use __builtin_thread_pointer() to maintain code's
>>> integrity, but it's not called at all.
>>>
>>> I think kvm/selftest is always relying on glibc if I'm correct.
>> All those are handled in the rseq selftests and in librseq. Why duplicate all that logic again?
> 
> Yeah, rseq_test should reuse librseq code.  The simplest way,
> if slightly hackish, is to do something like
> 
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 690b499c3471..6c192b0ec304 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -37,6 +37,7 @@ ifeq ($(ARCH),riscv)
>       UNAME_M := riscv
>   endif
> 
>   LIBKVM += lib/assert.c
>   LIBKVM += lib/elf.c
>   LIBKVM += lib/guest_modes.c
> @@ -198,7 +199,7 @@ endif
>   CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \
>       -fno-stack-protector -fno-PIE -I$(LINUX_TOOL_INCLUDE) \
>       -I$(LINUX_TOOL_ARCH_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude \
> -    -I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
> +    -I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES) -I../rseq
> 
>   no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \
>           $(CC) -Werror -no-pie -x c - -o "$$TMP", -no-pie)
> 
> 
> and just #include "../rseq/rseq.c" in rseq_test.c.
> 

Thank you. It's really a nice idea. I think it's best way to share
"../rseq/rseq.c". In this way, we needn't to rely on "../rseq/librseq.so",
which is compiled by "../rseq/Makefile".

I will modify the code accordingly in v2 :)

Thanks,
Gavin

----- On Aug 10, 2022, at 5:14 AM, Paolo Bonzini pbonzini@redhat.com wrote:

> On 8/9/22 14:21, Mathieu Desnoyers wrote:
>>> For kvm/selftests, there are 3 architectures involved actually. So we
>>> just need consider 4 cases: aarch64, x86, s390 and other. For other
>>> case, we just use __builtin_thread_pointer() to maintain code's
>>> integrity, but it's not called at all.
>>>
>>> I think kvm/selftest is always relying on glibc if I'm correct.
>> All those are handled in the rseq selftests and in librseq. Why duplicate all
>> that logic again?
> 
> Yeah, rseq_test should reuse librseq code.  The simplest way,
> if slightly hackish, is to do something like
> 
> diff --git a/tools/testing/selftests/kvm/Makefile
> b/tools/testing/selftests/kvm/Makefile
> index 690b499c3471..6c192b0ec304 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -37,6 +37,7 @@ ifeq ($(ARCH),riscv)
>  	UNAME_M := riscv
>  endif
>  
>  LIBKVM += lib/assert.c
>  LIBKVM += lib/elf.c
>  LIBKVM += lib/guest_modes.c
> @@ -198,7 +199,7 @@ endif
>  CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \
>  	-fno-stack-protector -fno-PIE -I$(LINUX_TOOL_INCLUDE) \
>  	-I$(LINUX_TOOL_ARCH_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude \
> -	-I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
> +	-I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES) -I../rseq
>  
>  no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \
>          $(CC) -Werror -no-pie -x c - -o "$$TMP", -no-pie)
> 
> 
> and just #include "../rseq/rseq.c" in rseq_test.c.

Hi Paolo,

Indeed, this hack seems to be a good approach to immediately fix things without
moving around all source files and headers. In the longer term, I'd prefer Sean's
proposal to move rseq.c to tools/lib/ (and to move rseq headers to tools/include/rseq/).
This can be done in a follow up phase though. I'll put a note on my todo list
for after I come back from vacation.

I'll be able to do this refactoring on top of this fix.

Thanks,

Mathieu

> 
> Thanks,
> 
> Paolo

----- On Aug 9, 2022, at 8:37 PM, Gavin Shan gshan@redhat.com wrote:

> Hi Mathieu and Sean,
> 
> On 8/10/22 7:38 AM, Sean Christopherson wrote:
>> On Tue, Aug 09, 2022, Mathieu Desnoyers wrote:
>>> ----- On Aug 9, 2022, at 8:21 AM, Mathieu Desnoyers
>>> mathieu.desnoyers@efficios.com wrote:
>>>> ----- Gavin Shan <gshan@redhat.com> wrote:
>>>>> On 8/9/22 5:16 PM, Florian Weimer wrote:
>>>>>>>> __builtin_thread_pointer doesn't work on all architectures/GCC
>>>>>>>> versions.
>>>>>>>> Is this a problem for selftests?
>>>>>>>>
>>>>>>>
>>>>>>> It's a problem as the test case is running on all architectures. I think I
>>>>>>> need introduce our own __builtin_thread_pointer() for where it's not
>>>>>>> supported: (1) PowerPC  (2) x86 without GCC 11
>>>>>>>
>>>>>>> Please let me know if I still have missed cases where
>>>>>>> __buitin_thread_pointer() isn't supported?
>>>>>>
>>>>>> As far as I know, these are the two outliers that also have rseq
>>>>>> support.  The list is a bit longer if we also consider non-rseq
>>>>>> architectures (csky, hppa, ia64, m68k, microblaze, sparc, don't know
>>>>>> about the Linux architectures without glibc support).
>>>>>>
>>>>>
>>>>> For kvm/selftests, there are 3 architectures involved actually. So we
>>>>> just need consider 4 cases: aarch64, x86, s390 and other. For other
>>>>> case, we just use __builtin_thread_pointer() to maintain code's
>>>>> integrity, but it's not called at all.
>>>>>
>>>>> I think kvm/selftest is always relying on glibc if I'm correct.
>>>>
>>>> All those are handled in the rseq selftests and in librseq. Why duplicate all
>>>> that logic again?
>>>
>>> More to the point, considering that we have all the relevant rseq registration
>>> code in tools/testing/selftests/rseq/rseq.c already, and the relevant thread
>>> pointer getter code in tools/testing/selftests/rseq/rseq-*thread-pointer.h,
>>> is there an easy way to get test applications in tools/testing/selftests/kvm
>>> and in tools/testing/selftests/rseq to share that common code ?
>>>
>>> Keeping duplicated compatibility code is bad for long-term maintainability.
>> 
>> Any reason not to simply add tools/lib/rseq.c and then expose a helper to get
>> the
>> registered rseq struct?
>> 
> 
> There are couple of reasons, not to share
> tools/testing/selftests/rseq/librseq.so
> or add tools/lib/librseq.so. Please let me know if the arguments making sense
> to you?
> 
> - By design, selftests/rseq and selftests/kvm are parallel. It's going to
> introduce
>   unnecessary dependency for selftests/kvm to use selftests/rseq/librseq.so. To
>   me,
>   it makes the maintainability even harder.

In terms of build system, yes, selftests/rseq and selftests/kvm are side-by-side,
and I agree it is odd to have a cross-dependency.

That's where moving rseq.c to tools/lib/ makes sense.

> 
> - What selftests/kvm needs is rseq-thread-pointer.h, which accounts for ~5% of
>   functionalities, provided by selftests/rseq/librseq.so.

I've never seen this type of argument used to prevent using a library before, except
on extremely memory-constrained devices, which is not our target here.

Even if you would only use 1% of the features of a library, it does not justify
reimplementing that 1% if that code already sits within the same project (kernel
selftests).

> 
> - I'm not too much familiar with selftests/rseq, but it seems it need heavy
>   rework before it can become tools/lib/librseq.so. However, I'm not sure if
>   the effort is worthwhile. The newly added library is fully used by
>   testtests/rseq. ~5% of that is going to be used by selftests/kvm.
>   In this case, we still have cross-dependency issue.

No, it's just moving files around and a bit of Makefile modifications. That's
the simple part.

> 
> I personally prefer not to use selftests/rseq/librseq.so or add
> tools/lib/librseq.so,
> but I need your feedback. Please share your thoughts.

I strongly favor that we use a two steps approach:

1) immediate fix: include ../rseq/rseq.c into your test code and use the headers,
   as proposed by Paolo.

2) I'll move librseq code into tools/lib/ and tools/include/rseq/, and adapt the
   users accordingly. (after the end of my vacation)

Thanks,

Mathieu

> Thanks,
> Gavin

On 8/10/22 14:29, Mathieu Desnoyers wrote:
>> - By design, selftests/rseq and selftests/kvm are parallel. It's going to
>> introduce
>>    unnecessary dependency for selftests/kvm to use selftests/rseq/librseq.so. To
>>    me,
>>    it makes the maintainability even harder.
> In terms of build system, yes, selftests/rseq and selftests/kvm are side-by-side,
> and I agree it is odd to have a cross-dependency.
> 
> That's where moving rseq.c to tools/lib/ makes sense.
> 
>> - What selftests/kvm needs is rseq-thread-pointer.h, which accounts for ~5% of
>>    functionalities, provided by selftests/rseq/librseq.so.
> I've never seen this type of argument used to prevent using a library before, except
> on extremely memory-constrained devices, which is not our target here.

I agree.

To me, the main argument against moving librseq to tools/lib is a 
variant of the build-system argument, namely that recursive Make 
sucks[1] and selftests/kvm right now does not use tools/lib.  So, for a 
single-file library, it may be simply not worth the hassle.

On the other hand, if "somebody else" does the work, I would have no 
problem with having selftests/kvm depend on tools/lib, not at all.

Thanks,

Paolo

[1] Kbuild is a marvel that makes it work, but it works because there 
are no such cross-subdirectory dependencies and anyway 
tools/testing/selftests does not use Kbuild.

Hi Paolo and Mathieu,

On 8/10/22 10:19 PM, Paolo Bonzini wrote:
> On 8/10/22 14:17, Mathieu Desnoyers wrote:
>> Indeed, this hack seems to be a good approach to immediately fix things without
>> moving around all source files and headers. In the longer term, I'd prefer Sean's
>> proposal to move rseq.c to tools/lib/ (and to move rseq headers to tools/include/rseq/).
>> This can be done in a follow up phase though. I'll put a note on my todo list
>> for after I come back from vacation.
> 
> Great, Gavin, are you going to repost using librseq?
> 

It seems you've merged v2. I will post additional patches to
use tools/lib/librseq.so, depending on what Mathieu will have.

Mathieu, Please let me know if there are anything I can help.

>>> Yeah, rseq_test should reuse librseq code.  The simplest way,
>>> if slightly hackish, is to do something like
>>>
>>> diff --git a/tools/testing/selftests/kvm/Makefile
>>> b/tools/testing/selftests/kvm/Makefile
>>> index 690b499c3471..6c192b0ec304 100644
>>> --- a/tools/testing/selftests/kvm/Makefile
>>> +++ b/tools/testing/selftests/kvm/Makefile
>>> @@ -37,6 +37,7 @@ ifeq ($(ARCH),riscv)
>>>      UNAME_M := riscv
>>>  endif
>>>
>>>  LIBKVM += lib/assert.c
>>>  LIBKVM += lib/elf.c
>>>  LIBKVM += lib/guest_modes.c
>>> @@ -198,7 +199,7 @@ endif
>>>  CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \
>>>      -fno-stack-protector -fno-PIE -I$(LINUX_TOOL_INCLUDE) \
>>>      -I$(LINUX_TOOL_ARCH_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude \
>>> -    -I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
>>> +    -I$(<D) -Iinclude/$(UNAME_M) -I.. $(EXTRA_CFLAGS) $(KHDR_INCLUDES) -I../rseq
>>>
>>>  no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \
>>>          $(CC) -Werror -no-pie -x c - -o "$$TMP", -no-pie)
>>>
>>>
>>> and just #include "../rseq/rseq.c" in rseq_test.c.

Thanks,
Gavin