| Message ID | 20220601180537.2329566-21-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | [PULL,01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages | expand |
On 01/06/2022 15:05, Alex Bennée wrote: > From: Paolo Bonzini <pbonzini@redhat.com> > > In preparation for handling more binaries than just cc, handle > the case of "probe_target_compiler $cpu" directly in the function, > setting the target_* variables based on the ones that are used to > build QEMU. The clang check also needs to be moved after this > fallback. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > Reviewed-by: Richard Henderson <richard.henderson@linaro.org> > Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org> Hi, After this patch, a clean build in ppc64le hosts will not build ppc64{,le}-linux-user tests with "make check-tcg" > > diff --git a/configure b/configure > index fbf6d39f96..217c8b3cac 100755 > --- a/configure > +++ b/configure > @@ -954,10 +954,6 @@ case $git_submodules_action in > ;; > esac > > -if eval test -z "\${cross_cc_$cpu}"; then > - eval "cross_cc_${cpu}=\$cc" > -fi > - > default_target_list="" > mak_wilds="" > > @@ -2008,13 +2004,6 @@ probe_target_compiler() { > if eval test -n "\"\${cross_cc_$1}\""; then > if eval has "\"\${cross_cc_$1}\""; then > eval "target_cc=\"\${cross_cc_$1}\"" > - case $1 in > - i386|x86_64) > - if $target_cc --version | grep -qi "clang"; then > - unset target_cc > - fi > - ;; > - esac > fi > fi > if eval test -n "\"\${cross_as_$1}\""; then > @@ -2027,6 +2016,20 @@ probe_target_compiler() { > eval "target_ld=\"\${cross_ld_$1}\"" > fi > fi > + if test "$1" = $cpu; then > + : ${target_cc:=$cc} > + : ${target_as:=$as} > + : ${target_ld:=$ld} > + fi $cpu is normalized[1] to ppc64 on little-endian hosts, so ppc64le-linux-user will not have $target_{cc,as,ld} set, and ppc64-linux-user will have them set to a toolchain that may not support -mbig-endian. I suppose we have a similar problem with MIPS targets on MIPS hosts. [1] https://gitlab.com/qemu-project/qemu/-/blob/2ad60f6f8c12ca0acd8834fdd70e088361b8791f/configure#L611
Matheus Kowalczuk Ferst <matheus.ferst@eldorado.org.br> writes: > On 01/06/2022 15:05, Alex Bennée wrote: >> From: Paolo Bonzini <pbonzini@redhat.com> >> >> In preparation for handling more binaries than just cc, handle >> the case of "probe_target_compiler $cpu" directly in the function, >> setting the target_* variables based on the ones that are used to >> build QEMU. The clang check also needs to be moved after this >> fallback. >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> >> Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org> > > Hi, > > After this patch, a clean build in ppc64le hosts will not build > ppc64{,le}-linux-user tests with "make check-tcg" > >> >> diff --git a/configure b/configure >> index fbf6d39f96..217c8b3cac 100755 >> --- a/configure >> +++ b/configure >> @@ -954,10 +954,6 @@ case $git_submodules_action in >> ;; >> esac >> >> -if eval test -z "\${cross_cc_$cpu}"; then >> - eval "cross_cc_${cpu}=\$cc" >> -fi >> - >> default_target_list="" >> mak_wilds="" >> >> @@ -2008,13 +2004,6 @@ probe_target_compiler() { >> if eval test -n "\"\${cross_cc_$1}\""; then >> if eval has "\"\${cross_cc_$1}\""; then >> eval "target_cc=\"\${cross_cc_$1}\"" >> - case $1 in >> - i386|x86_64) >> - if $target_cc --version | grep -qi "clang"; then >> - unset target_cc >> - fi >> - ;; >> - esac >> fi >> fi >> if eval test -n "\"\${cross_as_$1}\""; then >> @@ -2027,6 +2016,20 @@ probe_target_compiler() { >> eval "target_ld=\"\${cross_ld_$1}\"" >> fi >> fi >> + if test "$1" = $cpu; then > + : ${target_cc:=$cc} >> + : ${target_as:=$as} >> + : ${target_ld:=$ld} >> + fi > > $cpu is normalized[1] to ppc64 on little-endian hosts, so > ppc64le-linux-user will not have $target_{cc,as,ld} set, and > ppc64-linux-user will have them set to a toolchain that may not support > -mbig-endian. I suppose we have a similar problem with MIPS targets on > MIPS hosts. For now you can always explicitly tell configure about the host compiler with: --cross-cc-ppc64le=gcc but we should fix the broken detection. It seems the var cpu has an overloaded meaning so I wonder if we just need an explicit host_cpu setting when we normalize cpu? > > [1] > https://gitlab.com/qemu-project/qemu/-/blob/2ad60f6f8c12ca0acd8834fdd70e088361b8791f/configure#L611
Hi Matheus, could you please test the tests-tcg-next branch at https://gitlab.com/bonzini/qemu? Thanks, Paolo On Thu, Jun 16, 2022 at 3:23 AM Alex Bennée <alex.bennee@linaro.org> wrote: > > > Matheus Kowalczuk Ferst <matheus.ferst@eldorado.org.br> writes: > > > On 01/06/2022 15:05, Alex Bennée wrote: > >> From: Paolo Bonzini <pbonzini@redhat.com> > >> > >> In preparation for handling more binaries than just cc, handle > >> the case of "probe_target_compiler $cpu" directly in the function, > >> setting the target_* variables based on the ones that are used to > >> build QEMU. The clang check also needs to be moved after this > >> fallback. > >> > >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > >> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> > >> Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com> > >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > >> Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org> > > > > Hi, > > > > After this patch, a clean build in ppc64le hosts will not build > > ppc64{,le}-linux-user tests with "make check-tcg" > > > >> > >> diff --git a/configure b/configure > >> index fbf6d39f96..217c8b3cac 100755 > >> --- a/configure > >> +++ b/configure > >> @@ -954,10 +954,6 @@ case $git_submodules_action in > >> ;; > >> esac > >> > >> -if eval test -z "\${cross_cc_$cpu}"; then > >> - eval "cross_cc_${cpu}=\$cc" > >> -fi > >> - > >> default_target_list="" > >> mak_wilds="" > >> > >> @@ -2008,13 +2004,6 @@ probe_target_compiler() { > >> if eval test -n "\"\${cross_cc_$1}\""; then > >> if eval has "\"\${cross_cc_$1}\""; then > >> eval "target_cc=\"\${cross_cc_$1}\"" > >> - case $1 in > >> - i386|x86_64) > >> - if $target_cc --version | grep -qi "clang"; then > >> - unset target_cc > >> - fi > >> - ;; > >> - esac > >> fi > >> fi > >> if eval test -n "\"\${cross_as_$1}\""; then > >> @@ -2027,6 +2016,20 @@ probe_target_compiler() { > >> eval "target_ld=\"\${cross_ld_$1}\"" > >> fi > >> fi > >> + if test "$1" = $cpu; then > + : ${target_cc:=$cc} > >> + : ${target_as:=$as} > >> + : ${target_ld:=$ld} > >> + fi > > > > $cpu is normalized[1] to ppc64 on little-endian hosts, so > > ppc64le-linux-user will not have $target_{cc,as,ld} set, and > > ppc64-linux-user will have them set to a toolchain that may not support > > -mbig-endian. I suppose we have a similar problem with MIPS targets on > > MIPS hosts. > > For now you can always explicitly tell configure about the host compiler > with: > > --cross-cc-ppc64le=gcc > > but we should fix the broken detection. It seems the var cpu has an > overloaded meaning so I wonder if we just need an explicit host_cpu > setting when we normalize cpu? > > > > > [1] > > https://gitlab.com/qemu-project/qemu/-/blob/2ad60f6f8c12ca0acd8834fdd70e088361b8791f/configure#L611 > > > -- > Alex Bennée > On Wed, Jun 15, 2022 at 1:57 PM Matheus Kowalczuk Ferst <matheus.ferst@eldorado.org.br> wrote: > > On 01/06/2022 15:05, Alex Bennée wrote: > > From: Paolo Bonzini <pbonzini@redhat.com> > > > > In preparation for handling more binaries than just cc, handle > > the case of "probe_target_compiler $cpu" directly in the function, > > setting the target_* variables based on the ones that are used to > > build QEMU. The clang check also needs to be moved after this > > fallback. > > > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > > Reviewed-by: Richard Henderson <richard.henderson@linaro.org> > > Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com> > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > > Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org> > > Hi, > > After this patch, a clean build in ppc64le hosts will not build > ppc64{,le}-linux-user tests with "make check-tcg" > > > > > diff --git a/configure b/configure > > index fbf6d39f96..217c8b3cac 100755 > > --- a/configure > > +++ b/configure > > @@ -954,10 +954,6 @@ case $git_submodules_action in > > ;; > > esac > > > > -if eval test -z "\${cross_cc_$cpu}"; then > > - eval "cross_cc_${cpu}=\$cc" > > -fi > > - > > default_target_list="" > > mak_wilds="" > > > > @@ -2008,13 +2004,6 @@ probe_target_compiler() { > > if eval test -n "\"\${cross_cc_$1}\""; then > > if eval has "\"\${cross_cc_$1}\""; then > > eval "target_cc=\"\${cross_cc_$1}\"" > > - case $1 in > > - i386|x86_64) > > - if $target_cc --version | grep -qi "clang"; then > > - unset target_cc > > - fi > > - ;; > > - esac > > fi > > fi > > if eval test -n "\"\${cross_as_$1}\""; then > > @@ -2027,6 +2016,20 @@ probe_target_compiler() { > > eval "target_ld=\"\${cross_ld_$1}\"" > > fi > > fi > > + if test "$1" = $cpu; then > + : ${target_cc:=$cc} > > + : ${target_as:=$as} > > + : ${target_ld:=$ld} > > + fi > > $cpu is normalized[1] to ppc64 on little-endian hosts, so > ppc64le-linux-user will not have $target_{cc,as,ld} set, and > ppc64-linux-user will have them set to a toolchain that may not support > -mbig-endian. I suppose we have a similar problem with MIPS targets on > MIPS hosts. > > [1] > https://gitlab.com/qemu-project/qemu/-/blob/2ad60f6f8c12ca0acd8834fdd70e088361b8791f/configure#L611 > > -- > Matheus K. Ferst > Instituto de Pesquisas ELDORADO <http://www.eldorado.org.br/> > Analista de Software > Aviso Legal - Disclaimer <https://www.eldorado.org.br/disclaimer.html>
On 17/06/2022 07:12, Paolo Bonzini wrote: > Hi Matheus, > > could you please test the tests-tcg-next branch at > https://gitlab.com/bonzini/qemu? At be6090bcac10, it works if no BE toolchain is present. Otherwise, the script probes powerpc64-linux-gnu-gcc instead of the native tools for ppc64le-linux-user, and then do_compiler fails because the $target_cflags contains -mlittle-endian. > > On Thu, Jun 16, 2022 at 3:23 AM Alex Bennée <alex.bennee@linaro.org> wrote: >> >> >> Matheus Kowalczuk Ferst <matheus.ferst@eldorado.org.br> writes: >> >>> On 01/06/2022 15:05, Alex Bennée wrote: >>>> From: Paolo Bonzini <pbonzini@redhat.com> >>>> >>>> In preparation for handling more binaries than just cc, handle >>>> the case of "probe_target_compiler $cpu" directly in the function, >>>> setting the target_* variables based on the ones that are used to >>>> build QEMU. The clang check also needs to be moved after this >>>> fallback. >>>> >>>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >>>> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> >>>> Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com> >>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >>>> Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org> >>> >>> Hi, >>> >>> After this patch, a clean build in ppc64le hosts will not build >>> ppc64{,le}-linux-user tests with "make check-tcg" >>> >>>> >>>> diff --git a/configure b/configure >>>> index fbf6d39f96..217c8b3cac 100755 >>>> --- a/configure >>>> +++ b/configure >>>> @@ -954,10 +954,6 @@ case $git_submodules_action in >>>> ;; >>>> esac >>>> >>>> -if eval test -z "\${cross_cc_$cpu}"; then >>>> - eval "cross_cc_${cpu}=\$cc" >>>> -fi >>>> - >>>> default_target_list="" >>>> mak_wilds="" >>>> >>>> @@ -2008,13 +2004,6 @@ probe_target_compiler() { >>>> if eval test -n "\"\${cross_cc_$1}\""; then >>>> if eval has "\"\${cross_cc_$1}\""; then >>>> eval "target_cc=\"\${cross_cc_$1}\"" >>>> - case $1 in >>>> - i386|x86_64) >>>> - if $target_cc --version | grep -qi "clang"; then >>>> - unset target_cc >>>> - fi >>>> - ;; >>>> - esac >>>> fi >>>> fi >>>> if eval test -n "\"\${cross_as_$1}\""; then >>>> @@ -2027,6 +2016,20 @@ probe_target_compiler() { >>>> eval "target_ld=\"\${cross_ld_$1}\"" >>>> fi >>>> fi >>>> + if test "$1" = $cpu; then > + : ${target_cc:=$cc} >>>> + : ${target_as:=$as} >>>> + : ${target_ld:=$ld} >>>> + fi >>> >>> $cpu is normalized[1] to ppc64 on little-endian hosts, so >>> ppc64le-linux-user will not have $target_{cc,as,ld} set, and >>> ppc64-linux-user will have them set to a toolchain that may not support >>> -mbig-endian. I suppose we have a similar problem with MIPS targets on >>> MIPS hosts. >> >> For now you can always explicitly tell configure about the host compiler >> with: >> >> --cross-cc-ppc64le=gcc >> >> but we should fix the broken detection. With this workaround we can run the tests on master. However, it works because linux-user tests only use CC. I wonder if the --cross-cc-* options will remain useful in the future; it seems that only --cross-prefix-* should be used now that more tools are exported in the config-<target>.mak >> It seems the var cpu has an >> overloaded meaning so I wonder if we just need an explicit host_cpu >> setting when we normalize cpu? >> AFAICT, $cpu is always host-related, but after the normalization it seems more like the architecture than the CPU name... Maybe we should call them $host_cpu and $host_arch? >>> >>> [1] >>> https://gitlab.com/qemu-project/qemu/-/blob/2ad60f6f8c12ca0acd8834fdd70e088361b8791f/configure#L611 >> >> >> -- >> Alex Bennée >> Thanks, Matheus K. Ferst Instituto de Pesquisas ELDORADO <http://www.eldorado.org.br/> Analista de Software Aviso Legal - Disclaimer <https://www.eldorado.org.br/disclaimer.html>
On 6/20/22 09:41, Matheus Kowalczuk Ferst wrote: > On 17/06/2022 07:12, Paolo Bonzini wrote: >> Hi Matheus, >> >> could you please test the tests-tcg-next branch at >> https://gitlab.com/bonzini/qemu? > > At be6090bcac10, it works if no BE toolchain is present. Otherwise, the > script probes powerpc64-linux-gnu-gcc instead of the native tools for > ppc64le-linux-user, and then do_compiler fails because the > $target_cflags contains -mlittle-endian. Also testing this branch, -m32 may not be supported: make[1]: Entering directory `/home/rth/qemu/bld/pc-bios/vof' cc -m32 -mbig-endian -mcpu=power4 -c -o entry.o /home/rth/qemu/src/pc-bios/vof/entry.S cc1: error: ‘-m32’ not supported in this configuration make[1]: *** [entry.o] Error 1 This is with a self-built compiler, rather than distro packaged, but with default configure options to gcc. r~
On 6/20/22 18:41, Matheus Kowalczuk Ferst wrote: > On 17/06/2022 07:12, Paolo Bonzini wrote: >> Hi Matheus, >> >> could you please test the tests-tcg-next branch at >> https://gitlab.com/bonzini/qemu? > At be6090bcac10, it works if no BE toolchain is present. Otherwise, the > script probes powerpc64-linux-gnu-gcc instead of the native tools for > ppc64le-linux-user, and then do_compiler fails because the > $target_cflags contains -mlittle-endian. > So the problem in that case is that powerpc64-linux-gnu-gcc is _not_ biarch and thus does not support -mlittle-endian? (I thought PPC compilers were all 32/64-bit and bi-endian). If so, I guess this would fix your case diff --git a/configure b/configure index 0fd2838e82..6ad888ed97 100755 --- a/configure +++ b/configure @@ -2041,22 +2041,6 @@ probe_target_compiler() { : ${container_cross_strip:=${container_cross_prefix}strip} done - eval "target_cflags=\${cross_cc_cflags_$1}" - if eval test -n "\"\${cross_cc_$1}\""; then - if eval has "\"\${cross_cc_$1}\""; then - eval "target_cc=\"\${cross_cc_$1}\"" - fi - else - compute_target_variable $1 target_cc gcc - fi - target_ccas=$target_cc - compute_target_variable $1 target_ar ar - compute_target_variable $1 target_as as - compute_target_variable $1 target_ld ld - compute_target_variable $1 target_nm nm - compute_target_variable $1 target_objcopy objcopy - compute_target_variable $1 target_ranlib ranlib - compute_target_variable $1 target_strip strip case "$1:$cpu" in aarch64_be:aarch64 | \ armeb:arm | \ @@ -2065,17 +2049,35 @@ probe_target_compiler() { ppc*:ppc64 | \ sparc:sparc64 | \ "$cpu:$cpu") - : ${target_cc:=$cc} - : ${target_ccas:=$ccas} - : ${target_as:=$as} - : ${target_ld:=$ld} - : ${target_ar:=$ar} - : ${target_as:=$as} - : ${target_ld:=$ld} - : ${target_nm:=$nm} - : ${target_objcopy:=$objcopy} - : ${target_ranlib:=$ranlib} - : ${target_strip:=$strip} + target_cc=$cc + target_ccas=$ccas + target_as=$as + target_ld=$ld + target_ar=$ar + target_as=$as + target_ld=$ld + target_nm=$nm + target_objcopy=$objcopy + target_ranlib=$ranlib + target_strip=$strip + ;; + *) + eval "target_cflags=\${cross_cc_cflags_$1}" + if eval test -n "\"\${cross_cc_$1}\""; then + if eval has "\"\${cross_cc_$1}\""; then + eval "target_cc=\"\${cross_cc_$1}\"" + fi + else + compute_target_variable $1 target_cc gcc + fi + target_ccas=$target_cc + compute_target_variable $1 target_ar ar + compute_target_variable $1 target_as as + compute_target_variable $1 target_ld ld + compute_target_variable $1 target_nm nm + compute_target_variable $1 target_objcopy objcopy + compute_target_variable $1 target_ranlib ranlib + compute_target_variable $1 target_strip strip ;; esac if test -n "$target_cc"; then but it would break the opposite of compiling tests/tcg QEMU with a big-endian non-biarch compiler. In any case, the series is already a step forward so I've sent it out. Paolo
On 6/21/22 09:58, Paolo Bonzini wrote: > On 6/20/22 18:41, Matheus Kowalczuk Ferst wrote: >> On 17/06/2022 07:12, Paolo Bonzini wrote: >>> Hi Matheus, >>> >>> could you please test the tests-tcg-next branch at >>> https://gitlab.com/bonzini/qemu? >> At be6090bcac10, it works if no BE toolchain is present. Otherwise, the >> script probes powerpc64-linux-gnu-gcc instead of the native tools for >> ppc64le-linux-user, and then do_compiler fails because the >> $target_cflags contains -mlittle-endian. >> > > So the problem in that case is that powerpc64-linux-gnu-gcc is _not_ > biarch and thus does not support -mlittle-endian? (I thought PPC > compilers were all 32/64-bit and bi-endian). Nit: this is ppc64le-linux-gnu-gcc, built for gcc135, a power9 ppc64le host. It *does* support -mbig-endian, but not -m32. At least gcc11 as release was not biarch, with no special configure arguments. I can try upgrading to the current gcc12 release to see if that changed... r~
On 6/21/22 20:30, Richard Henderson wrote: > On 6/21/22 09:58, Paolo Bonzini wrote: >> On 6/20/22 18:41, Matheus Kowalczuk Ferst wrote: >>> On 17/06/2022 07:12, Paolo Bonzini wrote: >>>> Hi Matheus, >>>> >>>> could you please test the tests-tcg-next branch at >>>> https://gitlab.com/bonzini/qemu? >>> At be6090bcac10, it works if no BE toolchain is present. Otherwise, the >>> script probes powerpc64-linux-gnu-gcc instead of the native tools for >>> ppc64le-linux-user, and then do_compiler fails because the >>> $target_cflags contains -mlittle-endian. >>> >> >> So the problem in that case is that powerpc64-linux-gnu-gcc is _not_ >> biarch and thus does not support -mlittle-endian? (I thought PPC >> compilers were all 32/64-bit and bi-endian). > > Nit: this is ppc64le-linux-gnu-gcc, built for gcc135, a power9 ppc64le > host. It *does* support -mbig-endian, but not -m32. > > At least gcc11 as release was not biarch, with no special configure > arguments. I can try upgrading to the current gcc12 release to see if > that changed... Ok, yesterday I had tested with RHEL and there the ppc64le compiler is bi-arch and bi-endian, but multilibs are disabled. So it cannot build 32-bit hosted binaries like those for tests/tcg, only freestanding binaries (vof). On the other hand the powerpc64-linux-gnu-gcc binary from the cross-gcc package is theoretically multilib-friendly, but it cannot find the CRT files on the host because they are not in the .../le multilib subdirectory. The simplest way out is to just test both native and cross. There is already code to check for a working compiler in the tests/tcg stanza, and it can be moved to probe_target_compiler. Paolo
diff --git a/configure b/configure index fbf6d39f96..217c8b3cac 100755 --- a/configure +++ b/configure @@ -954,10 +954,6 @@ case $git_submodules_action in ;; esac -if eval test -z "\${cross_cc_$cpu}"; then - eval "cross_cc_${cpu}=\$cc" -fi - default_target_list="" mak_wilds="" @@ -2008,13 +2004,6 @@ probe_target_compiler() { if eval test -n "\"\${cross_cc_$1}\""; then if eval has "\"\${cross_cc_$1}\""; then eval "target_cc=\"\${cross_cc_$1}\"" - case $1 in - i386|x86_64) - if $target_cc --version | grep -qi "clang"; then - unset target_cc - fi - ;; - esac fi fi if eval test -n "\"\${cross_as_$1}\""; then @@ -2027,6 +2016,20 @@ probe_target_compiler() { eval "target_ld=\"\${cross_ld_$1}\"" fi fi + if test "$1" = $cpu; then + : ${target_cc:=$cc} + : ${target_as:=$as} + : ${target_ld:=$ld} + fi + if test -n "$target_cc"; then + case $1 in + i386|x86_64) + if $target_cc --version | grep -qi "clang"; then + unset target_cc + fi + ;; + esac + fi } write_target_makefile() {