| Message ID | 20201014135416.1290679-1-pbonzini@redhat.com |
|---|---|
| Headers | show |
| Series | build: replace ninjatool with ninja | expand |
On Wed, 14 Oct 2020 at 14:54, Paolo Bonzini <pbonzini@redhat.com> wrote: > > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > 27 files changed, 121 insertions(+), 1070 deletions(-) The diffstat is certainly persuasive :-) -- PMM
On Wed, Oct 14, 2020 at 09:54:10AM -0400, Paolo Bonzini wrote: > Remove from check-block the requirement that all TARGET_DIRS are built. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/Makefile.include | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel
On Wed, Oct 14, 2020 at 09:54:14AM -0400, Paolo Bonzini wrote: > ninja is included in the CentOS PowerTools repository. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/docker/dockerfiles/centos8.docker | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker > index 0fc2697491..e31d366fc1 100644 > --- a/tests/docker/dockerfiles/centos8.docker > +++ b/tests/docker/dockerfiles/centos8.docker > @@ -1,6 +1,9 @@ > FROM centos:8.1.1911 > > +RUN dnf -y install dnf-plugins-core > +RUN dnf config-manager --set-enabled PowerTools > RUN dnf -y update This results in multiple image layers which is undesirable. Instead do RUN dnf -y install dnf-plugins-core && \ dnf config-manager --set-enabled PowerTools && \ dnf -y update > + > ENV PACKAGES \ > SDL-devel \ > bzip2 \ Regards, Daniel
On Wed, Oct 14, 2020 at 3:57 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > > The advantages cover various areas: > > - maintainability: we drop scripts/ninjatool.py, which is > a large and hairy piece of code, and generally remove one > thing that can go wrong and one thing that is unique to QEMU; > > - platform support: we remove the requirement for GNU make > 3.82, which was annoying on Mac. We also avoid bugs on Windows > due to meson emitting Windows rather than POSIX escapes (as > expected by Ninja) and ninjatool ignoring the difference; > > - speed: invoking "configure" does not have to generate > 44k lines of rules, while invoking "Make" does not anymore have > to parse 44k lines of rules. > > - ease of use: Ninja tracks command lines, hence the problem > of static library changing the objects they hold goes away > > Paolo > > Hi, I still have some issues when compiling on Windows with msys2/mingw64 after this patch set. Command line: mkdir build && cd build && ../configure --cross-prefix=x86_64-w64-mingw32- --target-list=ppc-softmmu --enable-gtk --enable-sdl Traceback (most recent call last): File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/mesonmain.py", line 131, in run return options.run_func(options) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 245, in run app.generate() File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 159, in generate self._generate(env) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 215, in _generate intr.backend.generate() File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/backend/ninjabackend.py", line 483, in generate ninja = environment.detect_ninja_command_and_version(log=True) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/environment.py", line 177, in detect_ninja_command_and_version name = os.path.basename(n) File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 231, in basename return split(p)[1] File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 200, in split p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ERROR: meson setup failed When I set --ninja=ninja explicitly, this error does not occur: Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE However, compiling then runs into a problem: $ make -j8 /mingw64/bin/python3 -B /home/hsp/src/qemu-master/meson/meson.py introspect --targets --tests --benchmarks | /mingw64/bin/python3 -B scripts/mtest2make.py > Makefile.mtest { echo 'ninja-targets = \'; ninja -t targets all | sed 's/:.*//; $!s/$/ \\/'; } > Makefile.ninja ninja -j8 all | cat make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 make: *** Waiting for unfinished jobs.... The problem with optionrom was already present earlier and could be fixed by applying: https://patchwork.ozlabs.org/project/qemu-devel/list/?series=202798 That patch set does by now not apply cleanly anymore. Best, Howard <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 14, 2020 at 3:57 PM Paolo Bonzini <<a href="mailto:pbonzini@redhat.com" target="_blank">pbonzini@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This pull request is the last build system change for 5.2 from<br> me, and it is simple: similar to how we are invoking<br> ROM or tests/tcg "make" from the main Makefile, we now invoke ninja<br> to build QEMU. Unlike those cases, however, build.ninja targets are<br> forwarded transparently.<br> <br> The advantages cover various areas:<br> <br> - maintainability: we drop scripts/ninjatool.py, which is<br> a large and hairy piece of code, and generally remove one<br> thing that can go wrong and one thing that is unique to QEMU;<br> <br> - platform support: we remove the requirement for GNU make<br> 3.82, which was annoying on Mac. We also avoid bugs on Windows<br> due to meson emitting Windows rather than POSIX escapes (as<br> expected by Ninja) and ninjatool ignoring the difference;<br> <br> - speed: invoking "configure" does not have to generate<br> 44k lines of rules, while invoking "Make" does not anymore have<br> to parse 44k lines of rules.<br> <br> - ease of use: Ninja tracks command lines, hence the problem<br> of static library changing the objects they hold goes away<br> <br> Paolo<br> <br></blockquote><div> </div><div>Hi,</div><div><br></div><div>I still have some issues when compiling on Windows with msys2/mingw64 after this patch set. Command line:</div><div><br></div><div>mkdir build && cd build && ../configure --cross-prefix=x86_64-w64-mingw32- --target-list=ppc-softmmu --enable-gtk --enable-sdl</div><div><br></div><div>Traceback (most recent call last):<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/mesonmain.py", line 131, in run<br> return options.run_func(options)<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 245, in run<br> app.generate()<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 159, in generate<br> self._generate(env)<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 215, in _generate<br> intr.backend.generate()<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/backend/ninjabackend.py", line 483, in generate<br> ninja = environment.detect_ninja_command_and_version(log=True)<br> File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/environment.py", line 177, in detect_ninja_command_and_version<br> name = os.path.basename(n)<br> File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 231, in basename<br> return split(p)[1]<br> File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 200, in split<br> p = os.fspath(p)<br>TypeError: expected str, bytes or os.PathLike object, not NoneType<br><br>ERROR: meson setup failed<br></div><div><br></div><div>When I set --ninja=ninja explicitly, this error does not occur:</div><div>Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE</div><div><br></div><div>However, compiling then runs into a problem:<br></div><div>$ make -j8<br>/mingw64/bin/python3 -B /home/hsp/src/qemu-master/meson/meson.py introspect --targets --tests --benchmarks | /mingw64/bin/python3 -B scripts/mtest2make.py > Makefile.mtest<br>{ echo 'ninja-targets = \'; ninja -t targets all | sed 's/:.*//; $!s/$/ \\/'; } > Makefile.ninja<br>ninja -j8 all | cat<br>make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop.<br>make: *** [Makefile:171: pc-bios/optionrom/all] Error 2<br>make: *** Waiting for unfinished jobs....<br></div><div><br></div><div>The problem with optionrom was already present earlier and could be fixed by applying:</div><div> <div><a href="https://patchwork.ozlabs.org/project/qemu-devel/list/?series=202798" target="_blank">https://patchwork.ozlabs.org/project/qemu-devel/list/?series=202798</a></div><div><br></div><div>That patch set does by now not apply cleanly anymore.<br></div><div><br></div><div>Best,</div><div>Howard<br></div> </div></div></div>
On 15/10/20 08:15, Howard Spoelstra wrote: > ERROR: meson setup failed > > When I set --ninja=ninja explicitly, this error does not occur: > Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE That is fixed by Meson 0.55.2, I'll add a submodule update. Thanks for the reminder! Paolo
On 14/10/20 22:49, Cleber Rosa wrote: > On Wed, Oct 14, 2020 at 09:54:14AM -0400, Paolo Bonzini wrote: >> ninja is included in the CentOS PowerTools repository. >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> --- >> tests/docker/dockerfiles/centos8.docker | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker >> index 0fc2697491..e31d366fc1 100644 >> --- a/tests/docker/dockerfiles/centos8.docker >> +++ b/tests/docker/dockerfiles/centos8.docker >> @@ -1,6 +1,9 @@ >> FROM centos:8.1.1911 >> >> +RUN dnf -y install dnf-plugins-core >> +RUN dnf config-manager --set-enabled PowerTools > > Since config-manager won't be used besides here, I think it's safe and > cheaper to simply do: > > RUN sed -i /etc/yum.repos.d/CentOS-PowerTools.repo -e 's/^enabled=0/enabled=1/' > > It's also a single command, reducing the number of layers. Yeah, I'm lazy. :) I'll use Dan's suggestion. Paolo
On Thu, Oct 15, 2020 at 8:34 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > On 15/10/20 08:15, Howard Spoelstra wrote: > > ERROR: meson setup failed > > > > When I set --ninja=ninja explicitly, this error does not occur: > > Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE > > That is fixed by Meson 0.55.2, I'll add a submodule update. Thanks for > the reminder! > > Paolo > > Thanks Paolo, Then only the issue regarding the pcbios/optionrom stuff remains ;-) make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 make: *** Waiting for unfinished jobs.... Best, Howard <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 15, 2020 at 8:34 AM Paolo Bonzini <<a href="mailto:pbonzini@redhat.com" target="_blank">pbonzini@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 15/10/20 08:15, Howard Spoelstra wrote:<br> > ERROR: meson setup failed<br> > <br> > When I set --ninja=ninja explicitly, this error does not occur:<br> > Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE<br> <br> That is fixed by Meson 0.55.2, I'll add a submodule update. Thanks for<br> the reminder!<br> <br> Paolo<br> <br></blockquote><div><br></div><div>Thanks Paolo,<br></div><div><br></div><div>Then only the issue regarding the pcbios/optionrom stuff remains ;-)</div><div><br></div><div> make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop.<br>make: *** [Makefile:171: pc-bios/optionrom/all] Error 2<br>make: *** Waiting for unfinished jobs.... </div><div><br></div><div>Best,</div><div>Howard<br></div><div> <br></div></div></div>
On 15/10/20 09:39, Howard Spoelstra wrote: > > > Thanks Paolo, > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > make: *** Waiting for unfinished jobs.... It seems to work on cirrus CI: https://cirrus-ci.com/task/5383432727429120 Paolo
> > Thanks Paolo, > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > make: *** Waiting for unfinished jobs.... > > Best, > Howard > Hi Howard, one solution for this issue is to uncomment the following line in msys2_shell.cmd. rem To activate windows native symlinks uncomment next line set MSYS=winsymlinks:nativestrict Then tell Windows 10 it's okay to create symlinks without elevated rights. Here is a link with a description how to do this. https://www.joshkel.com/2018/01/18/symlinks-in-windows/ I think since commit bf708f3c4a "optionrom: simplify Makefile" pc-bios/optionrom/Makefile in your build directory has to be a symbolic link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the Makefile instead of creating a symbolic link. With best regards, Volker
On Wed, Oct 14, 2020 at 9:58 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > > The advantages cover various areas: > > - maintainability: we drop scripts/ninjatool.py, which is > a large and hairy piece of code, and generally remove one > thing that can go wrong and one thing that is unique to QEMU; > > - platform support: we remove the requirement for GNU make > 3.82, which was annoying on Mac. We also avoid bugs on Windows > due to meson emitting Windows rather than POSIX escapes (as > expected by Ninja) and ninjatool ignoring the difference; > > - speed: invoking "configure" does not have to generate > 44k lines of rules, while invoking "Make" does not anymore have > to parse 44k lines of rules. > > - ease of use: Ninja tracks command lines, hence the problem > of static library changing the objects they hold goes away > > Paolo > > Paolo Bonzini (7): > tests/Makefile.include: unbreak non-tcg builds > make: run shell with pipefail > tests: add missing generated sources to testqapi > configure: move QEMU_INCLUDES to meson > dockerfiles: enable Centos 8 PowerTools > add ninja to dockerfiles, CI configurations and test VMs > build: replace ninjatool with ninja > > .cirrus.yml | 6 +- > .travis.yml | 13 + > Makefile | 42 +- > configure | 29 +- > docs/devel/build-system.rst | 6 +- > meson.build | 34 +- > scripts/mtest2make.py | 4 +- > scripts/ninjatool.py | 1008 -------------------- > tests/Makefile.include | 2 +- > tests/docker/dockerfiles/centos7.docker | 1 + > tests/docker/dockerfiles/centos8.docker | 4 + > tests/docker/dockerfiles/debian10.docker | 1 + > tests/docker/dockerfiles/fedora.docker | 1 + > tests/docker/dockerfiles/travis.docker | 2 +- > tests/docker/dockerfiles/ubuntu.docker | 1 + > tests/docker/dockerfiles/ubuntu1804.docker | 1 + > tests/docker/dockerfiles/ubuntu2004.docker | 1 + > tests/include/meson.build | 8 +- > tests/meson.build | 14 +- > tests/vm/centos | 2 +- > tests/vm/centos.aarch64 | 2 +- > tests/vm/fedora | 2 +- > tests/vm/freebsd | 1 + > tests/vm/netbsd | 1 + > tests/vm/openbsd | 1 + > tests/vm/ubuntu.aarch64 | 2 +- > tests/vm/ubuntu.i386 | 2 +- > 27 files changed, 121 insertions(+), 1070 deletions(-) > delete mode 100755 scripts/ninjatool.py > > -- > 2.26.2 > > That's good, it's broken my locally rST doc build ``` [226/1564] Generating input-keymap-qnum-to-qcode.c.inc with a custom command (wrapped by meson to capture output) [227/1564] Generating input-keymap-qcode-to-qnum.c.inc with a custom command (wrapped by meson to capture output) [228/1564] Generating shared QAPI source files with a custom command [229/1564] Generating input-keymap-qcode-to-linux.c.inc with a custom command (wrapped by meson to capture output) [230/1564] Generating input-keymap-win32-to-qcode.c.inc with a custom command (wrapped by meson to capture output) [231/1564] Generating QAPI rST doc with a custom command FAILED: tests/qapi-schema/doc-good.txt "/mingw64/bin/sphinx-build" "-Dversion=5.1.50" "-Drelease=" "-W" "-b" "text" "-E" "-c" "C:/work/xemu/qemu/docs" "-D" "master_doc=doc-good" "C:/work/xemu/qemu/tests/qapi-schema" "C:/work/xemu/qemu/build/tests/qapi-schema" CreateProcess failed: The system cannot find the file specified. [232/1564] Generating input-keymap-qcode-to-atset1.c.inc with a custom command (wrapped by meson to capture output) [233/1564] Generating generated-helpers-wrappers.h with a custom command (wrapped by meson to capture output) ninja: fatal: GetOverlappedResult: �����Ч�� make: *** [Makefile:87：run-ninja] 错误 1 /mingw64/bin/ninja -v -j1 tests/check-block-qdict.exe tests/check-qdict.exe tests/check-qjson.exe tests/check-qlist.exe tests/check-qlit.exe tests/check-qnull.exe tests/check-qnum.exe tests/check-qobject.exe tests/check-qom-interface.exe tests/check-qom-proplist.exe tests/check-qstring.exe tests/ptimer-test.exe tests/qapi-schema/doc-good.ref.nocr tests/qapi-schema/doc-good.txt.nocr tests/rcutorture.exe tests/test-aio-multithread.exe tests/test-aio.exe tests/test-authz-list.exe tests/test-authz-listfile.exe tests/test-authz-simple.exe tests/test-base64.exe tests/test-bdrv-drain.exe tests/test-bdrv-graph-mod.exe tests/test-bitcnt.exe tests/test-bitmap.exe tests/test-bitops.exe tests/test-block-backend.exe tests/test-block-iothread.exe tests/test-blockjob-txn.exe tests/test-blockjob.exe tests/test-bufferiszero.exe tests/test-char.exe tests/test-clone-visitor.exe tests/test-coroutine.exe tests/test-crypto-afsplit.exe tests/test-crypto-block.exe tests/test-crypto-cipher.exe tests/test-crypto-hash.exe tests/test-crypto-hmac.exe tests/test-crypto-ivgen.exe tests/test-crypto-pbkdf.exe tests/test-crypto-secret.exe tests/test-cutils.exe tests/test-hbitmap.exe tests/test-int128.exe tests/test-io-channel-buffer.exe tests/test-io-channel-command.exe tests/test-io-channel-file.exe tests/test-io-channel-socket.exe tests/test-io-task.exe tests/test-iov.exe tests/test-keyval.exe tests/test-logging.exe tests/test-mul64.exe tests/test-opts-visitor.exe tests/test-qapi-util.exe tests/test-qdev-global-props.exe tests/test-qdist.exe tests/test-qemu-opts.exe tests/test-qgraph.exe tests/test-qht.exe tests/test-qmp-cmds.exe tests/test-qmp-event.exe tests/test-qobject-input-visitor.exe tests/test-qobject-output-visitor.exe tests/test-rcu-list.exe tests/test-rcu-simpleq.exe tests/test-rcu-slist.exe tests/test-rcu-tailq.exe tests/test-replication.exe tests/test-shift128.exe tests/test-string-input-visitor.exe tests/test-string-output-visitor.exe tests/test-thread-pool.exe tests/test-throttle.exe tests/test-timed-average.exe tests/test-util-sockets.exe tests/test-uuid.exe tests/test-visitor-serialization.exe tests/test-vmstate.exe tests/test-write-threshold.exe tests/test-x86-cpuid.exe tests/test-xbzrle.exe | cat ``` That's because currently sphinx-build are found in configure not in meson, maybe it's worth to move the sphinx related thing into meson totally. -- 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo <div dir="ltr"><br><br>On Wed, Oct 14, 2020 at 9:58 PM Paolo Bonzini <<a href="mailto:pbonzini@redhat.com">pbonzini@redhat.com</a>> wrote:<br>><br>> This pull request is the last build system change for 5.2 from<br>> me, and it is simple: similar to how we are invoking<br>> ROM or tests/tcg "make" from the main Makefile, we now invoke ninja<br>> to build QEMU. Unlike those cases, however, build.ninja targets are<br>> forwarded transparently.<br>><br>> The advantages cover various areas:<br>><br>> - maintainability: we drop scripts/ninjatool.py, which is<br>> a large and hairy piece of code, and generally remove one<br>> thing that can go wrong and one thing that is unique to QEMU;<br>><br>> - platform support: we remove the requirement for GNU make<br>> 3.82, which was annoying on Mac. We also avoid bugs on Windows<br>> due to meson emitting Windows rather than POSIX escapes (as<br>> expected by Ninja) and ninjatool ignoring the difference;<br>><br>> - speed: invoking "configure" does not have to generate<br>> 44k lines of rules, while invoking "Make" does not anymore have<br>> to parse 44k lines of rules.<br>><br>> - ease of use: Ninja tracks command lines, hence the problem<br>> of static library changing the objects they hold goes away<br>><br>> Paolo<br>><br>> Paolo Bonzini (7):<br>> tests/Makefile.include: unbreak non-tcg builds<br>> make: run shell with pipefail<br>> tests: add missing generated sources to testqapi<br>> configure: move QEMU_INCLUDES to meson<br>> dockerfiles: enable Centos 8 PowerTools<br>> add ninja to dockerfiles, CI configurations and test VMs<br>> build: replace ninjatool with ninja<br>><br>> .cirrus.yml | 6 +-<br>> .travis.yml | 13 +<br>> Makefile | 42 +-<br>> configure | 29 +-<br>> docs/devel/build-system.rst | 6 +-<br>> meson.build | 34 +-<br>> scripts/mtest2make.py | 4 +-<br>> scripts/ninjatool.py | 1008 --------------------<br>> tests/Makefile.include | 2 +-<br>> tests/docker/dockerfiles/centos7.docker | 1 +<br>> tests/docker/dockerfiles/centos8.docker | 4 +<br>> tests/docker/dockerfiles/debian10.docker | 1 +<br>> tests/docker/dockerfiles/fedora.docker | 1 +<br>> tests/docker/dockerfiles/travis.docker | 2 +-<br>> tests/docker/dockerfiles/ubuntu.docker | 1 +<br>> tests/docker/dockerfiles/ubuntu1804.docker | 1 +<br>> tests/docker/dockerfiles/ubuntu2004.docker | 1 +<br>> tests/include/meson.build | 8 +-<br>> tests/meson.build | 14 +-<br>> tests/vm/centos | 2 +-<br>> tests/vm/centos.aarch64 | 2 +-<br>> tests/vm/fedora | 2 +-<br>> tests/vm/freebsd | 1 +<br>> tests/vm/netbsd | 1 +<br>> tests/vm/openbsd | 1 +<br>> tests/vm/ubuntu.aarch64 | 2 +-<br>> tests/vm/ubuntu.i386 | 2 +-<br>> 27 files changed, 121 insertions(+), 1070 deletions(-)<br>> delete mode 100755 scripts/ninjatool.py<br>><br>> --<br>> 2.26.2<br>><br>><br>That's good, it's broken my locally rST doc build <div>```</div><div>[226/1564] Generating input-keymap-qnum-to-qcode.c.inc with a custom command (wrapped by meson to capture output)<br>[227/1564] Generating input-keymap-qcode-to-qnum.c.inc with a custom command (wrapped by meson to capture output)<br>[228/1564] Generating shared QAPI source files with a custom command<br>[229/1564] Generating input-keymap-qcode-to-linux.c.inc with a custom command (wrapped by meson to capture output)<br>[230/1564] Generating input-keymap-win32-to-qcode.c.inc with a custom command (wrapped by meson to capture output)<br>[231/1564] Generating QAPI rST doc with a custom command<br>FAILED: tests/qapi-schema/doc-good.txt<br>"/mingw64/bin/sphinx-build" "-Dversion=5.1.50" "-Drelease=" "-W" "-b" "text" "-E" "-c" "C:/work/xemu/qemu/docs" "-D" "master_doc=doc-good" "C:/work/xemu/qemu/tests/qapi-schema" "C:/work/xemu/qemu/build/tests/qapi-schema"<br>CreateProcess failed: The system cannot find the file specified.<br>[232/1564] Generating input-keymap-qcode-to-atset1.c.inc with a custom command (wrapped by meson to capture output)<br>[233/1564] Generating generated-helpers-wrappers.h with a custom command (wrapped by meson to capture output)<br>ninja: fatal: GetOverlappedResult: �����Ч��<br><br><br>make: *** [Makefile:87：run-ninja] 错误 1<br>/mingw64/bin/ninja -v -j1 tests/check-block-qdict.exe tests/check-qdict.exe tests/check-qjson.exe tests/check-qlist.exe tests/check-qlit.exe tests/check-qnull.exe tests/check-qnum.exe tests/check-qobject.exe tests/check-qom-interface.exe tests/check-qom-proplist.exe tests/check-qstring.exe tests/ptimer-test.exe tests/qapi-schema/doc-good.ref.nocr tests/qapi-schema/doc-good.txt.nocr tests/rcutorture.exe tests/test-aio-multithread.exe tests/test-aio.exe tests/test-authz-list.exe tests/test-authz-listfile.exe tests/test-authz-simple.exe tests/test-base64.exe tests/test-bdrv-drain.exe tests/test-bdrv-graph-mod.exe tests/test-bitcnt.exe tests/test-bitmap.exe tests/test-bitops.exe tests/test-block-backend.exe tests/test-block-iothread.exe tests/test-blockjob-txn.exe tests/test-blockjob.exe tests/test-bufferiszero.exe tests/test-char.exe tests/test-clone-visitor.exe tests/test-coroutine.exe tests/test-crypto-afsplit.exe tests/test-crypto-block.exe tests/test-crypto-cipher.exe tests/test-crypto-hash.exe tests/test-crypto-hmac.exe tests/test-crypto-ivgen.exe tests/test-crypto-pbkdf.exe tests/test-crypto-secret.exe tests/test-cutils.exe tests/test-hbitmap.exe tests/test-int128.exe tests/test-io-channel-buffer.exe tests/test-io-channel-command.exe tests/test-io-channel-file.exe tests/test-io-channel-socket.exe tests/test-io-task.exe tests/test-iov.exe tests/test-keyval.exe tests/test-logging.exe tests/test-mul64.exe tests/test-opts-visitor.exe tests/test-qapi-util.exe tests/test-qdev-global-props.exe tests/test-qdist.exe tests/test-qemu-opts.exe tests/test-qgraph.exe tests/test-qht.exe tests/test-qmp-cmds.exe tests/test-qmp-event.exe tests/test-qobject-input-visitor.exe tests/test-qobject-output-visitor.exe tests/test-rcu-list.exe tests/test-rcu-simpleq.exe tests/test-rcu-slist.exe tests/test-rcu-tailq.exe tests/test-replication.exe tests/test-shift128.exe tests/test-string-input-visitor.exe tests/test-string-output-visitor.exe tests/test-thread-pool.exe tests/test-throttle.exe tests/test-timed-average.exe tests/test-util-sockets.exe tests/test-uuid.exe tests/test-visitor-serialization.exe tests/test-vmstate.exe tests/test-write-threshold.exe tests/test-x86-cpuid.exe tests/test-xbzrle.exe | cat<br></div><div>```<br>That's because currently sphinx-build are found in configure not in meson, maybe it's worth to move the sphinx related thing into meson totally.<br>--</div><div><br> 此致<br>礼<br>罗勇刚<br>Yours<br> sincerely,<br>Yonggang Luo</div></div>
On 15/10/2020 18:39, Volker Rümelin wrote: >> Thanks Paolo, >> >> Then only the issue regarding the pcbios/optionrom stuff remains ;-) >> >> make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. >> make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 >> make: *** Waiting for unfinished jobs.... >> >> Best, >> Howard >> > > Hi Howard, > > one solution for this issue is to uncomment the following line in msys2_shell.cmd. > > rem To activate windows native symlinks uncomment next line > set MSYS=winsymlinks:nativestrict > > Then tell Windows 10 it's okay to create symlinks without elevated rights. Here is a link with a description how to do this. > https://www.joshkel.com/2018/01/18/symlinks-in-windows/ > > I think since commit bf708f3c4a "optionrom: simplify Makefile" pc-bios/optionrom/Makefile in your build directory has to be a symbolic link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the Makefile instead of creating a symbolic link. Wow thanks for this. I had the same issue as Howard, and I can confirm that this fixes the problem for me. Having said that it was quite fiddly to get this working correctly - is there any reason why https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html still can't be merged? ATB, Mark.
Il gio 15 ott 2020, 20:49 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> ha scritto: > Is there any reason why > https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html still > can't be > merged? > Because it's not the right approach. There is no reason why building firmware cannot be done with cross compilers, so moving those directories to Meson (not because Meson can't handle them; more specifically, the issue is tying the firmware build to the QEMU build system) is going in the wrong direction. The "Canadian cross" scenario, where you build on Linux a mingw GCC but the compiler is s390, is not even enough to describe the complexity in the case of QEMU, because there are multiple firmware for different machines. However we already have all the infrastructure to do such builds, we just don't use it for the firmware. So, instead of the patch you recalled above, the tests/tcg machinery should be extended into something that can be reused for firmware. As an aside, orchestrating this multi-compiler part of the build is what the Makefiles will keep on handling for the foreseeable future. As an aside to the aside, tests/tcg is more than underdocumented and I forget everything about it 5 minutes after looking at it. This is not something that I will be able to work on anytime soon. But still I don't think that going in the wrong direction is a good idea, even if temporarily. Paolo > > ATB, > > Mark. > > <div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il gio 15 ott 2020, 20:49 Mark Cave-Ayland <<a href="mailto:mark.cave-ayland@ilande.co.uk">mark.cave-ayland@ilande.co.uk</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Is there any reason why <br> <a href="https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html" rel="noreferrer noreferrer" target="_blank">https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html</a> still can't be <br> merged?<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Because it's not the right approach. There is no reason why building firmware cannot be done with cross compilers, so moving those directories to Meson (not because Meson can't handle them; more specifically, the issue is tying the firmware build to the QEMU build system) is going in the wrong direction.</div><div dir="auto"><br></div><div dir="auto">The "Canadian cross" scenario, where you build on Linux a mingw GCC but the compiler is s390, is not even enough to describe the complexity in the case of QEMU, because there are multiple firmware for different machines.</div><div dir="auto"><br></div><div dir="auto">However we already have all the infrastructure to do such builds, we just don't use it for the firmware. So, instead of the patch you recalled above, the tests/tcg machinery should be extended into something that can be reused for firmware. As an aside, orchestrating this multi-compiler part of the build is what the Makefiles will keep on handling for the foreseeable future. As an aside to the aside, tests/tcg is more than underdocumented and I forget everything about it 5 minutes after looking at it.</div><div dir="auto"><br></div><div dir="auto">This is not something that I will be able to work on anytime soon. But still I don't think that going in the wrong direction is a good idea, even if temporarily.</div><div dir="auto"><br></div><div dir="auto">Paolo</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> <br> ATB,<br> <br> Mark.<br> <br> </blockquote></div></div></div>
On Thu, Oct 15, 2020 at 7:39 PM Volker Rümelin <vr_qemu@t-online.de> wrote: > > > > > Thanks Paolo, > > > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. > Stop. > > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > > make: *** Waiting for unfinished jobs.... > > > > Best, > > Howard > > > > Hi Howard, > > one solution for this issue is to uncomment the following line in > msys2_shell.cmd. > > rem To activate windows native symlinks uncomment next line > set MSYS=winsymlinks:nativestrict > > Then tell Windows 10 it's okay to create symlinks without elevated rights. > Here is a link with a description how to do this. > https://www.joshkel.com/2018/01/18/symlinks-in-windows/ > > I think since commit bf708f3c4a "optionrom: simplify Makefile" > pc-bios/optionrom/Makefile in your build directory has to be a symbolic > link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the > Makefile instead of creating a symbolic link. > > Thanks Volker! I changed the Windows policy and setting in msys2_shell.cmd. However, I had to edit ming64.ini to uncomment the MSYS=winsymlinks:nativestrict there as well to get things going. While it is great to have this fix, I can't say I'm really happy with the need to change the Windows policy and to have to fix msys2. The patches Mark referred to and to which I also pointed earlier did fix this problem without changing Windows/Msys2/Mingw64 settings. Best regards, Howard <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 15, 2020 at 7:39 PM Volker Rümelin <<a href="mailto:vr_qemu@t-online.de" target="_blank">vr_qemu@t-online.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br> ><br> > Thanks Paolo,<br> ><br> > Then only the issue regarding the pcbios/optionrom stuff remains ;-)<br> ><br> > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop.<br> > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2<br> > make: *** Waiting for unfinished jobs....<br> ><br> > Best,<br> > Howard<br> ><br> <br> Hi Howard,<br> <br> one solution for this issue is to uncomment the following line in msys2_shell.cmd.<br> <br> rem To activate windows native symlinks uncomment next line<br> set MSYS=winsymlinks:nativestrict<br> <br> Then tell Windows 10 it's okay to create symlinks without elevated rights. Here is a link with a description how to do this.<br> <a href="https://www.joshkel.com/2018/01/18/symlinks-in-windows/" rel="noreferrer" target="_blank">https://www.joshkel.com/2018/01/18/symlinks-in-windows/</a><br> <br> I think since commit bf708f3c4a "optionrom: simplify Makefile" pc-bios/optionrom/Makefile in your build directory has to be a symbolic link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the Makefile instead of creating a symbolic link.<br> <br></blockquote><div><br></div><div>Thanks Volker!</div><div><br> </div><div>I changed the Windows policy and setting in msys2_shell.cmd. However, I had to edit ming64.ini to uncomment the MSYS=winsymlinks:nativestrict there as well to get things going. </div><div><br></div><div>While it is great to have this fix, I can't say I'm really happy with the need to change the Windows policy and to have to fix msys2.</div><div><br></div><div>The patches Mark referred to and to which I also pointed earlier did fix this problem without changing Windows/Msys2/Mingw64 settings.<br></div><div><br></div><div>Best regards,</div><div>Howard<br></div><div><br></div><div><br></div></div></div>
On 15/10/2020 22:41, Paolo Bonzini wrote: > Il gio 15 ott 2020, 20:49 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk > <mailto:mark.cave-ayland@ilande.co.uk>> ha scritto: > > Is there any reason why > https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html > <https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html> still can't be > merged? > > > Because it's not the right approach. There is no reason why building firmware cannot > be done with cross compilers, so moving those directories to Meson (not because Meson > can't handle them; more specifically, the issue is tying the firmware build to the > QEMU build system) is going in the wrong direction. > > The "Canadian cross" scenario, where you build on Linux a mingw GCC but the compiler > is s390, is not even enough to describe the complexity in the case of QEMU, because > there are multiple firmware for different machines. > > However we already have all the infrastructure to do such builds, we just don't use > it for the firmware. So, instead of the patch you recalled above, the tests/tcg > machinery should be extended into something that can be reused for firmware. As an > aside, orchestrating this multi-compiler part of the build is what the Makefiles will > keep on handling for the foreseeable future. As an aside to the aside, tests/tcg is > more than underdocumented and I forget everything about it 5 minutes after looking at it. > > This is not something that I will be able to work on anytime soon. But still I don't > think that going in the wrong direction is a good idea, even if temporarily. That's a shame, although I do appreciate the huge amount of time and effort that you've put into this release in order to get the Meson build up and running, and so why taking on another large task is going to be lower down the list :) At the moment OpenBIOS doesn't have a docker image capable of building the required binaries: I did experiment with trying to use the QEMU docker images for openbios-ppc but whilst the binary built successfully, it did not run compared to my hand-rolled compilers. So there's still some debugging to be done there... ATB, Mark.