[4/7] docs/devel: add information on how to setup build environments

MacOS and Linux are straightforward, but Windows needs a bit more
details.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 docs/about/build-platforms.rst   |   4 +-
 docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++
 docs/devel/index-build.rst       |   1 +
 3 files changed, 118 insertions(+), 1 deletion(-)
 create mode 100644 docs/devel/build-environment.rst

On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote:
> MacOS and Linux are straightforward, but Windows needs a bit more
> details.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> ---
>  docs/about/build-platforms.rst   |   4 +-
>  docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++
>  docs/devel/index-build.rst       |   1 +
>  3 files changed, 118 insertions(+), 1 deletion(-)
>  create mode 100644 docs/devel/build-environment.rst
> 
> diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
> index 6102f00aec0..c1ea53db834 100644
> --- a/docs/about/build-platforms.rst
> +++ b/docs/about/build-platforms.rst
> @@ -29,6 +29,9 @@ The `Repology`_ site is a useful resource to identify
>  currently shipped versions of software in various operating systems,
>  though it does not cover all distros listed below.
>  
> +You can find how to install build dependencies for different systems on the
> +:ref:`setup-build-env` page.
> +
>  Supported host architectures
>  ----------------------------
>  
> @@ -130,7 +133,6 @@ Optional build dependencies
>    cross compilation using ``docker`` or ``podman``, or to use pre-built
>    binaries distributed with QEMU.
>  
> -
>  Windows
>  -------
>  
> diff --git a/docs/devel/build-environment.rst b/docs/devel/build-environment.rst
> new file mode 100644
> index 00000000000..d9a66f5fcc6
> --- /dev/null
> +++ b/docs/devel/build-environment.rst
> @@ -0,0 +1,114 @@
> +
> +.. _setup-build-env:
> +
> +Setup build environment
> +=======================
> +
> +QEMU uses a lot of dependencies on the host system. glib2 is used everywhere in
> +the code base, and most of the other dependencies are optional.
> +
> +We present here simple instructions to enable native builds on most popular
> +systems.
> +
> +You can find additional instructions on `QEMU wiki <https://wiki.qemu.org/>`_:
> +
> +- `Linux <https://wiki.qemu.org/Hosts/Linux>`_
> +- `MacOS <https://wiki.qemu.org/Hosts/Mac>`_
> +- `Windows <https://wiki.qemu.org/Hosts/W32>`_
> +- `BSD <https://wiki.qemu.org/Hosts/BSD>`_


We generally suffer from having information spread over multiple sources,
giving us the burden of keeping the different places consistent, which we
pretty consistently fail at.

It is a good think to add build env docs to qemu.git where we actually
have oversight / review to catch mistakes. If we do this though, IMHO,
we should be deleting the wiki pages and making them into 302 redirects
to our new in-tree docs.

> +
> +Linux
> +-----
> +
> +Fedora
> +++++++
> +
> +::
> +
> +    sudo dnf update && sudo dnf builddep qemu
> +
> +Debian/Ubuntu
> ++++++++++++++
> +
> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_.
> +Then, use apt to install dependencies:
> +
> +::
> +
> +    sudo apt update && sudo apt build-dep qemu
> +
> +MacOS
> +-----
> +
> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to
> +install dependencies:
> +
> +::
> +
> +    brew update && brew install $(brew deps --include-build qemu)


The downside in recommending the 'build dep' approach is that it misses
out on deps that have been newly introduced in qemu.git, since whatever
old version of QEMU the distros are shipping.  It also misses deps for
any features the distro vendor has decided to exclude.

Can we put a caveat describing this limitation at the top, so that users
have pointer if things don't quite go the way we expected.

> +
> +Windows
> +-------
> +
> +You first need to install `MSYS2 <https://www.msys2.org/>`_.
> +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_.
> +x86_64 environments are based on GCC, while aarch64 is based on Clang.
> +
> +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64
> +(only available on windows-aarch64 hosts).

Does CLANGARM64 really work with QEMU ?   We go out of our way to actively
block the use of CLang for Windows because of its lack of support for
'gcc_struct' attributes, so I would have expected it to fail

> +
> +Then, you can open a windows shell, and enter msys2 env using:
> +
> +::
> +
> +    c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64
> +    # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments.
> +
> +MSYS2 package manager does not offer a built-in way to install build
> +dependencies. You can start with this list of packages using pacman:
> +

With regards,
Daniel

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote:
>> MacOS and Linux are straightforward, but Windows needs a bit more
>> details.
>> 
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>> ---
>>  docs/about/build-platforms.rst   |   4 +-
>>  docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++
>>  docs/devel/index-build.rst       |   1 +
>>  3 files changed, 118 insertions(+), 1 deletion(-)
>>  create mode 100644 docs/devel/build-environment.rst
>> 
<snip>
>> +Fedora
>> +++++++
>> +
>> +::
>> +
>> +    sudo dnf update && sudo dnf builddep qemu
>> +
>> +Debian/Ubuntu
>> ++++++++++++++
>> +
>> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_.
>> +Then, use apt to install dependencies:
>> +
>> +::
>> +
>> +    sudo apt update && sudo apt build-dep qemu
>> +
>> +MacOS
>> +-----
>> +
>> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to
>> +install dependencies:
>> +
>> +::
>> +
>> +    brew update && brew install $(brew deps --include-build qemu)
>
>
> The downside in recommending the 'build dep' approach is that it misses
> out on deps that have been newly introduced in qemu.git, since whatever
> old version of QEMU the distros are shipping.  It also misses deps for
> any features the distro vendor has decided to exclude.
>
> Can we put a caveat describing this limitation at the top, so that users
> have pointer if things don't quite go the way we expected.

At a recent QEMU workshop I presented I pointed to lcitool as a way to
list the minimal dependencies QEMU needs:

  ./tests/lcitool/libvirt-ci/bin/lcitool variables -f yaml debian-12 ./tests/lcitool/projects/qemu-minimal.yml

It wouldn't be the hardest thing to extend lcitool to generate a pkg
install line instead of a list. Although I notice it fails with the full
set of dependancies:

  ✗  ./tests/lcitool/libvirt-ci/bin/lcitool -d tests/lcitool variables -f shell debian-12 ./tests/lcitool/projects/qemu.yml
  Package generic name resolution error: Package libcbor not present in mappings

I note there is already an "install" action but I'm wary of what it does.

>
>> +
>> +Windows
>> +-------
>> +
>> +You first need to install `MSYS2 <https://www.msys2.org/>`_.
>> +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_.
>> +x86_64 environments are based on GCC, while aarch64 is based on Clang.
>> +
>> +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64
>> +(only available on windows-aarch64 hosts).
>
> Does CLANGARM64 really work with QEMU ?   We go out of our way to actively
> block the use of CLang for Windows because of its lack of support for
> 'gcc_struct' attributes, so I would have expected it to fail
>
>> +
>> +Then, you can open a windows shell, and enter msys2 env using:
>> +
>> +::
>> +
>> +    c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64
>> +    # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments.
>> +
>> +MSYS2 package manager does not offer a built-in way to install build
>> +dependencies. You can start with this list of packages using pacman:
>> +
>
> With regards,
> Daniel

On Tue, Nov 19, 2024 at 11:08:12AM +0000, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote:
> >> MacOS and Linux are straightforward, but Windows needs a bit more
> >> details.
> >> 
> >> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> >> ---
> >>  docs/about/build-platforms.rst   |   4 +-
> >>  docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++
> >>  docs/devel/index-build.rst       |   1 +
> >>  3 files changed, 118 insertions(+), 1 deletion(-)
> >>  create mode 100644 docs/devel/build-environment.rst
> >> 
> <snip>
> >> +Fedora
> >> +++++++
> >> +
> >> +::
> >> +
> >> +    sudo dnf update && sudo dnf builddep qemu
> >> +
> >> +Debian/Ubuntu
> >> ++++++++++++++
> >> +
> >> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_.
> >> +Then, use apt to install dependencies:
> >> +
> >> +::
> >> +
> >> +    sudo apt update && sudo apt build-dep qemu
> >> +
> >> +MacOS
> >> +-----
> >> +
> >> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to
> >> +install dependencies:
> >> +
> >> +::
> >> +
> >> +    brew update && brew install $(brew deps --include-build qemu)
> >
> >
> > The downside in recommending the 'build dep' approach is that it misses
> > out on deps that have been newly introduced in qemu.git, since whatever
> > old version of QEMU the distros are shipping.  It also misses deps for
> > any features the distro vendor has decided to exclude.
> >
> > Can we put a caveat describing this limitation at the top, so that users
> > have pointer if things don't quite go the way we expected.
> 
> At a recent QEMU workshop I presented I pointed to lcitool as a way to
> list the minimal dependencies QEMU needs:
> 
>   ./tests/lcitool/libvirt-ci/bin/lcitool variables -f yaml debian-12 ./tests/lcitool/projects/qemu-minimal.yml
> 
> It wouldn't be the hardest thing to extend lcitool to generate a pkg
> install line instead of a list. Although I notice it fails with the full
> set of dependancies:
> 
>   ✗  ./tests/lcitool/libvirt-ci/bin/lcitool -d tests/lcitool variables -f shell debian-12 ./tests/lcitool/projects/qemu.yml
>   Package generic name resolution error: Package libcbor not present in mappings

I think your libvirt-ci git submodule might be out of date. The current
generated Dockerfiles include libcbor and its in the mappings.

> I note there is already an "install" action but I'm wary of what it does.

That's related to lcitool functionality for provisioning VMs - kinda like
QEMUs tests/vm/ stuff.

With regards,
Daniel