diff mbox series

[1/9] tests/docker: Add debian-nios2-cross image

Message ID 20211001153347.1736014-2-richard.henderson@linaro.org
State New
Headers show
Series [1/9] tests/docker: Add debian-nios2-cross image | expand

Commit Message

Richard Henderson Oct. 1, 2021, 3:33 p.m. UTC 
Build the entire cross tool chain from source.
For this reason, default to caching.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

---
 tests/docker/Makefile.include                 | 19 ++++
 .../dockerfiles/debian-nios2-cross.docker     | 34 +++++++
 .../build-toolchain.sh                        | 97 +++++++++++++++++++
 3 files changed, 150 insertions(+)
 create mode 100644 tests/docker/dockerfiles/debian-nios2-cross.docker
 create mode 100755 tests/docker/dockerfiles/debian-nios2-cross.docker.d/build-toolchain.sh

-- 
2.25.1

Comments

Alex Bennée Oct. 12, 2021, 5:03 p.m. UTC | #1 
Richard Henderson <richard.henderson@linaro.org> writes:

> Build the entire cross tool chain from source.

> For this reason, default to caching.

>

> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

> ---

>  tests/docker/Makefile.include                 | 19 ++++

>  .../dockerfiles/debian-nios2-cross.docker     | 34 +++++++

>  .../build-toolchain.sh                        | 97 +++++++++++++++++++

>  3 files changed, 150 insertions(+)

>  create mode 100644 tests/docker/dockerfiles/debian-nios2-cross.docker

>  create mode 100755 tests/docker/dockerfiles/debian-nios2-cross.docker.d/build-toolchain.sh

>

> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include

> index ff5d732889..2ccd93caa4 100644

> --- a/tests/docker/Makefile.include

> +++ b/tests/docker/Makefile.include

> @@ -168,10 +168,28 @@ docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.dock

>  				qemu/debian-hexagon-cross --add-current-user,		\

>  			"PREPARE", "debian-hexagon-cross"))


We need^H^H^H^H (might need, see bellow) a separate patch with:

docker-image-debian10: NOUSER=1

to ensure the images we base our "handbuilt" compilers on don't include
a potentially clashing uid (which should only be added for local
builds).


> +docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-nios2-cross.docker

> +	$(if $(NOCACHE), 								\

> +		$(call quiet-command,							\

> +			$(DOCKER_SCRIPT) build -t qemu/debian-nios2-cross -f $< 	\

> +			$(if $V,,--quiet) --no-cache 					\

> +			--registry $(DOCKER_REGISTRY) --extra-files			\

> +			$(DOCKER_FILES_DIR)/debian-nios2-cross.docker.d/build-toolchain.sh, \

> +			"BUILD", "debian-nios2-cross"),				        \

> +		$(call quiet-command,							\

> +			$(DOCKER_SCRIPT) fetch $(if $V,,--quiet)			\

> +				qemu/debian-nios2-cross $(DOCKER_REGISTRY),		\

> +			"FETCH", "debian-nios2-cross")				        \

> +		$(call quiet-command,							\

> +			$(DOCKER_SCRIPT) update $(if $V,,--quiet) 			\

> +				qemu/debian-nios2-cross --add-current-user,		\

> +			"PREPARE", "debian-nios2-cross"))

> +


Could we update the comment above to something like:

  #
  # The build rule for these cross compilers are special in so far for
  # most of the time we don't want to build them. While dockers caching
  # does avoid this most of the time sometimes we want to force the
  # issue. Also we want to ensure the image they are based on hasn't
  # been polluted with our UID because it will fail when users build
  # their local version from what has been pushed upstream.
  #

>  # Specialist build images, sometimes very limited tools

>  docker-image-debian-tricore-cross: docker-image-debian10

>  docker-image-debian-all-test-cross: docker-image-debian10

>  docker-image-debian-arm64-test-cross: docker-image-debian11

> +docker-image-debian-nios2-cross: docker-image-debian10

>  docker-image-debian-powerpc-test-cross: docker-image-debian11

>  

>  # These images may be good enough for building tests but not for test builds

> @@ -180,6 +198,7 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross

>  DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross

>  DOCKER_PARTIAL_IMAGES += debian-hppa-cross

>  DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross

> +DOCKER_PARTIAL_IMAGES += debian-nios2-cross

>  DOCKER_PARTIAL_IMAGES += debian-riscv64-cross

>  DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross

>  DOCKER_PARTIAL_IMAGES += debian-tricore-cross

> diff --git a/tests/docker/dockerfiles/debian-nios2-cross.docker b/tests/docker/dockerfiles/debian-nios2-cross.docker

> new file mode 100644

> index 0000000000..208737fc5e

> --- /dev/null

> +++ b/tests/docker/dockerfiles/debian-nios2-cross.docker

> @@ -0,0 +1,34 @@

> +#

> +# Docker NIOS2 cross-compiler target

> +#

> +# This docker target is used for building tests. As it also needs to be

> +# able to build QEMU itself in CI we include it's build-deps. It is also

> +# a "stand-alone" image so as not to be triggered by re-builds on other

> +# base images given it takes a long time to build.

> +#

> +FROM qemu/debian10

> +

> +# Install build utilities for building gcc and glibc.

> +# ??? The build-dep isn't working, missing a number of

> +# minimal build dependiencies, e.g. libmpc.

> +

> +RUN apt update && \

> +    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \

> +    DEBIAN_FRONTEND=noninteractive eatmydata \

> +    apt install -y --no-install-recommends \

> +        bison \

> +        flex \

> +        gawk \

> +        libmpc-dev \

> +        libmpfr-dev \

> +        rsync \

> +        texinfo \

> +        wget \

> +        $(apt-get -s build-dep --arch-only gcc | egrep ^Inst | fgrep '[all]' | cut -d\  -f2) \

> +        $(apt-get -s build-dep --arch-only glibc | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)

> +

> +ADD build-toolchain.sh /root/build-toolchain.sh

> +

> +RUN cd /root && ./build-toolchain.sh


We need to split this like in hexagon and have a second stage which does
a:

  COPY --from=0 /usr/local /usr/local

This will limit the size of the final image (and also avoid duplicting
the UID in the hexagon build).

-- 
Alex Bennée
Richard Henderson Oct. 12, 2021, 6:24 p.m. UTC | #2 
On 10/12/21 10:03 AM, Alex Bennée wrote:
> We need to split this like in hexagon and have a second stage which does

> a:

> 

>    COPY --from=0 /usr/local /usr/local

> 

> This will limit the size of the final image (and also avoid duplicting

> the UID in the hexagon build).


Yeah, well, I had to take that out because it errors out.
I have no idea what that does or means.


r~
Alex Bennée Oct. 12, 2021, 6:33 p.m. UTC | #3 
It copies from the first stage. Is the compiler installed to a different
path?

On Tue, 12 Oct 2021, 19:24 Richard Henderson, <richard.henderson@linaro.org>
wrote:

> On 10/12/21 10:03 AM, Alex Bennée wrote:

> > We need to split this like in hexagon and have a second stage which does

> > a:

> >

> >    COPY --from=0 /usr/local /usr/local

> >

> > This will limit the size of the final image (and also avoid duplicting

> > the UID in the hexagon build).

>

> Yeah, well, I had to take that out because it errors out.

> I have no idea what that does or means.

>

>

> r~

>
<div dir="auto">It copies from the first stage. Is the compiler installed to a different path?</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 12 Oct 2021, 19:24 Richard Henderson, &lt;<a href="mailto:richard.henderson@linaro.org">richard.henderson@linaro.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 10/12/21 10:03 AM, Alex Bennée wrote:<br>
&gt; We need to split this like in hexagon and have a second stage which does<br>
&gt; a:<br>
&gt; <br>
&gt;    COPY --from=0 /usr/local /usr/local<br>
&gt; <br>
&gt; This will limit the size of the final image (and also avoid duplicting<br>
&gt; the UID in the hexagon build).<br>
<br>
Yeah, well, I had to take that out because it errors out.<br>
I have no idea what that does or means.<br>
<br>
<br>
r~<br>
</blockquote></div>
Alex Bennée Oct. 14, 2021, 9:34 a.m. UTC | #4 
Richard Henderson <richard.henderson@linaro.org> writes:

> On 10/12/21 10:03 AM, Alex Bennée wrote:

>> We need to split this like in hexagon and have a second stage which does

>> a:

>>    COPY --from=0 /usr/local /usr/local

>> This will limit the size of the final image (and also avoid

>> duplicting

>> the UID in the hexagon build).

>

> Yeah, well, I had to take that out because it errors out.

> I have no idea what that does or means.


Having this worked for me:

modified   tests/docker/dockerfiles/debian-nios2-cross.docker
@@ -31,4 +31,14 @@ ADD build-toolchain.sh /root/build-toolchain.sh
 
 RUN cd /root && ./build-toolchain.sh
 
+FROM debian:buster-slim
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+# Install QEMU build deps for use in CI
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+    DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy git ninja-build && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt build-dep -yy --arch-only qemu
+COPY --from=0 /usr/local /usr/local
 ENV PATH $PATH:/usr/local/bin/

-- 
Alex Bennée
Richard Henderson Oct. 14, 2021, 5:17 p.m. UTC | #5 
On 10/12/21 10:03 AM, Alex Bennée wrote:
> We need^H^H^H^H (might need, see bellow) a separate patch with:

> 

> docker-image-debian10: NOUSER=1

> 

> to ensure the images we base our "handbuilt" compilers on don't include

> a potentially clashing uid (which should only be added for local

> builds).


I think we ought to push this further back into docker.py.

I think that we should always have a separate image without the user installed.  When 
asking for a build with user installed, copy the nouser image (perhaps cached) and simply 
add the user.

> We need to split this like in hexagon and have a second stage which does

> a:

> 

>    COPY --from=0 /usr/local /usr/local

> 

> This will limit the size of the final image (and also avoid duplicting

> the UID in the hexagon build).


What are we eliminating from the image with the second FROM, since we then go back and add 
in all the build-dep for qemu.

There are one or two extra packages required for building gcc itself (e.g. libmpc) but 
they're not large.  Where does the major savings come from?


r~
Alex Bennée Oct. 14, 2021, 5:25 p.m. UTC | #6 
Richard Henderson <richard.henderson@linaro.org> writes:

> On 10/12/21 10:03 AM, Alex Bennée wrote:

>> We need^H^H^H^H (might need, see bellow) a separate patch with:

>> docker-image-debian10: NOUSER=1

>> to ensure the images we base our "handbuilt" compilers on don't

>> include

>> a potentially clashing uid (which should only be added for local

>> builds).

>

> I think we ought to push this further back into docker.py.

>

> I think that we should always have a separate image without the user

> installed.  When asking for a build with user installed, copy the

> nouser image (perhaps cached) and simply add the user.

>

>> We need to split this like in hexagon and have a second stage which does

>> a:

>>    COPY --from=0 /usr/local /usr/local

>> This will limit the size of the final image (and also avoid

>> duplicting

>> the UID in the hexagon build).

>

> What are we eliminating from the image with the second FROM, since we

> then go back and add in all the build-dep for qemu.

>

> There are one or two extra packages required for building gcc itself

> (e.g. libmpc) but they're not large.  Where does the major savings

> come from?


Hmm that is curious:

 $ docker history registry.gitlab.com/qemu-project/qemu/qemu/debian-nios2-cross:latest
 ...
  <missing>      2 weeks ago   RUN /bin/sh -c id rth 2>/dev/null || useradd…   330kB     buildkit.dockerfile.v0
  <missing>      2 weeks ago   LABEL com.qemu./home/rth/qemu/git-alt/tests/…   0B        buildkit.dockerfile.v0
  <missing>      2 weeks ago   LABEL com.qemu.dockerfile-checksum=fc3e779ae…   0B        buildkit.dockerfile.v0
  <missing>      2 weeks ago   ENV PATH=/usr/local/sbin:/usr/local/bin:/usr…   0B        buildkit.dockerfile.v0
  <missing>      2 weeks ago   RUN /bin/sh -c cd /root && ./build-toolchain…   973MB     buildkit.dockerfile.v0
  <missing>      2 weeks ago   ADD build-toolchain.sh /root/build-toolchain…   2.14kB    buildkit.dockerfile.v0
  <missing>      2 weeks ago   RUN /bin/sh -c apt update &&     DEBIAN_FRON…   17.3MB    buildkit.dockerfile.v0
  <missing>      2 weeks ago   /bin/sh -c #(nop)  LABEL com.qemu.dockerfile…   0B   
  <missing>      2 weeks ago   /bin/sh -c apt update &&     DEBIAN_FRONTEND…   715MB
  <missing>      2 weeks ago   /bin/sh -c cat /etc/apt/sources.list | sed "…   854B              
  <missing>      2 weeks ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B                               
  <missing>      2 weeks ago   /bin/sh -c #(nop) ADD file:99db7cfe7952a1c7a…   69.2MB                           

yet with the split build:

  $ docker image history qemu/debian-nios2-cross:latest                                                        
  IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT               
  24fbdeba931c   4 hours ago    LABEL com.qemu./home/alex.bennee/lsrc/qemu.g…   0B        buildkit.dockerfile.v0
  <missing>      4 hours ago    LABEL com.qemu.dockerfile-checksum=e7d3631e6…   0B        buildkit.dockerfile.v0
  <missing>      4 hours ago    ENV PATH=/usr/local/sbin:/usr/local/bin:/usr…   0B        buildkit.dockerfile.v0
  <missing>      4 hours ago    COPY /usr/local /usr/local # buildkit           1.77GB    buildkit.dockerfile.v0
  <missing>      4 hours ago    RUN /bin/sh -c apt update &&     DEBIAN_FRON…   781MB     buildkit.dockerfile.v0
  <missing>      4 hours ago    RUN /bin/sh -c cat /etc/apt/sources.list | s…   854B      buildkit.dockerfile.v0
  <missing>      7 months ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B                             
  <missing>      7 months ago   /bin/sh -c #(nop) ADD file:3c32f1cd03198e141…   69.2MB          

which makes it look like the artefacts are bigger than the whole build.

-- 
Alex Bennée
diff mbox series

Patch

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ff5d732889..2ccd93caa4 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -168,10 +168,28 @@  docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.dock
 				qemu/debian-hexagon-cross --add-current-user,		\
 			"PREPARE", "debian-hexagon-cross"))
 
+docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-nios2-cross.docker
+	$(if $(NOCACHE), 								\
+		$(call quiet-command,							\
+			$(DOCKER_SCRIPT) build -t qemu/debian-nios2-cross -f $< 	\
+			$(if $V,,--quiet) --no-cache 					\
+			--registry $(DOCKER_REGISTRY) --extra-files			\
+			$(DOCKER_FILES_DIR)/debian-nios2-cross.docker.d/build-toolchain.sh, \
+			"BUILD", "debian-nios2-cross"),				        \
+		$(call quiet-command,							\
+			$(DOCKER_SCRIPT) fetch $(if $V,,--quiet)			\
+				qemu/debian-nios2-cross $(DOCKER_REGISTRY),		\
+			"FETCH", "debian-nios2-cross")				        \
+		$(call quiet-command,							\
+			$(DOCKER_SCRIPT) update $(if $V,,--quiet) 			\
+				qemu/debian-nios2-cross --add-current-user,		\
+			"PREPARE", "debian-nios2-cross"))
+
 # Specialist build images, sometimes very limited tools
 docker-image-debian-tricore-cross: docker-image-debian10
 docker-image-debian-all-test-cross: docker-image-debian10
 docker-image-debian-arm64-test-cross: docker-image-debian11
+docker-image-debian-nios2-cross: docker-image-debian10
 docker-image-debian-powerpc-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
@@ -180,6 +198,7 @@  DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
 DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
 DOCKER_PARTIAL_IMAGES += debian-hppa-cross
 DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
+DOCKER_PARTIAL_IMAGES += debian-nios2-cross
 DOCKER_PARTIAL_IMAGES += debian-riscv64-cross
 DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
 DOCKER_PARTIAL_IMAGES += debian-tricore-cross
diff --git a/tests/docker/dockerfiles/debian-nios2-cross.docker b/tests/docker/dockerfiles/debian-nios2-cross.docker
new file mode 100644
index 0000000000..208737fc5e
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-nios2-cross.docker
@@ -0,0 +1,34 @@ 
+#
+# Docker NIOS2 cross-compiler target
+#
+# This docker target is used for building tests. As it also needs to be
+# able to build QEMU itself in CI we include it's build-deps. It is also
+# a "stand-alone" image so as not to be triggered by re-builds on other
+# base images given it takes a long time to build.
+#
+FROM qemu/debian10
+
+# Install build utilities for building gcc and glibc.
+# ??? The build-dep isn't working, missing a number of
+# minimal build dependiencies, e.g. libmpc.
+
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y --no-install-recommends \
+        bison \
+        flex \
+        gawk \
+        libmpc-dev \
+        libmpfr-dev \
+        rsync \
+        texinfo \
+        wget \
+        $(apt-get -s build-dep --arch-only gcc | egrep ^Inst | fgrep '[all]' | cut -d\  -f2) \
+        $(apt-get -s build-dep --arch-only glibc | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
+
+ADD build-toolchain.sh /root/build-toolchain.sh
+
+RUN cd /root && ./build-toolchain.sh
+
+ENV PATH $PATH:/usr/local/bin/
diff --git a/tests/docker/dockerfiles/debian-nios2-cross.docker.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-nios2-cross.docker.d/build-toolchain.sh
new file mode 100755
index 0000000000..d8cb428dab
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-nios2-cross.docker.d/build-toolchain.sh
@@ -0,0 +1,97 @@ 
+#!/bin/bash
+
+set -e
+
+TARGET=nios2-linux-gnu
+LINUX_ARCH=nios2
+
+J=$(expr $(nproc) / 2)
+TOOLCHAIN_INSTALL=/usr/local
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+CROSS_SYSROOT=${TOOLCHAIN_INSTALL}/$TARGET/sys-root
+
+export PATH=${TOOLCHAIN_BIN}:$PATH
+
+mkdir cross
+cd cross
+
+#
+# Grab all of the source for the toolchain bootstrap.
+#
+
+wget https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.xz
+wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz
+wget https://ftp.gnu.org/gnu/glibc/glibc-2.34.tar.xz
+wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.70.tar.xz
+
+tar axf binutils-2.37.tar.xz
+tar axf gcc-11.2.0.tar.xz
+tar axf glibc-2.34.tar.xz
+tar axf linux-5.10.70.tar.xz
+
+mv binutils-2.37 src-binu
+mv gcc-11.2.0 src-gcc
+mv glibc-2.34 src-glibc
+mv linux-5.10.70 src-linux
+
+mkdir -p bld-hdr bld-binu bld-gcc bld-glibc
+mkdir -p ${CROSS_SYSROOT}/usr/include
+
+#
+# Install kernel and glibc headers
+#
+
+cd src-linux
+make headers_install ARCH=${LINUX_ARCH} INSTALL_HDR_PATH=${CROSS_SYSROOT}/usr
+cd ..
+
+cd bld-hdr
+../src-glibc/configure --prefix=/usr --host=${TARGET}
+make install-headers DESTDIR=${CROSS_SYSROOT}
+touch ${CROSS_SYSROOT}/usr/include/gnu/stubs.h
+cd ..
+
+#
+# Build binutils
+#
+
+cd bld-binu
+../src-binu/configure --disable-werror \
+  --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET}
+make -j${J}
+make install
+cd ..
+
+#
+# Build gcc, without shared libraries, because we do not yet
+# have a shared libc against which to link.
+#
+
+cd bld-gcc
+../src-gcc/configure --disable-werror --disable-shared \
+  --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET} \
+  --enable-languages=c --disable-libssp --disable-libsanitizer \
+  --disable-libatomic --disable-libgomp --disable-libquadmath
+make -j${J}
+make install
+cd ..
+
+#
+# Build glibc
+# There are a few random things that use c++ but we didn't build that
+# cross-compiler.  We can get away without them.  Disable CXX so that
+# glibc doesn't try to use the host c++ compiler.
+#
+
+cd bld-glibc
+CXX=false ../src-glibc/configure --prefix=/usr --host=${TARGET}
+make -j${j}
+make install DESTDIR=${CROSS_SYSROOT}
+cd ..
+
+#
+# Clean up
+#
+
+cd ..
+rm -rf cross build-toolchain.sh