| Message ID | 20210914185830.1378771-2-richard.henderson@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | gitlab-ci: Add riscv64 cross builds | expand |
On 9/14/21 8:58 PM, Richard Henderson wrote: > From: Alex Bennée <alex.bennee@linaro.org> > > To be able to cross build QEMU itself we need to include a few more > libraries. These are only available in Debian's unstable ports repo > for now so we need to base the riscv64 image on sid with the the > minimal libs needed to build QEMU (glib/pixman). > > The result works but is not as clean as using build-dep to bring in > more dependencies. However sid is by definition a shifting pile of > sand and by keeping the list of libs minimal we reduce the chance of > having an image we can't build. It's good enough for a basic cross > build testing of TCG. > > Cc: "Daniel P. Berrangé" <berrange@redhat.com> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Signed-off-by: Richard Henderson <richard.henderson@linaro.org> > --- > .gitlab-ci.d/container-cross.yml | 1 - > tests/docker/Makefile.include | 2 - > .../dockerfiles/debian-riscv64-cross.docker | 46 +++++++++++++++++-- > 3 files changed, 41 insertions(+), 8 deletions(-) > > diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml > index 0fcebe363a..05996200e1 100644 > --- a/.gitlab-ci.d/container-cross.yml > +++ b/.gitlab-ci.d/container-cross.yml > @@ -134,7 +134,6 @@ ppc64el-debian-cross-container: > riscv64-debian-cross-container: > extends: .container_job_template > stage: containers-layer2 > - needs: ['amd64-debian10-container'] > variables: > NAME: debian-riscv64-cross > > diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include > index ff5d732889..3b03763186 100644 > --- a/tests/docker/Makefile.include > +++ b/tests/docker/Makefile.include > @@ -141,7 +141,6 @@ docker-image-debian-mips64-cross: docker-image-debian10 > docker-image-debian-mips64el-cross: docker-image-debian10 > docker-image-debian-mipsel-cross: docker-image-debian10 > docker-image-debian-ppc64el-cross: docker-image-debian10 > -docker-image-debian-riscv64-cross: docker-image-debian10 > docker-image-debian-s390x-cross: docker-image-debian10 > docker-image-debian-sh4-cross: docker-image-debian10 > docker-image-debian-sparc64-cross: docker-image-debian10 > @@ -180,7 +179,6 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross > DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross > DOCKER_PARTIAL_IMAGES += debian-hppa-cross > DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross > -DOCKER_PARTIAL_IMAGES += debian-riscv64-cross > DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross > DOCKER_PARTIAL_IMAGES += debian-tricore-cross > DOCKER_PARTIAL_IMAGES += debian-xtensa-cross > diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker > index 2bbff19772..594d97982c 100644 > --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker > +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker > @@ -1,12 +1,48 @@ > # > -# Docker cross-compiler target > +# Docker cross-compiler target for riscv64 > # > -# This docker target builds on the debian Buster base image. > +# Currently the only distro that gets close to cross compiling riscv64 > +# images is Debian Sid (with unofficial ports). As this is a moving > +# target we keep the library list minimal and are aiming to migrate > +# from this hack as soon as we are able. > # > -FROM qemu/debian10 > +FROM docker.io/library/debian:sid-slim > + > +# Add ports > +RUN apt update && \ > + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ > + DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \ > + DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy > + > +# Install common build utilities > +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ > + bc \ > + build-essential \ > + ca-certificates \ > + debian-ports-archive-keyring \ > + dpkg-dev \ > + gettext \ > + git \ > + ninja-build \ > + pkg-config \ > + python3 Alex, why not using a debian-sid intermediate image? > +# Add ports and riscv64 architecture > +RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list > +RUN dpkg --add-architecture riscv64 > + > +# Duplicate deb line as deb-src > +RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list > > RUN apt update && \ > DEBIAN_FRONTEND=noninteractive eatmydata \ > apt install -y --no-install-recommends \ > - gcc-riscv64-linux-gnu \ > - libc6-dev-riscv64-cross > + gcc-riscv64-linux-gnu \ > + libc6-dev-riscv64-cross \ > + libffi-dev:riscv64 \ > + libglib2.0-dev:riscv64 \ > + libpixman-1-dev:riscv64 > + > +# Specify the cross prefix for this image (see tests/docker/common.rc) > +ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu- > +ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user >
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > On 9/14/21 8:58 PM, Richard Henderson wrote: >> From: Alex Bennée <alex.bennee@linaro.org> >> >> To be able to cross build QEMU itself we need to include a few more >> libraries. These are only available in Debian's unstable ports repo >> for now so we need to base the riscv64 image on sid with the the >> minimal libs needed to build QEMU (glib/pixman). >> >> The result works but is not as clean as using build-dep to bring in >> more dependencies. However sid is by definition a shifting pile of >> sand and by keeping the list of libs minimal we reduce the chance of >> having an image we can't build. It's good enough for a basic cross >> build testing of TCG. >> <snip> >> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker >> index 2bbff19772..594d97982c 100644 >> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker >> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker >> @@ -1,12 +1,48 @@ >> # >> -# Docker cross-compiler target >> +# Docker cross-compiler target for riscv64 >> # >> -# This docker target builds on the debian Buster base image. >> +# Currently the only distro that gets close to cross compiling riscv64 >> +# images is Debian Sid (with unofficial ports). As this is a moving >> +# target we keep the library list minimal and are aiming to migrate >> +# from this hack as soon as we are able. >> # >> -FROM qemu/debian10 >> +FROM docker.io/library/debian:sid-slim >> + >> +# Add ports >> +RUN apt update && \ >> + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ >> + DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \ >> + DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy >> + >> +# Install common build utilities >> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ >> + bc \ >> + build-essential \ >> + ca-certificates \ >> + debian-ports-archive-keyring \ >> + dpkg-dev \ >> + gettext \ >> + git \ >> + ninja-build \ >> + pkg-config \ >> + python3 > > Alex, why not using a debian-sid intermediate image? Well we removed the original debian-sid image a while ago having excised images that used it. Basing things on sid does cause problems given it's such a moving feast. Rather than encouraging others to build sid based images we decided it made more sense to make this a pure standalone image which we can migrate away from at the first opportunity. -- Alex Bennée
On 9/15/21 10:15 AM, Alex Bennée wrote: > Philippe Mathieu-Daudé <f4bug@amsat.org> writes: >> On 9/14/21 8:58 PM, Richard Henderson wrote: >>> From: Alex Bennée <alex.bennee@linaro.org> >>> >>> To be able to cross build QEMU itself we need to include a few more >>> libraries. These are only available in Debian's unstable ports repo >>> for now so we need to base the riscv64 image on sid with the the >>> minimal libs needed to build QEMU (glib/pixman). >>> >>> The result works but is not as clean as using build-dep to bring in >>> more dependencies. However sid is by definition a shifting pile of >>> sand and by keeping the list of libs minimal we reduce the chance of >>> having an image we can't build. It's good enough for a basic cross >>> build testing of TCG. >>> > <snip> >>> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker >>> index 2bbff19772..594d97982c 100644 >>> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker >>> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker >>> @@ -1,12 +1,48 @@ >>> # >>> -# Docker cross-compiler target >>> +# Docker cross-compiler target for riscv64 >>> # >>> -# This docker target builds on the debian Buster base image. >>> +# Currently the only distro that gets close to cross compiling riscv64 >>> +# images is Debian Sid (with unofficial ports). As this is a moving >>> +# target we keep the library list minimal and are aiming to migrate >>> +# from this hack as soon as we are able. >>> # >>> -FROM qemu/debian10 >>> +FROM docker.io/library/debian:sid-slim >>> + >>> +# Add ports >>> +RUN apt update && \ >>> + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ >>> + DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \ >>> + DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy >>> + >>> +# Install common build utilities >>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ >>> + bc \ >>> + build-essential \ >>> + ca-certificates \ >>> + debian-ports-archive-keyring \ >>> + dpkg-dev \ >>> + gettext \ >>> + git \ >>> + ninja-build \ >>> + pkg-config \ >>> + python3 >> >> Alex, why not using a debian-sid intermediate image? > > Well we removed the original debian-sid image a while ago having excised > images that used it. Basing things on sid does cause problems given it's > such a moving feast. Rather than encouraging others to build sid based > images we decided it made more sense to make this a pure standalone > image which we can migrate away from at the first opportunity. Yeah, which is why I'm worried we'll get the same problem... So this image can't be used as gating IMO.
On Wed, Sep 15, 2021 at 05:01:23PM +0200, Philippe Mathieu-Daudé wrote: > On 9/15/21 10:15 AM, Alex Bennée wrote: > > Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > >> On 9/14/21 8:58 PM, Richard Henderson wrote: > >>> From: Alex Bennée <alex.bennee@linaro.org> > >>> > >>> To be able to cross build QEMU itself we need to include a few more > >>> libraries. These are only available in Debian's unstable ports repo > >>> for now so we need to base the riscv64 image on sid with the the > >>> minimal libs needed to build QEMU (glib/pixman). > >>> > >>> The result works but is not as clean as using build-dep to bring in > >>> more dependencies. However sid is by definition a shifting pile of > >>> sand and by keeping the list of libs minimal we reduce the chance of > >>> having an image we can't build. It's good enough for a basic cross > >>> build testing of TCG. > >>> > > <snip> > >>> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker > >>> index 2bbff19772..594d97982c 100644 > >>> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker > >>> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker > >>> @@ -1,12 +1,48 @@ > >>> # > >>> -# Docker cross-compiler target > >>> +# Docker cross-compiler target for riscv64 > >>> # > >>> -# This docker target builds on the debian Buster base image. > >>> +# Currently the only distro that gets close to cross compiling riscv64 > >>> +# images is Debian Sid (with unofficial ports). As this is a moving > >>> +# target we keep the library list minimal and are aiming to migrate > >>> +# from this hack as soon as we are able. > >>> # > >>> -FROM qemu/debian10 > >>> +FROM docker.io/library/debian:sid-slim > >>> + > >>> +# Add ports > >>> +RUN apt update && \ > >>> + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ > >>> + DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \ > >>> + DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy > >>> + > >>> +# Install common build utilities > >>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ > >>> + bc \ > >>> + build-essential \ > >>> + ca-certificates \ > >>> + debian-ports-archive-keyring \ > >>> + dpkg-dev \ > >>> + gettext \ > >>> + git \ > >>> + ninja-build \ > >>> + pkg-config \ > >>> + python3 > >> > >> Alex, why not using a debian-sid intermediate image? > > > > Well we removed the original debian-sid image a while ago having excised > > images that used it. Basing things on sid does cause problems given it's > > such a moving feast. Rather than encouraging others to build sid based > > images we decided it made more sense to make this a pure standalone > > image which we can migrate away from at the first opportunity. > > Yeah, which is why I'm worried we'll get the same problem... > > So this image can't be used as gating IMO. Yeah, IME from babysitting libvirt CI, 'sid' containers break in dep solving more often than is acceptable for a mandatory job, especially for non-x86 arches. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml index 0fcebe363a..05996200e1 100644 --- a/.gitlab-ci.d/container-cross.yml +++ b/.gitlab-ci.d/container-cross.yml @@ -134,7 +134,6 @@ ppc64el-debian-cross-container: riscv64-debian-cross-container: extends: .container_job_template stage: containers-layer2 - needs: ['amd64-debian10-container'] variables: NAME: debian-riscv64-cross diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index ff5d732889..3b03763186 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -141,7 +141,6 @@ docker-image-debian-mips64-cross: docker-image-debian10 docker-image-debian-mips64el-cross: docker-image-debian10 docker-image-debian-mipsel-cross: docker-image-debian10 docker-image-debian-ppc64el-cross: docker-image-debian10 -docker-image-debian-riscv64-cross: docker-image-debian10 docker-image-debian-s390x-cross: docker-image-debian10 docker-image-debian-sh4-cross: docker-image-debian10 docker-image-debian-sparc64-cross: docker-image-debian10 @@ -180,7 +179,6 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross DOCKER_PARTIAL_IMAGES += debian-hppa-cross DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross -DOCKER_PARTIAL_IMAGES += debian-riscv64-cross DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross DOCKER_PARTIAL_IMAGES += debian-tricore-cross DOCKER_PARTIAL_IMAGES += debian-xtensa-cross diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker index 2bbff19772..594d97982c 100644 --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker @@ -1,12 +1,48 @@ # -# Docker cross-compiler target +# Docker cross-compiler target for riscv64 # -# This docker target builds on the debian Buster base image. +# Currently the only distro that gets close to cross compiling riscv64 +# images is Debian Sid (with unofficial ports). As this is a moving +# target we keep the library list minimal and are aiming to migrate +# from this hack as soon as we are able. # -FROM qemu/debian10 +FROM docker.io/library/debian:sid-slim + +# Add ports +RUN apt update && \ + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ + DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \ + DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy + +# Install common build utilities +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ + bc \ + build-essential \ + ca-certificates \ + debian-ports-archive-keyring \ + dpkg-dev \ + gettext \ + git \ + ninja-build \ + pkg-config \ + python3 + +# Add ports and riscv64 architecture +RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list +RUN dpkg --add-architecture riscv64 + +# Duplicate deb line as deb-src +RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ apt install -y --no-install-recommends \ - gcc-riscv64-linux-gnu \ - libc6-dev-riscv64-cross + gcc-riscv64-linux-gnu \ + libc6-dev-riscv64-cross \ + libffi-dev:riscv64 \ + libglib2.0-dev:riscv64 \ + libpixman-1-dev:riscv64 + +# Specify the cross prefix for this image (see tests/docker/common.rc) +ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu- +ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user