diff mbox series

[v3,4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Message ID 20201014052140.1146924-5-crosa@redhat.com
State Accepted
Commit 4799c2102333cee5ddc51051c035b1c6ea83c7f5
Headers show
Series GitLab Custom Runners and Jobs (was: QEMU Gating CI) | expand

Commit Message

Cleber Rosa Oct. 14, 2020, 5:21 a.m. UTC 
The QEMU project has two machines (aarch64 and s390) that can be used
for jobs that do build and run tests.  This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)

Comments

Daniel P. Berrangé Oct. 14, 2020, 5:46 p.m. UTC | #1 
On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> The QEMU project has two machines (aarch64 and s390) that can be used

> for jobs that do build and run tests.  This introduces those jobs,

> which are a mapping of custom scripts used for the same purpose.

> 

> Signed-off-by: Cleber Rosa <crosa@redhat.com>

> ---

>  .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++

>  1 file changed, 192 insertions(+)

> 

> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml

> index 3004da2bda..5b51d1b336 100644

> --- a/.gitlab-ci.d/custom-runners.yml

> +++ b/.gitlab-ci.d/custom-runners.yml

> @@ -12,3 +12,195 @@

>  # strategy.

>  variables:

>    GIT_SUBMODULE_STRATEGY: recursive

> +

> +# All ubuntu-18.04 jobs should run successfully in an environment

> +# setup by the scripts/ci/setup/build-environment.yml task

> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"

> +ubuntu-18.04-s390x-all-linux-static:

> + needs: []

> + stage: build

> + tags:

> + - ubuntu_18.04

> + - s390x

> + rules:

> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'


IIRC, in the previous v2 (or was it v1) we discussed changing this
so that users who provide their own runners, don't have to always
use the "staging" branch name.

IIUC, the key thing is that we don't want the job running on the
"master" or "stable-*" branches in the primary QEMU git. So could
check

   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'

which would let it work on users forks no matter what branch names
they use

What happens to the job if the user doesn't have runners ? Is it
simply skipped, or does the pipeline stall and get marked as failed ?

If the jobs aren't auto-skiped, we would need to add an env variable

   (
   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'
   )
   ||
   $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'

and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
in the web UI for their fork

That all said, I don't mind if you postpone this rules change to a
followup patch.

> + script:

> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763


This bug links to

  https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh
arg from all these jobs

> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages

> + - mkdir build

> + - cd build

> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh

> + - make --output-sync -j`nproc`

> + - make --output-sync -j`nproc` check V=1

> + - make --output-sync -j`nproc` check-tcg V=1




Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
diff mbox series

Patch

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 3004da2bda..5b51d1b336 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -12,3 +12,195 @@ 
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-18.04-s390x-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-20.04-aarch64-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1