From patchwork Wed Jul 1 13:56:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 192153 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp528634ilg; Wed, 1 Jul 2020 07:09:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4PMAUuommB3rA+F2TIOet2HeuxB50cnSrzuW+z+37rZUaihDpB40SAnOIpO1tP8VRsRpL X-Received: by 2002:a25:7c45:: with SMTP id x66mr43200295ybc.325.1593612545011; Wed, 01 Jul 2020 07:09:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593612545; cv=none; d=google.com; s=arc-20160816; b=Hzka38pAmE1+JAtZgCOYLxLzrVWdc73Z88TTCGCEDQWpRe4zRibRNA0QXSK/4HKW5X 5mzv2NCwQh3qnPeNWUPWX9BiNSPEQ/lt05EsZi15PwiGovRZgwYM7tsDCfhI1iRhuqY9 XP//KbSO/JMTiXsEGtQpouDNrgnJdsxf2VZIXwXrFlVj2CRaZm/ab5v3lyyog/XZLtzM IwO7FkK3UrkhxdEGZs5U1VwBqB6AlNgLThPvl2jO7DdQDdp8RcXMBu/STbYCcQ5e+/wE itWRpDGrbTkq2Ujs0h6eQ1HfFb8PcQA177hTEb6iLsTmKtsAHHc25c6juNBTGtPgB5Yl DxXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XtbfSpBB9DtnKKoqK1hU0Y2RrtR1UeM6mQbJKmTSrNs=; b=g7aBotav9j3VP0UCDVkIeeFc5Pgc9KbmbaxTgRZFHbpjTEhNITZub4pcSZOPeEA7HE zQUFLQDjipM7VRs22ZklQq+Llwb9oWySF6P70f/x6KUd9DjLRgDDqLyQd8YNQm8mX4Rk AZPsv+3oGCodL+e9VU61pMly0UErlJY3bDU971pSLhgDpepiclH+PnBOfGNpiaiXHEjV PXSdiibJv6NQajfhVLIUMFgNOyS6ZpN4l+QvpPjwTVHFAAfFUfPH8MLKeMP9BXnfy1dT LgOm0FdpbcWaqmWR2KAmoB/nxvajdZNLiY2g+/AHS5r11uhnUoRdzdm630+KDk5Tkm2q pPWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=IdFcX18B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a131si1205914ybg.309.2020.07.01.07.09.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 01 Jul 2020 07:09:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=IdFcX18B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44422 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jqdQ8-00050J-CP for patch@linaro.org; Wed, 01 Jul 2020 10:09:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59040) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jqdEh-0002Xn-Kh for qemu-devel@nongnu.org; Wed, 01 Jul 2020 09:57:17 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:39335) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jqdEe-0005sm-62 for qemu-devel@nongnu.org; Wed, 01 Jul 2020 09:57:15 -0400 Received: by mail-wm1-x330.google.com with SMTP id w3so10905906wmi.4 for ; Wed, 01 Jul 2020 06:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XtbfSpBB9DtnKKoqK1hU0Y2RrtR1UeM6mQbJKmTSrNs=; b=IdFcX18BKqKlZRkc7+UIXoRZ+Zm7BZT00PJk5OvQxpHUSuURiZQCv2hlQTUDNcHcGS ynQSnRfAUUUB6WWc0xTCzSmXRgMGxehlXuw1guhlMbcEL9BCqFBb6wYSWEhxepECmdXW Vt5J/Bew/4bdgixynydeIkEnLbCETlauflU+Lxi446lR2cJhGy11uSodydr75ROSVNnl ddQDSnFjdHnHsGzwhxrfMXoiXr87BCJczxRTtlK6mMEMv+inPPNrEcjQbZ8jtYNksPst YNjYUqozwbW58a4Uy7riVwRSLtJcq6Bz76f7faa0GR1h628kdG095NUd/2lfitXVAsfo i55g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XtbfSpBB9DtnKKoqK1hU0Y2RrtR1UeM6mQbJKmTSrNs=; b=JhQHuu+tREb1BO61sfW2rx0nhvef1iXEpeKr3iSaPy9yc9dkSkqLvbEaAHoncjrabp d8XhPdIwrc91Itg5cnNEMb0uCD2tsIrO7Sns/7zQBGVf+jiKN5vkX29mEdaor/WZWDpU E40cdx1kgl5RROXz5BxvZiI/gslwnG/mVCJxaloDM27M4LqvorAMuqlNFobC1D1RHLtZ tStojHFldmDJ4RLS65p30uHmo+Qdd3uOqxPY66m0lnNDnlfPvvcf17poEI370RAKZfzz wf+aLKcuomwz1AR3y4DQZrkdyGYXqPiC6T3eV4Fgh8JZlDRCbuljvdHSl8AlDxXQsy4I MRWw== X-Gm-Message-State: AOAM5320CxJ9hGPaxOy8Ddd+7qEkd8paM3jbD+lIr7V8nmlaj/5lOIYx HEmRCWzA1tLJacWTL7PnwfRZBQ== X-Received: by 2002:a7b:c84d:: with SMTP id c13mr26769260wml.170.1593611830666; Wed, 01 Jul 2020 06:57:10 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id f12sm7501283wrw.53.2020.07.01.06.57.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jul 2020 06:57:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 198061FFAA; Wed, 1 Jul 2020 14:56:55 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v4 21/40] gitlab: build all container images during CI Date: Wed, 1 Jul 2020 14:56:33 +0100 Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200701135652.1366-1-alex.bennee@linaro.org> References: <20200701135652.1366-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x330.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée Acked-by: Thomas Huth --- v4 - rm leading and trailing ws --- .gitlab-ci.d/containers.yml | 245 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 248 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 00000000000..2afd01f1d17 --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,245 @@ +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea63..72d688875fd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' .update_apt_template: &before_script_apt before_script: