| Message ID | 20210310160002.11659-5-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | testing, docs, semihosting move and guest-loader | expand |
On 3/10/21 4:59 PM, Alex Bennée wrote: > From: Daniele Buono <dbuono@linux.vnet.ibm.com> > > QEMU has had options to enable control-flow integrity features > for a few months now. Add two sets of build/check/acceptance > jobs to ensure the binary produced is working fine. > > The three sets allow testing of x86_64 binaries for x86_64, s390x, > ppc64 and aarch64 targets > > [AJB: tweak job names to avoid brands] > > Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com> > Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org> > +build-cfi-x86_64: > + <<: *native_build_job_definition > + needs: > + - job: amd64-fedora-container > + variables: > + LD_JOBS: 1 > + AR: llvm-ar > + IMAGE: fedora > + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug > + --enable-safe-stack --enable-slirp=git > + TARGETS: x86_64-softmmu > + MAKE_CHECK_ARGS: check-build > + artifacts: > + expire_in: 2 days > + paths: > + - build FYI this job is timeouting: ERROR: Job failed: execution took longer than 1h0m0s seconds https://gitlab.com/qemu-project/qemu/-/jobs/1112829128 > + > +check-cfi-x86_64: > + <<: *native_test_job_definition > + needs: > + - job: build-cfi-x86_64 > + artifacts: true > + variables: > + IMAGE: fedora > + MAKE_CHECK_ARGS: check > + > +acceptance-cfi-x86_64: > + <<: *native_test_job_definition > + needs: > + - job: build-cfi-x86_64 > + artifacts: true > + variables: > + IMAGE: fedora > + MAKE_CHECK_ARGS: check-acceptance > + <<: *acceptance_definition > + > tsan-build: > <<: *native_build_job_definition > variables: >
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > On 3/10/21 4:59 PM, Alex Bennée wrote: >> From: Daniele Buono <dbuono@linux.vnet.ibm.com> >> >> QEMU has had options to enable control-flow integrity features >> for a few months now. Add two sets of build/check/acceptance >> jobs to ensure the binary produced is working fine. >> >> The three sets allow testing of x86_64 binaries for x86_64, s390x, >> ppc64 and aarch64 targets >> >> [AJB: tweak job names to avoid brands] >> >> Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> >> Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com> >> Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org> > >> +build-cfi-x86_64: >> + <<: *native_build_job_definition >> + needs: >> + - job: amd64-fedora-container >> + variables: >> + LD_JOBS: 1 >> + AR: llvm-ar >> + IMAGE: fedora >> + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug >> + --enable-safe-stack --enable-slirp=git >> + TARGETS: x86_64-softmmu >> + MAKE_CHECK_ARGS: check-build >> + artifacts: >> + expire_in: 2 days >> + paths: >> + - build > > FYI this job is timeouting: > > ERROR: Job failed: execution took longer than 1h0m0s seconds > > https://gitlab.com/qemu-project/qemu/-/jobs/1112829128 > I didn't see it fail in my tests (although visibility is somewhat masked by the current failing iotest). However I notice the builds are fairly close to the limit (56-58 mins). As they are already pared down maybe just increasing "timeout" is enough? >> + >> +check-cfi-x86_64: >> + <<: *native_test_job_definition >> + needs: >> + - job: build-cfi-x86_64 >> + artifacts: true >> + variables: >> + IMAGE: fedora >> + MAKE_CHECK_ARGS: check >> + >> +acceptance-cfi-x86_64: >> + <<: *native_test_job_definition >> + needs: >> + - job: build-cfi-x86_64 >> + artifacts: true >> + variables: >> + IMAGE: fedora >> + MAKE_CHECK_ARGS: check-acceptance >> + <<: *acceptance_definition >> + >> tsan-build: >> <<: *native_build_job_definition >> variables: >> -- Alex Bennée
Hi Philippe, I'm looking at the public QEMU pipelines and it seems that that job usually takes between 50 and 55 minutes, but there are higher spikes at 56, 57 and one where it failed at 1h. We could perhaps set the timeout a bit higher, like 1h 10m, to not terminate the outliers immediately? The job you linked was almost over, there were just about 20-ish tests to be linked, so it was probably next to completion. On 3/22/2021 9:39 AM, Philippe Mathieu-Daudé wrote: >> +build-cfi-x86_64: >> + <<: *native_build_job_definition >> + needs: >> + - job: amd64-fedora-container >> + variables: >> + LD_JOBS: 1 >> + AR: llvm-ar >> + IMAGE: fedora >> + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug >> + --enable-safe-stack --enable-slirp=git >> + TARGETS: x86_64-softmmu >> + MAKE_CHECK_ARGS: check-build >> + artifacts: >> + expire_in: 2 days >> + paths: >> + - build > FYI this job is timeouting: > > ERROR: Job failed: execution took longer than 1h0m0s seconds > > https://gitlab.com/qemu-project/qemu/-/jobs/1112829128 >
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 814f51873f..b23364bf3a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -483,6 +483,125 @@ clang-user: --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined MAKE_CHECK_ARGS: check-unit check-tcg +# Set LD_JOBS=1 because this requires LTO and ld consumes a large amount of memory. +# On gitlab runners, default value sometimes end up calling 2 lds concurrently and +# triggers an Out-Of-Memory error +# +# Since slirp callbacks are used in QEMU Timers, slirp needs to be compiled together +# with QEMU and linked as a static library to avoid false positives in CFI checks. +# This can be accomplished by using -enable-slirp=git, which avoids the use of +# a system-wide version of the library +# +# Split in three sets of build/check/acceptance to limit the execution time of each +# job +build-cfi-aarch64: + <<: *native_build_job_definition + needs: + - job: amd64-fedora-container + variables: + LD_JOBS: 1 + AR: llvm-ar + IMAGE: fedora + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug + --enable-safe-stack --enable-slirp=git + TARGETS: aarch64-softmmu + MAKE_CHECK_ARGS: check-build + artifacts: + expire_in: 2 days + paths: + - build + +check-cfi-aarch64: + <<: *native_test_job_definition + needs: + - job: build-cfi-aarch64 + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check + +acceptance-cfi-aarch64: + <<: *native_test_job_definition + needs: + - job: build-cfi-aarch64 + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + <<: *acceptance_definition + +build-cfi-ppc64-s390x: + <<: *native_build_job_definition + needs: + - job: amd64-fedora-container + variables: + LD_JOBS: 1 + AR: llvm-ar + IMAGE: fedora + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug + --enable-safe-stack --enable-slirp=git + TARGETS: ppc64-softmmu s390x-softmmu + MAKE_CHECK_ARGS: check-build + artifacts: + expire_in: 2 days + paths: + - build + +check-cfi-ppc64-s390x: + <<: *native_test_job_definition + needs: + - job: build-cfi-ppc64-s390x + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check + +acceptance-cfi-ppc64-s390x: + <<: *native_test_job_definition + needs: + - job: build-cfi-ppc64-s390x + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + <<: *acceptance_definition + +build-cfi-x86_64: + <<: *native_build_job_definition + needs: + - job: amd64-fedora-container + variables: + LD_JOBS: 1 + AR: llvm-ar + IMAGE: fedora + CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug + --enable-safe-stack --enable-slirp=git + TARGETS: x86_64-softmmu + MAKE_CHECK_ARGS: check-build + artifacts: + expire_in: 2 days + paths: + - build + +check-cfi-x86_64: + <<: *native_test_job_definition + needs: + - job: build-cfi-x86_64 + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check + +acceptance-cfi-x86_64: + <<: *native_test_job_definition + needs: + - job: build-cfi-x86_64 + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + <<: *acceptance_definition + tsan-build: <<: *native_build_job_definition variables: