From patchwork Wed Mar 10 15:59:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 396766 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp511875jai; Wed, 10 Mar 2021 08:23:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJxyGplu2z2EKF81CEac7ZglJ+TpUzOptAofvlJNRY7RIXogCDNWvaxfTGCXBXXiAzaSH+ZC X-Received: by 2002:a1f:9d12:: with SMTP id g18mr2040001vke.1.1615393407030; Wed, 10 Mar 2021 08:23:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615393407; cv=none; d=google.com; s=arc-20160816; b=hLHxmE95shximICugkJ8r/UdhVFQ13TWt+tcPEx/80GE7qhp6vwpBfxDEOxIcl77SK HwnN1+/58qCc1xwX03k7Fm8+uGCtkl6MM1jvRXmDEdDY/t0Q7OL+moz7fkwv0z+J8/OC jDGTdTGDPLTuoFSwnNESXxRbHsj0Un2quK4mFANCTlpKD7XUlcAIOeVefSBPQLLgS0f1 AZEJssR2jwZHEKXJwV9mjhzHsqWHEc78WE9BecD4nLzzHmjbNrNLoR8cCI2BDm2LqSEt 0ot3M20depFVvWvPZjHEP1AGfe+3i+v6yq2t6ahqJcZ9vA3x7hMQqYDaXv9XfRN2Bqey tR/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HHpsRwO6vST0yf6ATL7Kbq1Uv+qWDZ8yZxsOzY/cYlI=; b=09Q94AToVcl5lhc44vBnBHnWdm67BEv+UWDZ7Qe/ARVL6EmRTU5zVM+uQiArtolfrN CL4YhQmfPY3PDgfa81+zmgcivjkjRid2bCgpsACYZX554FCyRy8Md/sSKMIKiyfxXRWN G5lAYlOKibaEP6BS7ywehqv8WqOafpNz3g2gDpqi/aIZDoo1cI76AC6ced/PTiTp3yWA Mfv4VsitwY7i9LijUMh6PSzSOZ+GcKJx8Bh2Nv1ZsuXXr0xdqG93VQuewB1t/wh60AYe E/2CXfb56ePOgbpHiSoBfOhbH14fXOs2gXN/I8OHavlTb8u6v2CVuMi0HAXzdb3uSlED RBVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OUtskioK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i8si16432uak.154.2021.03.10.08.23.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 10 Mar 2021 08:23:27 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OUtskioK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lK1cM-0001KU-Af for patch@linaro.org; Wed, 10 Mar 2021 11:23:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34502) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lK1Fu-0004Q4-RA for qemu-devel@nongnu.org; Wed, 10 Mar 2021 11:00:14 -0500 Received: from mail-ej1-x634.google.com ([2a00:1450:4864:20::634]:34574) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lK1Fs-0002a0-NP for qemu-devel@nongnu.org; Wed, 10 Mar 2021 11:00:14 -0500 Received: by mail-ej1-x634.google.com with SMTP id hs11so39729793ejc.1 for ; Wed, 10 Mar 2021 08:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HHpsRwO6vST0yf6ATL7Kbq1Uv+qWDZ8yZxsOzY/cYlI=; b=OUtskioKpUT6gq7xHfrVqR3gBkb0hxh5udrgw9qDFPBUC+bH+zBwh345q0sWuOlANn pDEavZo5qw6Aw4Gw4f7nwlkdSpxR76m2PetagbbB/MDb6hn22cxC+kIS6M4UIyRuGPxP pUYvq/WxZd5DI1vdGsgimn+s6PhjIm7H1qA8wELtsLen2E1IIHtvlAzsJFe49kcdokjx xsyQnNHcglVVGrzc/hBAF/FdULMk1TWUCNMFMy2sIp0fVVud8V+Y+cRS1FnhNuvUOuoT yIkD3IJCX/yOvPCExX3Pqy5H46iF7bFZVGD0Y8IuzU4l3RFboBTUhGnHJ7bIXldxLMSX e1IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HHpsRwO6vST0yf6ATL7Kbq1Uv+qWDZ8yZxsOzY/cYlI=; b=S1GL9nMq9QUz43gPe8PCz3lbpsIuSTlDrLGc4amB3BG/zBQYb+YlTKtayezvHyQlLD YA5HLDDij2LvPqYePTveHFhP2myNo+g/74Jg50/FIWtiHWo9Jwzq3ged4tYpBBi4cMRK 5E+Fqf22zakT+gG2pvQkaQELFfF+nrBjvEdTQxc2CmGCvSRDMc0jB4Oij8wewQ7ei7d2 DEni+ElV5naZ3pK0yitMLyfbvs4v+rmrLXfrdTdQIyAnk2RkBvOKs90uXsfP4oGLMGsn 3fmpyiKtPBGyfKkltG7BpTHPePMSSR/laR3l6TgzK1Omx30lSM462TqjGVn6gIBf/PcZ ynvQ== X-Gm-Message-State: AOAM5336occPx4dzQFKSi21TojpteKFncyVVNPIiCwpjObucWTc3kBhT ZsCQhahUVpN+OquG/AhJDHdI8Q== X-Received: by 2002:a17:906:3388:: with SMTP id v8mr4487060eja.278.1615392008187; Wed, 10 Mar 2021 08:00:08 -0800 (PST) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id k26sm10149391ejk.29.2021.03.10.08.00.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Mar 2021 08:00:03 -0800 (PST) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D4E9C1FF8F; Wed, 10 Mar 2021 16:00:02 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL v2 03/15] gitlab-ci.yml: Allow custom # of parallel linkers Date: Wed, 10 Mar 2021 15:59:50 +0000 Message-Id: <20210310160002.11659-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210310160002.11659-1-alex.bennee@linaro.org> References: <20210310160002.11659-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::634; envelope-from=alex.bennee@linaro.org; helo=mail-ej1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Alex_Ben?= =?utf-8?q?n=C3=A9e?= , Daniele Buono Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniele Buono Define a new variable LD_JOBS, that can be used to select the maximum number of linking jobs to be executed in parallel. If the variable is not defined, maintain the default given by make -j Currently, make parallelism at build time is based on the number of cpus available. This doesn't work well with LTO at linking, because with LTO the linker has to load in memory all the intermediate object files for optimization. The end result is that, if the gitlab runner happens to run two linking processes at the same time, the job will fail with an out-of-memory error, This patch leverages the ability to maintain high parallelism at compile time, but limit the number of linkers executed in parallel. Signed-off-by: Daniele Buono Signed-off-by: Alex Bennée Reviewed-by: Daniel P. Berrangé Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com> Message-Id: <20210305092328.31792-8-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8b6d495288..814f51873f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -27,6 +27,10 @@ include: else ../configure --enable-werror $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; } + - if test -n "$LD_JOBS"; + then + meson configure . -Dbackend_max_links="$LD_JOBS" ; + fi || exit 1; - make -j"$JOBS" - if test -n "$MAKE_CHECK_ARGS"; then