From patchwork Mon Mar 8 13:50:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 395431 Delivered-To: patch@linaro.org Received: by 2002:a17:906:a383:0:0:0:0 with SMTP id k3csp1849684ejz; Mon, 8 Mar 2021 06:11:31 -0800 (PST) X-Google-Smtp-Source: ABdhPJwz2tGLSeRC30+EydwaQI+fs4QBILHPBQ5ImOfJWbXz46cHsHvXhbvN0tprN1cL5nnqXSyD X-Received: by 2002:aca:b48a:: with SMTP id d132mr6456363oif.11.1615212691475; Mon, 08 Mar 2021 06:11:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615212691; cv=none; d=google.com; s=arc-20160816; b=sFUFofonH9NFfEFzcKlGZ1S8IVXbzvslPQRbjF5G9sp0MAM5uV/QqMSTcuwJxhJvpI m+oFyZQmFO6u/WuYld4K/UV4tmYFIgPkpL8RN04uPb5cC5CTFGzzwfSANnWW5vxJUk0G MQdYOgA/Es+m2QyvVnaQ9U2EVmApBA9WwIPGI1n+xoMQVZu6h4/ec0jKsX2YfKpj1xRS xRnR/ZON6rTu4JHQ7eU/GdhfHsib+Zyesodfd98ZSTf1l5xVOf0j9KYmALY/ytWWTku+ p8o/2ttmKQ04OoGYJHtu4szgUdMHCmUPMWvvdl+j2cdHo74612NlDmmbp9iXIu5A+oIO 5EEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uTg3Tv32l8Xlt8XwDOEGmwz25MtkKlDZ8otFZdvh7to=; b=vlQiq416247d7BBdnHeqW2G3GlAkpKnuKZxhOK69/96czAz+m61SvFbV57uwhuC5sN HsvOZ7CZtRXXuKgTxbkD3Gt7rVmbWEesiCjVc/27mFuFX/QC/RMTWGnIFrVTgnAEaaha uZIqvw2UbDB1kKVrLZZhcfY7aUUbxzTzEoEa/B35EUCe9ilbkpF/7eXEQtd/YBvYQnF+ c8Ane2t2JS1xMmPIR082brU29YdbUyA/wwpEa6Cq7v7hfmZSGH2UnXcx2VIp1FH+madg 2NDdACjRwQSTsHAs5+cUVvp+IzwklkwbqzUkESF1OXkF3VDZ6/pq7hpugT+RCzvYmN1O rgMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=lCcvNqxQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j11si8642178otr.210.2021.03.08.06.11.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 08 Mar 2021 06:11:31 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=lCcvNqxQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJGba-0007bh-Nj for patch@linaro.org; Mon, 08 Mar 2021 09:11:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJGI5-0006Xk-E3 for qemu-devel@nongnu.org; Mon, 08 Mar 2021 08:51:22 -0500 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:42676) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJGI1-0001E8-Qt for qemu-devel@nongnu.org; Mon, 08 Mar 2021 08:51:21 -0500 Received: by mail-wm1-x32a.google.com with SMTP id b2-20020a7bc2420000b029010be1081172so3847642wmj.1 for ; Mon, 08 Mar 2021 05:51:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uTg3Tv32l8Xlt8XwDOEGmwz25MtkKlDZ8otFZdvh7to=; b=lCcvNqxQ1rLU7JVSPiokQwkYmfefY2HnkJLqQVH/NI+rYliIOISsTaMIX+KCA/Xxm9 PSpGRkNvtTce/gdfFp+bCd0LfUhm8zjY7IKtrTxdRPQKkAngxRTy2O+PYWtaAuuHVg5w AVjWH4QUfZ+6ait/jUfYzW8xGevVve0UQNOrzry6b+iHu2a8G3xkjxR70UZwNd3xHX+q ny0Jyh5clLL3S41/nCM9iN6D6ITR4msG82bwua6qccLJCyfXyhqb27xAQHQGq+8A7Ygz CU+GnlQPH4BLajon9rasJB7nEX+qHH52pFLFSaNYNSk3SxNwOJSGT/dWd7/D4YeN5B+q KeGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uTg3Tv32l8Xlt8XwDOEGmwz25MtkKlDZ8otFZdvh7to=; b=Vq1Rr3x++z6Gcd++Fq9XOqa2lz1R9EJ7hOcyJxoD7T/hEpgDv5GQHOKDXzcFF3B65l J9Q8KWCAVJnNunlRH3hsgIHByXMRjn6IXQkkt/BaXHHCdUEpcVJhkbIWbOrTfOqtqO82 RrbNks/VrduCIDWnnlyEQb+qMWyxZJpzZJg/n/Fsa0gZR36xtec+8svoySWZJJq2QX4p zxFyOrxaT4IRO59wl8dOG25Zgtrvxx5G6a3e3lu6e4yxAuFhyb4UlU8Kvzmgc04Nejky 81/ePKZhIfirdpSX2WLsB7fc/T3GdLPEaQeKhmhy0WSp2dD/1XFuVyyJGBj1UDNKMN6N j6Og== X-Gm-Message-State: AOAM531jdHyUrlU+GPRPdhCOmzVANsxL6SjTu9IEaOLQNWiEnEy9VdAf t27uaVTUXc3LoCmKkNv5s6atyA== X-Received: by 2002:a1c:3b02:: with SMTP id i2mr22233019wma.18.1615211476518; Mon, 08 Mar 2021 05:51:16 -0800 (PST) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id a17sm19496485wmj.9.2021.03.08.05.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Mar 2021 05:51:08 -0800 (PST) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 5BB811FF92; Mon, 8 Mar 2021 13:51:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 06/18] gitlab-ci.yml: Allow custom # of parallel linkers Date: Mon, 8 Mar 2021 13:50:52 +0000 Message-Id: <20210308135104.24903-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210308135104.24903-1-alex.bennee@linaro.org> References: <20210308135104.24903-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Alex_Ben?= =?utf-8?q?n=C3=A9e?= , Daniele Buono Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniele Buono Define a new variable LD_JOBS, that can be used to select the maximum number of linking jobs to be executed in parallel. If the variable is not defined, maintain the default given by make -j Currently, make parallelism at build time is based on the number of cpus available. This doesn't work well with LTO at linking, because with LTO the linker has to load in memory all the intermediate object files for optimization. The end result is that, if the gitlab runner happens to run two linking processes at the same time, the job will fail with an out-of-memory error, This patch leverages the ability to maintain high parallelism at compile time, but limit the number of linkers executed in parallel. Signed-off-by: Daniele Buono Signed-off-by: Alex Bennée Reviewed-by: Daniel P. Berrangé Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com> Message-Id: <20210305092328.31792-8-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c2c5196b8b..1ea944eb91 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -27,6 +27,10 @@ include: else ../configure --enable-werror $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; } + - if test -n "$LD_JOBS"; + then + meson configure . -Dbackend_max_links="$LD_JOBS" ; + fi || exit 1; - make -j"$JOBS" - if test -n "$MAKE_CHECK_ARGS"; then