From patchwork Tue Nov 14 11:38:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 118868 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2929909qgn; Tue, 14 Nov 2017 03:39:40 -0800 (PST) X-Google-Smtp-Source: AGs4zMYG8K0b9ejh0y+C+STOb6l57QvZHcawzIumozWbsxwte3j+Rpe2ObHEQl5kW2O5cKSLozkd X-Received: by 10.101.74.81 with SMTP id a17mr11605105pgu.392.1510659580871; Tue, 14 Nov 2017 03:39:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510659580; cv=none; d=google.com; s=arc-20160816; b=KsddTRlzigvgUmtcHY70MS4ABAQ5SZouyXB2mdSGB3IazOz/VzUK3zOdQs+1HvnrLm 3hrTze03oEIyap8QyNf0BnINkISxwLva0lJeyaPYo4e+FuYSEvlAQPLNHVcGwFT1OZG+ aDYPBO5iREotJAQ+0NGy2vdivQ2I5W70/3pshRyeCFrejXAprEfGIDBUETGgmqj9TR46 ki/+QlOlYxM1sJNExaQDC6hJzaob+OMB/ZCfPCAeGVeA+BnM/Un9TD2ntTzAGiw2YvMP hprpf3SdDJ3shvyEE411YGSlwihsCUC0Gs1JpTNKAvgUOvaeetU8fCQuvDACZjUpxmRy gc2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter:arc-authentication-results; bh=gb9TpTfPzygm4WlDnLw4WLFvoeCIOI1v07NhhJq00ao=; b=XrYhxTDctfMR4pagkXafYpslNMZSi0MZbRLhScBwcChtIQFr9zUfqL1e9ZVn4cdBkm toEa9Hb76DwiFo5xYKxgMWSSuOq8O9uPSX68fvYo/I5IWT2ZLerY4wi6xoVicpTatlfs tGMsPOvOri/ckGVkQ1oia+f8/aI43pX/5gwMIxE4ZJJVK0XHeD6t2zNmBOOmozDqsdem 5CXw0V7l81iGFHgBBgLXHCmu34wxK3zbfNwScYXK29vFsdOqQ5PzLUWEUKvIdI+cr6XJ y96cef77dqtOWXKPxuN3w0PQ0nu0dGtvVEc549vedXht0lN43W+doUgAdXGe/Tai6R7V OLkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=RLYShkc8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o18si15543549pgv.570.2017.11.14.03.39.40; Tue, 14 Nov 2017 03:39:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=RLYShkc8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754967AbdKNLji (ORCPT + 27 others); Tue, 14 Nov 2017 06:39:38 -0500 Received: from conuserg-08.nifty.com ([210.131.2.75]:48060 "EHLO conuserg-08.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754207AbdKNLja (ORCPT ); Tue, 14 Nov 2017 06:39:30 -0500 Received: from pug.e01.socionext.com (p14092-ipngnfx01kyoto.kyoto.ocn.ne.jp [153.142.97.92]) (authenticated) by conuserg-08.nifty.com with ESMTP id vAEBcBEl008527; Tue, 14 Nov 2017 20:38:11 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-08.nifty.com vAEBcBEl008527 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1510659492; bh=gb9TpTfPzygm4WlDnLw4WLFvoeCIOI1v07NhhJq00ao=; h=From:To:Cc:Subject:Date:From; b=RLYShkc8ezuYaFZACDFYGOo1ilIhZZW9EOovBPs8yR31eVLQElWLzCGMYhzuQ7F8Z Gn/ttaww9x19HFIuUMR7CJ7DiuQeM4acUCpdXECFIM/CRiDP2k+21CYhbE41KC5Dcq lhmyuOowX5VG9puxdkp09PdWmza2SIlTNrQbBXBBCpKtxfdSGEDtWMF7aE923aHzYb XW0QVTl63DqoS1aK4i9LlxZGn11qHG+pwdkOPrI5tM8iPbCiWqC/ZE6klpoKjFf7Ww f3YbUtul394Rq0IhyTGeUCzmZHiSAeSul+VVcbi2FxZumdhfuN9jS/f0UlRNbbI6nt UhmzrjPTvdgGQ== X-Nifty-SrcIP: [153.142.97.92] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: cocci@systeme.lip6.fr, Julia Lawall , Nicolas Palix , Gilles Muller , Masahiro Yamada , linux-kernel@vger.kernel.org, Michal Marek Subject: [PATCH v4] coccinelle: fix parallel build with CHECK=scripts/coccicheck Date: Tue, 14 Nov 2017 20:38:07 +0900 Message-Id: <1510659487-16361-1-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The command "make -j8 C=1 CHECK=scripts/coccicheck COCCI=..." produces lots of "coccicheck failed" error messages. Julia Lawall explained the Coccinelle behavior as follows: "The problem on the Coccinelle side is that it uses a subdirectory with the name of the semantic patch to store standard output and standard error for the different threads. I didn't want to use a name with the pid, so that one could easily find this information while Coccinelle is running. Normally the subdirectory is cleaned up when Coccinelle completes, so there is only one of them at a time. Maybe it is best to just add the pid. There is the risk that these subdirectories will accumulate if Coccinelle crashes in a way such that they don't get cleaned up, but Coccinelle could print a warning if it detects this case, rather than failing." When scripts/coccicheck is used as CHECK tool and -j option is given to Make, the whole of build process runs in parallel. So, multiple processes try to get access to the same subdirectory. I notice spatch creates the subdirectory only when it runs in parallel (i.e. --jobs is given and is greater than 1). Setting NPROC=1 is a reasonable solution; spatch does not create the subdirectory. Besides, ONLINE=1 mode takes a single file input for each spatch invocation, so there is no reason to parallelize it in the first place. Signed-off-by: Masahiro Yamada --- Changes in v4: - Remove unnecessary comments. One line comment will be enough. Changes in v3: - Set NPROC=1 because this is a more sensible solution given that there is no reason to run coccinelle in parallel for ONLINE=1 mode. - Move J= option handling to a proper place. - Add more detailed explanation Changes in v2: - Grep '-j' instead of '--jobserver-auth'. '--jobserver-*' is not a stable option flag. Make 4.2 change '--jobserver-fds' into '--jobserver-auth' - Add -q option to grep scripts/coccicheck | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) -- 2.7.4 Acked-by: Julia Lawall diff --git a/scripts/coccicheck b/scripts/coccicheck index 040a8b1..fefda40 100755 --- a/scripts/coccicheck +++ b/scripts/coccicheck @@ -30,12 +30,6 @@ else VERBOSE=0 fi -if [ -z "$J" ]; then - NPROC=$(getconf _NPROCESSORS_ONLN) -else - NPROC="$J" -fi - FLAGS="--very-quiet" # You can use SPFLAGS to append extra arguments to coccicheck or override any @@ -70,6 +64,9 @@ if [ "$C" = "1" -o "$C" = "2" ]; then # Take only the last argument, which is the C file to test shift $(( $# - 1 )) OPTIONS="$COCCIINCLUDE $1" + + # No need to parallelize Coccinelle since this mode takes one input file. + NPROC=1 else ONLINE=0 if [ "$KBUILD_EXTMOD" = "" ] ; then @@ -77,6 +74,12 @@ else else OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" fi + + if [ -z "$J" ]; then + NPROC=$(getconf _NPROCESSORS_ONLN) + else + NPROC="$J" + fi fi if [ "$KBUILD_EXTMOD" != "" ] ; then