From patchwork Thu Nov 9 15:41:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 118455 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp6746920qgn; Thu, 9 Nov 2017 07:43:00 -0800 (PST) X-Google-Smtp-Source: ABhQp+SLQDdwgv065MmQ0WHcxU3sTSWGm6ojj3uNSvMchFSrZVF4u8MYGuUs/bATHmV5/sGY+hoG X-Received: by 10.98.236.220 with SMTP id e89mr873349pfm.219.1510242180764; Thu, 09 Nov 2017 07:43:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510242180; cv=none; d=google.com; s=arc-20160816; b=SGI3OLst9ARGA/ZCuRvuYKEe7FMK+TzAxp1Lfnq0HgI+WKMF1L+BnC8ksFipuQVyLk t6OFyP7FnlYTdKc5BwDDEQq96TvFIf4GB8RYTTl4T5AZ9vZe//lzJxWInk3rXVd1N+G0 l/vmGbdrBE9YgR6hQbxWBKxMnLfyWBX9rhqHyfXEYOS6f85A78Y9/W1U4puMqSRBA0zJ D5Ms4VP4g+pRHZFgLp3FHRpYuGv6HOllF2uLpx99KdZGb5u+WFltkePInk0i+swkLYY3 AlBBg73fKfVHQh0oR9TmvfS+G5jicx+pqXIJV/Xc7VJVSTfRnrTEPH0soNWV26GRBETX HNxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter :arc-authentication-results; bh=PizTzENAZGokA8zYcC/aulwUXbUws9MjOFKj31l0zi0=; b=PSwhPZs1kryzFlBo1j3YVuyZE6aM5cI9tgIAVGIzs7UL5yEn89pyki5yAo3+qHTiuC 2z9lbuDYScJ58sC/0qQ10J00MujPz7UotbZ0BjsuV4IdRHe4JQqBvt2P/T0PT4DwpNtU NFfbyHo8DhpUl8PRpuZI8Ekkeds57U+v50t++yZfObkFcf/WN6hpIxAwE2V5Qn44uycg 5RvAegUnp7AdKPZRhYwEmoBXTlwihBoK4n0Dk0AdvSkCoPH9ng2n3tNKSECtCtYkmviy vNBLEjtglCUJMgrgSIymw8dMnPjx1vRTIOZ8NdXXIpLOEMTtl/Oha98VthiU2Ky6XSDW RtpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=zGUtHiam; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3si6578313plz.759.2017.11.09.07.42.59; Thu, 09 Nov 2017 07:43:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=zGUtHiam; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752535AbdKIPm6 (ORCPT + 22 others); Thu, 9 Nov 2017 10:42:58 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:23901 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750740AbdKIPm4 (ORCPT ); Thu, 9 Nov 2017 10:42:56 -0500 Received: from grover.sesame (FL1-125-199-20-195.osk.mesh.ad.jp [125.199.20.195]) (authenticated) by conuserg-11.nifty.com with ESMTP id vA9FfQvB007776; Fri, 10 Nov 2017 00:41:27 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com vA9FfQvB007776 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1510242088; bh=PizTzENAZGokA8zYcC/aulwUXbUws9MjOFKj31l0zi0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zGUtHiamurLG/psJ2Q02XrcDPBOdsGzFDeam2yKDS0fKTKn47dvukOLXirwXnLt9+ /Go8xVNOX8PB6rV2ijcS6QzYRb5mAbq9nvPdIb1MYNGAmFVF69Uy6Eg2OhKsywmOyc hiA7XMzBDx8u7lF7NtSzu+L/uplCK2zzL6kyEC2ZeDaditYOyORfuJ5jRxy95OVX3G K0o+4Gvg390+G53G3tWvo6EtYuD56mWqCEDrMbO9y7+xrvuNL+Qjt7jBH5vqSInUsf 037qyEWm46DDdL6/aUdZVqw6z+6PkojSAi4glQTqW2nx4BWZvBuXl8xD15TiFrA2yZ OJ1ZJ5oJKw9Tw== X-Nifty-SrcIP: [125.199.20.195] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Michal Marek , Sam Ravnborg , Douglas Anderson , Masahiro Yamada Subject: [PATCH 1/4] kbuild: create directory for make cache only when necessary Date: Fri, 10 Nov 2017 00:41:14 +0900 Message-Id: <1510242077-8122-2-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> References: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently, the existence of $(dir $(make-cache)) is always checked, and created if it is missing. We can avoid unnecessary system calls by some tricks. [1] If KBUILD_SRC is unset, we are building in the source tree. The output directory checks can be entirely skipped. [2] If at least one cache data is found, it means the cache file was included. Obiously its directory exists. Skip "mkdir -p". [3] If Makefile does not contain any call of __run-and-store, it will not create a cache file. No need to create its directory. [4] The "mkdir -p" should be only invoked by the first call of __run-and-store Signed-off-by: Masahiro Yamada --- scripts/Kbuild.include | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include index be1c9d6..4fb1be1 100644 --- a/scripts/Kbuild.include +++ b/scripts/Kbuild.include @@ -99,18 +99,19 @@ cc-cross-prefix = \ # Include values from last time make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk -ifeq ($(wildcard $(dir $(make-cache))),) -$(shell mkdir -p '$(dir $(make-cache))') -endif $(make-cache): ; -include $(make-cache) +cached-data := $(filter __cached_%, $(.VARIABLES)) + # If cache exceeds 1000 lines, shrink it down to 500. -ifneq ($(word 1000,$(filter __cached_%, $(.VARIABLES))),) +ifneq ($(word 1000,$(cached-data)),) $(shell tail -n 500 $(make-cache) > $(make-cache).tmp; \ mv $(make-cache).tmp $(make-cache)) endif +cache-dir := $(if $(KBUILD_SRC),$(if $(cache-data),,$(dir $(make-cache)))) + # Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios) # # Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_' @@ -136,6 +137,10 @@ __sanitize-opt = $(subst $$,_,$(subst $(right_paren),_,$(subst $(left_paren),_,$ define __run-and-store ifeq ($(origin $(1)),undefined) $$(eval $(1) := $$(shell $$(2))) +ifneq ($(cache-dir),) + $$(shell mkdir -p $(cache-dir)) + $$(eval cache-dir :=) +endif $$(shell echo '$(1) := $$($(1))' >> $(make-cache)) endif endef From patchwork Thu Nov 9 15:41:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 118457 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp6747415qgn; Thu, 9 Nov 2017 07:43:28 -0800 (PST) X-Google-Smtp-Source: ABhQp+S/DPDtEwh9zOU1cdpvjuQolm1+Jv15LvLuuD5ICW9Y1dnvc2DWbnwQieM1R6p7NqINDhpZ X-Received: by 10.98.220.220 with SMTP id c89mr866320pfl.167.1510242208100; Thu, 09 Nov 2017 07:43:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510242208; cv=none; d=google.com; s=arc-20160816; b=gCE6cvq7m376EPPoUO2bhuL5wiaz6UV305cYtdutd1Qaf84wZXhuSSQIVoN405b8kM Hc8GBRMrF0hnTZ8XKAsFfbyH7XgIDLf3wH3DO26AIdpx+KBwudYvb/AQRVZEucJ+xgma 1RAbU8p5ea73NOtlLFUqc/9sogHbGd2f3v2jRhBHyYafL2emmVDcLKNt0iFDu++fMggj R1p6YoAKc8KnfgyQ+6LDzFzymy7XbkP0h+cw7lXHZ3skyC+L3uxeAP8YQrQov19bYbYm cZNSvrIHK6xvAm6uI2OtW2Sx4Xo2H3OVfj9/aQm+diufvU++x7v/zk5QMYzEgITQ7oFR lVWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter :arc-authentication-results; bh=b1q8dwhyyqPlTOGywEzPG+Lsu7ungNwOpVxQYxgT/6M=; b=bhTPoj1B24+XSL2wrewE8r2gsgFJutVpZRPeleKS3QdDyD1lhwg3WfT1PDlx+HCGsx Ko7pP42mUstXt/NH3GYu1ze3gCSn7Z02NDTKlCoJXqyRVEi0wGj1GjslyRhiHaSYFuT5 cih8E4Z5601Nz2A7dvhJDLUTXbzIcvl4I4GJWCxgcA2E9zqJ0E0URudXxpIymr/Wuii+ T+9WDE/8v+tDnWJHpgsRnOT69LrBchKA85d5dAySBmoyToODzrBzFeqkrNePuK1eHvK2 9D68TC6HmwtRW6YGs4n4Je1SJMz858olaDFBduLUS4SM7aQmvWNPpuICieIUYBMBX1mU fdpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=Up4kDia6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3si6578313plz.759.2017.11.09.07.43.27; Thu, 09 Nov 2017 07:43:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=Up4kDia6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752889AbdKIPn0 (ORCPT + 22 others); Thu, 9 Nov 2017 10:43:26 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:23929 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750777AbdKIPm5 (ORCPT ); Thu, 9 Nov 2017 10:42:57 -0500 Received: from grover.sesame (FL1-125-199-20-195.osk.mesh.ad.jp [125.199.20.195]) (authenticated) by conuserg-11.nifty.com with ESMTP id vA9FfQvC007776; Fri, 10 Nov 2017 00:41:28 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com vA9FfQvC007776 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1510242088; bh=b1q8dwhyyqPlTOGywEzPG+Lsu7ungNwOpVxQYxgT/6M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Up4kDia69FY43vDU+19fNymKfLwVAJes3QLPIzFrTD5O2NRpBw/gcAi/i6+mx6a4j 8AAZyJzWFaQw8L6zIcvPtpkpiBGWaZtPjBUi9OkVbP1QsQyHgENLB5sAEviZPv0hgo 2fqCtC8Vwttps9Rbw7Zw0Et2aJ912YpNeGf7ay6Kn/3Uq3AjfJSN0LgpoQl1atu3Qy D9PP5MeviNbI98oH1vevk/Abb1VoYqE4JjCMXPrJrV3YzyTrM/sUbb5eHtWc+N17Rv g1M+zYZr1jTwRt1rWfqKvpMT0ljuMYNVnpsUTCJVNUvrTxNzMc2JvthWuMJWegHj/b 9EIMfEYl2SkBQ== X-Nifty-SrcIP: [125.199.20.195] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Michal Marek , Sam Ravnborg , Douglas Anderson , Masahiro Yamada , Michal Marek Subject: [PATCH 2/4] kbuild: remove redundant $(wildcard ...) for cmd_files calculation Date: Fri, 10 Nov 2017 00:41:15 +0900 Message-Id: <1510242077-8122-3-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> References: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I do not why $(wildcard ...) needs to be called twice for computing cmd_files. Remove the first one. Signed-off-by: Masahiro Yamada --- Makefile | 3 +-- scripts/Makefile.build | 3 +-- scripts/Makefile.headersinst | 3 +-- scripts/Makefile.modpost | 3 +-- 4 files changed, 4 insertions(+), 8 deletions(-) -- 2.7.4 diff --git a/Makefile b/Makefile index a7476e6..58dd245 100644 --- a/Makefile +++ b/Makefile @@ -1693,8 +1693,7 @@ cmd_crmodverdir = $(Q)mkdir -p $(MODVERDIR) \ # read all saved command lines -targets := $(wildcard $(sort $(targets))) -cmd_files := $(wildcard .*.cmd $(foreach f,$(targets),$(dir $(f)).$(notdir $(f)).cmd)) +cmd_files := $(wildcard .*.cmd $(foreach f,$(sort $(targets)),$(dir $(f)).$(notdir $(f)).cmd)) ifneq ($(cmd_files),) $(cmd_files): ; # Do not try to update included dependency files diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 061d0c3..62d5314 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -583,8 +583,7 @@ FORCE: # optimization, we don't need to read them if the target does not # exist, we will rebuild anyway in that case. -targets := $(wildcard $(sort $(targets))) -cmd_files := $(wildcard $(foreach f,$(targets),$(dir $(f)).$(notdir $(f)).cmd)) +cmd_files := $(wildcard $(foreach f,$(sort $(targets)),$(dir $(f)).$(notdir $(f)).cmd)) ifneq ($(cmd_files),) include $(cmd_files) diff --git a/scripts/Makefile.headersinst b/scripts/Makefile.headersinst index 5692d7a..2aa9181 100644 --- a/scripts/Makefile.headersinst +++ b/scripts/Makefile.headersinst @@ -114,9 +114,8 @@ $(check-file): scripts/headers_check.pl $(output-files) FORCE endif -targets := $(wildcard $(sort $(targets))) cmd_files := $(wildcard \ - $(foreach f,$(targets),$(dir $(f)).$(notdir $(f)).cmd)) + $(foreach f,$(sort $$(targets)),$(dir $(f)).$(notdir $(f)).cmd)) ifneq ($(cmd_files),) include $(cmd_files) diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index 16923ba..cf125c1 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -143,8 +143,7 @@ FORCE: # optimization, we don't need to read them if the target does not # exist, we will rebuild anyway in that case. -targets := $(wildcard $(sort $(targets))) -cmd_files := $(wildcard $(foreach f,$(targets),$(dir $(f)).$(notdir $(f)).cmd)) +cmd_files := $(wildcard $(foreach f,$(sort $(targets)),$(dir $(f)).$(notdir $(f)).cmd)) ifneq ($(cmd_files),) include $(cmd_files) From patchwork Thu Nov 9 15:41:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 118459 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp6748710qgn; Thu, 9 Nov 2017 07:44:39 -0800 (PST) X-Google-Smtp-Source: ABhQp+QlweB8VZLuF0CrAbrMmUQa/Ndppnyky1oV9wVcjnPJnoMgBTPAUNJnhtYRd/aAxj85kyjN X-Received: by 10.98.80.69 with SMTP id e66mr909361pfb.112.1510242279364; Thu, 09 Nov 2017 07:44:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510242279; cv=none; d=google.com; s=arc-20160816; b=nNdFwdO/xXbumvE0eYySQromzDfTG1ElqKeYiPpx8pal7K32vEtlN4vq0NcCJq4Gdk uQmWqzORt4T/1iLrkkssMt8l/v+LW6692/Er7aW1RbckeJH9mgZx7/TzWPYcNsuVkPlM FioTSSmk34oPRh3Le3M6jWXcdB5m8E43Dow9OuOvoApVl/s9Z+UF3tyfu7jBO2kbCCk5 B2Ly6XgKZljig/E2V5w+jchfpCff4m5yjgde8kKo9WSrwKmyz+EUaSwlIpRfs/vxRRSF QnW2o9xFlxcf6Opx4io3CwsnyingMvjgOVbwz6w+O8qoPgcbrvfieqk23s1Zh7awim4x iXqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter :arc-authentication-results; bh=HQ0BGAXco9VAAeYqP7FAdlrS6+Mrdeyd79pBTDfA5/I=; b=iC/ELw0CxVd41B+rhOu5rctxYTFuSqwubPFdiWvHRxeXv+5kUIOVm6L/uiuZDPhqdl HV6CUMZP4pMe1JK9OXneBmEBG1EO5StFXVWXrfV7tQovOVogTi3lGzyOeaIUNUWIq3r/ 0NSb63kD1bRZFvzx5AOwGQNekEG+He9WYTQuTfmeRFLW0c+lsPbxr+HCNzxirYfeog2b pW2vl6Z7z2jPUD/h7DAYpr4oiIeJ81rlEwE0i3py7Co25m/0QxDvnXhB1eSfYLKJFo+c 5zuHN/3bol+tJURWF3mKKRb8cI3LZA72dZ3fM0ZGHZftrDjqdj/95u8P57KpCz99bIe2 r77Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=smy7o9Kk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m6si7069001pfm.64.2017.11.09.07.44.39; Thu, 09 Nov 2017 07:44:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=smy7o9Kk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752958AbdKIPoh (ORCPT + 22 others); Thu, 9 Nov 2017 10:44:37 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:25757 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751915AbdKIPod (ORCPT ); Thu, 9 Nov 2017 10:44:33 -0500 Received: from grover.sesame (FL1-125-199-20-195.osk.mesh.ad.jp [125.199.20.195]) (authenticated) by conuserg-11.nifty.com with ESMTP id vA9FfQvD007776; Fri, 10 Nov 2017 00:41:29 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com vA9FfQvD007776 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1510242089; bh=HQ0BGAXco9VAAeYqP7FAdlrS6+Mrdeyd79pBTDfA5/I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=smy7o9KkzuTKrpitfozNdvyneO3W4ddtwGXWeh5D7PX2AoqT1Cj7EotyNSpKHQNBz AAN65HUm/P5AmL6iPuxg+rP76QFmTvBC3oxRiuVh1he0L2kezPyLB9v2gO/HxHDdm+ SmTjQX/RQeqmtZTePrAXr8xraPdAE/ICgslpvbNJgSjlKPqGW+4YAuzie8cUHMqS8L 5glvKmmvmoOYvTID/SLOacOWz9jTr2lQaP3zLl97sfnFa9WRwpW3aydRFaC3vwtvFb VJaB7/flQjZym712SsaD8Xm01PZhqxhdU756rP+n6k5jZbZOEYPCxE2YJ6EFa9oQ1r lnumfRVveZ+Pw== X-Nifty-SrcIP: [125.199.20.195] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Michal Marek , Sam Ravnborg , Douglas Anderson , Masahiro Yamada , Michal Marek , x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Andy Lutomirski , Ingo Molnar Subject: [PATCH 3/4] kbuild: create object directories simpler and faster Date: Fri, 10 Nov 2017 00:41:16 +0900 Message-Id: <1510242077-8122-4-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> References: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org For the out-of-tree build, scripts/Makefile.build creates output directories, but this operation is not efficient. scripts/Makefile.lib calculates obj-dirs as follows: obj-dirs := $(dir $(multi-objs) $(obj-y)) Please notice $(sort ...) is not used here. Usually the resulted obj-dirs is as many "./" as objects. For those duplicated paths, the following command is invoked. _dummy := $(foreach d,$(obj-dirs), $(shell [ -d $(d) ] || mkdir -p $(d))) Then, the costly shell command is run over and over again. I see many points for optimization: [1] Use $(sort ...) to cut down duplicated paths before passing them to system call [2] Use single $(shell ...) instead of repeating it with $(foreach ...) This will reduce forking. [3] We can calculate obj-dirs more simply. Most of objects are already accumulated in $(targets). So, $(dir $(targets)) is fine and more comprehensive. I also removed bad code in arch/x86/entry/vdso/Makefile. This is now really unnecessary. Signed-off-by: Masahiro Yamada --- arch/x86/entry/vdso/Makefile | 4 ---- scripts/Makefile.build | 15 ++++++--------- scripts/Makefile.host | 11 ----------- scripts/Makefile.lib | 5 ----- 4 files changed, 6 insertions(+), 29 deletions(-) -- 2.7.4 Acked-by: Ingo Molnar Tested-by: Douglas Anderson diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile index d540966..f8e3d85 100644 --- a/arch/x86/entry/vdso/Makefile +++ b/arch/x86/entry/vdso/Makefile @@ -129,10 +129,6 @@ $(obj)/vdsox32.so.dbg: $(src)/vdsox32.lds $(vobjx32s) FORCE CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds) VDSO_LDFLAGS_vdso32.lds = -m32 -Wl,-m,elf_i386 -Wl,-soname=linux-gate.so.1 -# This makes sure the $(obj) subdirectory exists even though vdso32/ -# is not a kbuild sub-make subdirectory. -override obj-dirs = $(dir $(obj)) $(obj)/vdso32/ - targets += vdso32/vdso32.lds targets += vdso32/note.o vdso32/system_call.o vdso32/sigreturn.o targets += vdso32/vclock_gettime.o diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 62d5314..89ac180 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -64,15 +64,6 @@ ifneq ($(hostprogs-y)$(hostprogs-m)$(hostlibs-y)$(hostlibs-m)$(hostcxxlibs-y)$(h include scripts/Makefile.host endif -ifneq ($(KBUILD_SRC),) -# Create output directory if not already present -_dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj)) - -# Create directories for object files if directory does not exist -# Needed when obj-y := dir/file.o syntax is used -_dummy := $(foreach d,$(obj-dirs), $(shell [ -d $(d) ] || mkdir -p $(d))) -endif - ifndef obj $(warning kbuild: Makefile.build is included improperly) endif @@ -589,6 +580,12 @@ ifneq ($(cmd_files),) include $(cmd_files) endif +ifneq ($(KBUILD_SRC),) +# Create directories for object files if directory does not exist +obj-dirs := $(sort $(obj) $(patsubst %/,%, $(dir $(targets)))) +$(shell mkdir -p $(obj-dirs)) +endif + # Declare the contents of the .PHONY variable as phony. We keep that # information in a variable se we can use it in if_changed and friends. diff --git a/scripts/Makefile.host b/scripts/Makefile.host index 9cfd5c8..3a5460d 100644 --- a/scripts/Makefile.host +++ b/scripts/Makefile.host @@ -48,15 +48,6 @@ host-cxxobjs := $(sort $(foreach m,$(host-cxxmulti),$($(m)-cxxobjs))) host-cshobjs := $(sort $(foreach m,$(host-cshlib),$($(m:.so=-objs)))) host-cxxshobjs := $(sort $(foreach m,$(host-cxxshlib),$($(m:.so=-objs)))) -# output directory for programs/.o files -# hostprogs-y := tools/build may have been specified. -# Retrieve also directory of .o files from prog-objs or prog-cxxobjs notation -host-objdirs := $(dir $(__hostprogs) $(host-cobjs) $(host-cxxobjs)) - -host-objdirs := $(strip $(sort $(filter-out ./,$(host-objdirs)))) - - -__hostprogs := $(addprefix $(obj)/,$(__hostprogs)) host-csingle := $(addprefix $(obj)/,$(host-csingle)) host-cmulti := $(addprefix $(obj)/,$(host-cmulti)) host-cobjs := $(addprefix $(obj)/,$(host-cobjs)) @@ -68,8 +59,6 @@ host-cshobjs := $(addprefix $(obj)/,$(host-cshobjs)) host-cxxshobjs := $(addprefix $(obj)/,$(host-cxxshobjs)) host-objdirs := $(addprefix $(obj)/,$(host-objdirs)) -obj-dirs += $(host-objdirs) - ##### # Handle options to gcc. Support building with separate output directory diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 4d88ad7..5fbc46d 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -50,15 +50,11 @@ single-used-m := $(sort $(filter-out $(multi-used-m),$(obj-m))) # objects depend on those (obviously) multi-objs-y := $(foreach m, $(multi-used-y), $($(m:.o=-objs)) $($(m:.o=-y))) multi-objs-m := $(foreach m, $(multi-used-m), $($(m:.o=-objs)) $($(m:.o=-y))) -multi-objs := $(multi-objs-y) $(multi-objs-m) # $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to # tell kbuild to descend subdir-obj-y := $(filter %/built-in.o, $(obj-y)) -# $(obj-dirs) is a list of directories that contain object files -obj-dirs := $(dir $(multi-objs) $(obj-y)) - # Replace multi-part objects by their individual parts, look at local dir only real-objs-y := $(foreach m, $(filter-out $(subdir-obj-y), $(obj-y)), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))),$($(m:.o=-objs)) $($(m:.o=-y)),$(m))) $(extra-y) real-objs-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m))) @@ -81,7 +77,6 @@ multi-used-m := $(addprefix $(obj)/,$(multi-used-m)) multi-objs-y := $(addprefix $(obj)/,$(multi-objs-y)) multi-objs-m := $(addprefix $(obj)/,$(multi-objs-m)) subdir-ym := $(addprefix $(obj)/,$(subdir-ym)) -obj-dirs := $(addprefix $(obj)/,$(obj-dirs)) # These flags are needed for modversions and compiling, so we define them here # $(modname_flags) defines KBUILD_MODNAME as the name of the module it will From patchwork Thu Nov 9 15:41:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 118456 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp6747075qgn; Thu, 9 Nov 2017 07:43:09 -0800 (PST) X-Google-Smtp-Source: ABhQp+S+Z0bsYvvg2uizja6oMp8i4ZDco/tr+7Do5f3ERDE3TaqQe+KZZ/884C6FdE6bct5ZgoRM X-Received: by 10.99.126.81 with SMTP id o17mr876570pgn.252.1510242189513; Thu, 09 Nov 2017 07:43:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510242189; cv=none; d=google.com; s=arc-20160816; b=HmzyLy+MGkVmSi38NsnkTErH+boMiplIkpxngTggXktcYq3V0QrKAlUVHJeiCcAKDB z8z7aiyN7t4VOHXYlcVLVFbg2WJmYCMwh7owAyYC3odVr4XSw7XZEx7TrhFeSrsLeOi+ tvcJZRP43hvUkaLKwcEm8TGmydE6wnXbU7BmZUu+YtXGZjMCYO9OIqYAkn6Cksf5uogz A8z/1HXLsX5WbVLhXT3DENSE8IXfAYN0FKoMgsGuL8uSJrncrGgpnmJ8ovyze+/cx8iz Aa9+buXDtIPPWhgYw9MWx9/SaMrEjBPfrlu99VlOC30EeNFxReNJ2kKpNT2StgGLEEJl LFxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter :arc-authentication-results; bh=mNXkCG/ZgjaELiZQWp3W2p9QJw1x/yVepfZbX90AtQw=; b=MAPYkxHAfYwVb/24sAM880N6kMEvCCtO3ynP1ZJtMZOM+lwLj0L+gTGY9ema/MmWzF skgDmkSPXn7I6PkJCcIoiA9HafdWPwKn31F3QMMpjYNgSpW6vpRa7qThAhcRV71aGcnb jZ+VDbuaYyc5bjlDWutwsvxC5ypl1f6CgD4cpbq8CYDA6FysV0ef1ywb9qnmiw4Kfaz2 RrgZIe7BSLrGdggulDkiej4s5dbxGdACOs623fQOnCHl0YFrICjxREDWG4ZwPknB+LHr bA9zdYMIb8cIMP061gxunXTmDKWw3hDFH0Arv2icOIR5IK9VkFqA3Qk3OXSMYWnTkBaV lsSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=Zp8E5m66; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3si6578313plz.759.2017.11.09.07.43.09; Thu, 09 Nov 2017 07:43:09 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=Zp8E5m66; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752738AbdKIPnH (ORCPT + 22 others); Thu, 9 Nov 2017 10:43:07 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:23930 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750909AbdKIPm5 (ORCPT ); Thu, 9 Nov 2017 10:42:57 -0500 Received: from grover.sesame (FL1-125-199-20-195.osk.mesh.ad.jp [125.199.20.195]) (authenticated) by conuserg-11.nifty.com with ESMTP id vA9FfQvE007776; Fri, 10 Nov 2017 00:41:30 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com vA9FfQvE007776 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1510242090; bh=mNXkCG/ZgjaELiZQWp3W2p9QJw1x/yVepfZbX90AtQw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zp8E5m66CQ14hbv2hN+I9pMdaUaaHG/9lhjI7NFk6yEiTGA+LwJ4VIJoA253mFbsR n5IKN6OtVt7i2e1ausdHKy6h4qOfkw/A4yr62bpJVZp7oC51OjGGuEC2+O/LOpMBmF 5kkhdhtU8EqMQELsBEFedAZGcIAxjPa7Kk9Oi0VB6BCZhTFQNi/D471S+ywGWD/a7u B/TjG5RGJE41B5wj3cK9p/ipFrEGqhLjKIOCPgiZiu2itY6OuTD1+bIbfh1uGAdMW/ wkrUzM9+DISYjQnpdbFNL8e7WuAW/DuTZGww7lspppjmUkuEGzwtAk/IJokmcO4nZG kwaLv9hjNGaqA== X-Nifty-SrcIP: [125.199.20.195] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Michal Marek , Sam Ravnborg , Douglas Anderson , Masahiro Yamada , Michal Marek Subject: [PATCH 4/4] kbuild: optimize object directory creation for incremental build Date: Fri, 10 Nov 2017 00:41:17 +0900 Message-Id: <1510242077-8122-5-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> References: <1510242077-8122-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The previous commit largely optimized the object directory creation. We can optimize it more for incremental build. There are already *.cmd files in the output directory. The existing *.cmd files have been picked up by $(wildcard ...). Obviously, directories containing them exist too, so we can skip "mkdir -p". With this, Kbuild runs almost zero "mkdir -p" in incremental building. Signed-off-by: Masahiro Yamada --- scripts/Makefile.build | 5 +++++ 1 file changed, 5 insertions(+) -- 2.7.4 diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 89ac180..90ea7a5 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -583,8 +583,13 @@ endif ifneq ($(KBUILD_SRC),) # Create directories for object files if directory does not exist obj-dirs := $(sort $(obj) $(patsubst %/,%, $(dir $(targets)))) +# If cmd_files exist, their directories apparently exist. Skip mkdir. +exist-dirs := $(sort $(patsubst %/,%, $(dir $(cmd_files)))) +obj-dirs := $(strip $(filter-out . $(exist-dirs), $(obj-dirs))) +ifneq ($(obj-dirs),) $(shell mkdir -p $(obj-dirs)) endif +endif # Declare the contents of the .PHONY variable as phony. We keep that # information in a variable se we can use it in if_changed and friends.