From patchwork Sat Dec 9 16:02:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 121286 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp838553qgn; Sat, 9 Dec 2017 08:04:36 -0800 (PST) X-Google-Smtp-Source: AGs4zMawxRImAfNzCTI2aCsxLoIvrj3/g3xzBNorzpSKcc5rPnhFJXRv2XzIL4thn4jbz1o4l0Xm X-Received: by 10.101.100.24 with SMTP id a24mr33950630pgv.239.1512835476209; Sat, 09 Dec 2017 08:04:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512835476; cv=none; d=google.com; s=arc-20160816; b=yfmKFy43CbCZa8fvsB55aTFG5CjWRg9YvKGj04sXsqd8hVWF10FekSBm3P/aweXr4A ReHotmG64A1sQsrHHWXzdh2vDnLuDUc6VIFc0Gwte1iEQ06eFD9ePg4K4CcUEWAx+v2C F+kItKt9TDls7fKkPH+nby/6/iau9IuUHghr57U+4FeR80k6b5rwXHnXmrg9sB7n8V+4 tjN/S5h3hkNGH24IjaJqVolmwzqUsYU5ap/skLs2X/EeVxvyIEy8mrkFIEi78MeGElmT IQCRcYAcI6tA7G9H0pa7ifFT1ZYL1Llo7i1xLcfhgdD4hyXeWhKoVpRuXudVYPKP3nkl DPzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter :arc-authentication-results; bh=uuN4pGsbTG95XeDS53rqGtHDxWdeS6NKatXw210JGPk=; b=uOOL27rwfvEPEbKjbTidDxcvFByQzVsvGyouIXe0pnte2kZirzKrz3tM4HdCs4HEm+ Auwu8zhyMdz3MUeOk2gyly37AqVTSaoIBrB25C71O8IOtcTMIdlrzZmWkE2DkiY4xyMz DXDyEH8bcsAM4RvPf90OrRwlUCwOp8bm/chPwJMuqg7OX1dzeFoua6GvEiiy+E9FfcXI yZ9DvVr3VJW/rqwCTWZnhDzrWfjch8M9ybyWN66/E0ewMp7f/D7aDFS+s7mn0NIzYefu qT+m9refxKqKSjtNoMsz/56ds/t05L/DGvc3pgSJrZeCBHfsH8xzvedTcxWGyU3Exdvq nEDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=zNxEeLPQ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x6si4537808plw.347.2017.12.09.08.04.34; Sat, 09 Dec 2017 08:04:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nifty.com header.s=dec2015msa header.b=zNxEeLPQ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751429AbdLIQEd (ORCPT + 20 others); Sat, 9 Dec 2017 11:04:33 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:36302 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750947AbdLIQE3 (ORCPT ); Sat, 9 Dec 2017 11:04:29 -0500 Received: from grover.sesame (FL1-125-199-20-195.osk.mesh.ad.jp [125.199.20.195]) (authenticated) by conuserg-11.nifty.com with ESMTP id vB9G2gja027275; Sun, 10 Dec 2017 01:02:45 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com vB9G2gja027275 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1512835367; bh=uuN4pGsbTG95XeDS53rqGtHDxWdeS6NKatXw210JGPk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zNxEeLPQJ+LCY0Iy+Ivn0/aH7cnilY8PP/1NmiPWA6wNFkrRsOhnDjfSoyC2gc0io I86qeVYPs2UMuvQzIh04N/jbZccNZQ1nwQtzoVrFAwWWJ6UohIeyhhvG9Af/70L2Xr v1ILBx+Z6HtvaVK8qvhGr+gJeO4eKbowX54sszp0Jeg5k0CSKSZI5TRyMhs89pUPGR xNf2tb20Pa6t92cmoIcfjSFU5pEEPJHiRUIEfR9VrqppgIBGW4zOs5mlDeblugu+A8 RSjW9rIEsCYKkYG3JATvwTwrrFwu2YEH1O9Oq7my9bPQa6ysIzX55PwpYDtDX5Sp/i ZNiIkgnRlVjuA== X-Nifty-SrcIP: [125.199.20.195] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: Ulf Magnusson , Sam Ravnborg , Linus Torvalds , Masahiro Yamada , Mauro Carvalho Chehab , Nicholas Piggin , linux-doc@vger.kernel.org, Markus Heiser , Jonathan Corbet , linux-kernel@vger.kernel.org, Michal Marek , SeongJae Park , Max Filippov Subject: [PATCH 2/3] kbuild: prepare to remove C files pre-generated by flex and bison Date: Sun, 10 Dec 2017 01:02:29 +0900 Message-Id: <1512835350-8646-3-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1512835350-8646-1-git-send-email-yamada.masahiro@socionext.com> References: <1512835350-8646-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In Linux build system convention, pre-generated files are version- controlled with a "_shipped" suffix. During the kernel building, they are simply shipped (copied) removing the suffix. This approach can reduce external tool dependency for the kernel build, but it is tedious to manually regenerate such artifacts from developers' point of view. (We need to do "make REGENERATE_PARSERS=1" every time we touch real source files such as *.l, *.y) Some months ago, I sent out RFC patches to run flex, bison, and gperf during the build. In the review and test, Linus noticed gperf-3.1 had changed the lookup function prototype. Then, the use of gperf in kernel was entirely removed by commit bb3290d91695 ("Remove gperf usage from toolchain"). I tested several versions of flex and bison, and I was not hit by any compatibility issue except a flaw in flex-2.6.3; if you generate lexer for dtc and genksyms with flex-2.6.3, you will see "yywrap redefined" warning. This was not intentional, but a bug fixed by flex-2.6.4. Otherwise, flex and bison look fairly stable for a long time. This commit prepares some build rules to start removing the _shipped files. Also, document minimal requirement for flex and bison. Rationale for the minimal version: The -Wmissing-prototypes option of GCC reports "no previous prototype" warnings for lexers generated by flex-2.5.34, so I chose 2.5.35 as the required version for flex. Flex-2.5.35 was released in 2008. Bison looked more stable. I did not see any problem with bison-2.0, released in 2004. I did not test bison-1.x, but bison-2.0 should be old enough. Tested flex versions: 2.5.35 2.5.36 2.5.37 2.5.39 2.6.0 2.6.1 2.6.2 2.6.3 (*) 2.6.4 (*) flex-2.6.3 causes "yywrap redefined" warning Tested bison versions: 2.0 2.1 2.2 2.3 2.4 2.4.1 2.5.1 2.6 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7 2.7.1 3.0 3.0.1 3.0.2 3.0.3 3.0.4 Signed-off-by: Masahiro Yamada --- Documentation/process/changes.rst | 25 +++++++++++++++++++++++++ scripts/Makefile.lib | 20 +++++++++++++++++--- 2 files changed, 42 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 560beae..fc9c7c3 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -32,6 +32,8 @@ you probably needn't concern yourself with isdn4k-utils. GNU C 3.2 gcc --version GNU make 3.81 make --version binutils 2.20 ld -v +flex 2.5.35 flex --version +bison 2.0 bison --version util-linux 2.10o fdformat --version module-init-tools 0.9.10 depmod -V e2fsprogs 1.41.4 e2fsck -V @@ -79,6 +81,19 @@ The build system has, as of 4.13, switched to using thin archives (`ar T`) rather than incremental linking (`ld -r`) for built-in.o intermediate steps. This requires binutils 2.20 or newer. +Flex +---- + +Since Linux 4.16, the build system generates lexical analisers +during build. This requires flex 2.5.35 or later. + + +Bison +----- + +Since Linux 4.16, the build system generates parsers +during build. This requires bison 2.0 or later. + Perl ---- @@ -333,6 +348,16 @@ Binutils - +Flex +---- + +- + +Bison +----- + +- + OpenSSL ------- diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index ee528e3..0f9ef3f 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -186,8 +186,6 @@ $(foreach m, $(notdir $1), \ $(addprefix $(obj)/, $(foreach s, $3, $($(m:%$(strip $2)=%$(s))))))) endef -ifdef REGENERATE_PARSERS - # LEX # --------------------------------------------------------------------------- LEX_PREFIX = $(if $(LEX_PREFIX_${baseprereq}),$(LEX_PREFIX_${baseprereq}),yy) @@ -195,9 +193,15 @@ LEX_PREFIX = $(if $(LEX_PREFIX_${baseprereq}),$(LEX_PREFIX_${baseprereq}),yy) quiet_cmd_flex = LEX $@ cmd_flex = $(LEX) -o$@ -L -P $(LEX_PREFIX) $< +ifdef REGENERATE_PARSERS .PRECIOUS: $(src)/%.lex.c_shipped $(src)/%.lex.c_shipped: $(src)/%.l $(call cmd,flex) +endif + +.PRECIOUS: $(obj)/%.lex.c +$(filter %.lex.c,$(targets)): $(obj)/%.lex.c: $(src)/%.l FORCE + $(call if_changed,flex) # YACC # --------------------------------------------------------------------------- @@ -206,19 +210,29 @@ YACC_PREFIX = $(if $(YACC_PREFIX_${baseprereq}),$(YACC_PREFIX_${baseprereq}),yy) quiet_cmd_bison = YACC $@ cmd_bison = $(YACC) -o$@ -t -l -p $(YACC_PREFIX) $< +ifdef REGENERATE_PARSERS .PRECIOUS: $(src)/%.tab.c_shipped $(src)/%.tab.c_shipped: $(src)/%.y $(call cmd,bison) +endif + +.PRECIOUS: $(obj)/%.tab.c +$(filter %.tab.c,$(targets)): $(obj)/%.tab.c: $(src)/%.y FORCE + $(call if_changed,bison) quiet_cmd_bison_h = YACC $@ cmd_bison_h = bison -o/dev/null --defines=$@ -t -l -p $(YACC_PREFIX) $< +ifdef REGENERATE_PARSERS .PRECIOUS: $(src)/%.tab.h_shipped $(src)/%.tab.h_shipped: $(src)/%.y $(call cmd,bison_h) - endif +.PRECIOUS: $(obj)/%.tab.h +$(filter %.tab.h,$(targets)): $(obj)/%.tab.h: $(src)/%.y FORCE + $(call if_changed,bison_h) + # Shipped files # ===========================================================================