| Message ID | 1554633831-10303-1-git-send-email-yamada.masahiro@socionext.com |
|---|---|
| State | Accepted |
| Commit | a788b2ed81abeb949325fd50d7527f45203c1b21 |
| Headers | show |
| Series | kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build | expand |
On Sun, Apr 7, 2019 at 7:44 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > After cross-compiling the kernel, "make mrproper" should be executed > with the proper ARCH= option. Otherwise, stale objects will remain > under arch/$(SRCARCH)/. > > One bad scenario is like this: > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > $ make mrproper # mrproper for host-arch (i.e. x86) > $ make ARCH=arm O=build_dir defconfig all > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > and ARCH= options, Kbuild will happily start to build, but may fail > due to stale objects in the srctree. > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > the out-of-tree build. To detect this, mrproper should clean only > arch/$(SRCARCH)/include/generated/. > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Applied to linux-kbuild. > --- > > Makefile | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/Makefile b/Makefile > index 10643c3..17945ce 100644 > --- a/Makefile > +++ b/Makefile > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > prepare3: include/config/kernel.release > ifneq ($(srctree),.) > @$(kecho) ' Using $(srctree) as source for kernel' > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > + $(Q)if [ -f $(srctree)/.config -o \ > + -d $(srctree)/include/config -o \ > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > echo >&2 " in the '$(srctree)' directory.";\ > /bin/false; \ > @@ -1350,7 +1352,7 @@ CLEAN_DIRS += $(MODVERDIR) include/ksym > > # Directories & files removed with 'make mrproper' > MRPROPER_DIRS += include/config usr/include include/generated \ > - arch/*/include/generated .tmp_objdiff > + arch/$(SRCARCH)/include/generated .tmp_objdiff > MRPROPER_FILES += .config .config.old .version \ > Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \ > signing_key.pem signing_key.priv signing_key.x509 \ > -- > 2.7.4 > -- Best Regards Masahiro Yamada
Hi Yamada-san, On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > After cross-compiling the kernel, "make mrproper" should be executed > with the proper ARCH= option. Otherwise, stale objects will remain > under arch/$(SRCARCH)/. > > One bad scenario is like this: > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > $ make mrproper # mrproper for host-arch (i.e. x86) > $ make ARCH=arm O=build_dir defconfig all > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > and ARCH= options, Kbuild will happily start to build, but may fail > due to stale objects in the srctree. > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > the out-of-tree build. To detect this, mrproper should clean only > arch/$(SRCARCH)/include/generated/. > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> > --- > > Makefile | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/Makefile b/Makefile > index 10643c3..17945ce 100644 > --- a/Makefile > +++ b/Makefile > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > prepare3: include/config/kernel.release > ifneq ($(srctree),.) > @$(kecho) ' Using $(srctree) as source for kernel' > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > + $(Q)if [ -f $(srctree)/.config -o \ > + -d $(srctree)/include/config -o \ > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > echo >&2 " in the '$(srctree)' directory.";\ This took me a bit to find out what was wrong... Usually I don't run "make mrproper", as it removes files I may want to keep (e.g. tags). Hence I ran "git ls-files -o | grep m68k | xargs rm" (I usually build in separate output directories), confirmed with "git ls-files -o" there were no remaining build artefacts, and was surprised to discover I still got the error message above?!? Apparently arch/m68k/include/generated was still present, but as "git ls-files -o" only shows files, not directories, it was not listed. Perhaps the directory checks above can be changed to directory exists _and_ is not empty? Thanks! Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Hi Geert, On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > Hi Yamada-san, > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada > <yamada.masahiro@socionext.com> wrote: > > After cross-compiling the kernel, "make mrproper" should be executed > > with the proper ARCH= option. Otherwise, stale objects will remain > > under arch/$(SRCARCH)/. > > > > One bad scenario is like this: > > > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > > $ make mrproper # mrproper for host-arch (i.e. x86) > > $ make ARCH=arm O=build_dir defconfig all > > > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > > and ARCH= options, Kbuild will happily start to build, but may fail > > due to stale objects in the srctree. > > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > > the out-of-tree build. To detect this, mrproper should clean only > > arch/$(SRCARCH)/include/generated/. > > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> > > --- > > > > Makefile | 6 ++++-- > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > diff --git a/Makefile b/Makefile > > index 10643c3..17945ce 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > > prepare3: include/config/kernel.release > > ifneq ($(srctree),.) > > @$(kecho) ' Using $(srctree) as source for kernel' > > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > > + $(Q)if [ -f $(srctree)/.config -o \ > > + -d $(srctree)/include/config -o \ > > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > > echo >&2 " in the '$(srctree)' directory.";\ > > This took me a bit to find out what was wrong... > > Usually I don't run "make mrproper", as it removes files I may want to > keep (e.g. tags). Hence I ran "git ls-files -o | grep m68k | xargs rm" > (I usually build in separate output directories), confirmed with "git > ls-files -o" there were no remaining build artefacts, and was surprised > to discover I still got the error message above?!? > > Apparently arch/m68k/include/generated was still present, but as "git > ls-files -o" only shows files, not directories, it was not listed. > Perhaps the directory checks above can be changed to directory exists > _and_ is not empty? No. Since you did not run mrproper, Kbuild _correctly_ showed error. This is the expected and correct behavior. :) The upstream kernel is not a place to be customized for your workflow. Sorry. Every developer has a set of handy custom commands. Since you are already running a long command, why don't you add one more line, and put in ~/.bash_aliases or somewhere? my_mrproper() { git ls-files -o | grep m68k | xargs rm rm -rf arch/m68k/include/generated } -- Best Regards Masahiro Yamada
Hi Yamada-san, On Tue, Jul 9, 2019 at 2:22 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada > > <yamada.masahiro@socionext.com> wrote: > > > After cross-compiling the kernel, "make mrproper" should be executed > > > with the proper ARCH= option. Otherwise, stale objects will remain > > > under arch/$(SRCARCH)/. > > > > > > One bad scenario is like this: > > > > > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > > > $ make mrproper # mrproper for host-arch (i.e. x86) > > > $ make ARCH=arm O=build_dir defconfig all > > > > > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > > > and ARCH= options, Kbuild will happily start to build, but may fail > > > due to stale objects in the srctree. > > > > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > > > the out-of-tree build. To detect this, mrproper should clean only > > > arch/$(SRCARCH)/include/generated/. > > > > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> > > > --- > > > > > > Makefile | 6 ++++-- > > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > > > diff --git a/Makefile b/Makefile > > > index 10643c3..17945ce 100644 > > > --- a/Makefile > > > +++ b/Makefile > > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > > > prepare3: include/config/kernel.release > > > ifneq ($(srctree),.) > > > @$(kecho) ' Using $(srctree) as source for kernel' > > > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > > > + $(Q)if [ -f $(srctree)/.config -o \ > > > + -d $(srctree)/include/config -o \ > > > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > > > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > > > echo >&2 " in the '$(srctree)' directory.";\ > > > > This took me a bit to find out what was wrong... > > > > Usually I don't run "make mrproper", as it removes files I may want to > > keep (e.g. tags). Hence I ran "git ls-files -o | grep m68k | xargs rm" > > (I usually build in separate output directories), confirmed with "git > > ls-files -o" there were no remaining build artefacts, and was surprised > > to discover I still got the error message above?!? > > > > Apparently arch/m68k/include/generated was still present, but as "git > > ls-files -o" only shows files, not directories, it was not listed. > > Perhaps the directory checks above can be changed to directory exists > > _and_ is not empty? > > No. > > Since you did not run mrproper, > Kbuild _correctly_ showed error. > This is the expected and correct behavior. :) > > The upstream kernel is not a place > to be customized for your workflow. Sorry. > > > Every developer has a set of handy custom commands. > > Since you are already running a long command, > why don't you add one more line, and put in ~/.bash_aliases or somewhere? > > my_mrproper() > { > git ls-files -o | grep m68k | xargs rm > rm -rf arch/m68k/include/generated > } Please note this was not part of my standard workflow, so I don't have a script for it. I just happened to had done a quick test build in my kernel source tree repository before, and had forgotten about that. So IMHO this is more of a usability issue: it is difficult to find out what is wrong, and how to solve it, as "git ls-files -o" doesn't give a clue. And running "make mrproper" doesn't help. Perhaps the message should be changed to $(srctree) is not clean, please run 'make ARCH=$(SRCARCH) mrproper'" instead? Thanks! Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Hi Geert, On Tue, Jul 9, 2019 at 9:45 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > Hi Yamada-san, > > On Tue, Jul 9, 2019 at 2:22 PM Masahiro Yamada > <yamada.masahiro@socionext.com> wrote: > > On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada > > > <yamada.masahiro@socionext.com> wrote: > > > > After cross-compiling the kernel, "make mrproper" should be executed > > > > with the proper ARCH= option. Otherwise, stale objects will remain > > > > under arch/$(SRCARCH)/. > > > > > > > > One bad scenario is like this: > > > > > > > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > > > > $ make mrproper # mrproper for host-arch (i.e. x86) > > > > $ make ARCH=arm O=build_dir defconfig all > > > > > > > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > > > > and ARCH= options, Kbuild will happily start to build, but may fail > > > > due to stale objects in the srctree. > > > > > > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > > > > the out-of-tree build. To detect this, mrproper should clean only > > > > arch/$(SRCARCH)/include/generated/. > > > > > > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> > > > > --- > > > > > > > > Makefile | 6 ++++-- > > > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/Makefile b/Makefile > > > > index 10643c3..17945ce 100644 > > > > --- a/Makefile > > > > +++ b/Makefile > > > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > > > > prepare3: include/config/kernel.release > > > > ifneq ($(srctree),.) > > > > @$(kecho) ' Using $(srctree) as source for kernel' > > > > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > > > > + $(Q)if [ -f $(srctree)/.config -o \ > > > > + -d $(srctree)/include/config -o \ > > > > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > > > > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > > > > echo >&2 " in the '$(srctree)' directory.";\ > > > > > > This took me a bit to find out what was wrong... > > > > > > Usually I don't run "make mrproper", as it removes files I may want to > > > keep (e.g. tags). Hence I ran "git ls-files -o | grep m68k | xargs rm" > > > (I usually build in separate output directories), confirmed with "git > > > ls-files -o" there were no remaining build artefacts, and was surprised > > > to discover I still got the error message above?!? > > > > > > Apparently arch/m68k/include/generated was still present, but as "git > > > ls-files -o" only shows files, not directories, it was not listed. > > > Perhaps the directory checks above can be changed to directory exists > > > _and_ is not empty? > > > > No. > > > > Since you did not run mrproper, > > Kbuild _correctly_ showed error. > > This is the expected and correct behavior. :) > > > > The upstream kernel is not a place > > to be customized for your workflow. Sorry. > > > > > > Every developer has a set of handy custom commands. > > > > Since you are already running a long command, > > why don't you add one more line, and put in ~/.bash_aliases or somewhere? > > > > my_mrproper() > > { > > git ls-files -o | grep m68k | xargs rm > > rm -rf arch/m68k/include/generated > > } > > Please note this was not part of my standard workflow, so I don't have a > script for it. I just happened to had done a quick test build in my > kernel source tree repository before, and had forgotten about that. > > So IMHO this is more of a usability issue: it is difficult to find out > what is wrong, and how to solve it, as "git ls-files -o" doesn't give a > clue. And running "make mrproper" doesn't help. > > Perhaps the message should be changed to > > $(srctree) is not clean, please run 'make ARCH=$(SRCARCH) mrproper'" Fair enough, but 'make ARCH=$(ARCH) mrproper' please. Will you send a patch? -- Best Regards Masahiro Yamada
Hi Yamada-san, On Tue, Jul 9, 2019 at 2:59 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > On Tue, Jul 9, 2019 at 9:45 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > On Tue, Jul 9, 2019 at 2:22 PM Masahiro Yamada > > <yamada.masahiro@socionext.com> wrote: > > > On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote: > > > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada > > > > <yamada.masahiro@socionext.com> wrote: > > > > > After cross-compiling the kernel, "make mrproper" should be executed > > > > > with the proper ARCH= option. Otherwise, stale objects will remain > > > > > under arch/$(SRCARCH)/. > > > > > > > > > > One bad scenario is like this: > > > > > > > > > > $ make ARCH=arm defconfig all # cross-compile the kernel for arm > > > > > $ make mrproper # mrproper for host-arch (i.e. x86) > > > > > $ make ARCH=arm O=build_dir defconfig all > > > > > > > > > > If you miss ARCH= for mrproper and cross-compile the kernel with O= > > > > > and ARCH= options, Kbuild will happily start to build, but may fail > > > > > due to stale objects in the srctree. > > > > > > > > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop > > > > > the out-of-tree build. To detect this, mrproper should clean only > > > > > arch/$(SRCARCH)/include/generated/. > > > > > > > > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> > > > > > --- > > > > > > > > > > Makefile | 6 ++++-- > > > > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/Makefile b/Makefile > > > > > index 10643c3..17945ce 100644 > > > > > --- a/Makefile > > > > > +++ b/Makefile > > > > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 > > > > > prepare3: include/config/kernel.release > > > > > ifneq ($(srctree),.) > > > > > @$(kecho) ' Using $(srctree) as source for kernel' > > > > > - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ > > > > > + $(Q)if [ -f $(srctree)/.config -o \ > > > > > + -d $(srctree)/include/config -o \ > > > > > + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ > > > > > echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ > > > > > echo >&2 " in the '$(srctree)' directory.";\ > > > > > > > > This took me a bit to find out what was wrong... > > > > > > > > Usually I don't run "make mrproper", as it removes files I may want to > > > > keep (e.g. tags). Hence I ran "git ls-files -o | grep m68k | xargs rm" > > > > (I usually build in separate output directories), confirmed with "git > > > > ls-files -o" there were no remaining build artefacts, and was surprised > > > > to discover I still got the error message above?!? > > > > > > > > Apparently arch/m68k/include/generated was still present, but as "git > > > > ls-files -o" only shows files, not directories, it was not listed. > > > > Perhaps the directory checks above can be changed to directory exists > > > > _and_ is not empty? > > > > > > No. > > > > > > Since you did not run mrproper, > > > Kbuild _correctly_ showed error. > > > This is the expected and correct behavior. :) > > > > > > The upstream kernel is not a place > > > to be customized for your workflow. Sorry. > > > > > > > > > Every developer has a set of handy custom commands. > > > > > > Since you are already running a long command, > > > why don't you add one more line, and put in ~/.bash_aliases or somewhere? > > > > > > my_mrproper() > > > { > > > git ls-files -o | grep m68k | xargs rm > > > rm -rf arch/m68k/include/generated > > > } > > > > Please note this was not part of my standard workflow, so I don't have a > > script for it. I just happened to had done a quick test build in my > > kernel source tree repository before, and had forgotten about that. > > > > So IMHO this is more of a usability issue: it is difficult to find out > > what is wrong, and how to solve it, as "git ls-files -o" doesn't give a > > clue. And running "make mrproper" doesn't help. > > > > Perhaps the message should be changed to > > > > $(srctree) is not clean, please run 'make ARCH=$(SRCARCH) mrproper'" > > Fair enough, but 'make ARCH=$(ARCH) mrproper' please. OK. > Will you send a patch? Sure, thanks! Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
diff --git a/Makefile b/Makefile index 10643c3..17945ce 100644 --- a/Makefile +++ b/Makefile @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3 prepare3: include/config/kernel.release ifneq ($(srctree),.) @$(kecho) ' Using $(srctree) as source for kernel' - $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \ + $(Q)if [ -f $(srctree)/.config -o \ + -d $(srctree)/include/config -o \ + -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \ echo >&2 " $(srctree) is not clean, please run 'make mrproper'"; \ echo >&2 " in the '$(srctree)' directory.";\ /bin/false; \ @@ -1350,7 +1352,7 @@ CLEAN_DIRS += $(MODVERDIR) include/ksym # Directories & files removed with 'make mrproper' MRPROPER_DIRS += include/config usr/include include/generated \ - arch/*/include/generated .tmp_objdiff + arch/$(SRCARCH)/include/generated .tmp_objdiff MRPROPER_FILES += .config .config.old .version \ Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \ signing_key.pem signing_key.priv signing_key.x509 \
After cross-compiling the kernel, "make mrproper" should be executed with the proper ARCH= option. Otherwise, stale objects will remain under arch/$(SRCARCH)/. One bad scenario is like this: $ make ARCH=arm defconfig all # cross-compile the kernel for arm $ make mrproper # mrproper for host-arch (i.e. x86) $ make ARCH=arm O=build_dir defconfig all If you miss ARCH= for mrproper and cross-compile the kernel with O= and ARCH= options, Kbuild will happily start to build, but may fail due to stale objects in the srctree. If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop the out-of-tree build. To detect this, mrproper should clean only arch/$(SRCARCH)/include/generated/. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> --- Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.7.4