| Message ID | 20201119085022.3606135-1-davidgow@google.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC] bpf: preload: Fix build error when O= is set | expand |
On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote: > > If BPF_PRELOAD is enabled, and an out-of-tree build is requested with > make O=<path>, compilation seems to fail with: > > tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. > make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 > make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 > make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 > make[2]: *** Waiting for unfinished jobs.... > make[1]: *** [.../Makefile:1799: kernel] Error 2 > make[1]: *** Waiting for unfinished jobs.... > make: *** [Makefile:185: __sub-make] Error 2 > > By the looks of things, this is because the (relative path) O= passed on > the command line is being passed to the libbpf Makefile, which then > can't find the directory. Given OUTPUT= is being passed anyway, we can > work around this by explicitly setting an empty O=, which will be > ignored in favour of OUTPUT= in tools/scripts/Makefile.include. Strange, but I can't repro it. I use make O=<absolute path> all the time with no issues. I just tried specifically with a make O=.build, where .build is inside Linux repo, and it still worked fine. See also be40920fbf10 ("tools: Let O= makes handle a relative path with -C option") which was supposed to address such an issue. So I'm wondering what exactly is causing this problem. > > Signed-off-by: David Gow <davidgow@google.com> > --- > > Hi all, > > I'm not 100% sure this is the correct fix here -- it seems to work for > me, and makes some sense, but let me know if there's a better way. > > One other thing worth noting is that I've been hitting this with > make allyesconfig on ARCH=um, but there's a comment in the Kconfig > suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that > maybe it shouldn't be being built at all. I figured that it was worth > trying to fix this anyway. > > Cheers, > -- David > > > kernel/bpf/preload/Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile > index 23ee310b6eb4..39848d296097 100644 > --- a/kernel/bpf/preload/Makefile > +++ b/kernel/bpf/preload/Makefile > @@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a > LIBBPF_OUT = $(abspath $(obj)) > > $(LIBBPF_A): > - $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a > + $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a > > userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \ > -I $(srctree)/tools/lib/ -Wno-unused-result > -- > 2.29.2.454.gaff20da3a2-goog >
On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote: > > On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote: > > > > If BPF_PRELOAD is enabled, and an out-of-tree build is requested with > > make O=<path>, compilation seems to fail with: > > > > tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. > > make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 > > make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 > > make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 > > make[2]: *** Waiting for unfinished jobs.... > > make[1]: *** [.../Makefile:1799: kernel] Error 2 > > make[1]: *** Waiting for unfinished jobs.... > > make: *** [Makefile:185: __sub-make] Error 2 > > > > By the looks of things, this is because the (relative path) O= passed on > > the command line is being passed to the libbpf Makefile, which then > > can't find the directory. Given OUTPUT= is being passed anyway, we can > > work around this by explicitly setting an empty O=, which will be > > ignored in favour of OUTPUT= in tools/scripts/Makefile.include. > > Strange, but I can't repro it. I use make O=<absolute path> all the > time with no issues. I just tried specifically with a make O=.build, > where .build is inside Linux repo, and it still worked fine. See also > be40920fbf10 ("tools: Let O= makes handle a relative path with -C > option") which was supposed to address such an issue. So I'm wondering > what exactly is causing this problem. > [+ linux-um list] Hmm... From a quick check, I can't reproduce this on x86, so it's possibly a UML-specific issue. The problem here seems to be that $PWD is, for whatever reason, equal to the srcdir on x86, but not on UML. In general, $PWD behaves pretty weirdly -- I don't fully understand it -- but if I add a tactical "PWD := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 as well. I guess this is because PWD only gets updated when set by a shell or something, and UML does this somewhere? Thoughts? Cheers, -- David > > > > Signed-off-by: David Gow <davidgow@google.com> > > --- > > > > Hi all, > > > > I'm not 100% sure this is the correct fix here -- it seems to work for > > me, and makes some sense, but let me know if there's a better way. > > > > One other thing worth noting is that I've been hitting this with > > make allyesconfig on ARCH=um, but there's a comment in the Kconfig > > suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that > > maybe it shouldn't be being built at all. I figured that it was worth > > trying to fix this anyway. > > > > Cheers, > > -- David > > > > > > kernel/bpf/preload/Makefile | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile > > index 23ee310b6eb4..39848d296097 100644 > > --- a/kernel/bpf/preload/Makefile > > +++ b/kernel/bpf/preload/Makefile > > @@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a > > LIBBPF_OUT = $(abspath $(obj)) > > > > $(LIBBPF_A): > > - $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a > > + $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a > > > > userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \ > > -I $(srctree)/tools/lib/ -Wno-unused-result > > -- > > 2.29.2.454.gaff20da3a2-goog > >
2020-11-21 17:48 UTC+0800 ~ David Gow <davidgow@google.com> > On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko > <andrii.nakryiko@gmail.com> wrote: >> >> On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote: >>> >>> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with >>> make O=<path>, compilation seems to fail with: >>> >>> tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. >>> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 >>> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 >>> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 >>> make[2]: *** Waiting for unfinished jobs.... >>> make[1]: *** [.../Makefile:1799: kernel] Error 2 >>> make[1]: *** Waiting for unfinished jobs.... >>> make: *** [Makefile:185: __sub-make] Error 2 >>> >>> By the looks of things, this is because the (relative path) O= passed on >>> the command line is being passed to the libbpf Makefile, which then >>> can't find the directory. Given OUTPUT= is being passed anyway, we can >>> work around this by explicitly setting an empty O=, which will be >>> ignored in favour of OUTPUT= in tools/scripts/Makefile.include. >> >> Strange, but I can't repro it. I use make O=<absolute path> all the >> time with no issues. I just tried specifically with a make O=.build, >> where .build is inside Linux repo, and it still worked fine. See also >> be40920fbf10 ("tools: Let O= makes handle a relative path with -C >> option") which was supposed to address such an issue. So I'm wondering >> what exactly is causing this problem. >> > [+ linux-um list] > > Hmm... From a quick check, I can't reproduce this on x86, so it's > possibly a UML-specific issue. > > The problem here seems to be that $PWD is, for whatever reason, equal > to the srcdir on x86, but not on UML. In general, $PWD behaves pretty > weirdly -- I don't fully understand it -- but if I add a tactical "PWD > := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 > as well. I guess this is because PWD only gets updated when set by a > shell or something, and UML does this somewhere? > > Thoughts? > > Cheers, > -- David Hi David, Andrii, David, did you use a different command for building for UML and x86? I'm asking because I reproduce on x86, but only for some targets, in particular when I tried bindeb-pkg. With "make O=.build vmlinux", I have: - $(O) for "dummy" check in tools/scripts/Makefile.include set to /linux/.build - $(PWD) for same check set to /linux/tools - Since $(O) is an absolute path, the "dummy" check passes With "make O=.build bindeb-pkg", I have instead: - $(O) set to .build (relative path) - $(PWD) set to /linux/.build - "dummy" check changes to /linux/.build and searches for .build in it, which fails and aborts the build (tools/scripts/Makefile.include is included from libbpf's Makefile, called from kernel/bpf/preload/Makefile.) I'm not sure how exactly the bindeb-pkg target ends up passing these values. For what it's worth, I have been solving this (before finding this thread) with a fix close to yours, I pass "O=$(abspath .)" on the command line for building libbpf in kernel/bpf/preload/Makefile. It looked consistent to me with the "tools/:" target from the main Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is "."). I hope this helps, Quentin
On Wed, Dec 16, 2020 at 10:53 PM Quentin Monnet <quentin@isovalent.com> wrote: > > 2020-11-21 17:48 UTC+0800 ~ David Gow <davidgow@google.com> > > On Sat, Nov 21, 2020 at 3:38 PM Andrii Nakryiko > > <andrii.nakryiko@gmail.com> wrote: > >> > >> On Thu, Nov 19, 2020 at 12:51 AM David Gow <davidgow@google.com> wrote: > >>> > >>> If BPF_PRELOAD is enabled, and an out-of-tree build is requested with > >>> make O=<path>, compilation seems to fail with: > >>> > >>> tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. > >>> make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 > >>> make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 > >>> make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 > >>> make[2]: *** Waiting for unfinished jobs.... > >>> make[1]: *** [.../Makefile:1799: kernel] Error 2 > >>> make[1]: *** Waiting for unfinished jobs.... > >>> make: *** [Makefile:185: __sub-make] Error 2 > >>> > >>> By the looks of things, this is because the (relative path) O= passed on > >>> the command line is being passed to the libbpf Makefile, which then > >>> can't find the directory. Given OUTPUT= is being passed anyway, we can > >>> work around this by explicitly setting an empty O=, which will be > >>> ignored in favour of OUTPUT= in tools/scripts/Makefile.include. > >> > >> Strange, but I can't repro it. I use make O=<absolute path> all the > >> time with no issues. I just tried specifically with a make O=.build, > >> where .build is inside Linux repo, and it still worked fine. See also > >> be40920fbf10 ("tools: Let O= makes handle a relative path with -C > >> option") which was supposed to address such an issue. So I'm wondering > >> what exactly is causing this problem. > >> > > [+ linux-um list] > > > > Hmm... From a quick check, I can't reproduce this on x86, so it's > > possibly a UML-specific issue. > > > > The problem here seems to be that $PWD is, for whatever reason, equal > > to the srcdir on x86, but not on UML. In general, $PWD behaves pretty > > weirdly -- I don't fully understand it -- but if I add a tactical "PWD > > := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 > > as well. I guess this is because PWD only gets updated when set by a > > shell or something, and UML does this somewhere? > > > > Thoughts? > > > > Cheers, > > -- David > > Hi David, Andrii, > > David, did you use a different command for building for UML and x86? I'm > asking because I reproduce on x86, but only for some targets, in > particular when I tried bindeb-pkg. I just ran "make ARCH={x86,um} O=.bpftest", with defconfig + enabling BPF_PRELOAD and its dependencies. UML fails, x86 works. (Though I can reproduce the failure if I make bindeb-pkg on x86). (It also shows up when building UML with the allyesconfig-based KUnit alltests option by running "./tools/testing/kunit/kunit.py run --alltests", though this understandably takes a long time and is less obvious) > > With "make O=.build vmlinux", I have: > - $(O) for "dummy" check in tools/scripts/Makefile.include set to > /linux/.build > - $(PWD) for same check set to /linux/tools > - Since $(O) is an absolute path, the "dummy" check passes > > With "make O=.build bindeb-pkg", I have instead: > - $(O) set to .build (relative path) > - $(PWD) set to /linux/.build > - "dummy" check changes to /linux/.build and searches for .build in it, > which fails and aborts the build > > (tools/scripts/Makefile.include is included from libbpf's Makefile, > called from kernel/bpf/preload/Makefile.) > > I'm not sure how exactly the bindeb-pkg target ends up passing these values. Yeah: I haven't been able to find where uml is changing them either: I'm assuming there's something which changes directory and/or spawns a shell/recursive make to change $(PWD) or something. > For what it's worth, I have been solving this (before finding this > thread) with a fix close to yours, I pass "O=$(abspath .)" on the > command line for building libbpf in kernel/bpf/preload/Makefile. It > looked consistent to me with the "tools/:" target from the main > Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is "."). Given that there are several targets being broken here, it's probably worth having a fix like this which overrides O= rather than trying to hunt down every target which could change $(PWD). I don't particularly mind whether we use O= or O=$(abspath .), both are working in the UML usecase as well. Does anyone object to basically accepting either this patch as-is, or using O=$(abspath .)? Cheers, -- David
diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile index 23ee310b6eb4..39848d296097 100644 --- a/kernel/bpf/preload/Makefile +++ b/kernel/bpf/preload/Makefile @@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a LIBBPF_OUT = $(abspath $(obj)) $(LIBBPF_A): - $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a + $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \ -I $(srctree)/tools/lib/ -Wno-unused-result
If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with: tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [.../Makefile:1799: kernel] Error 2 make[1]: *** Waiting for unfinished jobs.... make: *** [Makefile:185: __sub-make] Error 2 By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include. Signed-off-by: David Gow <davidgow@google.com> --- Hi all, I'm not 100% sure this is the correct fix here -- it seems to work for me, and makes some sense, but let me know if there's a better way. One other thing worth noting is that I've been hitting this with make allyesconfig on ARCH=um, but there's a comment in the Kconfig suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that maybe it shouldn't be being built at all. I figured that it was worth trying to fix this anyway. Cheers, -- David kernel/bpf/preload/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)