| Message ID | cover.1704565248.git.dxu@dxuuu.xyz |
|---|---|
| Headers | show |
| Series | Annotate kfuncs in .BTF_ids section | expand |
On Sat, Jan 6, 2024 at 7:25 PM Daniel Xu <dxu@dxuuu.xyz> wrote: > > === Description === > > This is a bpf-treewide change that annotates all kfuncs as such inside > .BTF_ids. This annotation eventually allows us to automatically generate > kfunc prototypes from bpftool. > > We store this metadata inside a yet-unused flags field inside struct > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. This is great, thanks for tackling this. With yout patches we can figure out the full set of kfuncs. Is there a way to extend it so that we can tell which program context a kfunc can be called from?
Hi Lorenz, On Mon, Jan 08, 2024 at 10:15:45AM +0100, Lorenz Bauer wrote: > On Sat, Jan 6, 2024 at 7:25 PM Daniel Xu <dxu@dxuuu.xyz> wrote: > > > > === Description === > > > > This is a bpf-treewide change that annotates all kfuncs as such inside > > .BTF_ids. This annotation eventually allows us to automatically generate > > kfunc prototypes from bpftool. > > > > We store this metadata inside a yet-unused flags field inside struct > > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > This is great, thanks for tackling this. With yout patches we can > figure out the full set of kfuncs. Is there a way to extend it so that > we can tell which program context a kfunc can be called from? I think a potential approach would be to extend BTF_ID_FLAGS() with some more flags if we want to continue with .BTF_ids parsing technique. But it has some issues with program-type-less helpers that are associated with attachpoints as well as struct_ops helpers. Since it looks like we're in rather early stages of program-type-less world, maybe it'd be good to defer solving this problem until more use cases are established and we can find a good cut point to design around. Even with uapi helpers there was no way before. Thanks, Daniel
On Sat, Jan 06, 2024 at 11:24:07AM -0700, Daniel Xu wrote: > === Description === > > This is a bpf-treewide change that annotates all kfuncs as such inside > .BTF_ids. This annotation eventually allows us to automatically generate > kfunc prototypes from bpftool. > > We store this metadata inside a yet-unused flags field inside struct > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > More details about the full chain of events are available in commit 3's > description. > > The accompanying pahole changes (still needs some cleanup) can be viewed > here on this "frozen" branch [0]. so the plan is to have bpftool support to generate header file with detected kfuncs? jirka > > [0]: https://github.com/danobi/pahole/tree/kfunc_btf-mailed > > === Changelog === > > Changes from v2: > * Only WARN() for vmlinux kfuncs > > Changes from v1: > * Move WARN_ON() up a call level > * Also return error when kfunc set is not properly tagged > * Use BTF_KFUNCS_START/END instead of flags > * Rename BTF_SET8_KFUNC to BTF_SET8_KFUNCS > > Daniel Xu (3): > bpf: btf: Support flags for BTF_SET8 sets > bpf: btf: Add BTF_KFUNCS_START/END macro pair > bpf: treewide: Annotate BPF kfuncs in BTF > > drivers/hid/bpf/hid_bpf_dispatch.c | 8 +++---- > fs/verity/measure.c | 4 ++-- > include/linux/btf_ids.h | 21 +++++++++++++++---- > kernel/bpf/btf.c | 8 +++++++ > kernel/bpf/cpumask.c | 4 ++-- > kernel/bpf/helpers.c | 8 +++---- > kernel/bpf/map_iter.c | 4 ++-- > kernel/cgroup/rstat.c | 4 ++-- > kernel/trace/bpf_trace.c | 8 +++---- > net/bpf/test_run.c | 8 +++---- > net/core/filter.c | 16 +++++++------- > net/core/xdp.c | 4 ++-- > net/ipv4/bpf_tcp_ca.c | 4 ++-- > net/ipv4/fou_bpf.c | 4 ++-- > net/ipv4/tcp_bbr.c | 4 ++-- > net/ipv4/tcp_cubic.c | 4 ++-- > net/ipv4/tcp_dctcp.c | 4 ++-- > net/netfilter/nf_conntrack_bpf.c | 4 ++-- > net/netfilter/nf_nat_bpf.c | 4 ++-- > net/xfrm/xfrm_interface_bpf.c | 4 ++-- > net/xfrm/xfrm_state_bpf.c | 4 ++-- > .../selftests/bpf/bpf_testmod/bpf_testmod.c | 8 +++---- > 22 files changed, 81 insertions(+), 60 deletions(-) > > -- > 2.42.1 >
On Fri, Jan 12, 2024 at 05:20:39PM +0100, Jiri Olsa wrote: > On Sat, Jan 06, 2024 at 11:24:07AM -0700, Daniel Xu wrote: > > === Description === > > > > This is a bpf-treewide change that annotates all kfuncs as such inside > > .BTF_ids. This annotation eventually allows us to automatically generate > > kfunc prototypes from bpftool. > > > > We store this metadata inside a yet-unused flags field inside struct > > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > > > More details about the full chain of events are available in commit 3's > > description. > > > > The accompanying pahole changes (still needs some cleanup) can be viewed > > here on this "frozen" branch [0]. > > so the plan is to have bpftool support to generate header file > with detected kfuncs? Yep, that's the major use case. But I see other use cases as well like precision probing of kfuncs. Rather than guess and check which progs can load (in the event of backwards incompatible kfunc changes), programs can look at kfunc type signature thru BTF.
On Fri, Jan 12, 2024 at 01:03:59PM -0700, Daniel Xu wrote: > On Fri, Jan 12, 2024 at 05:20:39PM +0100, Jiri Olsa wrote: > > On Sat, Jan 06, 2024 at 11:24:07AM -0700, Daniel Xu wrote: > > > === Description === > > > > > > This is a bpf-treewide change that annotates all kfuncs as such inside > > > .BTF_ids. This annotation eventually allows us to automatically generate > > > kfunc prototypes from bpftool. > > > > > > We store this metadata inside a yet-unused flags field inside struct > > > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > > > > > More details about the full chain of events are available in commit 3's > > > description. > > > > > > The accompanying pahole changes (still needs some cleanup) can be viewed > > > here on this "frozen" branch [0]. > > > > so the plan is to have bpftool support to generate header file > > with detected kfuncs? > > Yep, that's the major use case. But I see other use cases as well like ok, any chance you could already include it in the patchset? would be a great way to test this.. maybe we could change selftests to use that thanks, jirka > precision probing of kfuncs. Rather than guess and check which progs can > load (in the event of backwards incompatible kfunc changes), programs > can look at kfunc type signature thru BTF.
Hi Jiri, On Sat, Jan 13, 2024 at 02:57:40PM +0100, Jiri Olsa wrote: > On Fri, Jan 12, 2024 at 01:03:59PM -0700, Daniel Xu wrote: > > On Fri, Jan 12, 2024 at 05:20:39PM +0100, Jiri Olsa wrote: > > > On Sat, Jan 06, 2024 at 11:24:07AM -0700, Daniel Xu wrote: > > > > === Description === > > > > > > > > This is a bpf-treewide change that annotates all kfuncs as such inside > > > > .BTF_ids. This annotation eventually allows us to automatically generate > > > > kfunc prototypes from bpftool. > > > > > > > > We store this metadata inside a yet-unused flags field inside struct > > > > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > > > > > > > More details about the full chain of events are available in commit 3's > > > > description. > > > > > > > > The accompanying pahole changes (still needs some cleanup) can be viewed > > > > here on this "frozen" branch [0]. > > > > > > so the plan is to have bpftool support to generate header file > > > with detected kfuncs? > > > > Yep, that's the major use case. But I see other use cases as well like > > ok, any chance you could already include it in the patchset? > would be a great way to test this.. maybe we could change > selftests to use that I haven't start working on that code yet, but I can. Here is my plan FWIW: 1. Bump minimum required pahole version up. Or feature probe for kfunc decl tag support. Whatever is the standard practice here. 2. Teach bpftool to dump kfunc prototypes, guarded behind a flag. 3. Flip bpftool flag on in selftest build and remove all manual kfunc prototypes atomically in 1 commit. I thought it'd be nicer to do it incrementally given all the moving pieces. But if we want to land it all at once that is ok by me too. Thanks, Daniel
On Sat, Jan 13, 2024 at 09:17:44AM -0700, Daniel Xu wrote: > Hi Jiri, > > On Sat, Jan 13, 2024 at 02:57:40PM +0100, Jiri Olsa wrote: > > On Fri, Jan 12, 2024 at 01:03:59PM -0700, Daniel Xu wrote: > > > On Fri, Jan 12, 2024 at 05:20:39PM +0100, Jiri Olsa wrote: > > > > On Sat, Jan 06, 2024 at 11:24:07AM -0700, Daniel Xu wrote: > > > > > === Description === > > > > > > > > > > This is a bpf-treewide change that annotates all kfuncs as such inside > > > > > .BTF_ids. This annotation eventually allows us to automatically generate > > > > > kfunc prototypes from bpftool. > > > > > > > > > > We store this metadata inside a yet-unused flags field inside struct > > > > > btf_id_set8 (thanks Kumar!). pahole will be taught where to look. > > > > > > > > > > More details about the full chain of events are available in commit 3's > > > > > description. > > > > > > > > > > The accompanying pahole changes (still needs some cleanup) can be viewed > > > > > here on this "frozen" branch [0]. > > > > > > > > so the plan is to have bpftool support to generate header file > > > > with detected kfuncs? > > > > > > Yep, that's the major use case. But I see other use cases as well like > > > > ok, any chance you could already include it in the patchset? > > would be a great way to test this.. maybe we could change > > selftests to use that > > I haven't start working on that code yet, but I can. > > Here is my plan FWIW: > > 1. Bump minimum required pahole version up. Or feature probe for > kfunc decl tag support. Whatever is the standard practice here. > > 2. Teach bpftool to dump kfunc prototypes, guarded behind a flag. > > 3. Flip bpftool flag on in selftest build and remove all manual kfunc > prototypes atomically in 1 commit. > > I thought it'd be nicer to do it incrementally given all the moving > pieces. But if we want to land it all at once that is ok by me too. I think it'd be best to try the whole thing before we merge the change to pahole.. I guess the tests changes can come later, but would be great to try the kfunc dump and make sure it works as expected jirka