[bpf-next,0/2] Annotate kfuncs in .BTF_ids section

This is a bpf-treewide change that annotates all kfuncs as such inside
.BTF_ids. This annotation eventually allows us to automatically generate
kfunc prototypes from bpftool.

We store this metadata inside a yet-unused flags field inside struct
btf_id_set8 (thanks Kumar!). pahole will be taught where to look.

More details about the full chain of events are available in commit 2's
description.

Daniel Xu (2):
  bpf: btf: Support optional flags for BTF_SET8 sets
  bpf: treewide: Annotate BPF kfuncs in BTF

 drivers/hid/bpf/hid_bpf_dispatch.c              |  4 ++--
 fs/verity/measure.c                             |  2 +-
 include/linux/btf_ids.h                         | 17 ++++++++++++-----
 kernel/bpf/btf.c                                |  3 +++
 kernel/bpf/cpumask.c                            |  2 +-
 kernel/bpf/helpers.c                            |  4 ++--
 kernel/bpf/map_iter.c                           |  2 +-
 kernel/cgroup/rstat.c                           |  2 +-
 kernel/trace/bpf_trace.c                        |  4 ++--
 net/bpf/test_run.c                              |  4 ++--
 net/core/filter.c                               |  8 ++++----
 net/core/xdp.c                                  |  2 +-
 net/ipv4/bpf_tcp_ca.c                           |  2 +-
 net/ipv4/fou_bpf.c                              |  2 +-
 net/ipv4/tcp_bbr.c                              |  2 +-
 net/ipv4/tcp_cubic.c                            |  2 +-
 net/ipv4/tcp_dctcp.c                            |  2 +-
 net/netfilter/nf_conntrack_bpf.c                |  2 +-
 net/netfilter/nf_nat_bpf.c                      |  2 +-
 net/xfrm/xfrm_interface_bpf.c                   |  2 +-
 net/xfrm/xfrm_state_bpf.c                       |  2 +-
 .../selftests/bpf/bpf_testmod/bpf_testmod.c     |  2 +-
 22 files changed, 42 insertions(+), 32 deletions(-)

On Wed, Jan 03, 2024 at 04:31:56PM -0700, Daniel Xu wrote:

SNIP

> diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
> index 88f914579fa1..771e29762a2d 100644
> --- a/include/linux/btf_ids.h
> +++ b/include/linux/btf_ids.h
> @@ -8,6 +8,9 @@ struct btf_id_set {
>  	u32 ids[];
>  };
>  
> +/* This flag implies BTF_SET8 holds kfunc(s) */
> +#define BTF_SET8_KFUNC		(1 << 0)
> +
>  struct btf_id_set8 {
>  	u32 cnt;
>  	u32 flags;
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index 51e8b4bee0c8..b8ba00a4179f 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -7769,6 +7769,9 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook,
>  	struct btf *btf;
>  	int ret, i;
>  
> +	/* All kfuncs need to be tagged as such in BTF */
> +	WARN_ON(!(kset->set->flags & BTF_SET8_KFUNC));

__register_btf_kfunc_id_set gets called also from the 'hooks' path:

  bpf_mptcp_kfunc_init
    register_btf_fmodret_id_set
      __register_btf_kfunc_id_set

so it will hit the warn.. it should be probably in the register_btf_kfunc_id_set ?

also given that we can have modules calling register_btf_kfunc_id_set,
should we just return error instead of the warn?

SNIP

> diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> index 91907b321f91..32972334cd50 100644
> --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> @@ -341,7 +341,7 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = {
>  	.write = bpf_testmod_test_write,
>  };
>  
> -BTF_SET8_START(bpf_testmod_common_kfunc_ids)
> +BTF_SET8_START(bpf_testmod_common_kfunc_ids, BTF_SET8_KFUNC)
>  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_new, KF_ITER_NEW)
>  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL)
>  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY)

we need to change also bpf_testmod_check_kfunc_ids set

jirka

> -- 
> 2.42.1
>

Hi Jiri, 

On Thu, Jan 04, 2024 at 12:41:51PM +0100, Jiri Olsa wrote:
> On Wed, Jan 03, 2024 at 04:31:56PM -0700, Daniel Xu wrote:
> 
> SNIP
> 
> > diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
> > index 88f914579fa1..771e29762a2d 100644
> > --- a/include/linux/btf_ids.h
> > +++ b/include/linux/btf_ids.h
> > @@ -8,6 +8,9 @@ struct btf_id_set {
> >  	u32 ids[];
> >  };
> >  
> > +/* This flag implies BTF_SET8 holds kfunc(s) */
> > +#define BTF_SET8_KFUNC		(1 << 0)
> > +
> >  struct btf_id_set8 {
> >  	u32 cnt;
> >  	u32 flags;
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index 51e8b4bee0c8..b8ba00a4179f 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -7769,6 +7769,9 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook,
> >  	struct btf *btf;
> >  	int ret, i;
> >  
> > +	/* All kfuncs need to be tagged as such in BTF */
> > +	WARN_ON(!(kset->set->flags & BTF_SET8_KFUNC));
> 
> __register_btf_kfunc_id_set gets called also from the 'hooks' path:
> 
>   bpf_mptcp_kfunc_init
>     register_btf_fmodret_id_set
>       __register_btf_kfunc_id_set
> 
> so it will hit the warn.. it should be probably in the register_btf_kfunc_id_set ?

Yeah, good catch.

> 
> also given that we can have modules calling register_btf_kfunc_id_set,
> should we just return error instead of the warn?

It looks like quite a few registrations go through late_initcall(),
in which error codes are thrown away. I'm looking at
init/main.c:do_initcall_level:

        for (fn = initcall_levels[level]; fn < initcall_levels[level+1]; fn++)
                do_one_initcall(initcall_from_entry(fn));

Higher level question: if out of tree module does not follow convention,
it would still make sense to WARN(), right?

> 
> SNIP
> 
> > diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> > index 91907b321f91..32972334cd50 100644
> > --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> > +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
> > @@ -341,7 +341,7 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = {
> >  	.write = bpf_testmod_test_write,
> >  };
> >  
> > -BTF_SET8_START(bpf_testmod_common_kfunc_ids)
> > +BTF_SET8_START(bpf_testmod_common_kfunc_ids, BTF_SET8_KFUNC)
> >  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_new, KF_ITER_NEW)
> >  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL)
> >  BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY)
> 
> we need to change also bpf_testmod_check_kfunc_ids set

Good catch, thanks.

Daniel

On Thu, Jan 04, 2024 at 06:17:50PM -0700, Daniel Xu wrote:
[...]
> > 
> > also given that we can have modules calling register_btf_kfunc_id_set,
> > should we just return error instead of the warn?
> 
> It looks like quite a few registrations go through late_initcall(),
> in which error codes are thrown away. I'm looking at
> init/main.c:do_initcall_level:
> 
>         for (fn = initcall_levels[level]; fn < initcall_levels[level+1]; fn++)
>                 do_one_initcall(initcall_from_entry(fn));
> 
> Higher level question: if out of tree module does not follow convention,
> it would still make sense to WARN(), right?

Ah, I got what you meant now. I'd say returning error makes sense but
WARN() is also useful. I'll send v2 with both.

[...]