From patchwork Tue May 7 13:19:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795942 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06689156F20; Tue, 7 May 2024 13:19:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087988; cv=none; b=rSFjQnoV5nty1p9u9JvJDtw+kj3EQpWLY+dpwQTikhlXAPZIqcswiStuVokJl1vj7sjc1h/jwvMwkeqVMzeXpTlD65mLzx6tkfAYwFEujN60dTWyP/K+99979FLqKJFyqxK4kwRn8nupCqKDRQT7E01RNY/ys9wkveQZKwoafrg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087988; c=relaxed/simple; bh=isqHF8HeC8KRWX2yMwQXkLMgFKrRtCaj9HjYSmd5hgA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QwrAWQEgU6774txEwzOFpPQjbDOiu13hT55MXhNE0sJa7tPFCETyTJWxVaGkXhA7YGz4Vv2HSucMPAJOvpzHfOdialT/2RNiY0uDSrVh2BtN33fRsrtYouwgDWRIbQVSYpnvOiGE/BW9k1Bkl7bJn7xkMo5EJGJM9FgAX0yse3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OVfvXmDQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OVfvXmDQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9CB22C2BBFC; Tue, 7 May 2024 13:19:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715087987; bh=isqHF8HeC8KRWX2yMwQXkLMgFKrRtCaj9HjYSmd5hgA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OVfvXmDQMt7A0kehOhD20thI4YILIa2gsWLgP+ZQg2rnbPe0BuMSrdX/OfP6ZAJ3V CcipwCRBdjvZWaSl2O3WEfoMg/qph0ckxaQ4Ur97A37bkAEbfL3/W+AmVKuyTEu93Y Qs5Jb1Ma8AeyeI+I2VfTAzm1UYp4+sBF4QQ42127o7VDUbbh3vC/s3nZBiL4NOToQG C3fpyeYuzRbPQQ1xXYBJddInSf6grtVjkWvrratJKQQatYXOehprAw73COpQvNoCNy j6xrpEBJPMh1FmPrqNNv7xDx1qh1mXEkMqS6hcuxQSJZxG1KgDvBsHFXY0Jyct31CR A+Er9GRlfbrlQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:29 +0200 Subject: [PATCH RFC bpf-next 1/8] bpf: ignore sleepable prog parameter for kfuncs Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-1-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=1501; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=isqHF8HeC8KRWX2yMwQXkLMgFKrRtCaj9HjYSmd5hgA=; b=k9rUZ8hFqZ1a4Otx1zvx4YybpnqH1zKr1uzHypliJVTIkPrH8HhUQncGvx39vaTB8CsEPXwbL L9Lx6klBAfGCaUsqQIF6lcEu73FyZlo049Px7cFVg1Fd+5G54E7eMxj X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= There is no change of behavior: for each kfunc, we store the prog sleepable state. But this allows to declare an async non sleepable callback from a syscall, where everything is sleepable. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/verifier.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5d42db05315e..856cb77d0f87 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5288,8 +5288,7 @@ static int map_kptr_match_type(struct bpf_verifier_env *env, static bool in_sleepable(struct bpf_verifier_env *env) { - return env->prog->sleepable || - (env->cur_state && env->cur_state->in_sleepable); + return env->cur_state ? env->cur_state->in_sleepable : env->prog->sleepable; } /* The non-sleepable programs and sleepable programs with explicit bpf_rcu_read_lock() @@ -20722,6 +20721,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) state->curframe = 0; state->speculative = false; state->branches = 1; + state->in_sleepable = env->prog->sleepable; state->frame[0] = kzalloc(sizeof(struct bpf_func_state), GFP_KERNEL); if (!state->frame[0]) { kfree(state); From patchwork Tue May 7 13:19:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795371 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00754156F20; Tue, 7 May 2024 13:19:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087992; cv=none; b=jeEO1aZ8rs0VMfBDYzsI/MMynYHdJWI72N4rBseDVd9hLmrb8sByn2QxQQK+eiF1sswcWv+mbjObTFNZt5Nhm9bApNgdRRaqPm9GWIH7AUU7ahe0s3HDD6+VR7wVzSDaqVnqU0lL036Hd8V6CtJbf8f9oD6RCQeoavQFl2rRL88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087992; c=relaxed/simple; bh=mdcaTx8cr4EuijO3f+G4VMa/INC+vC+0vs1NUy/G+t8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ayi+d8AIM4UwNX1EvxaYP94MDKMXfBxlEk9PdY+cfKc1fotkRl34xVhapj3a6e9qGAgp4YSIkYHfdlqnwPgo1KvL3z2B7ONehKFpXWab9gIkkiCYFtaICEyM+VxNTs4YhjB3mTjx8Ohf51jsaxtPXTTlWOcoy+xowRuffPGYmfY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QP86DZPw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QP86DZPw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 501FEC2BBFC; Tue, 7 May 2024 13:19:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715087991; bh=mdcaTx8cr4EuijO3f+G4VMa/INC+vC+0vs1NUy/G+t8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QP86DZPwbNGpLP/IHwk/hD+pHWOwxkfMY+j24l9Nlc1+J/aJsU20LCEzGuHHDsDo3 RhiGP4IFGt9oNBtFWq05qHp9mwA2775Obrtb8yy2h900P/B4MJ1hZqwVR3hRirMFXE /Ka3aIxdQg6fIv0DhPVxsr8Zl6MaP6yC8oexq0IKX3Ic5b7ljomH2QDWvvxtZ9JLzR NnGWrmVni74xvdDnuaftOlXKwXcjAoLcHhorJ3Yh/EZ9Y1O/WADQuBHIdkNcIswiFG AILgL8lFz6W5sBWf6x6iGPkqKNbXd902tHTTkYU+dlnMvC9YiF5zDh89e8yzMtMLRS xw8zuZ/9vGEdQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:30 +0200 Subject: [PATCH RFC bpf-next 2/8] bpf: add kfunc_meta parameter to push_callback_call() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-2-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=8241; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=mdcaTx8cr4EuijO3f+G4VMa/INC+vC+0vs1NUy/G+t8=; b=1aGaI5o+lluCosEjLMQWR4X1KGGi0U1ytDRio/8X9irdTXH0RUpFLDy1DY/NzAX1F8ZGSOUft vxj90jRUoWGBBf14Ds+M/u/44zvYS1eIg0UQlefxV9yI7fAZsdCNBGj X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= No code change but is a preparatory patch for being able to declare async callbacks from bpf kfuncs. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/verifier.c | 48 +++++++++++++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 856cb77d0f87..2b1e24c440c5 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -9339,11 +9339,13 @@ static void clear_caller_saved_regs(struct bpf_verifier_env *env, typedef int (*set_callee_state_fn)(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx); + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta); static int set_callee_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, - struct bpf_func_state *callee, int insn_idx); + struct bpf_func_state *callee, int insn_idx, + struct bpf_kfunc_call_arg_meta *meta); static int setup_func_entry(struct bpf_verifier_env *env, int subprog, int callsite, set_callee_state_fn set_callee_state_cb, @@ -9381,7 +9383,7 @@ static int setup_func_entry(struct bpf_verifier_env *env, int subprog, int calls subprog /* subprog number within this prog */); /* Transfer references to the callee */ err = copy_reference_state(callee, caller); - err = err ?: set_callee_state_cb(env, caller, callee, callsite); + err = err ?: set_callee_state_cb(env, caller, callee, callsite, NULL); if (err) goto err_out; @@ -9518,7 +9520,8 @@ static int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int insn_idx, int subprog, - set_callee_state_fn set_callee_state_cb) + set_callee_state_fn set_callee_state_cb, + struct bpf_kfunc_call_arg_meta *kfunc_meta) { struct bpf_verifier_state *state = env->cur_state, *callback_state; struct bpf_func_state *caller, *callee; @@ -9560,7 +9563,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins callee->async_entry_cnt = caller->async_entry_cnt + 1; /* Convert bpf_timer_set_callback() args into timer callback args */ - err = set_callee_state_cb(env, caller, callee, insn_idx); + err = set_callee_state_cb(env, caller, callee, insn_idx, kfunc_meta); if (err) return err; @@ -9691,7 +9694,8 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env, static int set_callee_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, - struct bpf_func_state *callee, int insn_idx) + struct bpf_func_state *callee, int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { int i; @@ -9706,7 +9710,8 @@ static int set_callee_state(struct bpf_verifier_env *env, static int set_map_elem_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { struct bpf_insn_aux_data *insn_aux = &env->insn_aux_data[insn_idx]; struct bpf_map *map; @@ -9732,7 +9737,8 @@ static int set_map_elem_callback_state(struct bpf_verifier_env *env, static int set_loop_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { /* bpf_loop(u32 nr_loops, void *callback_fn, void *callback_ctx, * u64 flags); @@ -9754,7 +9760,8 @@ static int set_loop_callback_state(struct bpf_verifier_env *env, static int set_timer_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { struct bpf_map *map_ptr = caller->regs[BPF_REG_1].map_ptr; @@ -9784,7 +9791,8 @@ static int set_timer_callback_state(struct bpf_verifier_env *env, static int set_find_vma_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { /* bpf_find_vma(struct task_struct *task, u64 addr, * void *callback_fn, void *callback_ctx, u64 flags) @@ -9812,7 +9820,8 @@ static int set_find_vma_callback_state(struct bpf_verifier_env *env, static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { /* bpf_user_ringbuf_drain(struct bpf_map *map, void *callback_fn, void * callback_ctx, u64 flags); @@ -9835,7 +9844,8 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env, static int set_rbtree_add_callback_state(struct bpf_verifier_env *env, struct bpf_func_state *caller, struct bpf_func_state *callee, - int insn_idx) + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) { /* void bpf_rbtree_add_impl(struct bpf_rb_root *root, struct bpf_rb_node *node, * bool (less)(struct bpf_rb_node *a, const struct bpf_rb_node *b)); @@ -10411,15 +10421,15 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn break; case BPF_FUNC_for_each_map_elem: err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_map_elem_callback_state); + set_map_elem_callback_state, NULL); break; case BPF_FUNC_timer_set_callback: err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_timer_callback_state); + set_timer_callback_state, NULL); break; case BPF_FUNC_find_vma: err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_find_vma_callback_state); + set_find_vma_callback_state, NULL); break; case BPF_FUNC_snprintf: err = check_bpf_snprintf_call(env, regs); @@ -10434,7 +10444,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn return err; if (cur_func(env)->callback_depth < regs[BPF_REG_1].umax_value) { err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_loop_callback_state); + set_loop_callback_state, NULL); } else { cur_func(env)->callback_depth = 0; if (env->log.level & BPF_LOG_LEVEL2) @@ -10537,7 +10547,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn } case BPF_FUNC_user_ringbuf_drain: err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_user_ringbuf_callback_state); + set_user_ringbuf_callback_state, NULL); break; } @@ -12285,7 +12295,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, if (meta.func_id == special_kfunc_list[KF_bpf_rbtree_add_impl]) { err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_rbtree_add_callback_state); + set_rbtree_add_callback_state, &meta); if (err) { verbose(env, "kfunc %s#%d failed callback verification\n", func_name, meta.func_id); @@ -12295,7 +12305,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, if (is_bpf_wq_set_callback_impl_kfunc(meta.func_id)) { err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_timer_callback_state); + set_timer_callback_state, &meta); if (err) { verbose(env, "kfunc %s#%d failed callback verification\n", func_name, meta.func_id); From patchwork Tue May 7 13:19:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795941 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B121C1C65E2; Tue, 7 May 2024 13:19:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087995; cv=none; b=tgaLM8ToHNG/97FM4i3ZzwjA8JNPQD5bgIFn8A98M8ZNekf9Nfj5kWphVQEsTPpruGvCxVCVq2cZUAYHU+8MfUB/Xhkf8hcHDqaknWRlBOd4Oc+ObmzECL7BLPlRCtHEKJp2CKwNYLEJV84J8Iwxm1tftoaxkx94cBojQtpl15U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087995; c=relaxed/simple; bh=SLNZxKQweYv3yP5iFQGwSZpSSxqoUn6F/nU0+QfT/tc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s+PsECflp2Q78p+A07RJ4s9inY0562/Je28Q1ZKVcLjplYkc3byJdmp0qV7nqr4s9Ml89K6xeqXbCDR6J7pSAcRs1brkhm2peAQiqfCori5VbI5s44D2QIswxq8dLOYKAx7GwGHfNxaH/0jsyEeVzG3OLaeaTIZbRTCatVtTJ5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nMZU4K3x; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nMZU4K3x" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06438C4AF63; Tue, 7 May 2024 13:19:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715087995; bh=SLNZxKQweYv3yP5iFQGwSZpSSxqoUn6F/nU0+QfT/tc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nMZU4K3xeBwKduF0f4WUNc0+hzkmp+jLa30EJuMdzFfdwWKPdI6oBbB4SIf3juFjg LGVJsRrmaYQP8SZmkfukcO+w+z8euYmPB02gmNQSqUXYLei42TvLV0ZC9wgvxrcY30 RnDZoGL5tGi3EQDh4Zi6G5JJ7FjYHa22nsoqFWOmOvj7FrsFWWoq2i4hWU9B2FxhtV mnSkyq7pM6kS0f7Rpkbim3cedUpJcYw7HhCuKORmcKs5Zk7Nl0+EhFwWcsNthlTOko JMTPFfJuI9uMGmllbe0gIpFPVURqhBzYMNEL5FibQ0f2MoDDE5dydV+AqBgFJooYvV e/FE/tHfzCaCQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:31 +0200 Subject: [PATCH RFC bpf-next 3/8] bpf: implement __async and __s_async kfunc suffixes Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-3-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=11562; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=SLNZxKQweYv3yP5iFQGwSZpSSxqoUn6F/nU0+QfT/tc=; b=Wtmr6N4TDq2xf7r2wVjNOUPp5VYD2+9PVDqYdsfsCvJajcYup4BDU8c4qjtIt2RRn1h8TfJhv SgMRKdvX17ZAFFT6OWmX1IYV6BzBxl0pJG3ESr5TAWryj2h0hNr20IL X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= still mostly a WIP, but it seems to be working for the couple of tests. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/verifier.c | 223 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 206 insertions(+), 17 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2b1e24c440c5..cc4dab81b306 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -336,6 +336,16 @@ struct bpf_kfunc_call_arg_meta { struct bpf_map *ptr; int uid; } map; + struct { + bool enabled; + bool sleepable; + u32 nr_args; + struct { + // FIXME: should be enum kfunc_ptr_arg_type type; + int type; + u32 btf_id; + } args[5]; + } async_cb; u64 mem_size; }; @@ -9538,7 +9548,8 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins */ env->subprog_info[subprog].is_cb = true; if (bpf_pseudo_kfunc_call(insn) && - !is_callback_calling_kfunc(insn->imm)) { + !is_callback_calling_kfunc(insn->imm) && + !(kfunc_meta && kfunc_meta->async_cb.enabled)) { verbose(env, "verifier bug: kfunc %s#%d not marked as callback-calling\n", func_id_name(insn->imm), insn->imm); return -EFAULT; @@ -9549,14 +9560,15 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins return -EFAULT; } - if (is_async_callback_calling_insn(insn)) { + if (is_async_callback_calling_insn(insn) || (kfunc_meta && kfunc_meta->async_cb.enabled)) { struct bpf_verifier_state *async_cb; /* there is no real recursion here. timer and workqueue callbacks are async */ env->subprog_info[subprog].is_async_cb = true; async_cb = push_async_cb(env, env->subprog_info[subprog].start, insn_idx, subprog, - is_bpf_wq_set_callback_impl_kfunc(insn->imm)); + (is_bpf_wq_set_callback_impl_kfunc(insn->imm) || + (kfunc_meta && kfunc_meta->async_cb.sleepable))); if (!async_cb) return -EFAULT; callee = async_cb->frame[0]; @@ -10890,6 +10902,16 @@ static bool is_kfunc_arg_const_str(const struct btf *btf, const struct btf_param return btf_param_match_suffix(btf, arg, "__str"); } +static bool is_kfunc_arg_async_cb(const struct btf *btf, const struct btf_param *arg) +{ + return btf_param_match_suffix(btf, arg, "__async"); +} + +static bool is_kfunc_arg_sleepable_async_cb(const struct btf *btf, const struct btf_param *arg) +{ + return btf_param_match_suffix(btf, arg, "__s_async"); +} + static bool is_kfunc_arg_scalar_with_name(const struct btf *btf, const struct btf_param *arg, const char *name) @@ -11045,6 +11067,48 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_WORKQUEUE, }; +static const char *__kfunc_ptr_arg_type_str(enum kfunc_ptr_arg_type value) +{ + switch (value) { + case KF_ARG_PTR_TO_CTX: + return "KF_ARG_PTR_TO_CTX"; + case KF_ARG_PTR_TO_ALLOC_BTF_ID: + return "KF_ARG_PTR_TO_ALLOC_BTF_ID"; + case KF_ARG_PTR_TO_REFCOUNTED_KPTR: + return "KF_ARG_PTR_TO_REFCOUNTED_KPTR"; + case KF_ARG_PTR_TO_DYNPTR: + return "KF_ARG_PTR_TO_DYNPTR"; + case KF_ARG_PTR_TO_ITER: + return "KF_ARG_PTR_TO_ITER"; + case KF_ARG_PTR_TO_LIST_HEAD: + return "KF_ARG_PTR_TO_LIST_HEAD"; + case KF_ARG_PTR_TO_LIST_NODE: + return "KF_ARG_PTR_TO_LIST_NODE"; + case KF_ARG_PTR_TO_BTF_ID: + return "KF_ARG_PTR_TO_BTF_ID"; + case KF_ARG_PTR_TO_MEM: + return "KF_ARG_PTR_TO_MEM"; + case KF_ARG_PTR_TO_MEM_SIZE: + return "KF_ARG_PTR_TO_MEM_SIZE"; + case KF_ARG_PTR_TO_CALLBACK: + return "KF_ARG_PTR_TO_CALLBACK"; + case KF_ARG_PTR_TO_RB_ROOT: + return "KF_ARG_PTR_TO_RB_ROOT"; + case KF_ARG_PTR_TO_RB_NODE: + return "KF_ARG_PTR_TO_RB_NODE"; + case KF_ARG_PTR_TO_NULL: + return "KF_ARG_PTR_TO_NULL"; + case KF_ARG_PTR_TO_CONST_STR: + return "KF_ARG_PTR_TO_CONST_STR"; + case KF_ARG_PTR_TO_MAP: + return "KF_ARG_PTR_TO_MAP"; + case KF_ARG_PTR_TO_WORKQUEUE: + return "KF_ARG_PTR_TO_WORKQUEUE"; + } + + return "UNKNOWN"; +} + enum special_kfunc_type { KF_bpf_obj_new_impl, KF_bpf_obj_drop_impl, @@ -12151,6 +12215,39 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return -EINVAL; } meta->subprogno = reg->subprogno; + meta->async_cb.sleepable = is_kfunc_arg_sleepable_async_cb(meta->btf, &args[i]); + meta->async_cb.enabled = meta->async_cb.sleepable || + is_kfunc_arg_async_cb(meta->btf, &args[i]); + if (meta->async_cb.enabled) { + const struct btf_type *cb_proto; + const struct btf_param *cb_args; + u32 cb_type = args[i].type; + int i; + + cb_proto = btf_type_resolve_func_ptr(btf, cb_type, NULL); + if (cb_proto) { + meta->async_cb.nr_args = btf_type_vlen(cb_proto); + cb_args = btf_params(cb_proto); + for (i = 0; i < meta->async_cb.nr_args; i++) { + const struct btf_type *t, *ref_t; + const char *ref_tname; + u32 ref_id, t_id; + + t = btf_type_skip_modifiers(btf, cb_args[i].type, &t_id); + ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); + ref_tname = btf_name_by_offset(btf, ref_t->name_off); + meta->async_cb.args[i].type = get_kfunc_ptr_arg_type(env, meta, + t, ref_t, ref_tname, cb_args, i, meta->async_cb.nr_args); + + /* FIXME: we should not get an error from get_kfunc_ptr_arg_type() */ + if (meta->async_cb.args[i].type < 0) + meta->async_cb.args[i].type = KF_ARG_PTR_TO_BTF_ID; + meta->async_cb.args[i].btf_id = ref_id; + } + } else { + meta->async_cb.nr_args = 0; + } + } break; case KF_ARG_PTR_TO_REFCOUNTED_KPTR: if (!type_is_ptr_alloc_obj(reg->type)) { @@ -12248,6 +12345,71 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env, static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name); +static int set_generic_callback_state(struct bpf_verifier_env *env, + struct bpf_func_state *caller, + struct bpf_func_state *callee, + int insn_idx, + struct bpf_kfunc_call_arg_meta *meta) +{ + int i; + + for (i = 0; i < 5; i++) { + if (i < meta->async_cb.nr_args) { + u32 type = meta->async_cb.args[i].type; + + switch (type) { + case KF_ARG_PTR_TO_CTX: + case KF_ARG_PTR_TO_ALLOC_BTF_ID: + case KF_ARG_PTR_TO_REFCOUNTED_KPTR: + case KF_ARG_PTR_TO_DYNPTR: + case KF_ARG_PTR_TO_ITER: + case KF_ARG_PTR_TO_LIST_HEAD: + case KF_ARG_PTR_TO_LIST_NODE: + case KF_ARG_PTR_TO_CALLBACK: + case KF_ARG_PTR_TO_RB_ROOT: + case KF_ARG_PTR_TO_RB_NODE: + case KF_ARG_PTR_TO_NULL: + case KF_ARG_PTR_TO_CONST_STR: + verbose(env, "argument #%d of type %s is not supported in async callbacks\n", + i, __kfunc_ptr_arg_type_str(meta->async_cb.args[i].type)); + return -EINVAL; + case KF_ARG_PTR_TO_MEM: + case KF_ARG_PTR_TO_MEM_SIZE: + callee->regs[BPF_REG_1 + i].type = PTR_TO_MEM; + __mark_reg_known_zero(&callee->regs[BPF_REG_1 + i]); + callee->regs[BPF_REG_1 + i].mem_size = 8; // FIXME: should store the size while walking the arguments + break; + case KF_ARG_PTR_TO_MAP: + callee->regs[BPF_REG_1 + i].type = CONST_PTR_TO_MAP; + __mark_reg_known_zero(&callee->regs[BPF_REG_1 + i]); + callee->regs[BPF_REG_1 + i].map_ptr = meta->map.ptr; + break; + case KF_ARG_PTR_TO_WORKQUEUE: + callee->regs[BPF_REG_1 + i].type = PTR_TO_MAP_VALUE; + __mark_reg_known_zero(&callee->regs[BPF_REG_1 + i]); + callee->regs[BPF_REG_1 + i].map_ptr = meta->map.ptr; + break; + case KF_ARG_PTR_TO_BTF_ID: + callee->regs[BPF_REG_1 + i].type = PTR_TO_BTF_ID; + __mark_reg_known_zero(&callee->regs[BPF_REG_1 + i]); + callee->regs[BPF_REG_1 + i].btf = meta->btf; + callee->regs[BPF_REG_1 + i].btf_id = meta->async_cb.args[i].btf_id; + break; + default: + verbose(env, "verifier bug: unexpected arg#%d type (%d) in async callback\n", + i, type); + return -EFAULT; + } + } else { + __mark_reg_not_init(env, &callee->regs[BPF_REG_1 + i]); + } + } + callee->in_callback_fn = true; + // callee->callback_ret_range = retval_range(-MAX_ERRNO, ); + return 0; +} + + static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -12313,6 +12475,16 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } } + if (meta.async_cb.enabled) { + err = push_callback_call(env, insn, insn_idx, meta.subprogno, + set_generic_callback_state, &meta); + if (err) { + verbose(env, "kfunc %s#%d failed callback verification\n", + func_name, meta.func_id); + return err; + } + } + rcu_lock = is_kfunc_bpf_rcu_read_lock(&meta); rcu_unlock = is_kfunc_bpf_rcu_read_unlock(&meta); @@ -15918,22 +16090,39 @@ static int visit_insn(int t, struct bpf_verifier_env *env) } if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) { struct bpf_kfunc_call_arg_meta meta; + const struct btf_param *args; + u32 i, nargs; ret = fetch_kfunc_meta(env, insn, &meta, NULL); - if (ret == 0 && is_iter_next_kfunc(&meta)) { - mark_prune_point(env, t); - /* Checking and saving state checkpoints at iter_next() call - * is crucial for fast convergence of open-coded iterator loop - * logic, so we need to force it. If we don't do that, - * is_state_visited() might skip saving a checkpoint, causing - * unnecessarily long sequence of not checkpointed - * instructions and jumps, leading to exhaustion of jump - * history buffer, and potentially other undesired outcomes. - * It is expected that with correct open-coded iterators - * convergence will happen quickly, so we don't run a risk of - * exhausting memory. - */ - mark_force_checkpoint(env, t); + if (ret == 0) { + args = (const struct btf_param *)(meta.func_proto + 1); + nargs = btf_type_vlen(meta.func_proto); + + for (i = 0; i < nargs; i++) { + if (is_kfunc_arg_sleepable_async_cb(meta.btf, &args[i]) || + is_kfunc_arg_async_cb(meta.btf, &args[i])) + /* Mark this call insn as a prune point to trigger + * is_state_visited() check before call itself is + * processed by __check_func_call(). Otherwise new + * async state will be pushed for further exploration. + */ + mark_prune_point(env, t); + } + if (is_iter_next_kfunc(&meta)) { + mark_prune_point(env, t); + /* Checking and saving state checkpoints at iter_next() call + * is crucial for fast convergence of open-coded iterator loop + * logic, so we need to force it. If we don't do that, + * is_state_visited() might skip saving a checkpoint, causing + * unnecessarily long sequence of not checkpointed + * instructions and jumps, leading to exhaustion of jump + * history buffer, and potentially other undesired outcomes. + * It is expected that with correct open-coded iterators + * convergence will happen quickly, so we don't run a risk of + * exhausting memory. + */ + mark_force_checkpoint(env, t); + } } } return visit_func_call_insn(t, insns, env, insn->src_reg == BPF_PSEUDO_CALL); From patchwork Tue May 7 13:19:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795370 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13DF01C6893; Tue, 7 May 2024 13:19:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087999; cv=none; b=paGhE1bEagRqhEXm+URWQ4b526TnzSvWE3h5Zs6ltgknQQuTxhKHT407YGLN7LXCCza+zp+ZdTMEcS+9Spk7SDJQW7wGfxuff+Ur6k2QOujujO4w9mIhrlxzkE7h2Mc897q+u0bUsRmaQP7OgZF/40kmagfLe1siiOsVFrbvHBg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715087999; c=relaxed/simple; bh=xAtYvtTO7UEzvYvc5IhP8kn6hurkiD+l2/2akEDg1/4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CEbkt1tq88+s86D3wZ/pjXLNX3w/FGk6hUB9DE1u9nnCzyTW04fNtkxS6ou+5e2JqeXpJE/t1NiwX8lUA/FuhoK7Fagsj5XUHtOQxbSFTk8iNiwFGwjStG5s0/2AMBC1TFyRHouGeU/Ql66DISKGgGhlDO5rjbXzo3KPIj6in5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BDbde6Nw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BDbde6Nw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF2F7C4DDF5; Tue, 7 May 2024 13:19:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715087999; bh=xAtYvtTO7UEzvYvc5IhP8kn6hurkiD+l2/2akEDg1/4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BDbde6NwGP6cwSKBiTBhG14oNtmtY/tLjFqFuOtg1YwRGbae/iWzdzhUpJ3O+6mkf KRPU7h/N7GTCV7klV2PEiQDeayIN3YQota8Qr5nTWnW95fFIGyZlx5usw2r7UtG/h0 Jg3U738DniL1sQ0lTd69YRg2mF3wqxRMx3fAQw7geWBWGhtoDMohxg1ybeONEtViT+ EXBNOFzddIuXHXI6IfkcDrCSYmoQWi1uEDlrGxby9Asi6ym6al6pOhkauewYPPup8G ZY2lbDKSfttBp9AfzRg30uml5wKeHt0/0epG3c+A5e+/J1dlwRuNPsS5QMUWQ+Xriz 97Mr0jTjDvJBQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:32 +0200 Subject: [PATCH RFC bpf-next 4/8] bpf: typedef a type for the bpf_wq callbacks Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-4-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=1400; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=xAtYvtTO7UEzvYvc5IhP8kn6hurkiD+l2/2akEDg1/4=; b=TeUsxmzpFrNCSLyUPysWwfRNWA95zCbqELDbbNLlDh1H/j5v7+vFk44sUYgB3SLnbRPuTncxU O9VwlNPUat2AyjpkVaLG35uCtHlezG/8KFROGZusffhwVaPMYH/iSdn X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= This allows users to rely on it by using it from the vmlinux.h Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/helpers.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 2a69a9a36c0f..97628bcbd507 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2720,8 +2720,10 @@ __bpf_kfunc int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) return 0; } +typedef int (*wq_callback_fn_t)(struct bpf_map *map, int *key, struct bpf_wq *wq); + __bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, - int (callback_fn)(void *map, int *key, struct bpf_wq *wq), + wq_callback_fn_t callback_fn__s_async, unsigned int flags, void *aux__ign) { @@ -2731,7 +2733,7 @@ __bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, if (flags) return -EINVAL; - return __bpf_async_set_callback(async, callback_fn, aux, flags, BPF_ASYNC_TYPE_WQ); + return __bpf_async_set_callback(async, callback_fn__s_async, aux, flags, BPF_ASYNC_TYPE_WQ); } __bpf_kfunc void bpf_preempt_disable(void) From patchwork Tue May 7 13:19:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795940 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAAD31C68BE; Tue, 7 May 2024 13:20:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088002; cv=none; b=RRi5rgoDB4LgBrjWQ2lhtCf3xXdOLXs6+GjWS6Pv11drViiPUZ8SbiDLRT8/9KdPTiFm118fSO93lR3ew6+Vgv/wDb9Rf67vyv68z/svL6nBC+umtde6VKm4dIBR2lbqFgPQRFsIC2tATxmyRDOkUqNEu2FqxtP3ITDCumWJezI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088002; c=relaxed/simple; bh=0dEsk9xVdvoDP5+magmILPSKFXyPAEcbBanwvUnwrlY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=f5G8iwGCMIRY1m0qhLCZ2Iny2fO6BjtFn+CQi6ga03irCJfic8rDrXdX2FQCo+vssXslZQBRzhh7VKGjYesnpuhKQd59jJn3QCVvIYLis6NZmF55pp7itEkT7+9vXM4tLB6XtYc9QZTlZzau71FmpqvhC3VOOJO976RyaE+fqR0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nA3mioI8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nA3mioI8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62A7BC2BBFC; Tue, 7 May 2024 13:19:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715088002; bh=0dEsk9xVdvoDP5+magmILPSKFXyPAEcbBanwvUnwrlY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nA3mioI81UV3xgoC8RpC4Mu22zpMCKXYRlU/M0cIBQxJS5NYL2ihjDXIYn5pt0i5z cRFdDyZ28oIdOewfMakPEPhAYRiOP8MF1xPPMMc+FZ1sRWu29gcvhoBahZAyjQxNLN P6KN4YOvqktZPl5Xf/OST/2vejSVcJEyxfSYpsu0B1oM1H67Eg1qmsQDDVE3oGJobm 5ViRnwQXDxq81GQMwb39d/IBoZCuAQcOsJZTa6J9pN39UT+2sNF0njN7Nn9Kniw5// eMB3XuK6tRhI9v5MLYyGAlHin7SfLGuLjq9cTmZMg2FMUoFpBoZwcMxV6X2J8qdkcv 4OyFUh2ogaSNw== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:33 +0200 Subject: [PATCH RFC bpf-next 5/8] selftests/bpf: rely on wq_callback_fn_t Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-5-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=3917; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=0dEsk9xVdvoDP5+magmILPSKFXyPAEcbBanwvUnwrlY=; b=VNx/k5glZWhQsEmI/jClMdqC4UnM4gt56GY1CihI90O1YgryD8UCunolfxq3F1C4/J6ijV/H/ ZehySth33UECpOr7dY8Uh+dje+SWyor08VqDO+p7JMprnY5wk08PIgt X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= The type of bpf_wq callbacks changed. So adapt to it and make use of wq_callback_fn_t. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- tools/testing/selftests/bpf/bpf_experimental.h | 3 +-- tools/testing/selftests/bpf/progs/wq.c | 10 ++++------ tools/testing/selftests/bpf/progs/wq_failures.c | 4 ++-- 3 files changed, 7 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 8b9cc87be4c4..0a35e6efccae 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -494,8 +494,7 @@ extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; extern int bpf_wq_init(struct bpf_wq *wq, void *p__map, unsigned int flags) __weak __ksym; extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; -extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, - int (callback_fn)(void *map, int *key, struct bpf_wq *wq), +extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, wq_callback_fn_t cb, unsigned int flags__k, void *aux__ign) __ksym; #define bpf_wq_set_callback(timer, cb, flags) \ bpf_wq_set_callback_impl(timer, cb, flags, NULL) diff --git a/tools/testing/selftests/bpf/progs/wq.c b/tools/testing/selftests/bpf/progs/wq.c index 49e712acbf60..c8c88976baca 100644 --- a/tools/testing/selftests/bpf/progs/wq.c +++ b/tools/testing/selftests/bpf/progs/wq.c @@ -52,8 +52,7 @@ struct { __u32 ok; __u32 ok_sleepable; -static int test_elem_callback(void *map, int *key, - int (callback_fn)(void *map, int *key, struct bpf_wq *wq)) +static int test_elem_callback(void *map, int *key, wq_callback_fn_t callback_fn) { struct elem init = {}, *val; struct bpf_wq *wq; @@ -83,8 +82,7 @@ static int test_elem_callback(void *map, int *key, return 0; } -static int test_hmap_elem_callback(void *map, int *key, - int (callback_fn)(void *map, int *key, struct bpf_wq *wq)) +static int test_hmap_elem_callback(void *map, int *key, wq_callback_fn_t callback_fn) { struct hmap_elem init = {}, *val; struct bpf_wq *wq; @@ -114,7 +112,7 @@ static int test_hmap_elem_callback(void *map, int *key, } /* callback for non sleepable workqueue */ -static int wq_callback(void *map, int *key, struct bpf_wq *work) +static int wq_callback(struct bpf_map *map, int *key, struct bpf_wq *work) { bpf_kfunc_common_test(); ok |= (1 << *key); @@ -122,7 +120,7 @@ static int wq_callback(void *map, int *key, struct bpf_wq *work) } /* callback for sleepable workqueue */ -static int wq_cb_sleepable(void *map, int *key, struct bpf_wq *work) +static int wq_cb_sleepable(struct bpf_map *map, int *key, struct bpf_wq *work) { bpf_kfunc_call_test_sleepable(); ok_sleepable |= (1 << *key); diff --git a/tools/testing/selftests/bpf/progs/wq_failures.c b/tools/testing/selftests/bpf/progs/wq_failures.c index 4cbdb425f223..3d87ccb8286e 100644 --- a/tools/testing/selftests/bpf/progs/wq_failures.c +++ b/tools/testing/selftests/bpf/progs/wq_failures.c @@ -28,14 +28,14 @@ struct { } lru SEC(".maps"); /* callback for non sleepable workqueue */ -static int wq_callback(void *map, int *key, struct bpf_wq *work) +static int wq_callback(struct bpf_map *map, int *key, struct bpf_wq *work) { bpf_kfunc_common_test(); return 0; } /* callback for sleepable workqueue */ -static int wq_cb_sleepable(void *map, int *key, struct bpf_wq *work) +static int wq_cb_sleepable(struct bpf_map *map, int *key, struct bpf_wq *work) { bpf_kfunc_call_test_sleepable(); return 0; From patchwork Tue May 7 13:19:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795369 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 794C615F3F6; Tue, 7 May 2024 13:20:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088006; cv=none; b=Ufky3l0/AIgzcYB+mSV4/AkvqjeFp+YB2d/oAPJc2DulO1ZZfTnc91TXOchDskn7/qTBtBQ8bZPBv5HDcjlBT++oqDSTmXXroLDCYzphKGjZiiit46qHQD7PJeNNyJFX0PLbUPRkon37zMEsWD90U7JOVALZEdb+HNflMrgvUhs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088006; c=relaxed/simple; bh=ZHkUdCx51hxhBKatmpC3J91gdbyruA1Cb3+7Y+YPx9g=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=T8fsKhVNnC3bJfSpegl20lhUt0KvZWUDeWIuHkbH7/ah+e6nWIP34809oefxHBF1/1K4Oe0zY4cC6JtTNbVPSSaSifVe/T8bHtkY41ly1/BKVKUKyvDTGRqegeBOwDaWLz+GXx3oN0A2LhXkvouVp+WBJf1yaPMEueOMvMcXaZk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j8cLmKdY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j8cLmKdY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1839EC2BBFC; Tue, 7 May 2024 13:20:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715088006; bh=ZHkUdCx51hxhBKatmpC3J91gdbyruA1Cb3+7Y+YPx9g=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=j8cLmKdYaZvYjdWwLVoGorZkPFm1X6/PgT/qtThOSMp7ALVVSorihb8MLmuy2JQXB +9/62Vc2fDs5pzUZtFqGDnmw+x6ojwjkdeESRfSctA4tJZKXYpOv46AUSGdUCMZ/+C ZntOZHXuQYX+eSJqoyVEU0B+RyJsgUhBD3vd/PKknL8nl3br4dGK3UzKfswVkoQqZf cAJjggGBDllZ+XgyT9aSvmoyJMX9pjgDg4a55rkkLDbs0U5Mv3wzvRj4qEex3E0SgY iPkzJJ1q86GUZLZuveEQU3qiuC3H3zPI0fkXMcGD5/9x2c0Wfhg9wvuue+/5wwZdna yw3wGCBBKxMHg== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:34 +0200 Subject: [PATCH RFC bpf-next 6/8] bpf: remove one special case of is_bpf_wq_set_callback_impl_kfunc Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-6-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=3721; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=ZHkUdCx51hxhBKatmpC3J91gdbyruA1Cb3+7Y+YPx9g=; b=+psPvQ9U0lib8PoOYD0b0ezqcWWXbbmhqvnYoH0EeLMPirxH4lu7bdfYRXp0w/+KFXQGW7eH2 HfmBtwpdXeaDHz9Ay1pi/ODf7q7KQNcgbeoLKzBinovzJth7XPBrKEf X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= It looks like the generic implementation based on __s_async suffix works well enough. So let's use it. Note: - currently we lose the return value range - the second argument is not of type PTR_TO_MAP_KEY Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/verifier.c | 27 ++++----------------------- 1 file changed, 4 insertions(+), 23 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index cc4dab81b306..6fba9e2caa83 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -511,7 +511,6 @@ static bool is_dynptr_ref_function(enum bpf_func_id func_id) } static bool is_sync_callback_calling_kfunc(u32 btf_id); -static bool is_async_callback_calling_kfunc(u32 btf_id); static bool is_callback_calling_kfunc(u32 btf_id); static bool is_bpf_throw_kfunc(struct bpf_insn *insn); @@ -544,8 +543,7 @@ static bool is_sync_callback_calling_insn(struct bpf_insn *insn) static bool is_async_callback_calling_insn(struct bpf_insn *insn) { - return (bpf_helper_call(insn) && is_async_callback_calling_function(insn->imm)) || - (bpf_pseudo_kfunc_call(insn) && is_async_callback_calling_kfunc(insn->imm)); + return bpf_helper_call(insn) && is_async_callback_calling_function(insn->imm); } static bool is_may_goto_insn(struct bpf_insn *insn) @@ -9560,15 +9558,14 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins return -EFAULT; } - if (is_async_callback_calling_insn(insn) || (kfunc_meta && kfunc_meta->async_cb.enabled)) { + if (kfunc_meta && kfunc_meta->async_cb.enabled) { struct bpf_verifier_state *async_cb; /* there is no real recursion here. timer and workqueue callbacks are async */ env->subprog_info[subprog].is_async_cb = true; async_cb = push_async_cb(env, env->subprog_info[subprog].start, insn_idx, subprog, - (is_bpf_wq_set_callback_impl_kfunc(insn->imm) || - (kfunc_meta && kfunc_meta->async_cb.sleepable))); + kfunc_meta && kfunc_meta->async_cb.sleepable); if (!async_cb) return -EFAULT; callee = async_cb->frame[0]; @@ -11534,11 +11531,6 @@ static bool is_sync_callback_calling_kfunc(u32 btf_id) return btf_id == special_kfunc_list[KF_bpf_rbtree_add_impl]; } -static bool is_async_callback_calling_kfunc(u32 btf_id) -{ - return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl]; -} - static bool is_bpf_throw_kfunc(struct bpf_insn *insn) { return bpf_pseudo_kfunc_call(insn) && insn->off == 0 && @@ -11552,8 +11544,7 @@ static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id) static bool is_callback_calling_kfunc(u32 btf_id) { - return is_sync_callback_calling_kfunc(btf_id) || - is_async_callback_calling_kfunc(btf_id); + return is_sync_callback_calling_kfunc(btf_id); } static bool is_rbtree_lock_required_kfunc(u32 btf_id) @@ -12465,16 +12456,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } } - if (is_bpf_wq_set_callback_impl_kfunc(meta.func_id)) { - err = push_callback_call(env, insn, insn_idx, meta.subprogno, - set_timer_callback_state, &meta); - if (err) { - verbose(env, "kfunc %s#%d failed callback verification\n", - func_name, meta.func_id); - return err; - } - } - if (meta.async_cb.enabled) { err = push_callback_call(env, insn, insn_idx, meta.subprogno, set_generic_callback_state, &meta); From patchwork Tue May 7 13:19:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795939 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E1F31C9EA4; Tue, 7 May 2024 13:20:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088010; cv=none; b=aZW7AEDSYXH6b5yod1LKODtwyOTircouDRb1IrlxFq2DMfGLWr6ffCr/hGnPvdUI+uKnm5nyOxSsy5XBv6d6CtZAeaMaGN9V3Yzyljz5iOkQVWP2D8HMPkw/m8Eqi7VsvmpWGprr/cVEUIKTMCiKkVFT2iF58KPhRNCzHcon49w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088010; c=relaxed/simple; bh=egkWOz43eHGtMFIY140aoiyLqrnrPqlvPykxlJaPgaY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HcnZ+D7f296qmkbIU5CFTt/j48rNB8VUILummu1r9VhraVf7kTvH8h+IN45v5aDxGPgWd6wghxgdrWroMT8DHKeRL4fNgHzkjWjX9r6OYFR2cPqFuWiVj3qn9/04+d4Sx2We7+sJawAo0OjzhWMEexkHiZzgdPgKbwcyA8lIGBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KNcm/+Xy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KNcm/+Xy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C210FC4DDE8; Tue, 7 May 2024 13:20:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715088010; bh=egkWOz43eHGtMFIY140aoiyLqrnrPqlvPykxlJaPgaY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KNcm/+XyBrTLdm7oRT16OCMb3/Ap+qDbmQqv+Rp6gUHxh5DyiIXAwUGp2BmaDz7v0 durzb7hgX0XX+Nagbwdc9gc+66y4/uLfnF0/0fFcEAMppdNqi6oySRmeC29LEjggyb M+a84qUFh/pFTcKg14xZ65dTwyNT7YqY75ParDTdk0D6536ARVspSebS5ZBBsMWCWo M9x8b6yREDX4te0TwoMkIEoyRSBSLAioBYFubWsR5n0k4pZiSnWw+69JC6Uta3fFg2 mbhPoXK6Gm7sGdV7Jm15JdEC2pPof9EmHUeZPd3sQjj2xruBqjOHal8cNcHyLSJOcP 8vzR0dIEyJJkQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:35 +0200 Subject: [PATCH RFC bpf-next 7/8] bpf: implement __aux kfunc argument suffix to fetch prog_aux Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-7-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=2673; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=egkWOz43eHGtMFIY140aoiyLqrnrPqlvPykxlJaPgaY=; b=oJAmtA0r9hCIs8UUHfZHyxXeyHI1ooXguBzR9uFMgWiwn0xOIIL26fN4pkiWwxNCGDAFC1RfH wIqjJqkgIp0DKjQK28Gx8oOq7x3Y3Ne4Ttm9fOagKbeixtvtfxJxGGc X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= This allows kfunc to request the prog_aux environment in their implementation, to have access to the originated bpf_prog for example. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/verifier.c | 40 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6fba9e2caa83..33b108db0025 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10909,6 +10909,11 @@ static bool is_kfunc_arg_sleepable_async_cb(const struct btf *btf, const struct return btf_param_match_suffix(btf, arg, "__s_async"); } +static bool is_kfunc_arg_prog_aux(const struct btf *btf, const struct btf_param *arg) +{ + return btf_param_match_suffix(btf, arg, "__aux"); +} + static bool is_kfunc_arg_scalar_with_name(const struct btf *btf, const struct btf_param *arg, const char *name) @@ -11807,7 +11812,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ t = btf_type_skip_modifiers(btf, args[i].type, NULL); - if (is_kfunc_arg_ignore(btf, &args[i])) + if (is_kfunc_arg_ignore(btf, &args[i]) || + is_kfunc_arg_prog_aux(btf, &args[i])) continue; if (btf_type_is_scalar(t)) { @@ -19950,6 +19956,38 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_buf[1] = ld_addrs[1]; insn_buf[2] = *insn; *cnt = 3; + } else { + struct bpf_kfunc_call_arg_meta meta; + struct bpf_insn kfunc_insn = *insn; + const struct btf_param *args; + u32 i, nargs, prog_aux_arg; + const char *func_name; + int ret; + + /* imm might not have func_id, so create a fake insn with the expected args */ + kfunc_insn.imm = desc->func_id; + + ret = fetch_kfunc_meta(env, &kfunc_insn, &meta, &func_name); + if (ret == 0) { + args = (const struct btf_param *)(meta.func_proto + 1); + nargs = btf_type_vlen(meta.func_proto); + prog_aux_arg = nargs; + + for (i = 0; i < nargs; i++) { + if (is_kfunc_arg_prog_aux(meta.btf, &args[i])) + prog_aux_arg = i; + } + + if (prog_aux_arg < nargs) { + struct bpf_insn ld_addrs[2] = { BPF_LD_IMM64(BPF_REG_1 + prog_aux_arg, + (long)env->prog->aux) }; + + insn_buf[0] = ld_addrs[0]; + insn_buf[1] = ld_addrs[1]; + insn_buf[2] = *insn; + *cnt = 3; + } + } } return 0; } From patchwork Tue May 7 13:19:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Tissoires X-Patchwork-Id: 795368 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21AC21CB304; Tue, 7 May 2024 13:20:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088014; cv=none; b=jpD2WEGmhd8Ucr1QP5HTb+jJswEcrLLju/xPp+6O8YY74Dt/CClRdxtY2+8jiCFKjFRPIE5hgVKVN+R7NLOU4k4JWF7LKpNIFEnmzyTH0D1MfUNB3ZWT+J+Hie3wYMvvk7V2B4LS77kwIqE4E0MdJGGlULZtIdUGRQoi4QQHs+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715088014; c=relaxed/simple; bh=xqxP5tviAL2S96IuIPgdlhvR3scECTl5ZJSxLsbtWpE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qWNtSjdgCG/a3FYdCrGzSHNON0mM3Nb/y1BH3SxMybcOtaHybphatCgnNvpypzEKEgkNhGVxJeIQD7rJ8AbtZ6qw7EOzT9sEEMnsJ6Azr0mgad7ETaIIAge8fZ+8rsPnRs25IhI06aHgxfA0l9EANGkw0oq+TWtAleLjnvfsPjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gd4Oiq2g; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gd4Oiq2g" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7A9D3C4DDE7; Tue, 7 May 2024 13:20:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715088013; bh=xqxP5tviAL2S96IuIPgdlhvR3scECTl5ZJSxLsbtWpE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=gd4Oiq2gEdgU1xGUmMmSAo2Ok46t/iK3YslvSMCr1C8RAhH7YNv7NP5q/cKDFNCli vtBGyNL1bMnsbMz2JVDcctY3aDWQq/FmHWsw1K88IV8ITMv6MtE1MJ3pFWU4yvItaf BwwhfsZzg1sA76I4bnzpQGw3HjWmkpZ8pUwQ9P3SA9QfbhoXfOm24631lAwLGzDRdR UPscs38LStKDElMYm0Ze3gz7IUva3/sMFooxIwcSIKxo+qsGbxffEv4jyTWSsTOgae ENsvws/vunKkdMSsTnGyMpja8gceb9WxHOTmq5vKXTP5FHJn/Cf8MhtruG8VHWn50W mB+BeOuEeokQQ== From: Benjamin Tissoires Date: Tue, 07 May 2024 15:19:36 +0200 Subject: [PATCH RFC bpf-next 8/8] bpf: rely on __aux suffix for bpf_wq_set_callback_impl Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240507-bpf_async-v1-8-b4df966096d8@kernel.org> References: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> In-Reply-To: <20240507-bpf_async-v1-0-b4df966096d8@kernel.org> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Benjamin Tissoires X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715087980; l=3513; i=bentiss@kernel.org; s=20230215; h=from:subject:message-id; bh=xqxP5tviAL2S96IuIPgdlhvR3scECTl5ZJSxLsbtWpE=; b=gLKwDN8r6bRspBt809EY3fwZaVCUxwSCKzAOrVkEzq+C8Ab8+unf1PTiflvMQqRSlCcJAZyt9 XIcu3gAUTtSBKFD4QWf7tuSKVgxC+E6WpF36p1+A7nM8EvjE/Pq1B+E X-Developer-Key: i=bentiss@kernel.org; a=ed25519; pk=7D1DyAVh6ajCkuUTudt/chMuXWIJHlv2qCsRkIizvFw= And then cleanup the verifier about the special cases about this kfunc. Signed-off-by: Benjamin Tissoires --- This is an RFC, and is not meant to be fully reviewed/applied as it is. I'm posting this to show what I wanted to explain in https://lore.kernel.org/bpf/mhkzkf4e23uvljtmwizwcxyuyat2tmfxn33xb4t7waafgmsa66@mcrzpj3b6ssx/ --- kernel/bpf/helpers.c | 4 ++-- kernel/bpf/verifier.c | 17 ----------------- 2 files changed, 2 insertions(+), 19 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 97628bcbd507..03524fa5feef 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2725,9 +2725,9 @@ typedef int (*wq_callback_fn_t)(struct bpf_map *map, int *key, struct bpf_wq *wq __bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, wq_callback_fn_t callback_fn__s_async, unsigned int flags, - void *aux__ign) + void *prog__aux) { - struct bpf_prog_aux *aux = (struct bpf_prog_aux *)aux__ign; + struct bpf_prog_aux *aux = (struct bpf_prog_aux *)prog__aux; struct bpf_async_kern *async = (struct bpf_async_kern *)wq; if (flags) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 33b108db0025..829a234832d9 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -514,8 +514,6 @@ static bool is_sync_callback_calling_kfunc(u32 btf_id); static bool is_callback_calling_kfunc(u32 btf_id); static bool is_bpf_throw_kfunc(struct bpf_insn *insn); -static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id); - static bool is_sync_callback_calling_function(enum bpf_func_id func_id) { return func_id == BPF_FUNC_for_each_map_elem || @@ -11134,7 +11132,6 @@ enum special_kfunc_type { KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, KF_bpf_throw, - KF_bpf_wq_set_callback_impl, KF_bpf_preempt_disable, KF_bpf_preempt_enable, KF_bpf_iter_css_task_new, @@ -11161,7 +11158,6 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) -BTF_ID(func, bpf_wq_set_callback_impl) #ifdef CONFIG_CGROUPS BTF_ID(func, bpf_iter_css_task_new) #endif @@ -11190,7 +11186,6 @@ BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_throw) -BTF_ID(func, bpf_wq_set_callback_impl) BTF_ID(func, bpf_preempt_disable) BTF_ID(func, bpf_preempt_enable) #ifdef CONFIG_CGROUPS @@ -11542,11 +11537,6 @@ static bool is_bpf_throw_kfunc(struct bpf_insn *insn) insn->imm == special_kfunc_list[KF_bpf_throw]; } -static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id) -{ - return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl]; -} - static bool is_callback_calling_kfunc(u32 btf_id) { return is_sync_callback_calling_kfunc(btf_id); @@ -19949,13 +19939,6 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, desc->func_id == special_kfunc_list[KF_bpf_rdonly_cast]) { insn_buf[0] = BPF_MOV64_REG(BPF_REG_0, BPF_REG_1); *cnt = 1; - } else if (is_bpf_wq_set_callback_impl_kfunc(desc->func_id)) { - struct bpf_insn ld_addrs[2] = { BPF_LD_IMM64(BPF_REG_4, (long)env->prog->aux) }; - - insn_buf[0] = ld_addrs[0]; - insn_buf[1] = ld_addrs[1]; - insn_buf[2] = *insn; - *cnt = 3; } else { struct bpf_kfunc_call_arg_meta meta; struct bpf_insn kfunc_insn = *insn;