From patchwork Thu Aug 26 19:38:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC9E3C4320A for ; Thu, 26 Aug 2021 19:39:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A5D1261040 for ; Thu, 26 Aug 2021 19:39:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243495AbhHZTkV (ORCPT ); Thu, 26 Aug 2021 15:40:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42237 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243407AbhHZTkU (ORCPT ); Thu, 26 Aug 2021 15:40:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t8XYUgR9kHcOYFdSNrWNV6bYolumP9ymhRdSASBvoKM=; b=J4TD16JPNXes7ncQkhVlXA103mslDdrHwNveGgWlQL1CffLKe1ANTWj93MbmteKA9FHpUD tKW4tJaMbpdd2zIK0ads27ujFJPJBSvNm8STzIHwpJEV3F6yB6LcKXyxfE26yEsiuJIqyl 9wo7C8LpYJGu3CiebMOH8/NarcAfrlw= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-531-BYXGoto1PSeYbaFSTb_yPA-1; Thu, 26 Aug 2021 15:39:30 -0400 X-MC-Unique: BYXGoto1PSeYbaFSTb_yPA-1 Received: by mail-wm1-f69.google.com with SMTP id z186-20020a1c7ec30000b02902e6a27a9962so4767001wmc.3 for ; Thu, 26 Aug 2021 12:39:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t8XYUgR9kHcOYFdSNrWNV6bYolumP9ymhRdSASBvoKM=; b=ZlYdiGioWqyf7oahc96ENtiuTlgY521d5phcx9BRquoqmzTkByCkepMseFotmrREV+ g8NjPJzwn7t5ZjhC3hxYL2AfOUMSREYGHFdEMuqumbqGTqckkTnSl5yIvirVKgcFAo9H xpHnDT9vE/3ItpMSpx8EUnROTtcEW/5LgJehHtdeGCUNFg4sDvK22jcVxW8FkZbxwgxT fnSx9CtFJNjXsMUpP+0cQRTTkVnchlIZWk6rxGl4D1AlOyb1w9Xe6nNUYx6bTiY22oAy 9BKw8TZFBKpFyecaY7swO0q8t35yNCHKYk3NMVSromvazuda+sUVjOdXoln0LJ3roZ1M qRxg== X-Gm-Message-State: AOAM532GiGCSf/gcYnWvXHQ8WjHJbNZq3pMtiZf4MLim/VAELekUGP59 VEl1RfMAk5iX6AkvYaR98yW7cew0rxlKeohMwVAHQCJ2RK3CytEmIYy7RH+VJyUDcCwqOti+uVa 6M2mnxmgKGdVAyFiH X-Received: by 2002:a5d:4142:: with SMTP id c2mr6091476wrq.340.1630006769760; Thu, 26 Aug 2021 12:39:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxBL1FafyptrQyoXeyUcyn0ennz0cwN+Ah0kJWNT5IknqO/e8yH2B5sBSVQLTvUBYm9pq1F/Q== X-Received: by 2002:a5d:4142:: with SMTP id c2mr6091460wrq.340.1630006769637; Thu, 26 Aug 2021 12:39:29 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id r1sm8012572wmn.46.2021.08.26.12.39.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:39:29 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 01/27] x86/ftrace: Remove extra orig rax move Date: Thu, 26 Aug 2021 21:38:56 +0200 Message-Id: <20210826193922.66204-2-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There's identical move 2 lines earlier. Signed-off-by: Jiri Olsa --- arch/x86/kernel/ftrace_64.S | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index 7c273846c687..a8eb084a7a9a 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -251,7 +251,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL) * If ORIG_RAX is anything but zero, make this a call to that. * See arch_ftrace_set_direct_caller(). */ - movq ORIG_RAX(%rsp), %rax testq %rax, %rax SYM_INNER_LABEL(ftrace_regs_caller_jmp, SYM_L_GLOBAL) jnz 1f From patchwork Thu Aug 26 19:38:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59699C432BE for ; Thu, 26 Aug 2021 19:39:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 414B761052 for ; Thu, 26 Aug 2021 19:39:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243506AbhHZTkf (ORCPT ); Thu, 26 Aug 2021 15:40:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23585 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243491AbhHZTke (ORCPT ); Thu, 26 Aug 2021 15:40:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sLqTTRxVL8xfk55K+9eSTs+Utqi1uRVMWYGVEA7msME=; b=J+gunPJmeIbku8bzKRygcTunPznLueaAREQJkQixSJFzDdXsBgtXH2BRQ+JotyqCOB/1Fb ZruvFU61zRVKgKhGWPWfT6f4XnEXLTbxfZWZu/AJ9ABTOqj3DCu07v0YkFzXDLiE2I61vO PQGmvHxgxsjUgSEjN9XbbFl8o40Jhrw= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-403-ny_1C4h4NEGkbETC0OutnA-1; Thu, 26 Aug 2021 15:39:43 -0400 X-MC-Unique: ny_1C4h4NEGkbETC0OutnA-1 Received: by mail-wm1-f70.google.com with SMTP id j33-20020a05600c48a100b002e879427915so1267234wmp.5 for ; Thu, 26 Aug 2021 12:39:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sLqTTRxVL8xfk55K+9eSTs+Utqi1uRVMWYGVEA7msME=; b=DJ4Dm3YrQ5oUlWebrN95mq7CbuHqE9C/M4QDkSQHaAayVw883LuWE3vRMNyrrjxJ5u HArZfRErcq2hOivuqAkgyHDpVyreBnRuiW0DCh1g78uH864LlrEJuIcgjFcslsQxrFTs 4588cRA/5tgSmDo9aAS4SdAuVvByGKBFfzWUAYk6I1hVkTOXmGNdOX+rTUfiAVRuBq/J Z/EjJLr9RAN6hNckeINnd1Vi3PJ+6nUuS9jyWIH2Dq83/SN8uod4VLmuiznrq+uLEj2j OBaFsBze/xdk3arBQDyZ+Gmx2N9tNpGOBfZBpptYe3NiLShRxEzqZREYJEr1HX0D1R0A QI3w== X-Gm-Message-State: AOAM533avDcc87Jyhtoss2E+mCBvjn0pSgVlDBerlCD7cwhwiqKKHLdu OUbKFRJzDudgFtU5Y/VvdrbXSqHtbOq9Hu9wYrfWeAXOjq7rceKdhzIcS8WxGY0uT1Uc34KJ8g0 pD1WrJ6AT0me2DO+T X-Received: by 2002:a05:600c:1ca7:: with SMTP id k39mr5210880wms.162.1630006782217; Thu, 26 Aug 2021 12:39:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVCSYcOLJeoO9HAhJSY06pVgY7DESSfJAk8Q4dTP9qnnmFI+O8vT7InR93U6Zg+d5eIZNF4A== X-Received: by 2002:a05:600c:1ca7:: with SMTP id k39mr5210858wms.162.1630006782038; Thu, 26 Aug 2021 12:39:42 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id g76sm8944197wme.16.2021.08.26.12.39.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:39:41 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 03/27] x86/ftrace: Make function graph use ftrace directly Date: Thu, 26 Aug 2021 21:38:58 +0200 Message-Id: <20210826193922.66204-4-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Steven Rostedt (VMware)" We don't need special hook for graph tracer entry point, but instead we can use graph_ops::func function to install the return_hooker. This moves the graph tracing setup _before_ the direct trampoline prepares the stack, so the return_hooker will be called when the direct trampoline is finished. This simplifies the code, because we don't need to take into account the direct trampoline setup when preparing the graph tracer hooker and we can allow function graph tracer on entries registered with direct trampoline. Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Jiri Olsa --- arch/x86/include/asm/ftrace.h | 9 +++++++-- arch/x86/kernel/ftrace.c | 37 ++++++++++++++++++++++++++++++++--- arch/x86/kernel/ftrace_64.S | 29 +-------------------------- include/linux/ftrace.h | 6 ++++++ kernel/trace/fgraph.c | 6 ++++-- 5 files changed, 52 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h index 9f3130f40807..024d9797646e 100644 --- a/arch/x86/include/asm/ftrace.h +++ b/arch/x86/include/asm/ftrace.h @@ -57,6 +57,13 @@ arch_ftrace_get_regs(struct ftrace_regs *fregs) #define ftrace_instruction_pointer_set(fregs, _ip) \ do { (fregs)->regs.ip = (_ip); } while (0) + +struct ftrace_ops; +#define ftrace_graph_func ftrace_graph_func +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs); +#else +#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR #endif #ifdef CONFIG_DYNAMIC_FTRACE @@ -65,8 +72,6 @@ struct dyn_arch_ftrace { /* No extra data needed for x86 */ }; -#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR - #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* __ASSEMBLY__ */ #endif /* CONFIG_FUNCTION_TRACER */ diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index c555624da989..804fcc6ef2c7 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -527,7 +527,7 @@ static void *addr_from_call(void *ptr) return ptr + CALL_INSN_SIZE + call.disp; } -void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, +void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer); /* @@ -541,7 +541,8 @@ static void *static_tramp_func(struct ftrace_ops *ops, struct dyn_ftrace *rec) void *ptr; if (ops && ops->trampoline) { -#ifdef CONFIG_FUNCTION_GRAPH_TRACER +#if !defined(CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS) && \ + defined(CONFIG_FUNCTION_GRAPH_TRACER) /* * We only know about function graph tracer setting as static * trampoline. @@ -589,8 +590,9 @@ void arch_ftrace_trampoline_free(struct ftrace_ops *ops) #ifdef CONFIG_FUNCTION_GRAPH_TRACER #ifdef CONFIG_DYNAMIC_FTRACE -extern void ftrace_graph_call(void); +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +extern void ftrace_graph_call(void); static const char *ftrace_jmp_replace(unsigned long ip, unsigned long addr) { return text_gen_insn(JMP32_INSN_OPCODE, (void *)ip, (void *)addr); @@ -618,7 +620,17 @@ int ftrace_disable_ftrace_graph_caller(void) return ftrace_mod_jmp(ip, &ftrace_stub); } +#else /* !CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ +int ftrace_enable_ftrace_graph_caller(void) +{ + return 0; +} +int ftrace_disable_ftrace_graph_caller(void) +{ + return 0; +} +#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* !CONFIG_DYNAMIC_FTRACE */ /* @@ -629,6 +641,7 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer) { unsigned long return_hooker = (unsigned long)&return_to_handler; + int bit; /* * When resuming from suspend-to-ram, this function can be indirectly @@ -648,7 +661,25 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, if (unlikely(atomic_read(¤t->tracing_graph_pause))) return; + bit = ftrace_test_recursion_trylock(ip, *parent); + if (bit < 0) + return; + if (!function_graph_enter(*parent, ip, frame_pointer, parent)) *parent = return_hooker; + + ftrace_test_recursion_unlock(bit); +} + +#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs) +{ + struct pt_regs *regs = &fregs->regs; + unsigned long *stack = (unsigned long *)kernel_stack_pointer(regs); + + prepare_ftrace_return(ip, (unsigned long *)stack, 0); } +#endif + #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index a8eb084a7a9a..7a879901f103 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -174,11 +174,6 @@ SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL) SYM_FUNC_END(ftrace_caller); SYM_FUNC_START(ftrace_epilogue) -#ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - jmp ftrace_stub -#endif - /* * This is weak to keep gas from relaxing the jumps. * It is also used to copy the retq for trampolines. @@ -288,15 +283,6 @@ SYM_FUNC_START(__fentry__) cmpq $ftrace_stub, ftrace_trace_function jnz trace -fgraph_trace: -#ifdef CONFIG_FUNCTION_GRAPH_TRACER - cmpq $ftrace_stub, ftrace_graph_return - jnz ftrace_graph_caller - - cmpq $ftrace_graph_entry_stub, ftrace_graph_entry - jnz ftrace_graph_caller -#endif - SYM_INNER_LABEL(ftrace_stub, SYM_L_GLOBAL) retq @@ -314,25 +300,12 @@ trace: CALL_NOSPEC r8 restore_mcount_regs - jmp fgraph_trace + jmp ftrace_stub SYM_FUNC_END(__fentry__) EXPORT_SYMBOL(__fentry__) #endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_FUNC_START(ftrace_graph_caller) - /* Saves rbp into %rdx and fills first parameter */ - save_mcount_regs - - leaq MCOUNT_REG_SIZE+8(%rsp), %rsi - movq $0, %rdx /* No framepointers needed */ - call prepare_ftrace_return - - restore_mcount_regs - - retq -SYM_FUNC_END(ftrace_graph_caller) - SYM_FUNC_START(return_to_handler) subq $24, %rsp diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a69f363b61bf..9b218e59a608 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -614,6 +614,12 @@ void ftrace_modify_all_code(int command); extern void ftrace_graph_caller(void); extern int ftrace_enable_ftrace_graph_caller(void); extern int ftrace_disable_ftrace_graph_caller(void); +#ifndef ftrace_graph_func +#define ftrace_graph_func ftrace_stub +#define FTRACE_OPS_GRAPH_STUB FTRACE_OPS_FL_STUB +#else +#define FTRACE_OPS_GRAPH_STUB 0 +#endif #else static inline int ftrace_enable_ftrace_graph_caller(void) { return 0; } static inline int ftrace_disable_ftrace_graph_caller(void) { return 0; } diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index b8a0d1d564fb..22061d38fc00 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -115,6 +115,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, { struct ftrace_graph_ent trace; +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS /* * Skip graph tracing if the return location is served by direct trampoline, * since call sequence and return addresses are unpredictable anyway. @@ -124,6 +125,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, if (ftrace_direct_func_count && ftrace_find_rec_direct(ret - MCOUNT_INSN_SIZE)) return -EBUSY; +#endif trace.func = func; trace.depth = ++current->curr_ret_depth; @@ -333,10 +335,10 @@ unsigned long ftrace_graph_ret_addr(struct task_struct *task, int *idx, #endif /* HAVE_FUNCTION_GRAPH_RET_ADDR_PTR */ static struct ftrace_ops graph_ops = { - .func = ftrace_stub, + .func = ftrace_graph_func, .flags = FTRACE_OPS_FL_INITIALIZED | FTRACE_OPS_FL_PID | - FTRACE_OPS_FL_STUB, + FTRACE_OPS_GRAPH_STUB, #ifdef FTRACE_GRAPH_TRAMP_ADDR .trampoline = FTRACE_GRAPH_TRAMP_ADDR, /* trampoline_size is only needed for dynamically allocated tramps */ From patchwork Thu Aug 26 19:39:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86EB1C4320E for ; Thu, 26 Aug 2021 19:40:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6EDD86108F for ; Thu, 26 Aug 2021 19:40:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243529AbhHZTku (ORCPT ); Thu, 26 Aug 2021 15:40:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22330 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243525AbhHZTkp (ORCPT ); Thu, 26 Aug 2021 15:40:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uK5Gw3luJndbFn1Fa9N5OtQXbRTUWbF3Z2jHUj38PoM=; b=JgdpZARtUwyw7ldeEnZIwFaeGUpSAl5jSTsPRlj2NFResSU6bM1Y+ko82atoMrNG0yyQ7k y6xMQlfWsYnq+ZO9AO8kUVNGPBWvUXZ6jkvIqoeO64AYJhrm18cc4+AZxeL4ulQwdCTHr0 g1ypXfq8CsfRPUIyHMFbri5dDk+SGOY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-230-VmDiL5gROqGtE5_dWAuUow-1; Thu, 26 Aug 2021 15:39:55 -0400 X-MC-Unique: VmDiL5gROqGtE5_dWAuUow-1 Received: by mail-wm1-f70.google.com with SMTP id 201-20020a1c01d2000000b002e72ba822dcso4761642wmb.6 for ; Thu, 26 Aug 2021 12:39:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uK5Gw3luJndbFn1Fa9N5OtQXbRTUWbF3Z2jHUj38PoM=; b=h+ap0dX5jJ9OjRQSypy+K24PkQVHlEmw9TxPGj6uFUHTl/iOYnJs4571WAR1XhA9wt HzzxfcbaMkkOnAjymoFqofnPp3z4/fBlBOmQqyqdI5cVHKJJpPhml+AkEdj4ZY+8q0rC 7+ERFY5X2TXiAfouCv+Gzo6j5XXSeoByohWRIoZqDGsm8qprAie75CG2cQCzve4i0X3A jIJB801hg2l9ydedQthCumgV+4c6TmTWs1G8n8wiuykRrmAEo4MeMlprNFNnGE8LqGRE Ltn2KDZtpTvfMCzwhIbMNBRVWhK9Grg48LpK2uv622r55RgZ1r1noxQRQQeK32dplSSF CSnQ== X-Gm-Message-State: AOAM532xTzsDXfVQFeMxkdfXvH+upcEUqf411F6lKLPZHoeF+ARfrHyy jeT7Vu/nGB4U4hYkhyWQs7UeyVc8TZFmYde6g6aBYnbSBnh1jSs+8T9YUwOYpsL+3cg/rutVetv w9o/cTFpZdgMFCtNK X-Received: by 2002:adf:fb8f:: with SMTP id a15mr6051140wrr.92.1630006794547; Thu, 26 Aug 2021 12:39:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyFi9NXcOpnIV36WGLOJUq/THYNfhH3OgZ+38gkY2C+jDnkQzuS/bc0x0iDcjDVsSvlBt7Aug== X-Received: by 2002:adf:fb8f:: with SMTP id a15mr6051116wrr.92.1630006794343; Thu, 26 Aug 2021 12:39:54 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id b18sm4107493wrr.89.2021.08.26.12.39.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:39:54 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 05/27] ftrace: Add ftrace_add_rec_direct function Date: Thu, 26 Aug 2021 21:39:00 +0200 Message-Id: <20210826193922.66204-6-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Factor out the code that adds (ip, addr) tuple to direct_functions hash in new ftrace_add_rec_direct function. It will be used in following patches. Signed-off-by: Jiri Olsa --- kernel/trace/ftrace.c | 60 ++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 24 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 7b180f61e6d3..c60217d81040 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2394,6 +2394,39 @@ unsigned long ftrace_find_rec_direct(unsigned long ip) return entry->direct; } +static struct ftrace_func_entry* +ftrace_add_rec_direct(unsigned long ip, unsigned long addr, + struct ftrace_hash **free_hash) +{ + struct ftrace_func_entry *entry; + + if (ftrace_hash_empty(direct_functions) || + direct_functions->count > 2 * (1 << direct_functions->size_bits)) { + struct ftrace_hash *new_hash; + int size = ftrace_hash_empty(direct_functions) ? 0 : + direct_functions->count + 1; + + if (size < 32) + size = 32; + + new_hash = dup_hash(direct_functions, size); + if (!new_hash) + return NULL; + + *free_hash = direct_functions; + direct_functions = new_hash; + } + + entry = kmalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return NULL; + + entry->ip = ip; + entry->direct = addr; + __add_hash_entry(direct_functions, entry); + return entry; +} + static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { @@ -5110,27 +5143,6 @@ int register_ftrace_direct(unsigned long ip, unsigned long addr) } ret = -ENOMEM; - if (ftrace_hash_empty(direct_functions) || - direct_functions->count > 2 * (1 << direct_functions->size_bits)) { - struct ftrace_hash *new_hash; - int size = ftrace_hash_empty(direct_functions) ? 0 : - direct_functions->count + 1; - - if (size < 32) - size = 32; - - new_hash = dup_hash(direct_functions, size); - if (!new_hash) - goto out_unlock; - - free_hash = direct_functions; - direct_functions = new_hash; - } - - entry = kmalloc(sizeof(*entry), GFP_KERNEL); - if (!entry) - goto out_unlock; - direct = ftrace_find_direct_func(addr); if (!direct) { direct = ftrace_alloc_direct_func(addr); @@ -5140,9 +5152,9 @@ int register_ftrace_direct(unsigned long ip, unsigned long addr) } } - entry->ip = ip; - entry->direct = addr; - __add_hash_entry(direct_functions, entry); + entry = ftrace_add_rec_direct(ip, addr, &free_hash); + if (!entry) + goto out_unlock; ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); if (ret) From patchwork Thu Aug 26 19:39:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA6B6C432BE for ; Thu, 26 Aug 2021 19:40:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6A4D610A6 for ; Thu, 26 Aug 2021 19:40:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243535AbhHZTlA (ORCPT ); Thu, 26 Aug 2021 15:41:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59704 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243531AbhHZTk7 (ORCPT ); Thu, 26 Aug 2021 15:40:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cz5FUM/QUQ2HTCjVFzxMUjYkqvve+4iMlPHPGJoE+bY=; b=CT4LoXzI2fEBxOaAOuJArOeQ4qou0Ut9dEZAz+F20HZieMYRo09jswr3BF1fyMXjGfj7Pq 9p70/0VGHHhOyAFKQ9p+QjWD6FswcG/n+ycqXHrty3hak04Fluw731SjGow2JvfYDM7VBM srFa7v3vdK9a5WqIBAts8hSbtA5wGI4= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-398-PHIni-rSMpyTWe7DZjwk_Q-1; Thu, 26 Aug 2021 15:40:08 -0400 X-MC-Unique: PHIni-rSMpyTWe7DZjwk_Q-1 Received: by mail-wm1-f71.google.com with SMTP id x125-20020a1c3183000000b002e73f079eefso3447935wmx.0 for ; Thu, 26 Aug 2021 12:40:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Cz5FUM/QUQ2HTCjVFzxMUjYkqvve+4iMlPHPGJoE+bY=; b=C5Nzb1xZPwO5f+jdRgXMwMya9DakzdhzsDl/M6jC+epniiQo5uZ++/IJjT1nBIE+z2 GwEIMDqqcwx9s2aADzn8x7yT96lijZbt4GdX3JUwPIGVMePaHAyjD7ToYYGnpZss5YZ8 IWjGmkpr5vuDNLmV+SDBcBAh4bI9krR/+ghXatHeg6k66Cw0pz1DuwP2LjfViEMSgpbf eloYR9jZzY44u/Kj0PLoomc41bNflkgvD4n8WYHdq2csh+ntKy+zxmCq6uE21v1WDq4y UoD7/JaDALEJ1aVN9O0zzBvxDpZUDfGJuTse0PYXhc40bwDQ+OnBlo5c7i4EEfBB8hYn YbFQ== X-Gm-Message-State: AOAM532n87Svs9DW8dDzj0wzgoDYtm1NztCgEm2CO7irsoBf3BH4uH+N seQrv3RVouAZ8a5ZkMU7fQyqwAqofSI4rBzVku5M4T2t0YjB+mPtCNGhWxcO/vhIsVIR3kGyLUE L0D7FAyxCQ3E0nAGt X-Received: by 2002:adf:e4c5:: with SMTP id v5mr6082129wrm.1.1630006806980; Thu, 26 Aug 2021 12:40:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+oapw3UzB0tUkEstf0tZaeuwgr3wd1znhLKDrE5AJuTWzq9xPA/RbcYKzFpfOJzGc1FS3SA== X-Received: by 2002:adf:e4c5:: with SMTP id v5mr6082103wrm.1.1630006806810; Thu, 26 Aug 2021 12:40:06 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id f2sm4002494wru.31.2021.08.26.12.40.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:40:06 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 07/27] ftrace: Add multi direct modify interface Date: Thu, 26 Aug 2021 21:39:02 +0200 Message-Id: <20210826193922.66204-8-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding interface to modify registered direct function for ftrace_ops. Adding following function: modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) The function changes the currently registered direct function for all attached functions. Signed-off-by: Jiri Olsa --- include/linux/ftrace.h | 6 ++++++ kernel/trace/ftrace.c | 43 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 93d8f12e70b3..63ca0a424947 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -318,6 +318,8 @@ int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, unsigned long ftrace_find_rec_direct(unsigned long ip); int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); int unregister_ftrace_direct_multi(struct ftrace_ops *ops); +int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); + #else # define ftrace_direct_func_count 0 static inline int register_ftrace_direct(unsigned long ip, unsigned long addr) @@ -356,6 +358,10 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops) { return -ENODEV; } +int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + return -ENODEV; +} #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ #ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 7243769493c9..59940a6a907c 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5518,6 +5518,49 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops) return err; } EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); + +int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + struct ftrace_hash *hash = ops->func_hash->filter_hash; + struct ftrace_func_entry *entry, *iter; + int i, size; + int err; + + if (check_direct_multi(ops)) + return -EINVAL; + if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) + return -EINVAL; + + mutex_lock(&direct_mutex); + mutex_lock(&ftrace_lock); + + /* + * Shutdown the ops, change 'direct' pointer for each + * ops entry in direct_functions hash and startup the + * ops back again. + */ + err = ftrace_shutdown(ops, 0); + if (err) + goto out_unlock; + + size = 1 << hash->size_bits; + for (i = 0; i < size; i++) { + hlist_for_each_entry(iter, &hash->buckets[i], hlist) { + entry = __ftrace_lookup_ip(direct_functions, iter->ip); + if (!entry) + continue; + entry->direct = addr; + } + } + + err = ftrace_startup(ops, 0); + + out_unlock: + mutex_unlock(&ftrace_lock); + mutex_unlock(&direct_mutex); + return err; +} +EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi); #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** From patchwork Thu Aug 26 19:39:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73727C432BE for ; Thu, 26 Aug 2021 19:40:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D406610CB for ; Thu, 26 Aug 2021 19:40:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243545AbhHZTlP (ORCPT ); Thu, 26 Aug 2021 15:41:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51890 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243536AbhHZTlK (ORCPT ); Thu, 26 Aug 2021 15:41:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v00APJ6meHaZsL60TP0qAulcqbDujycB2Ey6wbdiEio=; b=ZQM3UD9L8j7OkZnunNNDNHYNbKPoQYjffFQuASNDjH8zfmHjGz5nZ3K5rXSW/VAVkZJso5 JLQK5fxEtueue28VdCunygKIATzh28+c0UrY/lBoYLzOqVZyc685baDbqA/Issp7kvBlAs fOeZrFjBZXVRtnLs4wJDjzYz8C/eTe8= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-2-LO_VR75RMy-XfRG0WFxP5A-1; Thu, 26 Aug 2021 15:40:20 -0400 X-MC-Unique: LO_VR75RMy-XfRG0WFxP5A-1 Received: by mail-wm1-f71.google.com with SMTP id z186-20020a1c7ec30000b02902e6a27a9962so4767817wmc.3 for ; Thu, 26 Aug 2021 12:40:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v00APJ6meHaZsL60TP0qAulcqbDujycB2Ey6wbdiEio=; b=ID6bEZQqIn22yea1ZV5tzsiicfD3aMJZvTb8mLpIR1gwba9E915bfbnpfJbj5cQqAI /2AYTjGQauCGRj7uvjhBvM17163DSCwwxwqmkm3eBomtEZi/ztz6W1VR8uIyhm/Aqd2e qEpUFFP8kSca5cKSbaxdaPGqvGnAfLRpcEbiSuDmy2dcYnVtlmE3fDdzK+6KXKppjjZg nhba6m85nagZe1orJXIGwo4Ov9PrsVAsuBvp88NQECCrjk/2JDlWMbozthO6gUYaIm/f bRG1YU9/mPnFJtiPIwiGuSPMKWxXATod6b0573O/zqn5RtDrgHM9I2oTNG4NuaEaLcta SJrw== X-Gm-Message-State: AOAM5332T3fMj7n4JwwToJKGD8B6eFhblD9/xVD4ub2NlH6zNqBD51ub pmThiv5WxrwsOR/WNJSIN6cQIRAEF5b0fyQpWeakEdNSUzjr1t1cIDMXgxmORG3TrhIsTJdTx08 sqk2oqAtY90t3aK7l X-Received: by 2002:a05:600c:154e:: with SMTP id f14mr16035638wmg.162.1630006819453; Thu, 26 Aug 2021 12:40:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxMzN3kbJMLDo29fhXdu4MghVFwWLrCtOYz0OBBkS+zzgjXQaVK4AOPIsKHCwBqu0N6kFRmgw== X-Received: by 2002:a05:600c:154e:: with SMTP id f14mr16035614wmg.162.1630006819193; Thu, 26 Aug 2021 12:40:19 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id n14sm4016308wrx.10.2021.08.26.12.40.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:40:18 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 09/27] bpf: Add support to load multi func tracing program Date: Thu, 26 Aug 2021 21:39:04 +0200 Message-Id: <20210826193922.66204-10-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to load tracing program with new BPF_F_MULTI_FUNC flag, that allows the program to be loaded without specific function to be attached to. Such program will be allowed to be attached to multiple functions in following patches. Signed-off-by: Jiri Olsa --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 7 +++++++ kernel/bpf/syscall.c | 35 +++++++++++++++++++++++++++++----- kernel/bpf/verifier.c | 3 ++- tools/include/uapi/linux/bpf.h | 7 +++++++ 5 files changed, 47 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4c16f19f83e..dc9838d741ac 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -865,6 +865,7 @@ struct bpf_prog_aux { bool func_proto_unreliable; bool sleepable; bool tail_call_reachable; + bool multi_func; struct hlist_node tramp_hlist; /* BTF_KIND_FUNC_PROTO for valid attach_btf_id */ const struct btf_type *attach_func_proto; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 791f31dd0abe..1f9d336861f0 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1110,6 +1110,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 4e50c0bfdb7d..fa3f93c423d8 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -31,6 +31,7 @@ #include #include #include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -2012,7 +2013,8 @@ static int bpf_prog_load_check_attach(enum bpf_prog_type prog_type, enum bpf_attach_type expected_attach_type, struct btf *attach_btf, u32 btf_id, - struct bpf_prog *dst_prog) + struct bpf_prog *dst_prog, + bool multi_func) { if (btf_id) { if (btf_id > BTF_MAX_TYPE) @@ -2032,6 +2034,14 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type, } } + if (multi_func) { + if (prog_type != BPF_PROG_TYPE_TRACING) + return -EINVAL; + if (!attach_btf || btf_id) + return -EINVAL; + return 0; + } + if (attach_btf && (!btf_id || dst_prog)) return -EINVAL; @@ -2155,6 +2165,16 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type) } } +#define DEFINE_BPF_MULTI_FUNC(args...) \ + extern int bpf_multi_func(args); \ + int __init bpf_multi_func(args) { return 0; } + +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, + unsigned long a5, unsigned long a6) + +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) + /* last field in 'union bpf_attr' used by this command */ #define BPF_PROG_LOAD_LAST_FIELD fd_array @@ -2165,6 +2185,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) struct btf *attach_btf = NULL; int err; char license[128]; + bool multi_func; bool is_gpl; if (CHECK_ATTR(BPF_PROG_LOAD)) @@ -2174,7 +2195,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) BPF_F_ANY_ALIGNMENT | BPF_F_TEST_STATE_FREQ | BPF_F_SLEEPABLE | - BPF_F_TEST_RND_HI32)) + BPF_F_TEST_RND_HI32 | + BPF_F_MULTI_FUNC)) return -EINVAL; if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && @@ -2205,6 +2227,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) if (is_perfmon_prog_type(type) && !perfmon_capable()) return -EPERM; + multi_func = attr->prog_flags & BPF_F_MULTI_FUNC; + /* attach_prog_fd/attach_btf_obj_fd can specify fd of either bpf_prog * or btf, we need to check which one it is */ @@ -2223,7 +2247,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) return -ENOTSUPP; } } - } else if (attr->attach_btf_id) { + } else if (attr->attach_btf_id || multi_func) { /* fall back to vmlinux BTF, if BTF type ID is specified */ attach_btf = bpf_get_btf_vmlinux(); if (IS_ERR(attach_btf)) @@ -2236,7 +2260,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) bpf_prog_load_fixup_attach_type(attr); if (bpf_prog_load_check_attach(type, attr->expected_attach_type, attach_btf, attr->attach_btf_id, - dst_prog)) { + dst_prog, multi_func)) { if (dst_prog) bpf_prog_put(dst_prog); if (attach_btf) @@ -2256,10 +2280,11 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) prog->expected_attach_type = attr->expected_attach_type; prog->aux->attach_btf = attach_btf; - prog->aux->attach_btf_id = attr->attach_btf_id; + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; prog->aux->dst_prog = dst_prog; prog->aux->offload_requested = !!attr->prog_ifindex; prog->aux->sleepable = attr->prog_flags & BPF_F_SLEEPABLE; + prog->aux->multi_func = multi_func; err = security_bpf_prog_alloc(prog->aux); if (err) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 206c221453cf..e9e84adfb974 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13628,7 +13628,8 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) if (!bpf_iter_prog_supported(prog)) return -EINVAL; return 0; - } + } else if (prog->aux->multi_func) + return prog->type == BPF_PROG_TYPE_TRACING ? 0 : -EINVAL; if (prog->type == BPF_PROG_TYPE_LSM) { ret = bpf_lsm_verify_prog(&env->log, prog); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 791f31dd0abe..1f9d336861f0 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1110,6 +1110,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * From patchwork Thu Aug 26 19:39:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 479CFC4320A for ; Thu, 26 Aug 2021 19:40:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 29B9160F4C for ; Thu, 26 Aug 2021 19:40:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243572AbhHZTlm (ORCPT ); Thu, 26 Aug 2021 15:41:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22924 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243561AbhHZTlW (ORCPT ); Thu, 26 Aug 2021 15:41:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSIDNuxSNboWGja6JJJ//OQqFnAgGxrRV3fj8oXduYs=; b=JAZ2tTRDSxMLX26HYqGRw7tJDdH6QN0amlb2gbT2+79JIPbkxUtd2SIZ64dsYsy3gOEf4E 8dJyDHC9XOW5tLASQ15rMxq0brZl79Fo0Ex/N3J9cNJe3MQprvUhGzeBz+yL8/NghBMOL9 6X/bfdBg1KL5t2wBmz9bsVV3ot0tZU8= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-542--qtjpH4tNIadqKuXAcrDfQ-1; Thu, 26 Aug 2021 15:40:33 -0400 X-MC-Unique: -qtjpH4tNIadqKuXAcrDfQ-1 Received: by mail-wm1-f69.google.com with SMTP id m22-20020a7bca56000000b002e7508f3faeso1273952wml.2 for ; Thu, 26 Aug 2021 12:40:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PSIDNuxSNboWGja6JJJ//OQqFnAgGxrRV3fj8oXduYs=; b=qKBJpTaZtB3iXVxAqoXs2jUHTCvmQg+G/+c7ofITeEW6AUcjsfomj8h3FONUTfD1yZ ruiI6TRA6WVPmjc1eJpPmWXxIjZpc+ivO/TCXoif65hvUFQaIpNKWrJoek4jCzYWc3QR qheYdesXteh5dKFU9JNb3Hu0Pi+7TlyegociIV425IzTa0W0gwR9UF5nEVUaoDRN6Xem mpWLCnhWSGNIvGHKcGB8BaZKYHJ5NC3C46XIyrmQur552/wJafIyQBY+OiTxOzU1o797 QlqIsAR9tzjUXGJYkmAG2JDm7onCm1iVrUekaBRbIeGg+cbNOMvc8pOyCF3N5Q5lCpxx a5dg== X-Gm-Message-State: AOAM5303WOM+DfowhYcjcNwHYeGTEKDjzek7YM0u5BynOuPUOVM3v5G6 dDpHPusf+7fApRdj6HMcswMtfZfbwqKOvMQKO6qoUGJodoqVB9Jtwl3SXl8UQBo6jH3z7MNkgl+ ll/M2wyoft9S4Z26E X-Received: by 2002:a5d:4d03:: with SMTP id z3mr6008082wrt.229.1630006832019; Thu, 26 Aug 2021 12:40:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyuQjRMB3r13599nxIcsa8dPQw9RicOFY+s2ocnwBE1cxvcGUhEESYSLd5EulO1V4gbP89M8g== X-Received: by 2002:a5d:4d03:: with SMTP id z3mr6008069wrt.229.1630006831880; Thu, 26 Aug 2021 12:40:31 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id g11sm4016811wrx.30.2021.08.26.12.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:40:31 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 11/27] bpf: Factor out bpf_trampoline_init function Date: Thu, 26 Aug 2021 21:39:06 +0200 Message-Id: <20210826193922.66204-12-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Separating out bpf_trampoline_init function, so it can be used from other places in following patches. Signed-off-by: Jiri Olsa --- kernel/bpf/trampoline.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 525fa74c2f62..f44899c9698a 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -58,11 +58,22 @@ void bpf_image_ksym_del(struct bpf_ksym *ksym) PAGE_SIZE, true, ksym->name); } +static void bpf_trampoline_init(struct bpf_trampoline *tr, u64 key) +{ + int i; + + tr->key = key; + INIT_HLIST_NODE(&tr->hlist); + refcount_set(&tr->refcnt, 1); + mutex_init(&tr->mutex); + for (i = 0; i < BPF_TRAMP_MAX; i++) + INIT_HLIST_HEAD(&tr->progs_hlist[i]); +} + static struct bpf_trampoline *bpf_trampoline_lookup(u64 key) { struct bpf_trampoline *tr; struct hlist_head *head; - int i; mutex_lock(&trampoline_mutex); head = &trampoline_table[hash_64(key, TRAMPOLINE_HASH_BITS)]; @@ -75,14 +86,8 @@ static struct bpf_trampoline *bpf_trampoline_lookup(u64 key) tr = kzalloc(sizeof(*tr), GFP_KERNEL); if (!tr) goto out; - - tr->key = key; - INIT_HLIST_NODE(&tr->hlist); + bpf_trampoline_init(tr, key); hlist_add_head(&tr->hlist, head); - refcount_set(&tr->refcnt, 1); - mutex_init(&tr->mutex); - for (i = 0; i < BPF_TRAMP_MAX; i++) - INIT_HLIST_HEAD(&tr->progs_hlist[i]); out: mutex_unlock(&trampoline_mutex); return tr; From patchwork Thu Aug 26 19:39:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0C6FC432BE for ; Thu, 26 Aug 2021 19:41:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A989660F4C for ; Thu, 26 Aug 2021 19:41:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243587AbhHZTlq (ORCPT ); Thu, 26 Aug 2021 15:41:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44538 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243538AbhHZTlf (ORCPT ); Thu, 26 Aug 2021 15:41:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006847; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z2XITBrOxDpGRyjAJ/GeVEmFFxm+Vz1j/C47GlqXHzU=; b=Lw1A6wOP8nC5CRP2xAJw+SRdOfhZH3wJmGw1aDOk/5Pxz3xEiidK3hAwBHwDEoioGXLkjO WMfE9FrNiwMHl0CgUp/koZf3/4amyvrGptmW8sIUXLTtmLfwsf7nI0t/dbAPJSc3rN/8Gg BWmNpzBzpuicFcae2JrWA068olSsLR4= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-248-gwJ_cUiSO9mf6jR66IYLGw-1; Thu, 26 Aug 2021 15:40:45 -0400 X-MC-Unique: gwJ_cUiSO9mf6jR66IYLGw-1 Received: by mail-wm1-f70.google.com with SMTP id z186-20020a1c7ec30000b02902e6a27a9962so4768274wmc.3 for ; Thu, 26 Aug 2021 12:40:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z2XITBrOxDpGRyjAJ/GeVEmFFxm+Vz1j/C47GlqXHzU=; b=WkAh7VPiiPpztWU2PO8I8P8wTlFdKzMqrl//ISRWuxqg5/YuFcpyYgm77Rcncwkya+ jBWjz50oVVcdUhNygeUbkmSjYuKmw69TRsXw+zS1wsp6dUHS1kNLp4cmgqSuiW4y0S++ 980aur/SIfNbScRLF3AFZyx2iHJOcbmz2XUQi/XSZ/xPKCjvROBDFEVYACfl04y2UI9R gvwMrTGRq6236Ie882z5eimAlNDMQ+KKl2aW5bScON8IjoDIb1pcLIJyRxK5k0NEtg3s Tix8PrjzKZ4FSik6ucc7PusKsb779aXwP4OFWcm5IHORVB1DmnXMqwG4odPYawhUgDLR ZgmQ== X-Gm-Message-State: AOAM530HmKux7ZlENRbpV9nffutdgkKoMhbwk2pTLtHUW6fqW+teNwCL 1YjKS72LpI3Tc8xH9tO5LYUuTj/8Z4GJsVpk4x00mVzAelVUnCH3UsXlBZISixkFXSLgVJot34H lMA3ZJY4bUmjzU0UC X-Received: by 2002:a1c:ed10:: with SMTP id l16mr15751505wmh.8.1630006844418; Thu, 26 Aug 2021 12:40:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzitQXnasWLsuJudMdZeXK+TS2WTVT2UhHDl/35tfbJIVjoTYJHVpOxEFwQ00OumMXLgAsD+g== X-Received: by 2002:a1c:ed10:: with SMTP id l16mr15751486wmh.8.1630006844263; Thu, 26 Aug 2021 12:40:44 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id f7sm9136006wmh.20.2021.08.26.12.40.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:40:43 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 13/27] bpf: Factor out __bpf_trampoline_put function Date: Thu, 26 Aug 2021 21:39:08 +0200 Message-Id: <20210826193922.66204-14-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Separating out __bpf_trampoline_put function, so it can be used from other places in following patches. Signed-off-by: Jiri Olsa --- kernel/bpf/trampoline.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 6dba43266e0b..8aa0aca38b3a 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -522,18 +522,16 @@ struct bpf_trampoline *bpf_trampoline_get(u64 key, return tr; } -void bpf_trampoline_put(struct bpf_trampoline *tr) +static void __bpf_trampoline_put(struct bpf_trampoline *tr) { - if (!tr) - return; - mutex_lock(&trampoline_mutex); + lockdep_assert_held(&trampoline_mutex); if (!refcount_dec_and_test(&tr->refcnt)) - goto out; + return; WARN_ON_ONCE(mutex_is_locked(&tr->mutex)); if (WARN_ON_ONCE(!hlist_empty(&tr->progs_hlist[BPF_TRAMP_FENTRY]))) - goto out; + return; if (WARN_ON_ONCE(!hlist_empty(&tr->progs_hlist[BPF_TRAMP_FEXIT]))) - goto out; + return; /* This code will be executed even when the last bpf_tramp_image * is alive. All progs are detached from the trampoline and the * trampoline image is patched with jmp into epilogue to skip @@ -542,7 +540,14 @@ void bpf_trampoline_put(struct bpf_trampoline *tr) */ hlist_del(&tr->hlist); kfree(tr); -out: +} + +void bpf_trampoline_put(struct bpf_trampoline *tr) +{ + if (!tr) + return; + mutex_lock(&trampoline_mutex); + __bpf_trampoline_put(tr); mutex_unlock(&trampoline_mutex); } From patchwork Thu Aug 26 19:39:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D28CAC4320E for ; Thu, 26 Aug 2021 19:41:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BDDAA60F11 for ; Thu, 26 Aug 2021 19:41:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243593AbhHZTl5 (ORCPT ); Thu, 26 Aug 2021 15:41:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35321 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243605AbhHZTlt (ORCPT ); Thu, 26 Aug 2021 15:41:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aB69aifL1zI8vrW5XNajMWuHmZ5MVDONIdA/rCm6EsU=; b=azZvIdztmZcWTeF9AInXVzFb3l/RSj5oe4Zk7b4yasehrXT75eKzRDl6AqGayuZs3lyvW2 sbA9Igakf3ePmVnMY/XawaANoOU+tP4nYZ5kWLTQDaSw2VfbjObBbfa7fkzNQAr+I9gKVO 2AfK5BHtc2zBADGIzwKxQKYV1DQb1Z0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-593-zW3QaOpUNLe6yOYF1HzUtw-1; Thu, 26 Aug 2021 15:40:59 -0400 X-MC-Unique: zW3QaOpUNLe6yOYF1HzUtw-1 Received: by mail-wr1-f71.google.com with SMTP id r11-20020a5d4e4b000000b001575c5ed4b4so1206744wrt.4 for ; Thu, 26 Aug 2021 12:40:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aB69aifL1zI8vrW5XNajMWuHmZ5MVDONIdA/rCm6EsU=; b=IdiOHa5VbwLrwIftw2w6dgtlBPSZEcsakfjDZaeXyLnOFjf9rRROnrpaPPESEfBvi9 ktoodgVpI5oAxKpttJBC5mIskR0dbEM0x5QeY42k88ejhlTJhr3IRLlOSpmYLpWNsXFX wUhbxodTzzeu6+gWWMepv1FSuzrgvnSPxor0hqX8LQCyyzTkNsmSJacAa4A3zIeqoetv 0saudLBGLvzcR4v7yYz/SEu39UvCjP4cTTz4+/xyXajz221GxYlfctW8s1DgO28Tch56 sYmpQ2MwWg1wufP9Ib94c54Ci1xY9OGm11kUOlNsVzpPXuZAPvuQn/q5Fe18nYIjOMaB 67rw== X-Gm-Message-State: AOAM53285QhOtbnEZNqwAle2jpz5Gw+s4+eZj1a2LoO9gyI0c0/5Hly7 BdYxYKekVuKkIbdAOjBgpY0R6uxJGgyx4jOpHaNi39B0FeOy0ONQA5ZEa81sdEAGq8IWkrVXdMQ UFYORlb9gHJX7wY2k X-Received: by 2002:a1c:27c2:: with SMTP id n185mr5120212wmn.20.1630006856789; Thu, 26 Aug 2021 12:40:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJximqozMr0/Jm9W8qaJJ+TOp3iu2uH+xdipTy7sLT+cli5O3p4fnfXEZvgGgFYa01EeiwtxAQ== X-Received: by 2002:a1c:27c2:: with SMTP id n185mr5120188wmn.20.1630006856586; Thu, 26 Aug 2021 12:40:56 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id x21sm2507342wmi.15.2021.08.26.12.40.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:40:56 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 15/27] bpf, x64: Allow to use caller address from stack Date: Thu, 26 Aug 2021 21:39:10 +0200 Message-Id: <20210826193922.66204-16-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Currently we call the original function by using the absolute address given at the JIT generation. That's not usable when having trampoline attached to multiple functions. In this case we need to take the return address from the stack. Adding support to retrieve the original function address from the stack by adding new BPF_TRAMP_F_ORIG_STACK flag for arch_prepare_bpf_trampoline function. Basically we take the return address of the 'fentry' call: function + 0: call fentry # stores 'function + 5' address on stack function + 5: ... The 'function + 5' address will be used as the address for the original function to call. Signed-off-by: Jiri Olsa --- arch/x86/net/bpf_jit_comp.c | 13 +++++++++---- include/linux/bpf.h | 5 +++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 0fe6aacef3db..9f31197780ae 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -2024,10 +2024,15 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (flags & BPF_TRAMP_F_CALL_ORIG) { restore_regs(m, &prog, nr_args, stack_size); - /* call original function */ - if (emit_call(&prog, orig_call, prog)) { - ret = -EINVAL; - goto cleanup; + if (flags & BPF_TRAMP_F_ORIG_STACK) { + emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); + EMIT2(0xff, 0xd0); /* call *rax */ + } else { + /* call original function */ + if (emit_call(&prog, orig_call, prog)) { + ret = -EINVAL; + goto cleanup; + } } /* remember return value in a stack for bpf prog to access */ emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8); diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f0f548f8f391..a5c3307d49c6 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -584,6 +584,11 @@ struct btf_func_model { */ #define BPF_TRAMP_F_IP_ARG BIT(3) +/* Get original function from stack instead of from provided direct address. + * Makes sense for fexit programs only. + */ +#define BPF_TRAMP_F_ORIG_STACK BIT(4) + /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. Pick a number to fit into BPF_IMAGE_SIZE / 2 */ From patchwork Thu Aug 26 19:39:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7629C4320A for ; Thu, 26 Aug 2021 19:41:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 934E460F4C for ; Thu, 26 Aug 2021 19:41:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243609AbhHZTmH (ORCPT ); Thu, 26 Aug 2021 15:42:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:40104 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243607AbhHZTmA (ORCPT ); Thu, 26 Aug 2021 15:42:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006872; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k7Y1Ok4cyox0DsYgmFPhX/i9rcG3AZ5Q61TPBm3wVYA=; b=bGk3jkt8mcTUrsG/oL/7yHE0xkKK9P1/wjvKMEOZKmNmluwDvQCNkkKZhmsxwCndTslaD+ qOdOqkBLG8I1hp8l20FyF8d0/U27DHne4TIvh8OcYBFELxxai5wZmWl1Tq/99cm1jzKlMK 1SgvBb1Fu9mUI02BGkwRTUcDO8aCBDs= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-508-9FLyMrY9PDa_1dKXRNRNQA-1; Thu, 26 Aug 2021 15:41:11 -0400 X-MC-Unique: 9FLyMrY9PDa_1dKXRNRNQA-1 Received: by mail-wr1-f69.google.com with SMTP id h14-20020a056000000e00b001575b00eb08so1184837wrx.13 for ; Thu, 26 Aug 2021 12:41:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k7Y1Ok4cyox0DsYgmFPhX/i9rcG3AZ5Q61TPBm3wVYA=; b=hOXpr/wK3QICC6l05pSYppLYe3iOaQy0Ub7JFaQJ2Cr4ADAfuRQt5uXr+W4f/jVagP LHbfQaHwWXyv+iyyisQOrAGywanaFrdH9LvIB2AX5S0CIiWKYK9LIkiotpxnj4Lj2wJQ FM3ckVMRU2pKJj7uK9LFPy+U2IzJ5sde4gQ3wRtD7Yuu3n6H1GRp3S+7zFNJVIAg61LW 8K4cPNBP35hko2yP1GXz9LhURpirhIMrqE4ih34k8IiXn4ULukCtdfgXlPoifr0y55Gu xCdBR8PoEbecrEqanXFWkd5KhwJj5A60Wm6zYtGvDSShmrj9Ag5o3sFaJ+JmbSJvGpTm fcEg== X-Gm-Message-State: AOAM5303sclmSTv+H/JhCv1+c544e2w6I2IsWJxwHs9ApqqNpgLHniPW 5d+RJuYxwx9L/sNLA7Gdqe1LFQ/h9XvwdSqvNr0lCWJsv/ygSZIuk3FfVK9RbkVE9yPeYBWTNnx zMri/uaIt+YUw0RIF X-Received: by 2002:adf:fb8f:: with SMTP id a15mr6055948wrr.92.1630006869551; Thu, 26 Aug 2021 12:41:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJykopySwjKrdyJqFYHgjK0fph8uXk7OdmrjR6/62gjo9cD7ouKp4mlKRx/OXYmQv49Cpaib6w== X-Received: by 2002:adf:fb8f:: with SMTP id a15mr6055922wrr.92.1630006869317; Thu, 26 Aug 2021 12:41:09 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id p9sm9445594wmq.40.2021.08.26.12.41.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:41:09 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 17/27] bpf: Add multi trampoline attach support Date: Thu, 26 Aug 2021 21:39:12 +0200 Message-Id: <20210826193922.66204-18-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding new multi trampoline link (BPF_LINK_TYPE_TRACING_MULTI) as an interface to attach program to multiple functions. The link_create bpf_attr interface already has 'bpf_prog' file descriptor, that defines the program to be attached. It must be loaded with BPF_F_MULTI_FUNC flag. Adding new multi_btf_ids/multi_btf_ids_cnt link_create bpf_attr fields that provides BTF ids. The new link gets multi trampoline (via bpf_trampoline_multi_get) and links the provided program with embedded trampolines and the 'main' trampoline with new multi link/unlink functions: int bpf_trampoline_multi_link_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *tr); int bpf_trampoline_multi_unlink_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *tr); If embedded trampoline contains fexit programs, we need to switch its model to the multi trampoline model (because of the final 'ret' argument). We keep the count of attached multi func programs for each trampoline, so we can tell when to switch the model. Signed-off-by: Jiri Olsa --- include/linux/bpf.h | 5 ++ include/uapi/linux/bpf.h | 5 ++ kernel/bpf/core.c | 1 + kernel/bpf/syscall.c | 120 +++++++++++++++++++++++++++++++++ kernel/bpf/trampoline.c | 87 ++++++++++++++++++++++-- tools/include/uapi/linux/bpf.h | 5 ++ 6 files changed, 219 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 678b9cd2fa21..3ce4656e2057 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -693,6 +693,7 @@ struct bpf_trampoline { struct module *mod; struct { struct bpf_trampoline *tr; + int count; } multi; }; @@ -747,6 +748,8 @@ void bpf_trampoline_put(struct bpf_trampoline *tr); struct bpf_trampoline_multi *bpf_trampoline_multi_get(struct bpf_prog *prog, u32 *ids, u32 ids_cnt); void bpf_trampoline_multi_put(struct bpf_trampoline_multi *multi); +int bpf_trampoline_multi_link_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *tr); +int bpf_trampoline_multi_unlink_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *tr); #define BPF_DISPATCHER_INIT(_name) { \ .mutex = __MUTEX_INITIALIZER(_name.mutex), \ .func = &_name##_func, \ @@ -892,6 +895,8 @@ struct bpf_prog_aux { bool tail_call_reachable; bool multi_func; struct bpf_tramp_node tramp_node; + struct bpf_tramp_node *multi_node; + struct mutex multi_node_mutex; /* BTF_KIND_FUNC_PROTO for valid attach_btf_id */ const struct btf_type *attach_func_proto; /* function name for valid attach_btf_id */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 1f9d336861f0..9533200ffadf 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1008,6 +1008,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_TRACING_MULTI = 8, MAX_BPF_LINK_TYPE, }; @@ -1462,6 +1463,10 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 multi_btf_ids; /* addresses to attach */ + __u32 multi_btf_ids_cnt; /* addresses count */ + }; }; } link_create; diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index bad03dde97a2..6c16ac43dd91 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -109,6 +109,7 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag INIT_LIST_HEAD_RCU(&fp->aux->ksym.lnode); mutex_init(&fp->aux->used_maps_mutex); mutex_init(&fp->aux->dst_mutex); + mutex_init(&fp->aux->multi_node_mutex); return fp; } diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 537687664bdf..8f1f934a8f26 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -32,6 +32,7 @@ #include #include #include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -2851,6 +2852,121 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog, return err; } +struct bpf_tracing_multi_link { + struct bpf_link link; + enum bpf_attach_type attach_type; + struct bpf_trampoline_multi *multi; +}; + +static void bpf_tracing_multi_link_release(struct bpf_link *link) +{ + struct bpf_tracing_multi_link *tr_link = + container_of(link, struct bpf_tracing_multi_link, link); + + bpf_trampoline_multi_unlink_prog(link->prog, tr_link->multi); +} + +static void bpf_tracing_multi_link_dealloc(struct bpf_link *link) +{ + struct bpf_tracing_multi_link *tr_link = + container_of(link, struct bpf_tracing_multi_link, link); + + bpf_trampoline_multi_put(tr_link->multi); + kfree(tr_link); +} + +static void bpf_tracing_multi_link_show_fdinfo(const struct bpf_link *link, + struct seq_file *seq) +{ + struct bpf_tracing_multi_link *tr_link = + container_of(link, struct bpf_tracing_multi_link, link); + + seq_printf(seq, "attach_type:\t%d\n", tr_link->attach_type); +} + +static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, + struct bpf_link_info *info) +{ + struct bpf_tracing_multi_link *tr_link = + container_of(link, struct bpf_tracing_multi_link, link); + + info->tracing.attach_type = tr_link->attach_type; + return 0; +} + +static int check_multi_prog_type(struct bpf_prog *prog) +{ + if (prog->expected_attach_type != BPF_TRACE_FENTRY && + prog->expected_attach_type != BPF_TRACE_FEXIT) + return -EINVAL; + return 0; +} + +static const struct bpf_link_ops bpf_tracing_multi_link_lops = { + .release = bpf_tracing_multi_link_release, + .dealloc = bpf_tracing_multi_link_dealloc, + .show_fdinfo = bpf_tracing_multi_link_show_fdinfo, + .fill_link_info = bpf_tracing_multi_link_fill_link_info, +}; + +static int bpf_tracing_multi_attach(struct bpf_prog *prog, + const union bpf_attr *attr) +{ + void __user *ubtf_ids = u64_to_user_ptr(attr->link_create.multi_btf_ids); + u32 size, cnt = attr->link_create.multi_btf_ids_cnt; + struct bpf_tracing_multi_link *link = NULL; + struct bpf_link_primer link_primer; + struct bpf_trampoline_multi *multi = NULL; + int err = -EINVAL; + u32 *btf_ids; + + if (check_multi_prog_type(prog)) + return -EINVAL; + if (!cnt) + return -EINVAL; + + size = cnt * sizeof(*btf_ids); + btf_ids = kmalloc(size, GFP_USER | __GFP_NOWARN); + if (!btf_ids) + return -ENOMEM; + + err = -EFAULT; + if (ubtf_ids && copy_from_user(btf_ids, ubtf_ids, size)) + goto out_free_ids; + + multi = bpf_trampoline_multi_get(prog, btf_ids, cnt); + if (IS_ERR(multi)) { + err = PTR_ERR(multi); + goto out_free_ids; + } + + link = kzalloc(sizeof(*link), GFP_USER); + if (!link) { + err = -ENOMEM; + goto out_free; + } + + bpf_link_init(&link->link, BPF_LINK_TYPE_TRACING_MULTI, + &bpf_tracing_multi_link_lops, prog); + link->attach_type = prog->expected_attach_type; + link->multi = multi; + + err = bpf_link_prime(&link->link, &link_primer); + if (err) + goto out_free; + err = bpf_trampoline_multi_link_prog(prog, multi); + if (err) + goto out_free; + return bpf_link_settle(&link_primer); + +out_free: + bpf_trampoline_multi_put(multi); + kfree(link); +out_free_ids: + kfree(btf_ids); + return err; +} + struct bpf_raw_tp_link { struct bpf_link link; struct bpf_raw_event_map *btp; @@ -3157,6 +3273,8 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type) case BPF_CGROUP_SETSOCKOPT: return BPF_PROG_TYPE_CGROUP_SOCKOPT; case BPF_TRACE_ITER: + case BPF_TRACE_FENTRY: + case BPF_TRACE_FEXIT: return BPF_PROG_TYPE_TRACING; case BPF_SK_LOOKUP: return BPF_PROG_TYPE_SK_LOOKUP; @@ -4213,6 +4331,8 @@ static int tracing_bpf_link_attach(const union bpf_attr *attr, bpfptr_t uattr, if (prog->expected_attach_type == BPF_TRACE_ITER) return bpf_iter_link_attach(attr, uattr, prog); + else if (prog->aux->multi_func) + return bpf_tracing_multi_attach(prog, attr); else if (prog->type == BPF_PROG_TYPE_EXT) return bpf_tracing_prog_attach(prog, attr->link_create.target_fd, diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index d66b76c23d74..6ff5c2512f91 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -30,6 +30,11 @@ static LIST_HEAD(trampoline_multi); /* serializes access to trampoline_table */ static DEFINE_MUTEX(trampoline_mutex); +static bool is_multi_trampoline(struct bpf_trampoline *tr) +{ + return tr->key == 0; +} + void *bpf_jit_alloc_exec_page(void) { void *image; @@ -384,8 +389,21 @@ static struct bpf_tramp_image *bpf_tramp_image_alloc(u64 key, u32 idx) return ERR_PTR(err); } +static bool needs_multi_model(struct bpf_trampoline *tr, struct btf_func_model *new) +{ + struct bpf_trampoline *multi = tr->multi.tr; + + if (!tr->multi.count || !multi) + return false; + if (tr->func.model.nr_args >= multi->func.model.nr_args) + return false; + memcpy(new, &multi->func.model, sizeof(*new)); + return true; +} + static int bpf_trampoline_update(struct bpf_trampoline *tr) { + struct btf_func_model model_multi, *model = &tr->func.model; struct bpf_tramp_image *im; struct bpf_tramp_progs *tprogs; u32 flags = BPF_TRAMP_F_RESTORE_REGS; @@ -411,15 +429,19 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr) } if (tprogs[BPF_TRAMP_FEXIT].nr_progs || - tprogs[BPF_TRAMP_MODIFY_RETURN].nr_progs) + tprogs[BPF_TRAMP_MODIFY_RETURN].nr_progs) { flags = BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_SKIP_FRAME; + if (is_multi_trampoline(tr)) + flags |= BPF_TRAMP_F_ORIG_STACK; + if (needs_multi_model(tr, &model_multi)) + model = &model_multi; + } if (ip_arg) flags |= BPF_TRAMP_F_IP_ARG; err = arch_prepare_bpf_trampoline(im, im->image, im->image + PAGE_SIZE, - &tr->func.model, flags, tprogs, - tr->func.addr); + model, flags, tprogs, tr->func.addr); if (err < 0) goto out; @@ -507,7 +529,8 @@ int bpf_trampoline_link_prog(struct bpf_tramp_node *node, struct bpf_trampoline if (err) { hlist_del_init(&node->hlist); tr->progs_cnt[kind]--; - } + } else if (prog->aux->multi_func) + tr->multi.count++; out: mutex_unlock(&tr->mutex); return err; @@ -531,6 +554,8 @@ int bpf_trampoline_unlink_prog(struct bpf_tramp_node *node, struct bpf_trampolin } hlist_del_init(&node->hlist); tr->progs_cnt[kind]--; + if (prog->aux->multi_func) + tr->multi.count--; err = bpf_trampoline_update(tr); out: mutex_unlock(&tr->mutex); @@ -732,6 +757,60 @@ void bpf_trampoline_multi_put(struct bpf_trampoline_multi *multi) mutex_unlock(&trampoline_mutex); } +int bpf_trampoline_multi_link_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *multi) +{ + struct bpf_tramp_node *multi_node = NULL; + int i, j, err = 0; + + multi_node = kzalloc(sizeof(*multi_node) * multi->tr_cnt, GFP_KERNEL); + if (!multi_node) + return -ENOMEM; + + mutex_lock(&prog->aux->multi_node_mutex); + if (prog->aux->multi_node) + err = -EBUSY; + else + prog->aux->multi_node = multi_node; + mutex_unlock(&prog->aux->multi_node_mutex); + if (err) + goto out_free; + + for (i = 0; i < multi->tr_cnt; i++) { + multi_node[i].prog = prog; + err = bpf_trampoline_link_prog(&multi_node[i], multi->tr[i]); + if (err) + goto out_unlink; + } + + err = bpf_trampoline_link_prog(&prog->aux->tramp_node, &multi->main); + if (!err) + return 0; + +out_unlink: + for (j = 0; j < i; j++) + WARN_ON_ONCE(bpf_trampoline_unlink_prog(&multi_node[j], multi->tr[j])); + +out_free: + kfree(multi_node); + return err; +} + +int bpf_trampoline_multi_unlink_prog(struct bpf_prog *prog, struct bpf_trampoline_multi *multi) +{ + struct bpf_tramp_node *multi_node = prog->aux->multi_node; + int i; + + for (i = 0; i < multi->tr_cnt; i++) + WARN_ON_ONCE(bpf_trampoline_unlink_prog(&multi_node[i], multi->tr[i])); + + mutex_lock(&prog->aux->multi_node_mutex); + prog->aux->multi_node = NULL; + mutex_unlock(&prog->aux->multi_node_mutex); + + kfree(multi_node); + return bpf_trampoline_unlink_prog(&prog->aux->tramp_node, &multi->main); +} + #define NO_START_TIME 1 static u64 notrace bpf_prog_start_time(void) { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 1f9d336861f0..9533200ffadf 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1008,6 +1008,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_TRACING_MULTI = 8, MAX_BPF_LINK_TYPE, }; @@ -1462,6 +1463,10 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 multi_btf_ids; /* addresses to attach */ + __u32 multi_btf_ids_cnt; /* addresses count */ + }; }; } link_create; From patchwork Thu Aug 26 19:39:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74828C4320A for ; Thu, 26 Aug 2021 19:41:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60BB060F4C for ; Thu, 26 Aug 2021 19:41:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243611AbhHZTmP (ORCPT ); Thu, 26 Aug 2021 15:42:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42721 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243574AbhHZTmN (ORCPT ); Thu, 26 Aug 2021 15:42:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/wdRRB3BbU0zTebKEzNMBHGnZzjSnqwp9yN08+1iPjU=; b=XTCAJin1neNHFLzJvlySZ/iYT40ae8hFSZJkZURFEHf6vx9IVA08Yk6gXyNzsdp/hXFjPf HS8+ckPLzzqvCR7Tsjsm/usVX0QU5nEvkHHHvsslnZnLPq3jFoqeV1zaxmONl5KIn25uRj W8StGtpN6vLep/qQ8VemdSyCFsAKgFU= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-498-P-sgEwFcNKGVcLvjhEtTmg-1; Thu, 26 Aug 2021 15:41:24 -0400 X-MC-Unique: P-sgEwFcNKGVcLvjhEtTmg-1 Received: by mail-wr1-f71.google.com with SMTP id b7-20020a5d4d87000000b001575d1053d2so1180801wru.23 for ; Thu, 26 Aug 2021 12:41:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/wdRRB3BbU0zTebKEzNMBHGnZzjSnqwp9yN08+1iPjU=; b=Qg0ZInbm1yKXJ4gtBQeZpPq4McvB8qqUXD2Iu3SfR4Ooi7iTjguO/liILGiR23u6Tj FypV52ijpd32CYLE4g3DSszwBLAdqdeOqK3s2OGYYVNqBvBXE93+/zYsmL4xnHgJ7MOO UdyQeQ58u7Fn7FDhY6de5tllhjwFUCPSSVzpUuPK3rXxW5MBGUQKoeoLWL9dUdX68QS8 mZh3c/vaUIqPYrtGm6dJ113oakJdxuVOoDBpEpCZqIKzohSRs91ZW+TEvcclkvHrwbJK ufsIX8xlzSwEfmFwz9qxxKJ5OazXfG2SpLaa3i+ll0wyB/ezMOyO3WZqLh5l2geCtppC mQXg== X-Gm-Message-State: AOAM531tHUWuoFNl+BIKazTtXUqXYvZhn6epdLv4JTJSAfhIXVIvc5VS ylsyhFQhh1zUuVL6tg/VzOJQBQVdLK8lprd3xn4N29n2LmhM1sLRISG9QEG8xF+NElmeCp1ePyP eVRUI09VlLM6d1U5+ X-Received: by 2002:a05:6000:1808:: with SMTP id m8mr5842527wrh.272.1630006882351; Thu, 26 Aug 2021 12:41:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzhi28xOo8akRuWYI1q+XExG6E1iNF4UXfPNErBo+zAHkd2rbEd372edPEF5IYB7n7bBmLlLA== X-Received: by 2002:a05:6000:1808:: with SMTP id m8mr5842503wrh.272.1630006882147; Thu, 26 Aug 2021 12:41:22 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id l21sm3356742wmh.31.2021.08.26.12.41.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:41:21 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 19/27] bpf: Attach multi trampoline with ftrace_ops Date: Thu, 26 Aug 2021 21:39:14 +0200 Message-Id: <20210826193922.66204-20-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding struct ftrace_ops object to bpf_trampoline_multi struct and setting it up with all the requested function addresses. Adding is_multi_trampoline(tr) hooks to installing functions to actually install multiple bpf trampoline via ftrace_ops. I had to add -DCC_USING_FENTRY to several places because arch/x86/include/asm/ftrace.h would fail the compilation if it's not defined. Perhaps there's a better way. Signed-off-by: Jiri Olsa --- arch/x86/Makefile | 7 +++++ arch/x86/boot/compressed/Makefile | 4 +++ drivers/firmware/efi/libstub/Makefile | 3 +++ include/linux/bpf.h | 2 ++ kernel/bpf/trampoline.c | 37 +++++++++++++++++++++++++++ 5 files changed, 53 insertions(+) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 307fd0000a83..72986eb38d0b 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -34,6 +34,9 @@ REALMODE_CFLAGS += -fno-stack-protector REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), -Wno-address-of-packed-member) REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), $(cc_stack_align4)) REALMODE_CFLAGS += $(CLANG_FLAGS) +ifdef CONFIG_DYNAMIC_FTRACE + REALMODE_CFLAGS += -DCC_USING_FENTRY +endif export REALMODE_CFLAGS # BITS is used as extension for files which are available in a 32 bit @@ -54,6 +57,10 @@ KBUILD_CFLAGS += $(call cc-option,-mno-avx,) # Intel CET isn't enabled in the kernel KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none) +ifdef CONFIG_DYNAMIC_FTRACE + KBUILD_CFLAGS += -DCC_USING_FENTRY +endif + ifeq ($(CONFIG_X86_32),y) BITS := 32 UTS_MACHINE := i386 diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 431bf7f846c3..a93dcbda4de2 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -49,6 +49,10 @@ KBUILD_CFLAGS += $(call as-option,-Wa$(comma)-mrelax-relocations=no) KBUILD_CFLAGS += -include $(srctree)/include/linux/hidden.h KBUILD_CFLAGS += $(CLANG_FLAGS) +ifdef CONFIG_DYNAMIC_FTRACE + KBUILD_CFLAGS += -DCC_USING_FENTRY +endif + # sev.c indirectly inludes inat-table.h which is generated during # compilation and stored in $(objtree). Add the directory to the includes so # that the compiler finds it even with out-of-tree builds (make O=/some/path). diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index d0537573501e..2fd5d6f55e24 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -15,6 +15,9 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ \ $(call cc-disable-warning, gnu) \ -fno-asynchronous-unwind-tables \ $(CLANG_FLAGS) +ifdef CONFIG_DYNAMIC_FTRACE + cflags-$(CONFIG_X86) += -DCC_USING_FENTRY +endif # arm64 uses the full KBUILD_CFLAGS so it's necessary to explicitly # disable the stackleak plugin diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 373f45ae7dce..52cbec23665c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -23,6 +23,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -703,6 +704,7 @@ struct bpf_trampoline_multi { struct list_head list; u32 *ids; u32 ids_cnt; + struct ftrace_ops ops; int tr_cnt; struct bpf_trampoline *tr[]; }; diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 308d58e698be..82e7545a7426 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -179,11 +179,38 @@ static int is_ftrace_location(void *ip) return 1; } +static int register_ftrace_multi(struct bpf_trampoline *tr, void *new_addr) +{ + struct bpf_trampoline_multi *multi; + + multi = container_of(tr, struct bpf_trampoline_multi, main); + return register_ftrace_direct_multi(&multi->ops, (long)new_addr); +} + +static int unregister_ftrace_multi(struct bpf_trampoline *tr, void *old_addr) +{ + struct bpf_trampoline_multi *multi; + + multi = container_of(tr, struct bpf_trampoline_multi, main); + return unregister_ftrace_direct_multi(&multi->ops); +} + +static int modify_ftrace_multi(struct bpf_trampoline *tr, void *new_addr) +{ + struct bpf_trampoline_multi *multi; + + multi = container_of(tr, struct bpf_trampoline_multi, main); + return modify_ftrace_direct_multi(&multi->ops, (long)new_addr); +} + static int unregister_fentry(struct bpf_trampoline *tr, void *old_addr) { void *ip = tr->func.addr; int ret; + if (is_multi_trampoline(tr)) + return unregister_ftrace_multi(tr, old_addr); + if (tr->func.ftrace_managed) ret = unregister_ftrace_direct((long)ip, (long)old_addr); else @@ -199,6 +226,9 @@ static int modify_fentry(struct bpf_trampoline *tr, void *old_addr, void *new_ad void *ip = tr->func.addr; int ret; + if (is_multi_trampoline(tr)) + return modify_ftrace_multi(tr, new_addr); + if (tr->func.ftrace_managed) ret = modify_ftrace_direct((long)ip, (long)old_addr, (long)new_addr); else @@ -212,6 +242,9 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr) void *ip = tr->func.addr; int ret; + if (is_multi_trampoline(tr)) + return register_ftrace_multi(tr, new_addr); + ret = is_ftrace_location(ip); if (ret < 0) return ret; @@ -703,6 +736,10 @@ struct bpf_trampoline_multi *bpf_trampoline_multi_get(struct bpf_prog *prog, if (!is_ftrace_location((void *) tgt_info.tgt_addr)) goto out_free; + err = ftrace_set_filter_ip(&multi->ops, tgt_info.tgt_addr, 0, 0); + if (err) + goto out_free; + if (nr_args < tgt_info.fmodel.nr_args) nr_args = tgt_info.fmodel.nr_args; } From patchwork Thu Aug 26 19:39:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5431C432BE for ; Thu, 26 Aug 2021 19:41:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B4C7610CD for ; Thu, 26 Aug 2021 19:41:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243622AbhHZTm0 (ORCPT ); Thu, 26 Aug 2021 15:42:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55718 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243599AbhHZTmZ (ORCPT ); Thu, 26 Aug 2021 15:42:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sNOph7fDuFs97Eha5R8U+AbH+ZbOQ2fdynr7Ppgl9xo=; b=HxD7TpEwAgWEK8u+k4YHnIuO2q37GNdLTB1m3ee/Hp/cSBXsM9C8lrbQMP1maI6Y6F8Tsc DfC5VOKGx3QKJ29/6Jh94RX/CTo4QlmhbMYXhAYPD7nXG4aOokmWsnV/dFyLPQKJs8o++G Vz/R/PSPTonuPUjN0Kss1zNQ7xMs+3Y= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-602-04rQY9AVN86oDjX3ZSkPHQ-1; Thu, 26 Aug 2021 15:41:35 -0400 X-MC-Unique: 04rQY9AVN86oDjX3ZSkPHQ-1 Received: by mail-wm1-f72.google.com with SMTP id r125-20020a1c2b830000b0290197a4be97b7so1125517wmr.9 for ; Thu, 26 Aug 2021 12:41:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sNOph7fDuFs97Eha5R8U+AbH+ZbOQ2fdynr7Ppgl9xo=; b=E7hP+LItLJGVYHyGRMHJkl+rhJYHdgVKJNXRqa0Hze79qN491pf7Dq9ZD7IBvSXwGV bqr1PV7QOTpcX/QlkzulXCxLzt/3vUgLWM7bEDVfkmIVG8bmZB6bs6HvKFZ7+pE/fN// DogYAUNjXYrIhBi1CztCNikI6aQG0E3KpfPJ0huuuiQbkkSFhgcgWyx3p5yx2Cb39zEu RUZdDd7yN1ZIUDYypQ7twWF+H9/Ft+ze/GJr2MSsf+sZi2FnqEb7ZKip5xt9fhVXH2CO vnIA21sEifJpBYivMqQ4ytdbHPjydfj4vKlZY5DEaEy3rUYrqXk0Bw1hrwNVXRyWCKof w/cQ== X-Gm-Message-State: AOAM531FhA/81kijNvolDqvQxmGkpBYKPb2/2HlfNv4V2PQXEWZuacTS 3hi6zzQuPjsVCy1eKvpIVImPPhDNGPZaEk5P4Tfcs8NlNGa7nCOf9OcAbFNdUlYFCx9rplqtFZ0 214HVk24ieBjc1QFW X-Received: by 2002:a5d:65cc:: with SMTP id e12mr6218292wrw.266.1630006894649; Thu, 26 Aug 2021 12:41:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz1VfBS9lN1U/9ZDy+tjTNLSfUPE3NMeOcXS6gjf3AKHcPmgkRW6/Zzyv3AhZ1r8BdB6dgFyg== X-Received: by 2002:a5d:65cc:: with SMTP id e12mr6218280wrw.266.1630006894474; Thu, 26 Aug 2021 12:41:34 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id q11sm9133336wmc.41.2021.08.26.12.41.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:41:34 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 21/27] libbpf: Add support to link multi func tracing program Date: Thu, 26 Aug 2021 21:39:16 +0200 Message-Id: <20210826193922.66204-22-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to link multi func tracing program through link_create interface. Adding special types for multi func programs: fentry.multi fexit.multi so you can define multi func programs like: SEC("fentry.multi/bpf_fentry_test*") int BPF_PROG(test1, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) that defines test1 to be attached to bpf_fentry_test* functions. The test1 program is loaded with BPF_F_MULTI_FUNC flag. If functions are not specified the program needs to be attached manually. Adding new btf_ids/btf_ids_cnt fields to bpf_link_create_opts, that define functions to attach the program to. Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 8 +++++ tools/lib/bpf/bpf.h | 6 +++- tools/lib/bpf/libbpf.c | 72 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 85 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2401fad090c5..20422d58b945 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -713,12 +713,20 @@ int bpf_link_create(int prog_fd, int target_fd, if (!OPTS_ZEROED(opts, perf_event)) return libbpf_err(-EINVAL); break; + case BPF_TRACE_FENTRY: + case BPF_TRACE_FEXIT: + attr.link_create.multi_btf_ids = (__u64) OPTS_GET(opts, multi.btf_ids, 0); + attr.link_create.multi_btf_ids_cnt = OPTS_GET(opts, multi.btf_ids_cnt, 0); + if (!OPTS_ZEROED(opts, multi)) + return libbpf_err(-EINVAL); + break; default: if (!OPTS_ZEROED(opts, flags)) return libbpf_err(-EINVAL); break; } proceed: + fd = sys_bpf(BPF_LINK_CREATE, &attr, sizeof(attr)); return libbpf_err_errno(fd); } diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index 6fffb3cdf39b..2eff99be7069 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -181,10 +181,14 @@ struct bpf_link_create_opts { struct { __u64 bpf_cookie; } perf_event; + struct { + __u32 *btf_ids; + __u32 btf_ids_cnt; + } multi; }; size_t :0; }; -#define bpf_link_create_opts__last_field perf_event +#define bpf_link_create_opts__last_field multi LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type, diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 88d8825fc6f6..7f717b7755be 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -230,6 +230,7 @@ struct bpf_sec_def { bool is_attachable; bool is_attach_btf; bool is_sleepable; + bool is_multi_func; attach_fn_t attach_fn; }; @@ -6446,6 +6447,8 @@ __bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz, if (prog->sec_def->is_sleepable) prog->prog_flags |= BPF_F_SLEEPABLE; + if (prog->sec_def->is_multi_func) + prog->prog_flags |= BPF_F_MULTI_FUNC; bpf_program__set_type(prog, prog->sec_def->prog_type); bpf_program__set_expected_attach_type(prog, prog->sec_def->expected_attach_type); @@ -7915,6 +7918,8 @@ static struct bpf_link *attach_raw_tp(const struct bpf_sec_def *sec, struct bpf_program *prog); static struct bpf_link *attach_trace(const struct bpf_sec_def *sec, struct bpf_program *prog); +static struct bpf_link *attach_trace_multi(const struct bpf_sec_def *sec, + struct bpf_program *prog); static struct bpf_link *attach_lsm(const struct bpf_sec_def *sec, struct bpf_program *prog); static struct bpf_link *attach_iter(const struct bpf_sec_def *sec, @@ -7991,6 +7996,14 @@ static const struct bpf_sec_def section_defs[] = { .attach_fn = attach_iter), SEC_DEF("syscall", SYSCALL, .is_sleepable = true), + SEC_DEF("fentry.multi/", TRACING, + .expected_attach_type = BPF_TRACE_FENTRY, + .is_multi_func = true, + .attach_fn = attach_trace_multi), + SEC_DEF("fexit.multi/", TRACING, + .expected_attach_type = BPF_TRACE_FEXIT, + .is_multi_func = true, + .attach_fn = attach_trace_multi), BPF_EAPROG_SEC("xdp_devmap/", BPF_PROG_TYPE_XDP, BPF_XDP_DEVMAP), BPF_EAPROG_SEC("xdp_cpumap/", BPF_PROG_TYPE_XDP, @@ -8435,6 +8448,9 @@ static int libbpf_find_attach_btf_id(struct bpf_program *prog, int *btf_obj_fd, if (!name) return -EINVAL; + if (prog->prog_flags & BPF_F_MULTI_FUNC) + return 0; + for (i = 0; i < ARRAY_SIZE(section_defs); i++) { if (!section_defs[i].is_attach_btf) continue; @@ -9523,6 +9539,62 @@ static struct bpf_link *bpf_program__attach_btf_id(struct bpf_program *prog) return (struct bpf_link *)link; } +static struct bpf_link *bpf_program__attach_multi(struct bpf_program *prog) +{ + char *pattern = prog->sec_name + prog->sec_def->len; + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + enum bpf_attach_type attach_type; + int prog_fd, link_fd, cnt, err; + struct bpf_link *link = NULL; + __u32 *ids = NULL; + + prog_fd = bpf_program__fd(prog); + if (prog_fd < 0) { + pr_warn("prog '%s': can't attach before loaded\n", prog->name); + return ERR_PTR(-EINVAL); + } + + err = bpf_object__load_vmlinux_btf(prog->obj, true); + if (err) + return ERR_PTR(err); + + cnt = btf__find_by_glob_kind(prog->obj->btf_vmlinux, BTF_KIND_FUNC, + pattern, NULL, &ids); + if (cnt <= 0) + return ERR_PTR(-EINVAL); + + link = calloc(1, sizeof(*link)); + if (!link) { + err = -ENOMEM; + goto out_err; + } + link->detach = &bpf_link__detach_fd; + + opts.multi.btf_ids = ids; + opts.multi.btf_ids_cnt = cnt; + + attach_type = bpf_program__get_expected_attach_type(prog); + link_fd = bpf_link_create(prog_fd, 0, attach_type, &opts); + if (link_fd < 0) { + err = -errno; + goto out_err; + } + link->fd = link_fd; + free(ids); + return link; + +out_err: + free(link); + free(ids); + return ERR_PTR(err); +} + +static struct bpf_link *attach_trace_multi(const struct bpf_sec_def *sec, + struct bpf_program *prog) +{ + return bpf_program__attach_multi(prog); +} + struct bpf_link *bpf_program__attach_trace(struct bpf_program *prog) { return bpf_program__attach_btf_id(prog); From patchwork Thu Aug 26 19:39:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91656C4320A for ; Thu, 26 Aug 2021 19:41:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B5FC610CB for ; Thu, 26 Aug 2021 19:41:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243630AbhHZTmj (ORCPT ); Thu, 26 Aug 2021 15:42:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24949 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243629AbhHZTmh (ORCPT ); Thu, 26 Aug 2021 15:42:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006909; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L4uQfRC0gVRi3xhE8qT4qNuckZwO4Pkcm7ceoMb3k98=; b=ghh22P8zIihrI4MNK0mLuZ/2Gkic5DmGe8gjn7qQ68yZO5OJ/etx5bwk5QqFyCozc4Vav5 krXNuP2+79t6Xue87pqmQCL6ASEWiFX8Q7CQT9xneq0BuqVO4UcxmDkij4Q02d9xHUqIyZ WCZcfmbwhM1SULGCQBZb94qV1maDBBg= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-545-ta6KoC_9OwaDQ3QOOMz98A-1; Thu, 26 Aug 2021 15:41:48 -0400 X-MC-Unique: ta6KoC_9OwaDQ3QOOMz98A-1 Received: by mail-wr1-f70.google.com with SMTP id h14-20020a056000000e00b001575b00eb08so1185378wrx.13 for ; Thu, 26 Aug 2021 12:41:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L4uQfRC0gVRi3xhE8qT4qNuckZwO4Pkcm7ceoMb3k98=; b=hkuNMEuZCL1iiKvBJ+Kk/HmBWkrIRj+9CrMQqoSEWqcSYAbS6uaV4vutTOzELf3Qz0 lHN+niVaYp7KVcNwQKtlxI6rIm/0oocBOr880duvlxbOgO2gWdbv+FplpAlYeaPIqnvi e+7epRmzjoXGQSO8GQPavAlP2xiSy56EsSmb4WyMo34Vfn++YC3ALHZjzyj7AjvEezFf s2fBsw0UiOiOc2E3D8B2+gE2TIqCPsjj/OEvsPZyGjRwf+SiNUu4rUtWKHaZAswss/0B JUKIuGYCRXF1jJ9i/bQFG2/GNoItC8/4NRXSBn+1ayexKNuj4x6WkfSrFr8BCNGFT78i sGGA== X-Gm-Message-State: AOAM533Ir4DTET3FrBRL4Efn4k/WxEZs1cU6Io+y4tLNFgtkc7gUfLXw pDEXbssuUp9s1ymIybY+J9v3YCyts8H+rlJjj8BEf8h62I0lgFgaeLyIZytThF7DOowwY0Z8M6h Tm20lezmJgaf7By3H X-Received: by 2002:adf:916f:: with SMTP id j102mr2112222wrj.422.1630006907161; Thu, 26 Aug 2021 12:41:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVnhRZe4NqfZXpIVeTTiL8K0dGI7pHdPxTHEUchEkXPBEeXA2GfmitEE+opVev1Ae+keafIw== X-Received: by 2002:adf:916f:: with SMTP id j102mr2112205wrj.422.1630006906931; Thu, 26 Aug 2021 12:41:46 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id u10sm4137538wrt.14.2021.08.26.12.41.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:41:46 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 23/27] selftests/bpf: Add fexit multi func test Date: Thu, 26 Aug 2021 21:39:18 +0200 Message-Id: <20210826193922.66204-24-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding selftest for fexit multi func test that attaches to bpf_fentry_test* functions and checks argument values based on the processed function. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/Makefile | 3 +- .../bpf/prog_tests/multi_fexit_test.c | 31 +++++++++++++++++++ .../testing/selftests/bpf/progs/multi_check.c | 22 +++++++++++++ .../testing/selftests/bpf/progs/multi_fexit.c | 20 ++++++++++++ 4 files changed, 75 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/multi_fexit_test.c create mode 100644 tools/testing/selftests/bpf/progs/multi_fexit.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 013d41c8edae..8da3be0972de 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -313,7 +313,7 @@ SKEL_BLACKLIST := btf__% test_pinning_invalid.c test_sk_assign.c LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \ linked_vars.skel.h linked_maps.skel.h \ - multi_fentry_test.skel.h + multi_fentry_test.skel.h multi_fexit_test.skel.h LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \ test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c @@ -324,6 +324,7 @@ linked_funcs.skel.h-deps := linked_funcs1.o linked_funcs2.o linked_vars.skel.h-deps := linked_vars1.o linked_vars2.o linked_maps.skel.h-deps := linked_maps1.o linked_maps2.o multi_fentry_test.skel.h-deps := multi_fentry.o multi_check.o +multi_fexit_test.skel.h-deps := multi_fexit.o multi_check.o LINKED_BPF_SRCS := $(patsubst %.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(skel)-deps))) diff --git a/tools/testing/selftests/bpf/prog_tests/multi_fexit_test.c b/tools/testing/selftests/bpf/prog_tests/multi_fexit_test.c new file mode 100644 index 000000000000..d9b0eedd9f45 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/multi_fexit_test.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "multi_fexit_test.skel.h" +#include "trace_helpers.h" + +void test_multi_fexit_test(void) +{ + struct multi_fexit_test *skel = NULL; + __u32 duration = 0, retval; + int err, prog_fd; + + skel = multi_fexit_test__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fexit_multi_skel_load")) + goto cleanup; + + err = multi_fexit_test__attach(skel); + if (!ASSERT_OK(err, "fexit_attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test_arg_result, 8, "fexit_multi_arg_result"); + ASSERT_EQ(skel->bss->test_ret_result, 8, "fexit_multi_ret_result"); + +cleanup: + multi_fexit_test__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/multi_check.c b/tools/testing/selftests/bpf/progs/multi_check.c index 415c96684a30..80ebbefaa8cd 100644 --- a/tools/testing/selftests/bpf/progs/multi_check.c +++ b/tools/testing/selftests/bpf/progs/multi_check.c @@ -61,3 +61,25 @@ void multi_arg_check(__u64 *ctx, __u64 *test_result) *test_result += 1; } } + +void multi_ret_check(void *ctx, int ret, __u64 *test_result) +{ + void *ip = (void *) bpf_get_func_ip(ctx); + + if (ip == &bpf_fentry_test1) + *test_result += ret == 2; + else if (ip == &bpf_fentry_test2) + *test_result += ret == 5; + else if (ip == &bpf_fentry_test3) + *test_result += ret == 15; + else if (ip == &bpf_fentry_test4) + *test_result += ret == 34; + else if (ip == &bpf_fentry_test5) + *test_result += ret == 65; + else if (ip == &bpf_fentry_test6) + *test_result += ret == 111; + else if (ip == &bpf_fentry_test7) + *test_result += ret == 0; + else if (ip == &bpf_fentry_test8) + *test_result += ret == 0; +} diff --git a/tools/testing/selftests/bpf/progs/multi_fexit.c b/tools/testing/selftests/bpf/progs/multi_fexit.c new file mode 100644 index 000000000000..29c49aa3bfc2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/multi_fexit.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__hidden extern void multi_arg_check(__u64 *ctx, __u64 *test_result); +__hidden extern void multi_ret_check(void *ctx, int ret, __u64 *test_result); + +__u64 test_arg_result = 0; +__u64 test_ret_result = 0; + +SEC("fexit.multi/bpf_fentry_test*") +int BPF_PROG(test, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) +{ + multi_arg_check(ctx, &test_arg_result); + multi_ret_check(ctx, ret, &test_ret_result); + return 0; +} From patchwork Thu Aug 26 19:39:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C94A1C4320E for ; Thu, 26 Aug 2021 19:43:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B380360F4C for ; Thu, 26 Aug 2021 19:43:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243537AbhHZTnu (ORCPT ); Thu, 26 Aug 2021 15:43:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21888 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243496AbhHZTnt (ORCPT ); Thu, 26 Aug 2021 15:43:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cg3b6pCUYgk1DCEc/G+FSd+XekVAu5S9ccZ75etSNB8=; b=AkYK2YxQHAxyo69U8XHC9X4RwLhXjPp4jzVROzSaIbid0Z3g3Y4q0yFndsblkYhsnSD+6q 8AO11e0qz1qUtRD2ZjZ1+ICFAfcspdEsSJQzFKwUb5DA5L7PKP/sArUZnIUpqJuiyGV5K3 +FdQDoPUmCxFIeZIjMB3RUV708AnXYQ= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-7-SOk0tu6WOS-8PCV0IIIYXQ-1; Thu, 26 Aug 2021 15:41:54 -0400 X-MC-Unique: SOk0tu6WOS-8PCV0IIIYXQ-1 Received: by mail-wm1-f70.google.com with SMTP id u1-20020a05600c210100b002e74fc5af71so1277544wml.1 for ; Thu, 26 Aug 2021 12:41:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cg3b6pCUYgk1DCEc/G+FSd+XekVAu5S9ccZ75etSNB8=; b=rqN9n1JfR+mUc8FPVkC97uIQFbX1Uaulen+vkCarLvsx2ZEgr2VS2Tfmn6AKcgXOIU 1mpoZ9P+MxxrZOo/nkMzJnD25/UAmJ9+Y1yWgIgnSujbmZX6KojcSQ9wXiVJlcUX/DRA DjvdQSHla1FYO/kvtqhoRzWlSRRxoRmQQg+DkasypCyYj9Fa2Z84pGXUXQkeKV3sjzM1 opMtCjcD4ugxE3xzeNg/cAGIOfcmv0zxxWRTeJo0jxYYrIeBBUuPE6aWziwMMDCx/c+o Xbmd2jdTaBvcEcGljggCRm3D1jnxTUjLaUQRD2ZWxVvHGABLvK8/qvZANGGlgBdk7V6v iTsA== X-Gm-Message-State: AOAM532jNn9Q/Xx2RQmLQbUNjan0KnQKo2yRd1oQK5+7fHHQDqmeFyiy oyjXfdJnRIHqwwwl3Ngf5i5GzgKR03w/29+CXx30XThzS+4iAgrjjpaWiLpj91g10dY9ICujz3O 4/w1iK/fSnbTz/SzJ X-Received: by 2002:a7b:cb44:: with SMTP id v4mr5315779wmj.169.1630006913398; Thu, 26 Aug 2021 12:41:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz9WybcY+dg6NCte2Tc5bpGkGSGSc32LlQkKDXkHz9yizHUZoqK2YAG3KJWGGtS9Cqwax+XXA== X-Received: by 2002:a7b:cb44:: with SMTP id v4mr5315763wmj.169.1630006913198; Thu, 26 Aug 2021 12:41:53 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id l2sm10371542wmi.1.2021.08.26.12.41.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:41:52 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 24/27] selftests/bpf: Add fentry/fexit multi func test Date: Thu, 26 Aug 2021 21:39:19 +0200 Message-Id: <20210826193922.66204-25-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding selftest for fentry/fexit multi func tests that attaches to bpf_fentry_test* functions and checks argument values based on the processed function. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/Makefile | 4 ++- .../bpf/prog_tests/multi_fentry_fexit_test.c | 32 +++++++++++++++++++ .../selftests/bpf/progs/multi_fentry_fexit.c | 28 ++++++++++++++++ 3 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/multi_fentry_fexit_test.c create mode 100644 tools/testing/selftests/bpf/progs/multi_fentry_fexit.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 8da3be0972de..6272d9c166f9 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -313,7 +313,8 @@ SKEL_BLACKLIST := btf__% test_pinning_invalid.c test_sk_assign.c LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \ linked_vars.skel.h linked_maps.skel.h \ - multi_fentry_test.skel.h multi_fexit_test.skel.h + multi_fentry_test.skel.h multi_fexit_test.skel.h \ + multi_fentry_fexit_test.skel.h LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \ test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c @@ -325,6 +326,7 @@ linked_vars.skel.h-deps := linked_vars1.o linked_vars2.o linked_maps.skel.h-deps := linked_maps1.o linked_maps2.o multi_fentry_test.skel.h-deps := multi_fentry.o multi_check.o multi_fexit_test.skel.h-deps := multi_fexit.o multi_check.o +multi_fentry_fexit_test.skel.h-deps := multi_fentry_fexit.o multi_check.o LINKED_BPF_SRCS := $(patsubst %.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(skel)-deps))) diff --git a/tools/testing/selftests/bpf/prog_tests/multi_fentry_fexit_test.c b/tools/testing/selftests/bpf/prog_tests/multi_fentry_fexit_test.c new file mode 100644 index 000000000000..d54abf36ab2f --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/multi_fentry_fexit_test.c @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "multi_fentry_fexit_test.skel.h" + +void test_multi_fentry_fexit_test(void) +{ + DECLARE_LIBBPF_OPTS(bpf_link_update_opts, link_upd_opts); + struct multi_fentry_fexit_test *skel = NULL; + __u32 duration = 0, retval; + int err, prog_fd; + + skel = multi_fentry_fexit_test__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fentry_multi_skel_load")) + goto cleanup; + + err = multi_fentry_fexit_test__attach(skel); + if (!ASSERT_OK(err, "fentry_fexit_attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test2); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test1_arg_result, 8, "test1_arg_result"); + ASSERT_EQ(skel->bss->test2_arg_result, 8, "test2_arg_result"); + ASSERT_EQ(skel->bss->test2_ret_result, 8, "test2_ret_result"); + +cleanup: + multi_fentry_fexit_test__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/multi_fentry_fexit.c b/tools/testing/selftests/bpf/progs/multi_fentry_fexit.c new file mode 100644 index 000000000000..aac7df8c9211 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/multi_fentry_fexit.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 test1_arg_result = 0; +__u64 test2_arg_result = 0; +__u64 test2_ret_result = 0; + +__hidden extern void multi_arg_check(__u64 *ctx, __u64 *test_result); +__hidden extern void multi_ret_check(void *ctx, int ret, __u64 *test_result); + +SEC("fentry.multi/bpf_fentry_test*") +int BPF_PROG(test1, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) +{ + multi_arg_check(ctx, &test1_arg_result); + return 0; +} + +SEC("fexit.multi/bpf_fentry_test*") +int BPF_PROG(test2, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) +{ + multi_arg_check(ctx, &test2_arg_result); + multi_ret_check(ctx, ret, &test2_ret_result); + return 0; +} From patchwork Thu Aug 26 19:39:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 503249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A1CDC4320A for ; Thu, 26 Aug 2021 19:42:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7036D60F11 for ; Thu, 26 Aug 2021 19:42:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243523AbhHZTm5 (ORCPT ); Thu, 26 Aug 2021 15:42:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43870 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243519AbhHZTm4 (ORCPT ); Thu, 26 Aug 2021 15:42:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630006928; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bokwZZdBG1B1kLTNIjydZjB2m8CruKIWTf2DTwAUs3E=; b=i9qKK11CdsE0bvDecLOA8Bi4HJCl/L8viGm5yGD0BENngzRVwt6r47Hn3x3Ff0AzBCMRp/ 3qGMA/mOkPtNrB4rHoyXSBQmdPIEEvIKhhWfYFmW3W6TAtTDLIDcJ6nfCQLbcBZARIXW9S AUzhWL+62Jcpk0FS/Sq7D72kVm4+e/M= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-389-y_RyhoCdNoSDeF3xHpZBBw-1; Thu, 26 Aug 2021 15:42:07 -0400 X-MC-Unique: y_RyhoCdNoSDeF3xHpZBBw-1 Received: by mail-wr1-f69.google.com with SMTP id d12-20020a056000186cb02901548bff164dso1190008wri.18 for ; Thu, 26 Aug 2021 12:42:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bokwZZdBG1B1kLTNIjydZjB2m8CruKIWTf2DTwAUs3E=; b=Q4r2oUZHRkxHpZzYyU5nK9vzDEOiXtGUfCSEdSCiiQq98yitljPi1J76ZLp05anLqN rxMWQZqeWLzNv/8EskhaRG2Hc4AyqpC2Lz1vDJtwkYJxKCiC4wNUO6RuFiN7xg19ca2Q pPtOzFUosKAMgut4tdyDOTg6cJ69guj5EOA43B4IQnV++jbPKZrV0WEDvgbJGQh0ehar tRnpYcm7Vn8LMIpIpR4dkhX+8pWSETuTuP4eckM4d1ko0nV+BOrBGZIS2aEAGcTOvpkK IIewkmRHQSEJcygJPPJE++isxyh3R+mHv0a98ZjJd4/8hQCnY1KXyW7eAVV57NJyEuf4 aArQ== X-Gm-Message-State: AOAM532uW1s3pdQQiXAur62bDXE4VCLxkRl8/SwC2eauZqI0hkgZCiv0 tlIdzVAbErQD2/2dxD9NSXDB+xkzTwsAUgpdbAb7u33At3LOdfUleqzxJhTT3SxgTwEy2D+4rSF r1O3jCaTgqueRxp6L X-Received: by 2002:adf:e28a:: with SMTP id v10mr5805327wri.289.1630006925723; Thu, 26 Aug 2021 12:42:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzFN+ZboqGkMatDOw0B/DLmI+FQdpqDxpOj7fXgQ9Xb7hPg5qMWaB/EhxUaqgiLxJEfevUbow== X-Received: by 2002:adf:e28a:: with SMTP id v10mr5805306wri.289.1630006925500; Thu, 26 Aug 2021 12:42:05 -0700 (PDT) Received: from krava.redhat.com ([83.240.63.86]) by smtp.gmail.com with ESMTPSA id c1sm9108499wml.33.2021.08.26.12.42.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Aug 2021 12:42:05 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH bpf-next v4 26/27] selftests/bpf: Add attach multi func test Date: Thu, 26 Aug 2021 21:39:21 +0200 Message-Id: <20210826193922.66204-27-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210826193922.66204-1-jolsa@kernel.org> References: <20210826193922.66204-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding selftest to check attaching rules for multi func programs. - attach 2 programs: fentry/bpf_fentry_test1 fexit/bpf_fentry_test2 - check that we can attach multi func program on top of them: fentry.multi/bpf_fentry_test* - check that we cannot attach another multi funct program that does not cover the same BTF ids (one less): fentry.multi/bpf_fentry_test[1-7] fexit.multi/bpf_fentry_test[1-7] - check that we can no longer attach standard trampoline programs (below) on top of attached multi func program: fentry/bpf_fentry_test1 fexit/bpf_fentry_test3 Because the supported wildcards do not allow us to match just limited set of bpf_fentry_test*, adding extra code to look it up in kernel's BTF. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/multi_attach_check_test.c | 115 ++++++++++++++++++ .../selftests/bpf/progs/multi_attach_check.c | 36 ++++++ .../bpf/progs/multi_attach_check_extra1.c | 12 ++ .../bpf/progs/multi_attach_check_extra2.c | 12 ++ 4 files changed, 175 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/multi_attach_check_test.c create mode 100644 tools/testing/selftests/bpf/progs/multi_attach_check.c create mode 100644 tools/testing/selftests/bpf/progs/multi_attach_check_extra1.c create mode 100644 tools/testing/selftests/bpf/progs/multi_attach_check_extra2.c diff --git a/tools/testing/selftests/bpf/prog_tests/multi_attach_check_test.c b/tools/testing/selftests/bpf/prog_tests/multi_attach_check_test.c new file mode 100644 index 000000000000..32b23718437d --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/multi_attach_check_test.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include "multi_attach_check.skel.h" +#include "multi_attach_check_extra1.skel.h" +#include "multi_attach_check_extra2.skel.h" +#include + +static __u32 btf_ids[7]; + +static int load_btf_ids(void) +{ + __u32 i, nr_types, cnt; + struct btf *btf; + + btf = btf__load_vmlinux_btf(); + if (!ASSERT_OK_PTR(btf, "btf__load_vmlinux_btf")) + return -1; + + nr_types = btf__get_nr_types(btf); + + for (i = 1, cnt = 0; i <= nr_types && cnt < 7; i++) { + const struct btf_type *t = btf__type_by_id(btf, i); + const char *name; + + if (!btf_is_func(t)) + continue; + + name = btf__name_by_offset(btf, t->name_off); + if (!name) + continue; + if (strncmp(name, "bpf_fentry_test", sizeof("bpf_fentry_test") - 1)) + continue; + + btf_ids[cnt] = i; + cnt++; + } + + btf__free(btf); + return ASSERT_EQ(cnt, 7, "bpf_fentry_test_cnt") ? 0 : -1; +} + +void test_multi_attach_check_test(void) +{ + struct bpf_link *link1 = NULL, *link2 = NULL, *link3 = NULL; + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + struct multi_attach_check_extra1 *skel_extra1 = NULL; + struct multi_attach_check_extra2 *skel_extra2 = NULL; + struct multi_attach_check *skel; + int link_fd, prog_fd; + + /* Load/attach standard trampolines and on top of it multi + * func program. It should succeed. + */ + skel = multi_attach_check__open_and_load(); + if (!ASSERT_OK_PTR(skel, "multi_attach_check__load")) + return; + + link1 = bpf_program__attach(skel->progs.test1); + if (!ASSERT_OK_PTR(link1, "multi_attach_check__test1_attach")) + goto cleanup; + + link2 = bpf_program__attach(skel->progs.test2); + if (!ASSERT_OK_PTR(link2, "multi_attach_check__test2_attach")) + goto cleanup; + + link3 = bpf_program__attach(skel->progs.test3); + if (!ASSERT_OK_PTR(link3, "multi_attach_check__test3_attach")) + goto cleanup; + + if (!ASSERT_OK(load_btf_ids(), "load_btf_ids")) + goto cleanup; + + /* There's 8 bpf_fentry_test* functions, get BTF ids for 7 of them + * and try to load/link multi func program with them. It should fail + * both for fentry.multi ... + */ + opts.multi.btf_ids = btf_ids; + opts.multi.btf_ids_cnt = 7; + + prog_fd = bpf_program__fd(skel->progs.test4); + + link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FENTRY, &opts); + if (!ASSERT_LT(link_fd, 0, "bpf_link_create")) + goto cleanup; + + close(link_fd); + + /* ... and fexit.multi */ + prog_fd = bpf_program__fd(skel->progs.test5); + + link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FEXIT, &opts); + if (!ASSERT_LT(link_fd, 0, "bpf_link_create")) + goto cleanup; + + close(link_fd); + + /* Try to load/attach extra programs on top of multi func programs, + * it should fail for both fentry ... + */ + skel_extra1 = multi_attach_check_extra1__open_and_load(); + if (!ASSERT_ERR_PTR(skel_extra1, "multi_attach_check_extra1__load")) + multi_attach_check_extra1__destroy(skel_extra1); + + /* ... and fexit */ + skel_extra2 = multi_attach_check_extra2__open_and_load(); + if (!ASSERT_ERR_PTR(skel_extra2, "multi_attach_check_extra2__load")) + multi_attach_check_extra2__destroy(skel_extra2); + +cleanup: + bpf_link__destroy(link1); + bpf_link__destroy(link2); + bpf_link__destroy(link3); + multi_attach_check__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/multi_attach_check.c b/tools/testing/selftests/bpf/progs/multi_attach_check.c new file mode 100644 index 000000000000..cf0f25c69556 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/multi_attach_check.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test1, int a) +{ + return 0; +} + +SEC("fexit/bpf_fentry_test2") +int BPF_PROG(test2, int a, __u64 b, int ret) +{ + return 0; +} + +SEC("fentry.multi/bpf_fentry_test*") +int BPF_PROG(test3, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) +{ + return 0; +} + +SEC("fentry.multi/bpf_fentry_test1-7") +int BPF_PROG(test4, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) +{ + return 0; +} + +SEC("fexit.multi/bpf_fentry_test1-7") +int BPF_PROG(test5, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) +{ + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/multi_attach_check_extra1.c b/tools/testing/selftests/bpf/progs/multi_attach_check_extra1.c new file mode 100644 index 000000000000..c1d816a0206a --- /dev/null +++ b/tools/testing/selftests/bpf/progs/multi_attach_check_extra1.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test1, int a) +{ + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/multi_attach_check_extra2.c b/tools/testing/selftests/bpf/progs/multi_attach_check_extra2.c new file mode 100644 index 000000000000..cd66abb4b848 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/multi_attach_check_extra2.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("fexit/bpf_fentry_test3") +int BPF_PROG(test3, int a, __u64 b, int ret) +{ + return 0; +}