From patchwork Tue Jun 22 19:55:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 466279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BCEAC48BDF for ; Tue, 22 Jun 2021 19:57:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2896B610C7 for ; Tue, 22 Jun 2021 19:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232807AbhFVT70 (ORCPT ); Tue, 22 Jun 2021 15:59:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232796AbhFVT7Z (ORCPT ); Tue, 22 Jun 2021 15:59:25 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9571BC061574; Tue, 22 Jun 2021 12:57:08 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id m2so17965314pgk.7; Tue, 22 Jun 2021 12:57:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7EK4KCzWQjnwZZfmTqzpOsXvyTUA9SISmaf19z0hRsA=; b=UOz6Wrsz4pUBdKXM3vBYeg/faqwU8ekW41vkrks+bhqlJFYNa7HutVkxXX8sFl1q6b bVF1rsDzlZO0ZJ8xqISXAdiMicBJq00pykpJOTSjG9StqhI56eazOyROdGUuc1avUUqf 1HfmWZfDGS/svvOqPEVB+nPNPalgxSU2TwkacGXZItIeCgoAuY2FKq3uRgttMPKj2Bsg Ro2L2vF39Ol9fsQwOF9fg1VUtJYIEqn0tkqw/jq4+1MhnyIHhy9gicLkvicgujfm9C8D WMDbSsI3OXIrxO6VyzcjdfqITorp5uJPGRtKguclLEzpPsG2X8iVO4FGkO/L0UETlCTx Y3/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7EK4KCzWQjnwZZfmTqzpOsXvyTUA9SISmaf19z0hRsA=; b=rhd09X80xxkLDfkxNr8LOdJL1Ba3t+AVS6KbIl9rE5d1wY0LwGZXQ+LcVboxC33V2H RPwf84JYYH2QUkvqSP0+3iTOjAaCAUvMIfUakkEmL72stal/XcthEISrMDZQLsM8L3xC GjlFDAfVl/n/SjUMheVDkDJCyO3zfl7O96AHdC0LJq3q8JkYYzm/wUa4CuD6N4457WVU b5xtUemCFhRi9MjvUi2STEAZeXkJSqu2DXvmOGsoWkIFsjIPQ8kUiK1Uz6VSz9J5slhi QYkxAGY0fdVEdNT9WGWDWo8NUcsAOc440jtO2mJa1saA8/xL1oxgytrEF+md+uxhVreW uv1w== X-Gm-Message-State: AOAM530umw1ErZAV1Em8qysAuqUgZ57P5nrY/8CAoGqV7AhSAzkSxF6+ OGmxDace6KT5lK5vvKvjPPD8ngbynck= X-Google-Smtp-Source: ABdhPJy9kbfjaKFpBwCsOzNP7vSMUM7W9Eh0jhG7o8s80ZmM4dIAcaOXLrd3UdZzfIxs0ijjq+QjrA== X-Received: by 2002:a62:dd8b:0:b029:2e9:731a:e22e with SMTP id w133-20020a62dd8b0000b02902e9731ae22emr5238665pff.69.1624391827975; Tue, 22 Jun 2021 12:57:07 -0700 (PDT) Received: from localhost ([2402:3a80:11bb:33b3:7f0c:3646:8bde:417e]) by smtp.gmail.com with ESMTPSA id a197sm137891pfa.220.2021.06.22.12.57.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 12:57:07 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: netdev@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesper Dangaard Brouer , "David S. Miller" , Jakub Kicinski , John Fastabend , Martin KaFai Lau , bpf@vger.kernel.org Subject: [PATCH net-next v2 1/5] net: core: split out code to run generic XDP prog Date: Wed, 23 Jun 2021 01:25:23 +0530 Message-Id: <20210622195527.1110497-2-memxor@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210622195527.1110497-1-memxor@gmail.com> References: <20210622195527.1110497-1-memxor@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This helper can later be utilized in code that runs cpumap and devmap programs in generic redirect mode and adjust skb based on changes made to xdp_buff. When returning XDP_REDIRECT/XDP_TX, it invokes __skb_push, so whenever a generic redirect path invokes devmap/cpumap prog if set, it must __skb_pull again as we expect mac header to be pulled. It also drops the skb_reset_mac_len call after do_xdp_generic, as the mac_header and network_header are advanced by the same offset, so the difference (mac_len) remains constant. Signed-off-by: Kumar Kartikeya Dwivedi --- NB: I am not too sure why the skb_reset_mac_len was there, so I removed it since the offset addition/subtraction should be same for network_header and mac_header, but I could be missing something important... --- include/linux/netdevice.h | 2 + net/core/dev.c | 84 ++++++++++++++++++++++++--------------- 2 files changed, 55 insertions(+), 31 deletions(-) -- 2.31.1 diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index be1dcceda5e4..90472ea70db2 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3984,6 +3984,8 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) __dev_kfree_skb_any(skb, SKB_REASON_CONSUMED); } +u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog); void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb); int netif_rx(struct sk_buff *skb); diff --git a/net/core/dev.c b/net/core/dev.c index 50531a2d0b20..c34ff1dbf6e6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4717,45 +4717,18 @@ static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb) return rxqueue; } -static u32 netif_receive_generic_xdp(struct sk_buff *skb, - struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { void *orig_data, *orig_data_end, *hard_start; struct netdev_rx_queue *rxqueue; - u32 metalen, act = XDP_DROP; bool orig_bcast, orig_host; u32 mac_len, frame_sz; __be16 orig_eth_type; struct ethhdr *eth; + u32 metalen, act; int off; - /* Reinjected packets coming from act_mirred or similar should - * not get XDP generic processing. - */ - if (skb_is_redirected(skb)) - return XDP_PASS; - - /* XDP packets must be linear and must have sufficient headroom - * of XDP_PACKET_HEADROOM bytes. This is the guarantee that also - * native XDP provides, thus we need to do it here as well. - */ - if (skb_cloned(skb) || skb_is_nonlinear(skb) || - skb_headroom(skb) < XDP_PACKET_HEADROOM) { - int hroom = XDP_PACKET_HEADROOM - skb_headroom(skb); - int troom = skb->tail + skb->data_len - skb->end; - - /* In case we have to go down the path and also linearize, - * then lets do the pskb_expand_head() work just once here. - */ - if (pskb_expand_head(skb, - hroom > 0 ? ALIGN(hroom, NET_SKB_PAD) : 0, - troom > 0 ? troom + 128 : 0, GFP_ATOMIC)) - goto do_drop; - if (skb_linearize(skb)) - goto do_drop; - } - /* The XDP program wants to see the packet starting at the MAC * header. */ @@ -4810,6 +4783,13 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, skb->protocol = eth_type_trans(skb, skb->dev); } + /* Redirect/Tx gives L2 packet, code that will reuse skb must __skb_pull + * before calling us again on redirect path. We do not call do_redirect + * as we leave that up to the caller. + * + * Caller is responsible for managing lifetime of skb (i.e. calling + * kfree_skb in response to actions it cannot handle/XDP_DROP). + */ switch (act) { case XDP_REDIRECT: case XDP_TX: @@ -4820,6 +4800,49 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, if (metalen) skb_metadata_set(skb, metalen); break; + } + + return act; +} + +static u32 netif_receive_generic_xdp(struct sk_buff *skb, + struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) +{ + u32 act = XDP_DROP; + + /* Reinjected packets coming from act_mirred or similar should + * not get XDP generic processing. + */ + if (skb_is_redirected(skb)) + return XDP_PASS; + + /* XDP packets must be linear and must have sufficient headroom + * of XDP_PACKET_HEADROOM bytes. This is the guarantee that also + * native XDP provides, thus we need to do it here as well. + */ + if (skb_cloned(skb) || skb_is_nonlinear(skb) || + skb_headroom(skb) < XDP_PACKET_HEADROOM) { + int hroom = XDP_PACKET_HEADROOM - skb_headroom(skb); + int troom = skb->tail + skb->data_len - skb->end; + + /* In case we have to go down the path and also linearize, + * then lets do the pskb_expand_head() work just once here. + */ + if (pskb_expand_head(skb, + hroom > 0 ? ALIGN(hroom, NET_SKB_PAD) : 0, + troom > 0 ? troom + 128 : 0, GFP_ATOMIC)) + goto do_drop; + if (skb_linearize(skb)) + goto do_drop; + } + + act = bpf_prog_run_generic_xdp(skb, xdp, xdp_prog); + switch (act) { + case XDP_REDIRECT: + case XDP_TX: + case XDP_PASS: + break; default: bpf_warn_invalid_xdp_action(act); fallthrough; @@ -5285,7 +5308,6 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc, ret = NET_RX_DROP; goto out; } - skb_reset_mac_len(skb); } if (eth_type_vlan(skb->protocol)) { From patchwork Tue Jun 22 19:55:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 465553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C44AC2B9F4 for ; Tue, 22 Jun 2021 19:57:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 56006611CE for ; Tue, 22 Jun 2021 19:57:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232797AbhFVT7e (ORCPT ); Tue, 22 Jun 2021 15:59:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232799AbhFVT73 (ORCPT ); Tue, 22 Jun 2021 15:59:29 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2628C061756; Tue, 22 Jun 2021 12:57:12 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id a2so671703pgi.6; Tue, 22 Jun 2021 12:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jivxYGcJYbE7eSkXipGR1sWBMuZY0V3FzEQSe7d96iA=; b=A75ck4VPX7rTv9NjGMspsJ0S593UQnVRZ4sKsdQXfjYPLIeNS8593s8IM47v4zaTh0 0YMRqfYrc5+lZpasb2BPdeKSMzhtB4zBt/88hdtI30wu+q4BXqQIxLQLJpwWqxI042JS R2Fc3x/x1SwQdCkLlTLa2HRWToG/aVxevPgEcgchwIC2XKoKK+tVxKopWQ8IcH+JVlWq WWEBy3AOIwL4eqjcJnB4jyfrZvHKO5n4Lucs3qEPQ0o1sJI0qu6HWc6jbce4diJou6ca wFlCKquq0qZOawAlYuUpM3EYic5jwzWGpcFztZjD6pJd533pUVcaoMa659yW73tHrHnE PTHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jivxYGcJYbE7eSkXipGR1sWBMuZY0V3FzEQSe7d96iA=; b=Q4uHpmmQz1AlZuzOdxRRUTQMSvlpf35FFn9xm4ICt9mErzBlF1HQpzlw8ORo38baCi Xom3MES1f6YUPH3VItaM2N0F0sWREjOpI72pVVEhI5VpC1CSxtNSVS3rRqNOkFsOt7/c D+dRa/kZsd3YT1wYb2sD6ed7GispZBlB+jpDeLvrNLABamY7q4hyV2JzlL83XKd5c6jg lfdCtxl1MGTmxXu25miI0MCNzSV2+ozYou+8gi7A6IBnasMkF94T8ki+CjeDMF2i8olL lDDGFnGR3RSf7Nk4HcbiWCLddjf7oj/DS97O4g33ElcI+9+1OD3gLA7ylXQUJ4YyXfmp vq/A== X-Gm-Message-State: AOAM532r+ijpjSV+UIQaXw2ljJr5yyrgIuYkspgs0wGGhHfCw7JrOKCc G9Uo9pv5H/uTzZjpbfDpIADpUnvh+Y0= X-Google-Smtp-Source: ABdhPJy5j9g+S5puLLOhpPjSWLlolaeh0UWmsPV2Gyc5MLLREOEK9tyNHsO4/63m1pYaPyYVeJK97A== X-Received: by 2002:a63:1460:: with SMTP id 32mr304427pgu.163.1624391832073; Tue, 22 Jun 2021 12:57:12 -0700 (PDT) Received: from localhost ([2402:3a80:11bb:33b3:7f0c:3646:8bde:417e]) by smtp.gmail.com with ESMTPSA id t14sm158820pfe.45.2021.06.22.12.57.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 12:57:11 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: netdev@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesper Dangaard Brouer , "David S. Miller" , Jakub Kicinski , John Fastabend , Martin KaFai Lau , bpf@vger.kernel.org Subject: [PATCH net-next v2 2/5] bitops: add non-atomic bitops for pointers Date: Wed, 23 Jun 2021 01:25:24 +0530 Message-Id: <20210622195527.1110497-3-memxor@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210622195527.1110497-1-memxor@gmail.com> References: <20210622195527.1110497-1-memxor@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org cpumap needs to set, clear, and test the lowest bit in skb pointer in various places. To make these checks less noisy, add pointer friendly bitop macros that also do some typechecking to sanitize the argument. These wrap the non-atomic bitops __set_bit, __clear_bit, and test_bit but for pointer arguments. Pointer's address has to be passed in and it is treated as an unsigned long *, since width and representation of pointer and unsigned long match on targets Linux supports. They are prefixed with double underscore to indicate lack of atomicity. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bitops.h | 19 +++++++++++++++++++ include/linux/typecheck.h | 10 ++++++++++ 2 files changed, 29 insertions(+) diff --git a/include/linux/bitops.h b/include/linux/bitops.h index 26bf15e6cd35..a9e336b9fa4d 100644 --- a/include/linux/bitops.h +++ b/include/linux/bitops.h @@ -4,6 +4,7 @@ #include #include +#include #include @@ -253,6 +254,24 @@ static __always_inline void __assign_bit(long nr, volatile unsigned long *addr, __clear_bit(nr, addr); } +#define __ptr_set_bit(nr, addr) \ + ({ \ + typecheck_pointer(*(addr)); \ + __set_bit(nr, (unsigned long *)(addr)); \ + }) + +#define __ptr_clear_bit(nr, addr) \ + ({ \ + typecheck_pointer(*(addr)); \ + __clear_bit(nr, (unsigned long *)(addr)); \ + }) + +#define __ptr_test_bit(nr, addr) \ + ({ \ + typecheck_pointer(*(addr)); \ + test_bit(nr, (unsigned long *)(addr)); \ + }) + #ifdef __KERNEL__ #ifndef set_mask_bits diff --git a/include/linux/typecheck.h b/include/linux/typecheck.h index 20d310331eb5..33c78f27147a 100644 --- a/include/linux/typecheck.h +++ b/include/linux/typecheck.h @@ -22,4 +22,14 @@ (void)__tmp; \ }) +/* + * Check at compile that something is a pointer type. + * Always evaluates to 1 so you may use it easily in comparisons. + */ +#define typecheck_pointer(x) \ +({ typeof(x) __dummy; \ + (void)sizeof(*__dummy); \ + 1; \ +}) + #endif /* TYPECHECK_H_INCLUDED */ From patchwork Tue Jun 22 19:55:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 465552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FEEAC48BDF for ; Tue, 22 Jun 2021 19:57:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 39E486100B for ; Tue, 22 Jun 2021 19:57:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232376AbhFVT7n (ORCPT ); Tue, 22 Jun 2021 15:59:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232832AbhFVT7j (ORCPT ); Tue, 22 Jun 2021 15:59:39 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BD10C061574; Tue, 22 Jun 2021 12:57:22 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id a2so671960pgi.6; Tue, 22 Jun 2021 12:57:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2M+/jke7oI/v1bHKeQ1t1eI5gEmwcggeXPdID1191qU=; b=teWAd6YuPkd333afwFpTpH0QbS2uOehze7j9VxgKDhHdO1tpcV22V4zREcUyGo1iOU rV0oAKdAIbZQA8qMUGSp5tYx8+kQDC7XvkfdFG3hLHDajWfaxb6inM7aNI4kAYEF6QSC m77FcJmv1NDBrlsTOHPtPiVxKZPQ2G+Pj3RebL3L39ajb4kxKBSkhjlzF5+y/MOiVlFA 4Ty9IbFoCAl/6070xER/mqDh56UF10fuDBdHjuxc5ExDoQ8oOxzhyPqbDv01aE4dkunu 7F/P5zp321rJVLzY1NudaFtLkJuQrzkyI1VV8vtY3TmaSY2YaOf9Y3RaqcQKxwc7Z+rs 5UOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2M+/jke7oI/v1bHKeQ1t1eI5gEmwcggeXPdID1191qU=; b=sx/8cP3jlqw6j48CPMgBBzHWVCg/7IGHKpjFoBaqI0t8+7t9LmYHAZGZRAO/OQE2Ij TPpfhKzQGIv9he/ricjq/jkh8QKOopUwm+C88nzWGdYROG/QoksvUm/fXdYRzlFAi8W7 vKkh34DR57PFSsnKM2P4ID006kIgTCbBx5FcIg4q+/XgOX0hM7RuTXqXc4uU9fV/T7pr ZyXDNI5oiC2pYhWDhfusfkLpOaimYEG7JdN9TGFGy3Ii47XbU7b11IrrAely+Y30exp1 E4qAhGzqin9fb4VuTlkZzhNY/GCWhpELa6EiWIwa02tPu4g6FfEPBFKNdeQjnOPd44p5 maUg== X-Gm-Message-State: AOAM5316DqHptYBd2zUAtmAsqgxN7SZ+uKS8iNRRvbN42CVn0ifIlT6i pAMVmpFv91xNsr0K1ASsAQfsZfpr2gA= X-Google-Smtp-Source: ABdhPJxuGUAnvy+Dg+4LAsJz1+8jVMJPJvXg30/RguGRkv7I72e7VAFrc+fxXp8n12ZBi01cUM9oYQ== X-Received: by 2002:a65:520a:: with SMTP id o10mr334564pgp.172.1624391841788; Tue, 22 Jun 2021 12:57:21 -0700 (PDT) Received: from localhost ([2402:3a80:11bb:33b3:7f0c:3646:8bde:417e]) by smtp.gmail.com with ESMTPSA id z6sm20829937pgs.24.2021.06.22.12.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 12:57:21 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: netdev@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesper Dangaard Brouer , "David S. Miller" , Jakub Kicinski , John Fastabend , Martin KaFai Lau , bpf@vger.kernel.org Subject: [PATCH net-next v2 4/5] bpf: devmap: implement devmap prog execution for generic XDP Date: Wed, 23 Jun 2021 01:25:26 +0530 Message-Id: <20210622195527.1110497-5-memxor@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210622195527.1110497-1-memxor@gmail.com> References: <20210622195527.1110497-1-memxor@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This lifts the restriction on running devmap BPF progs in generic redirect mode. To match native XDP behavior, it is invoked right before generic_xdp_tx is called, and only supports XDP_PASS/XDP_ABORTED/ XDP_DROP actions. We also return 0 even if devmap program drops the packet, as semantically redirect has already succeeded and the devmap prog is the last point before TX of the packet to device where it can deliver a verdict on the packet. This also means it must take care of freeing the skb, as xdp_do_generic_redirect callers only do that in case an error is returned. Since devmap entry prog is supported, remove the check in generic_xdp_install entirely. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 1 - kernel/bpf/devmap.c | 49 ++++++++++++++++++++++++++++++++++++--------- net/core/dev.c | 18 ----------------- 3 files changed, 39 insertions(+), 29 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 095aaa104c56..4afbff308ca3 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1508,7 +1508,6 @@ int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb, int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb, struct bpf_prog *xdp_prog, struct bpf_map *map, bool exclude_ingress); -bool dev_map_can_have_prog(struct bpf_map *map); void __cpu_map_flush(void); int cpu_map_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_buff *xdp, diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 2a75e6c2d27d..49f03e8e5561 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -318,16 +318,6 @@ static int dev_map_hash_get_next_key(struct bpf_map *map, void *key, return -ENOENT; } -bool dev_map_can_have_prog(struct bpf_map *map) -{ - if ((map->map_type == BPF_MAP_TYPE_DEVMAP || - map->map_type == BPF_MAP_TYPE_DEVMAP_HASH) && - map->value_size != offsetofend(struct bpf_devmap_val, ifindex)) - return true; - - return false; -} - static int dev_map_bpf_prog_run(struct bpf_prog *xdp_prog, struct xdp_frame **frames, int n, struct net_device *dev) @@ -499,6 +489,37 @@ static inline int __xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp, return 0; } +static u32 dev_map_bpf_prog_run_skb(struct sk_buff *skb, struct bpf_dtab_netdev *dst) +{ + struct xdp_txq_info txq = { .dev = dst->dev }; + struct xdp_buff xdp; + u32 act; + + if (!dst->xdp_prog) + return XDP_PASS; + + __skb_pull(skb, skb->mac_len); + xdp.txq = &txq; + + act = bpf_prog_run_generic_xdp(skb, &xdp, dst->xdp_prog); + switch (act) { + case XDP_PASS: + __skb_push(skb, skb->mac_len); + break; + default: + bpf_warn_invalid_xdp_action(act); + fallthrough; + case XDP_ABORTED: + trace_xdp_exception(dst->dev, dst->xdp_prog, act); + fallthrough; + case XDP_DROP: + kfree_skb(skb); + break; + } + + return act; +} + int dev_xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp, struct net_device *dev_rx) { @@ -614,6 +635,14 @@ int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb, err = xdp_ok_fwd_dev(dst->dev, skb->len); if (unlikely(err)) return err; + + /* Redirect has already succeeded semantically at this point, so we just + * return 0 even if packet is dropped. Helper below takes care of + * freeing skb. + */ + if (dev_map_bpf_prog_run_skb(skb, dst) != XDP_PASS) + return 0; + skb->dev = dst->dev; generic_xdp_tx(skb, xdp_prog); diff --git a/net/core/dev.c b/net/core/dev.c index a00421e9ee16..9d9c78496459 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5633,24 +5633,6 @@ static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp) struct bpf_prog *new = xdp->prog; int ret = 0; - if (new) { - u32 i; - - mutex_lock(&new->aux->used_maps_mutex); - - /* generic XDP does not work with DEVMAPs that can - * have a bpf_prog installed on an entry - */ - for (i = 0; i < new->aux->used_map_cnt; i++) { - if (dev_map_can_have_prog(new->aux->used_maps[i])) { - mutex_unlock(&new->aux->used_maps_mutex); - return -EINVAL; - } - } - - mutex_unlock(&new->aux->used_maps_mutex); - } - switch (xdp->command) { case XDP_SETUP_PROG: rcu_assign_pointer(dev->xdp_prog, new); From patchwork Tue Jun 22 19:55:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 466278 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B814C48BE5 for ; Tue, 22 Jun 2021 19:57:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3664F6100B for ; Tue, 22 Jun 2021 19:57:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232860AbhFVT7o (ORCPT ); Tue, 22 Jun 2021 15:59:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232823AbhFVT7m (ORCPT ); Tue, 22 Jun 2021 15:59:42 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F13F3C061574; Tue, 22 Jun 2021 12:57:25 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id b3so3349187plg.2; Tue, 22 Jun 2021 12:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ho8mpG9beAl+0ggB96aMls1JfFIVbnP4cgBTDr5kAic=; b=ZaJhdTpZCheyIDvl8SMfyCgl7gpCHTjgh05riTZ4lTYuUs85WJubXFuoFYY6GlmH+x t9wRsYMhKTN84r0qcL5ZPgb9gSqjn9+sGmWtKuHXQZJDQO5IojFsQVdQVjNOHFxh/DFb 3/j2BlUsYH0Ifmu1ABOBrMIZ3OSiT/azqFaYYx1cFEWQ48PaRRwgJihExZFoEecwSjxT ru67K3J58rBCXrS5gpJV8MtXEyegxkK5NGQiLxndgo11pf9GdubOYYaq/HNBp0r/RuuY +dPL755LOcJYP/RWn2FPJnmP8IUvylFktR55BBJ2uf4M4YmepnA5mF0Sq8WcCBZTXGWC feJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ho8mpG9beAl+0ggB96aMls1JfFIVbnP4cgBTDr5kAic=; b=rSFy0lS24bLs1nT4/smGakysNFfEVEgVJwPx7lsaL8HD3UXCJB4D/2nIKd8HN8/JM8 lVfXEE86yHAucrKSNcEeutk6dR6vRnRhS6aSMC2Og/0zc09b5Va+y0VVa5zjMzDrZqPk kKje2M2Z+7Bn4rpN1NMpIIh5Fj9kIHxXX+YGU/lIp9g9v5sOcR+o6o7cKb8ED7JZL5ty +r3O7UPuHEBkfUbfp/EKNV95UqY3zjdMdRw+BD9THHRIlA6gbgLVJHGEcawr2Mhn4Kk5 3rCpZOjjrwT1rR8JHVub+jNZuZrqEBhADhQA0RqpU6139KafAmXmw+aMKSSc9YcBjEQQ gRRQ== X-Gm-Message-State: AOAM533TWI0PzRVGf8AP7Emu7hMLwPYeX8C3G/YEiFGsJN+LRrTDNL7e kBU5w999Eh1JMSABnRGKMppZkgE+Pfo= X-Google-Smtp-Source: ABdhPJwhisqMbi48Ece0jNwR2Vxewbk+DgYiPBAXSC3f4exv6GNV4jKd+tWmQSlpt0ncX+T2e0rYwg== X-Received: by 2002:a17:90a:8b0d:: with SMTP id y13mr5612234pjn.88.1624391845414; Tue, 22 Jun 2021 12:57:25 -0700 (PDT) Received: from localhost ([2402:3a80:11bb:33b3:7f0c:3646:8bde:417e]) by smtp.gmail.com with ESMTPSA id b18sm160712pft.1.2021.06.22.12.57.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Jun 2021 12:57:25 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: netdev@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesper Dangaard Brouer , "David S. Miller" , Jakub Kicinski , John Fastabend , Martin KaFai Lau , bpf@vger.kernel.org Subject: [PATCH net-next v2 5/5] bpf: update XDP selftests to not fail with generic XDP Date: Wed, 23 Jun 2021 01:25:27 +0530 Message-Id: <20210622195527.1110497-6-memxor@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210622195527.1110497-1-memxor@gmail.com> References: <20210622195527.1110497-1-memxor@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Generic XDP devmaps and cpumaps now allow setting value_size to 8 bytes (so that prog_fd can be specified) and XDP progs using them succeed in SKB mode now. Adjust the checks. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/prog_tests/xdp_cpumap_attach.c | 4 ++-- tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_cpumap_attach.c b/tools/testing/selftests/bpf/prog_tests/xdp_cpumap_attach.c index 0176573fe4e7..42e46d2ae349 100644 --- a/tools/testing/selftests/bpf/prog_tests/xdp_cpumap_attach.c +++ b/tools/testing/selftests/bpf/prog_tests/xdp_cpumap_attach.c @@ -29,8 +29,8 @@ void test_xdp_with_cpumap_helpers(void) */ prog_fd = bpf_program__fd(skel->progs.xdp_redir_prog); err = bpf_set_link_xdp_fd(IFINDEX_LO, prog_fd, XDP_FLAGS_SKB_MODE); - CHECK(err == 0, "Generic attach of program with 8-byte CPUMAP", - "should have failed\n"); + CHECK(err, "Generic attach of program with 8-byte CPUMAP", + "shouldn't have failed\n"); prog_fd = bpf_program__fd(skel->progs.xdp_dummy_cm); map_fd = bpf_map__fd(skel->maps.cpu_map); diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c b/tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c index 88ef3ec8ac4c..861db508ace2 100644 --- a/tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c +++ b/tools/testing/selftests/bpf/prog_tests/xdp_devmap_attach.c @@ -31,8 +31,8 @@ void test_xdp_with_devmap_helpers(void) */ dm_fd = bpf_program__fd(skel->progs.xdp_redir_prog); err = bpf_set_link_xdp_fd(IFINDEX_LO, dm_fd, XDP_FLAGS_SKB_MODE); - CHECK(err == 0, "Generic attach of program with 8-byte devmap", - "should have failed\n"); + CHECK(err, "Generic attach of program with 8-byte devmap", + "shouldn't have failed\n"); dm_fd = bpf_program__fd(skel->progs.xdp_dummy_dm); map_fd = bpf_map__fd(skel->maps.dm_ports);