From patchwork Mon Apr 27 22:46:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85571C81857 for ; Mon, 27 Apr 2020 22:46:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5DAFF21BE5 for ; Mon, 27 Apr 2020 22:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027611; bh=QmSY0DTkfhy13FYDnKDgFH45jP01Djrm+1DRtCEGZA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=oosu+XN2u4U3rwwx9UXzSMYX+kPxuyBCd1AsnYMyGyGxmEmSV3HEVqC4+QJFh0ATE mo5VN0DUXXrecbSOawO9zT0+yR3AYlVp5HBWrW8+xyjD/FpHfdqwmtkrpYmWkVmUVX wjtTeTWWj2tAKHD11YO84XEmKdnBbICxF8/0GiOs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726284AbgD0Wqi (ORCPT ); Mon, 27 Apr 2020 18:46:38 -0400 Received: from mail.kernel.org ([198.145.29.99]:35894 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726251AbgD0Wqh (ORCPT ); Mon, 27 Apr 2020 18:46:37 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 462BD2087E; Mon, 27 Apr 2020 22:46:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027597; bh=QmSY0DTkfhy13FYDnKDgFH45jP01Djrm+1DRtCEGZA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H4J0+vjGlmOiVWkDynVS7SDj0fS+AF3pS5oqWwgAdRcizfn4bUnhy03TqV0y3C81j gaD+LPlFAruk3L0cND4jRjIFm3d+aZpJ0DqC7pVQp9Z3jXMai9cSufTVue3WhF7T14 55NuK/1Rm8irjPyZ656YV7WU8kVcq9eerDMk2x+4= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 01/15] net: Refactor convert_to_xdp_frame Date: Mon, 27 Apr 2020 16:46:19 -0600 Message-Id: <20200427224633.15627-2-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Move the guts of convert_to_xdp_frame to a new helper, update_xdp_frame so it can be reused in a later patch. Suggested-by: Jesper Dangaard Brouer Signed-off-by: David Ahern Acked-by: Jesper Dangaard Brouer Acked-by: John Fastabend --- include/net/xdp.h | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 3cc6d5d84aa4..3264fa882de3 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -93,32 +93,42 @@ static inline void xdp_scrub_frame(struct xdp_frame *frame) struct xdp_frame *xdp_convert_zc_to_xdp_frame(struct xdp_buff *xdp); -/* Convert xdp_buff to xdp_frame */ static inline -struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +bool update_xdp_frame(struct xdp_buff *xdp, struct xdp_frame *xdp_frame) { - struct xdp_frame *xdp_frame; int metasize; int headroom; - if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) - return xdp_convert_zc_to_xdp_frame(xdp); - /* Assure headroom is available for storing info */ headroom = xdp->data - xdp->data_hard_start; metasize = xdp->data - xdp->data_meta; metasize = metasize > 0 ? metasize : 0; if (unlikely((headroom - metasize) < sizeof(*xdp_frame))) - return NULL; - - /* Store info in top of packet */ - xdp_frame = xdp->data_hard_start; + return false; xdp_frame->data = xdp->data; xdp_frame->len = xdp->data_end - xdp->data; xdp_frame->headroom = headroom - sizeof(*xdp_frame); xdp_frame->metasize = metasize; + return true; +} + +/* Convert xdp_buff to xdp_frame */ +static inline +struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +{ + struct xdp_frame *xdp_frame; + + if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) + return xdp_convert_zc_to_xdp_frame(xdp); + + /* Store info in top of packet */ + xdp_frame = xdp->data_hard_start; + + if (unlikely(!update_xdp_frame(xdp, xdp_frame))) + return NULL; + /* rxq only valid until napi_schedule ends, convert to xdp_mem_info */ xdp_frame->mem = xdp->rxq->mem; From patchwork Mon Apr 27 22:46:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9BC0C82C01 for ; Mon, 27 Apr 2020 22:46:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 959E421BE5 for ; Mon, 27 Apr 2020 22:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027611; bh=8b4+XTz2mLq34mnoxFs6xIkw/tB+P66T3si0vrPxh1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=K+Rvg066pRr0P+Xvvwdzisy27qmBVEWxR83tV1gRMbKt3xBmFV9IVgC1GbVr5yAkq 2827f7rLvSy1qQkhGI/hk38ihfnI+3CUAPAuWh2xrLQaQoHATDjfKiXBPA3A2TkQqJ 92XcnGaZnIkrKKvQF8YSAcmfF6ONrc6qZvLtzHs8= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726345AbgD0Wqk (ORCPT ); Mon, 27 Apr 2020 18:46:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:35954 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726309AbgD0Wqk (ORCPT ); Mon, 27 Apr 2020 18:46:40 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4EB2D21835; Mon, 27 Apr 2020 22:46:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027599; bh=8b4+XTz2mLq34mnoxFs6xIkw/tB+P66T3si0vrPxh1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iKtMzQ7pfcsJcE7PcC2LJeBua2wMnZi6bV0ZNrlC767Fp6zmFWZ8g2UtgqQcIIbk5 GIhVU+iXdVO+od/I7GDbmOiJkgTrLcQ+23jaAlCWzwEZ5yTZoT5LdOXZJ++MW1hEsJ 30PJ+Ow1TkUdjcDek7UR41oGnCYZqi2k9aL1BU8w= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 03/15] net: Add XDP setup and query commands for Tx programs Date: Mon, 27 Apr 2020 16:46:21 -0600 Message-Id: <20200427224633.15627-4-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add new netdev commands, XDP_SETUP_PROG_EGRESS and XDP_QUERY_PROG_EGRESS, to query and setup egress programs. Update dev_change_xdp_fd and dev_xdp_install to check for egress mode via XDP_FLAGS_EGRESS_MODE in the flags. If egress bool is set, then use XDP_SETUP_PROG_EGRESS in dev_xdp_install and XDP_QUERY_PROG_EGRESS in dev_change_xdp_fd. Signed-off-by: David Ahern Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- include/linux/netdevice.h | 2 ++ net/core/dev.c | 20 +++++++++++++++----- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 594c13d4cd00..ee0cb73ca18a 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -873,8 +873,10 @@ enum bpf_netdev_command { */ XDP_SETUP_PROG, XDP_SETUP_PROG_HW, + XDP_SETUP_PROG_EGRESS, XDP_QUERY_PROG, XDP_QUERY_PROG_HW, + XDP_QUERY_PROG_EGRESS, /* BPF program for offload callbacks, invoked at program load time. */ BPF_OFFLOAD_MAP_ALLOC, BPF_OFFLOAD_MAP_FREE, diff --git a/net/core/dev.c b/net/core/dev.c index afff16849c26..c0455e764f97 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -8600,13 +8600,16 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct bpf_prog *prog) { bool non_hw = !(flags & XDP_FLAGS_HW_MODE); + bool egress = flags & XDP_FLAGS_EGRESS_MODE; struct bpf_prog *prev_prog = NULL; struct netdev_bpf xdp; int err; if (non_hw) { - prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, - XDP_QUERY_PROG)); + enum bpf_netdev_command cmd; + + cmd = egress ? XDP_QUERY_PROG_EGRESS : XDP_QUERY_PROG; + prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, cmd)); if (IS_ERR(prev_prog)) prev_prog = NULL; } @@ -8615,7 +8618,7 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, if (flags & XDP_FLAGS_HW_MODE) xdp.command = XDP_SETUP_PROG_HW; else - xdp.command = XDP_SETUP_PROG; + xdp.command = egress ? XDP_SETUP_PROG_EGRESS : XDP_SETUP_PROG; xdp.extack = extack; xdp.flags = flags; xdp.prog = prog; @@ -8677,12 +8680,18 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, bpf_op_t bpf_op, bpf_chk; struct bpf_prog *prog; bool offload; + bool egress; int err; ASSERT_RTNL(); offload = flags & XDP_FLAGS_HW_MODE; - query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + egress = flags & XDP_FLAGS_EGRESS_MODE; + if (egress) + query = XDP_QUERY_PROG_EGRESS; + else + query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + bpf_op = bpf_chk = ops->ndo_bpf; if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { @@ -8712,7 +8721,8 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, } } if (fd >= 0) { - if (!offload && __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { + if (!offload && !egress && + __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { NL_SET_ERR_MSG(extack, "native and generic XDP can't be active at the same time"); return -EEXIST; } From patchwork Mon Apr 27 22:46:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 123B7C82C01 for ; Mon, 27 Apr 2020 22:47:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E416520661 for ; Mon, 27 Apr 2020 22:47:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027625; bh=MIV5Gf4ovOe8mpnafd9volB6feu0H3KJg8e6ERaVn/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=fhASfy4z0xTye1Hpuq5Y+c7WZXL5ds+272qBhM6vDYu5l9+gQq6urOv9wGvg0tWga +gyrUNml8ioH5DoCaHXo+2Hssd13FEu4Joi/lL4BdmgkO+WuIcCXkXIXPYBKNGGO/x 26Vwc34PNhgC25YA7yQxE2A4n0juMwqgppqiHYv0= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726402AbgD0Wqr (ORCPT ); Mon, 27 Apr 2020 18:46:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:36114 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726378AbgD0Wqp (ORCPT ); Mon, 27 Apr 2020 18:46:45 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F030921974; Mon, 27 Apr 2020 22:46:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027604; bh=MIV5Gf4ovOe8mpnafd9volB6feu0H3KJg8e6ERaVn/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YEhIvcsTlbp+6gNLA0Lk3ilFviWqkDhUzxaRKyZleIJ+F2O23ZCxsiWofhF1jdLk/ mimuyx7uUlnuKDACsgcfHnfjyvKyZxBGpLLwNJmwHN/IfTdJYy0zuo6eFjzWz56hla INSghXSWrJWjnDtJ/kXksmjh5F1cdUbbuMOIgvzg= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 08/15] net: set XDP egress program on netdevice Date: Mon, 27 Apr 2020 16:46:26 -0600 Message-Id: <20200427224633.15627-9-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern This patch handles the plumbing for installing an XDP egress program on a net_device by handling XDP_SETUP_PROG_EGRESS and XDP_QUERY_PROG_EGRESS in generic_xdp_install handler. New static key is added to signal when an egress program has been installed. Update dev_xdp_uninstall to remove egress programs. Signed-off-by: David Ahern --- include/linux/netdevice.h | 2 ++ net/core/dev.c | 48 +++++++++++++++++++++++++++------------ 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index f4d24d9ea4f9..2b552c29e188 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -752,6 +752,8 @@ struct netdev_rx_queue { #endif } ____cacheline_aligned_in_smp; +extern struct static_key_false xdp_egress_needed_key; + /* * RX queue sysfs structures and functions. */ diff --git a/net/core/dev.c b/net/core/dev.c index 5bbdbc0c0a92..14ce8e25e3d3 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4620,6 +4620,7 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) } static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); +DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { @@ -5335,12 +5336,12 @@ static void __netif_receive_skb_list(struct list_head *head) static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp) { - struct bpf_prog *old = rtnl_dereference(dev->xdp_prog); - struct bpf_prog *new = xdp->prog; + struct bpf_prog *old, *new = xdp->prog; int ret = 0; switch (xdp->command) { case XDP_SETUP_PROG: + old = rtnl_dereference(dev->xdp_prog); rcu_assign_pointer(dev->xdp_prog, new); if (old) bpf_prog_put(old); @@ -5353,11 +5354,25 @@ static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp) dev_disable_gro_hw(dev); } break; + case XDP_SETUP_PROG_EGRESS: + old = rtnl_dereference(dev->xdp_egress_prog); + rcu_assign_pointer(dev->xdp_egress_prog, new); + if (old) + bpf_prog_put(old); + if (old && !new) + static_branch_dec(&xdp_egress_needed_key); + else if (new && !old) + static_branch_inc(&xdp_egress_needed_key); + break; case XDP_QUERY_PROG: + old = rtnl_dereference(dev->xdp_prog); + xdp->prog_id = old ? old->aux->id : 0; + break; + case XDP_QUERY_PROG_EGRESS: + old = rtnl_dereference(dev->xdp_egress_prog); xdp->prog_id = old ? old->aux->id : 0; break; - default: ret = -EINVAL; break; @@ -8640,6 +8655,10 @@ static void dev_xdp_uninstall(struct net_device *dev) /* Remove generic XDP */ WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, 0, NULL)); + /* Remove XDP egress */ + WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, + XDP_FLAGS_EGRESS_MODE, NULL)); + /* Remove from the driver */ ndo_bpf = dev->netdev_ops->ndo_bpf; if (!ndo_bpf) @@ -8686,21 +8705,22 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, offload = flags & XDP_FLAGS_HW_MODE; egress = flags & XDP_FLAGS_EGRESS_MODE; - if (egress) + if (egress) { query = XDP_QUERY_PROG_EGRESS; - else + bpf_op = bpf_chk = generic_xdp_install; + } else { query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; - - bpf_op = bpf_chk = ops->ndo_bpf; - if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { - NL_SET_ERR_MSG(extack, "underlying driver does not support XDP in native mode"); - return -EOPNOTSUPP; + bpf_op = bpf_chk = ops->ndo_bpf; + if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { + NL_SET_ERR_MSG(extack, "underlying driver does not support XDP in native mode"); + return -EOPNOTSUPP; + } + if (!bpf_op || (flags & XDP_FLAGS_SKB_MODE)) + bpf_op = generic_xdp_install; + if (bpf_op == bpf_chk) + bpf_chk = generic_xdp_install; } - if (!bpf_op || (flags & XDP_FLAGS_SKB_MODE)) - bpf_op = generic_xdp_install; - if (bpf_op == bpf_chk) - bpf_chk = generic_xdp_install; prog_id = __dev_xdp_query(dev, bpf_op, query); if (flags & XDP_FLAGS_REPLACE) { From patchwork Mon Apr 27 22:46:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D4EEC82C03 for ; Mon, 27 Apr 2020 22:46:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 157A920661 for ; Mon, 27 Apr 2020 22:46:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027612; bh=vEHabIriy15AdTLfBBBxnveDOqMzjzZP7OAIjAscyDk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=TtFXYPOUFpsAcOe9ZGhFtUN+5EBHSNHnUkKRoSbJa2M9QNmdLVNLc1boNFM3iRX7p tb5Nrnr/kaewDQ7nRXbrohqQ/yyDBBDMavGiqsM7zfoe+alNZEKHFJn6I4+OBJfIhQ GjSKx536U8DJ6O0W/JtLoOrhlgjZVCDUfHTkEx4w= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726415AbgD0Wqt (ORCPT ); Mon, 27 Apr 2020 18:46:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:36138 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726396AbgD0Wqq (ORCPT ); Mon, 27 Apr 2020 18:46:46 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F3FD02078C; Mon, 27 Apr 2020 22:46:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027605; bh=vEHabIriy15AdTLfBBBxnveDOqMzjzZP7OAIjAscyDk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E1ELxRFEXWdrgmKsCfGQsgo0zfU19H4zcwqgwfIKf5VMxd9au/qapmX0HPWgWZzyZ cRipl4CFtkaMIGkTFnQsrmur5ZDR/2jEC7p0GyaAJcb+1lDvsqYQo4/6n6f1HaakJU 5eJULg5r9gWDVevLpryiufJy89VCefTQ0foSZXSA= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 09/15] net: Support xdp in the Tx path for packets as an skb Date: Mon, 27 Apr 2020 16:46:27 -0600 Message-Id: <20200427224633.15627-10-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on packets about to hit the ndo_start_xmit function for a device. Only XDP_DROP and XDP_PASS are supported now. Conceptually, XDP_REDIRECT for this path can work the same as it does for the Rx path, but that support is left for a follow on series. Signed-off-by: David Ahern --- include/linux/netdevice.h | 11 +++++++++ net/core/dev.c | 52 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 2b552c29e188..33a09396444f 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3717,6 +3717,7 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); @@ -4577,6 +4578,16 @@ static inline netdev_tx_t __netdev_start_xmit(const struct net_device_ops *ops, struct sk_buff *skb, struct net_device *dev, bool more) { + if (static_branch_unlikely(&xdp_egress_needed_key)) { + u32 act; + + rcu_read_lock(); + act = do_xdp_egress_skb(dev, skb); + rcu_read_unlock(); + if (act == XDP_DROP) + return NET_XMIT_DROP; + } + __this_cpu_write(softnet_data.xmit.more, more); return ops->ndo_start_xmit(skb, dev); } diff --git a/net/core/dev.c b/net/core/dev.c index 14ce8e25e3d3..4d98189548c7 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4620,7 +4620,6 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) } static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); -DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { @@ -4671,6 +4670,57 @@ int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_generic_rx); +DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); +EXPORT_SYMBOL_GPL(xdp_egress_needed_key); + +static u32 handle_xdp_egress_act(u32 act, struct net_device *dev, + struct bpf_prog *xdp_prog) +{ + switch (act) { + case XDP_DROP: + /* fall through */ + case XDP_PASS: + break; + case XDP_TX: + /* fall through */ + case XDP_REDIRECT: + /* fall through */ + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(dev, xdp_prog, act); + act = XDP_DROP; + break; + } + + return act; +} + +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb) +{ + struct bpf_prog *xdp_prog; + u32 act = XDP_PASS; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + struct xdp_buff xdp; + + xdp.txq = &txq; + act = do_xdp_generic_core(skb, &xdp, xdp_prog); + act = handle_xdp_egress_act(act, dev, xdp_prog); + if (act == XDP_DROP) { + atomic_long_inc(&dev->tx_dropped); + skb_tx_error(skb); + kfree_skb(skb); + } + } + + return act; +} +EXPORT_SYMBOL_GPL(do_xdp_egress_skb); + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Mon Apr 27 22:46:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F0EDC82C01 for ; Mon, 27 Apr 2020 22:47:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1846220661 for ; Mon, 27 Apr 2020 22:47:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027629; bh=o6bxY+wu5tMIqMCl4XpCbRXZMLE7m3Vj7RBysk7OZVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=QhVwIxkuRrsOuXcApqiWEoyPYmJOTYdQKD2mir7GfItvT522chESZ1P/2FG6FntjO Y6NnZQUi8AbdDzC4ty2KpDfeieispfBoE71ovE5mWKHYJoFHoVYesuYZIGmX/IpRa6 Lu1Oj2Yi18tfA/PUHcrzzisyqm2kXFQTKtz67yvk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726458AbgD0WrI (ORCPT ); Mon, 27 Apr 2020 18:47:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:36174 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726401AbgD0Wqr (ORCPT ); Mon, 27 Apr 2020 18:46:47 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 055F721D79; Mon, 27 Apr 2020 22:46:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027606; bh=o6bxY+wu5tMIqMCl4XpCbRXZMLE7m3Vj7RBysk7OZVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BwoVSUD01eF4q/IaCB8KTmLXvlVwXu0YE/ualhMrD/DGROjbEg/ritub7lbaRMQr5 GGwTWrQGdkze2fDbOlIJyjX51AE+5wwDw5RKhBhwadPFsj0wW+VOrWvuqwVSpZl179 Sz2Fe/WQkuoWEkZnsz0miVIYGsge97ZK0aOawLSs= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 10/15] net: Support xdp in the Tx path for xdp_frames Date: Mon, 27 Apr 2020 16:46:28 -0600 Message-Id: <20200427224633.15627-11-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on xdp_frames by adding a hook to bq_xmit_all before xdp_frames are passed to ndo_xdp_xmit for the device. If an xdp_frame is dropped by the program, it is removed from the xdp_frames array with subsequent entries moved up. Signed-off-by: David Ahern --- include/linux/netdevice.h | 3 ++ kernel/bpf/devmap.c | 19 +++++++++---- net/core/dev.c | 59 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 33a09396444f..8c707ce9ab65 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3718,6 +3718,9 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb); +unsigned int do_xdp_egress_frame(struct net_device *dev, + struct xdp_frame **frames, + unsigned int count); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index a51d9fb7a359..3add36d697a8 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -321,24 +321,33 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) { struct net_device *dev = bq->dev; int sent = 0, drops = 0, err = 0; + unsigned int count = bq->count; int i; - if (unlikely(!bq->count)) + if (unlikely(!count)) return 0; - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; prefetch(xdpf); } - sent = dev->netdev_ops->ndo_xdp_xmit(dev, bq->count, bq->q, flags); + if (static_branch_unlikely(&xdp_egress_needed_key)) { + count = do_xdp_egress_frame(dev, bq->q, count); + drops += bq->count - count; + /* all frames consumed by the xdp program? */ + if (!count) + goto out; + } + + sent = dev->netdev_ops->ndo_xdp_xmit(dev, count, bq->q, flags); if (sent < 0) { err = sent; sent = 0; goto error; } - drops = bq->count - sent; + drops += count - sent; out: bq->count = 0; @@ -350,7 +359,7 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) /* If ndo_xdp_xmit fails with an errno, no frames have been * xmit'ed and it's our responsibility to them free all. */ - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; xdp_return_frame_rx_napi(xdpf); diff --git a/net/core/dev.c b/net/core/dev.c index 4d98189548c7..62ef6bf80998 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4721,6 +4721,65 @@ u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_egress_skb); +static u32 __xdp_egress_frame(struct net_device *dev, + struct bpf_prog *xdp_prog, + struct xdp_frame *xdp_frame, + struct xdp_txq_info *txq) +{ + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = xdp_frame->data - xdp_frame->headroom + - sizeof(*xdp_frame); + xdp.data = xdp_frame->data; + xdp.data_end = xdp.data + xdp_frame->len; + xdp.data_meta = xdp.data - xdp_frame->metasize; + xdp.txq = txq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + act = handle_xdp_egress_act(act, dev, xdp_prog); + + /* if not dropping frame, readjust pointers in case + * program made changes to the buffer + */ + if (act != XDP_DROP) { + if (unlikely(!update_xdp_frame(&xdp, xdp_frame))) + return XDP_DROP; + } + + return act; +} + +unsigned int do_xdp_egress_frame(struct net_device *dev, + struct xdp_frame **frames, + unsigned int count) +{ + struct bpf_prog *xdp_prog; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + unsigned int i, j; + u32 act; + + for (i = 0, j = 0; i < count; i++) { + struct xdp_frame *frame = frames[i]; + + act = __xdp_egress_frame(dev, xdp_prog, frame, &txq); + if (act == XDP_DROP) { + xdp_return_frame_rx_napi(frame); + continue; + } + + frames[j] = frame; + j++; + } + count = j; + } + + return count; +} + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Mon Apr 27 22:46:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A749C82C00 for ; Mon, 27 Apr 2020 22:46:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 34ED320661 for ; Mon, 27 Apr 2020 22:46:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027619; bh=XZap2EEumrmCAj5GHjnK0BMGpW3p3efAnFk5mMSvEgY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=K5hyC1EJXq0ecyJvC7F1JD3hOtAfY4s+YK6aPTG8jnnRwZsGLi5hxH8gGpmA96qQH DVVMRXguwoO4ACCXIEX8sridOthdeUMLfzdSEzhi7gQleuaWCACicEi4rTVyMxOR5E 2+DT4vWuUcQxsqKm/T07WWI87aN+w8f/CjzBjSIc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726436AbgD0Wq5 (ORCPT ); Mon, 27 Apr 2020 18:46:57 -0400 Received: from mail.kernel.org ([198.145.29.99]:36280 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726410AbgD0Wqu (ORCPT ); Mon, 27 Apr 2020 18:46:50 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3764620B80; Mon, 27 Apr 2020 22:46:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027610; bh=XZap2EEumrmCAj5GHjnK0BMGpW3p3efAnFk5mMSvEgY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ALDUoTdB5j3G2WTYNA+u8/2Kxut6rK78GTYRUwZGDta0DoJp+qUr9/BJmGs2FSKNy kpwOJ1t7IJm3BF7+rGLW+EKBJbfT7NeP6lknCpNd5sjv9QU7eV9qErBWOGIImwchj7 fmYvxEh3oyr/hDFvSv0xxQcTIQTpH+a9Ak8ivwCg= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 13/15] selftest: Add test for xdp_egress Date: Mon, 27 Apr 2020 16:46:31 -0600 Message-Id: <20200427224633.15627-14-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add selftest for xdp_egress. Add xdp_drop program to veth connecting a namespace to drop packets and break connectivity. Signed-off-by: David Ahern --- tools/testing/selftests/bpf/Makefile | 1 + tools/testing/selftests/bpf/progs/xdp_drop.c | 25 +++ .../testing/selftests/bpf/test_xdp_egress.sh | 160 ++++++++++++++++++ 3 files changed, 186 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/xdp_drop.c create mode 100755 tools/testing/selftests/bpf/test_xdp_egress.sh diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 7729892e0b04..5dae18ebac13 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -50,6 +50,7 @@ TEST_PROGS := test_kmod.sh \ test_xdp_redirect.sh \ test_xdp_meta.sh \ test_xdp_veth.sh \ + test_xdp_egress.sh \ test_offload.py \ test_sock_addr.sh \ test_tunnel.sh \ diff --git a/tools/testing/selftests/bpf/progs/xdp_drop.c b/tools/testing/selftests/bpf/progs/xdp_drop.c new file mode 100644 index 000000000000..cffabc53a5e1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/xdp_drop.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +SEC("drop") +int xdp_drop(struct xdp_md *ctx) +{ + void *data_end = (void *)(long)ctx->data_end; + void *data = (void *)(long)ctx->data; + struct ethhdr *eth = data; + void *nh; + + nh = data + sizeof(*eth); + if (nh > data_end) + return XDP_DROP; + + if (eth->h_proto == 0x0008) + return XDP_DROP; + + return XDP_PASS; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/test_xdp_egress.sh b/tools/testing/selftests/bpf/test_xdp_egress.sh new file mode 100755 index 000000000000..7efa59fdf823 --- /dev/null +++ b/tools/testing/selftests/bpf/test_xdp_egress.sh @@ -0,0 +1,160 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# XDP egress tests. + +# Kselftest framework requirement - SKIP code is 4. +ksft_skip=4 + +TESTNAME=xdp_egress +BPF_FS=$(awk '$3 == "bpf" {print $2; exit}' /proc/mounts) + +ret=0 + +################################################################################ +# +log_test() +{ + local rc=$1 + local expected=$2 + local msg="$3" + + if [ ${rc} -eq ${expected} ]; then + printf "TEST: %-60s [ OK ]\n" "${msg}" + else + ret=1 + printf "TEST: %-60s [FAIL]\n" "${msg}" + fi +} + +################################################################################ +# create namespaces and connect them + +create_ns() +{ + local ns=$1 + local addr=$2 + local addr6=$3 + + ip netns add ${ns} + + ip -netns ${ns} link set lo up + ip -netns ${ns} addr add dev lo ${addr} + ip -netns ${ns} -6 addr add dev lo ${addr6} + + ip -netns ${ns} ro add unreachable default metric 8192 + ip -netns ${ns} -6 ro add unreachable default metric 8192 + + ip netns exec ${ns} sysctl -qw net.ipv4.ip_forward=1 + ip netns exec ${ns} sysctl -qw net.ipv6.conf.all.keep_addr_on_down=1 + ip netns exec ${ns} sysctl -qw net.ipv6.conf.all.forwarding=1 + ip netns exec ${ns} sysctl -qw net.ipv6.conf.all.forwarding=1 + ip netns exec ${ns} sysctl -qw net.ipv6.conf.all.accept_dad=0 +} + +connect_ns() +{ + local ns1=$1 + local ns1_dev=$2 + local ns1_addr=$3 + local ns1_addr6=$4 + local ns2=$5 + local ns2_dev=$6 + local ns2_addr=$7 + local ns2_addr6=$8 + local ns1arg + local ns2arg + + if [ -n "${ns1}" ]; then + ns1arg="-netns ${ns1}" + fi + if [ -n "${ns2}" ]; then + ns2arg="-netns ${ns2}" + fi + + ip ${ns1arg} li add ${ns1_dev} type veth peer name tmp + ip ${ns1arg} li set ${ns1_dev} up + ip ${ns1arg} li set tmp netns ${ns2} name ${ns2_dev} + ip ${ns2arg} li set ${ns2_dev} up + + ip ${ns1arg} addr add dev ${ns1_dev} ${ns1_addr} + ip ${ns2arg} addr add dev ${ns2_dev} ${ns2_addr} + + ip ${ns1arg} addr add dev ${ns1_dev} ${ns1_addr6} nodad + ip ${ns2arg} addr add dev ${ns2_dev} ${ns2_addr6} nodad +} + +################################################################################ +# + +setup() +{ + create_ns host 172.16.101.1/32 2001:db8:101::1/128 + connect_ns "" veth-host 172.16.1.1/24 2001:db8:1::1/64 host eth0 172.16.1.2/24 2001:db8:1::2/64 + ip ro add 172.16.101.1 via 172.16.1.2 + ip -6 ro add 2001:db8:101::1 via 2001:db8:1::2 + ping -c1 -w1 172.16.101.1 >/dev/null 2>&1 + ping -c1 -w1 2001:db8:101::1 >/dev/null 2>&1 +} + +cleanup() +{ + ip li del veth-host 2>/dev/null + ip netns del host 2>/dev/null + rm -f $BPF_FS/test_$TESTNAME +} + +################################################################################ +# main + +if [ $(id -u) -ne 0 ]; then + echo "selftests: $TESTNAME [SKIP] Need root privileges" + exit $ksft_skip +fi + +if ! ip link set dev lo xdp off > /dev/null 2>&1; then + echo "selftests: $TESTNAME [SKIP] Could not run test without the ip xdp support" + exit $ksft_skip +fi + +if [ -z "$BPF_FS" ]; then + echo "selftests: $TESTNAME [SKIP] Could not run test without bpffs mounted" + exit $ksft_skip +fi + +if ! bpftool version > /dev/null 2>&1; then + echo "selftests: $TESTNAME [SKIP] Could not run test without bpftool" + exit $ksft_skip +fi + +cleanup +trap cleanup EXIT + +set -e +setup +set +e + +bpftool prog load xdp_drop.o $BPF_FS/test_$TESTNAME type xdp_egress || exit 1 +ID=$(bpftool prog show name xdp_drop | awk '$4 == "xdp_drop" {print $1}') + +# attach egress program +bpftool net attach xdp_egress id ${ID/:/} dev veth-host +ping -c1 -w1 172.16.101.1 >/dev/null 2>&1 +log_test $? 1 "IPv4 connectivity disabled by xdp_egress" +ping -c1 -w1 2001:db8:101::1 >/dev/null 2>&1 +log_test $? 0 "IPv6 connectivity not disabled by egress drop program" + +# detach program should restore connectivity +bpftool net detach xdp_egress dev veth-host +ping -c1 -w1 172.16.101.1 >/dev/null 2>&1 +log_test $? 0 "IPv4 connectivity restored" + +# cleanup on delete +ip netns exec host bpftool net attach xdp_egress id ${ID/:/} dev eth0 +bpftool net attach xdp_egress id ${ID/:/} dev veth-host +ip li del veth-host +rm -f $BPF_FS/test_$TESTNAME +sleep 5 # rcu grace pass; verify program is dropped +bpftool prog show name xdp_drop + +exit $ret From patchwork Mon Apr 27 22:46:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9445C82C00 for ; Mon, 27 Apr 2020 22:47:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D67020661 for ; Mon, 27 Apr 2020 22:47:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027622; bh=y6cYodOnqiQ7E1VdSwSuqVLLoLlzcN3HGsrUIcZG4y0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=tJmFE5iJEOP7fBXVWwTFY9Wfe27nKeUusUGl003GghXNjXRW+UHlt5A1BzokCcX2e x6C3wbwuLmY/kwzK5S+dKZzUCG6BAXTQOEU8QkPVkUeiZZKLYsJyNVwXaixJROkcjZ XSjRZj8PXxkPk9fvsZ6coGoU1WFMpgZO5AizSJ8g= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726450AbgD0WrB (ORCPT ); Mon, 27 Apr 2020 18:47:01 -0400 Received: from mail.kernel.org ([198.145.29.99]:36344 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726423AbgD0Wqw (ORCPT ); Mon, 27 Apr 2020 18:46:52 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5BC6C218AC; Mon, 27 Apr 2020 22:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588027612; bh=y6cYodOnqiQ7E1VdSwSuqVLLoLlzcN3HGsrUIcZG4y0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X0yo6wzaTqMNnQWJpOlmF4/cCbY58hED8KuR2JILZmqQDqtAfR9pMv9dEjSWfWsxQ WOomMi3aq6OxL6++J58ze48JvYnY5TTPYExI9NkHzE7KcqmHE/c4PRETFTgXnRPSt4 LwIQ4DkrBDRz73RHj6ff8n5wFQbCUoW6maPyLIA0= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v4 bpf-next 15/15] samples/bpf: add XDP egress support to xdp1 Date: Mon, 27 Apr 2020 16:46:33 -0600 Message-Id: <20200427224633.15627-16-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200427224633.15627-1-dsahern@kernel.org> References: <20200427224633.15627-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern xdp1 and xdp2 now accept -E flag to set XDP program in the egress path. Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- samples/bpf/xdp1_user.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/samples/bpf/xdp1_user.c b/samples/bpf/xdp1_user.c index c447ad9e3a1d..bb104f4d8c5e 100644 --- a/samples/bpf/xdp1_user.c +++ b/samples/bpf/xdp1_user.c @@ -73,7 +73,8 @@ static void usage(const char *prog) "OPTS:\n" " -S use skb-mode\n" " -N enforce native mode\n" - " -F force loading prog\n", + " -F force loading prog\n" + " -E egress path program\n", prog); } @@ -85,7 +86,7 @@ int main(int argc, char **argv) }; struct bpf_prog_info info = {}; __u32 info_len = sizeof(info); - const char *optstr = "FSN"; + const char *optstr = "FSNE"; int prog_fd, map_fd, opt; struct bpf_object *obj; struct bpf_map *map; @@ -103,13 +104,17 @@ int main(int argc, char **argv) case 'F': xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; break; + case 'E': + xdp_flags |= XDP_FLAGS_EGRESS_MODE; + prog_load_attr.expected_attach_type = BPF_XDP_EGRESS; + break; default: usage(basename(argv[0])); return 1; } } - if (!(xdp_flags & XDP_FLAGS_SKB_MODE)) + if (!(xdp_flags & (XDP_FLAGS_SKB_MODE | XDP_FLAGS_EGRESS_MODE))) xdp_flags |= XDP_FLAGS_DRV_MODE; if (optind == argc) {