From patchwork Sat Sep 29 11:28:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilias Apalodimas X-Patchwork-Id: 147836 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1815187lji; Sat, 29 Sep 2018 04:28:18 -0700 (PDT) X-Google-Smtp-Source: ACcGV6136ffC//JcnnyrMG0y2n00aIDbugJKM82muA89ePztM0iu0xC4sR14NzWFzR9Gv0x0JK8u X-Received: by 2002:a63:5445:: with SMTP id e5-v6mr2615361pgm.316.1538220497939; Sat, 29 Sep 2018 04:28:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538220497; cv=none; d=google.com; s=arc-20160816; b=oP90NbaO/0z/bRcEmEgMtyq5tGOkb048Iz7agRslOyZiUDA+cCovvdxPfxwyW064pE CH2lLPS4wvMPiyLlH1W6Sr7SXHKfXoMQeXN5qw+vPDAJ94zDu7448eC6f/D+zoNTCl0T /FfcAXBtbkPWKmH9Sj2nxTRly6hRKJ4rqSPXPcGtxf3475cBYKIJX3MMizG6g9su1XBQ RLp2tSn+1AZlv4Snn8ym9pFzvF5LP4qHB+vmeECG0LyHbYIALiqnk+1CLDhO+JZHRGw4 nnGkzPhVMp64LYd6z58uLbV6fIivqBdPXZwmafzCdf60EFpNn0/h6fLAGhQa4gn+x+Rh yDIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=/O1Qc58Y8ZoDoCSyTMy0cxjicF0L4uRy8MQT4/cGw0w=; b=zRm6Cx8Gqsw579HkMS2YenRC42FYlymzXUDZhDBwIAwEDO7pEQclE0fZ45p9mZxGWD rLxNxUGvLuKJCALj05Sk+k7ipPg2gx2oED9SqgxPuXTSqAupNGGXfvno5Ci4ThAukXQS FmRgu2FsLtPEB0r1oZzlbUDKe8tPONI4KopS2swXDpVIj3cZcOmVrhdxuAEcRA3dPhAv Qqt1HDE1SFnzIP/IHRW+lVX+ZscbDrkxcPkq0FgQE4Ddxgyn84zm/A3Kk1lOadllPZ/5 Lsr+0fm8lk1de59JCNDaKF1fQ+NkSQ2IUWNAuviSceWBqA9+/5svQ8qHowFHPs5+T19J a64w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fQGALSAs; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m76-v6si7494789pfg.323.2018.09.29.04.28.17; Sat, 29 Sep 2018 04:28:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fQGALSAs; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728164AbeI2R4W (ORCPT + 10 others); Sat, 29 Sep 2018 13:56:22 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:53535 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727961AbeI2R4V (ORCPT ); Sat, 29 Sep 2018 13:56:21 -0400 Received: by mail-wm1-f65.google.com with SMTP id b19-v6so4492919wme.3 for ; Sat, 29 Sep 2018 04:28:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/O1Qc58Y8ZoDoCSyTMy0cxjicF0L4uRy8MQT4/cGw0w=; b=fQGALSAs0JeEnZ73JnjjohmgL8WC8a2Jy3QUDg4s3+oS6F6hFYt5amj0myER2I24h6 a1cQN4/npZ1ShkJ6j8St+bY4d4EGF57TYmPXFhqP2dRzi5mpIP0bvP/vUJBD2YR3VSTk MqWuYpLgIswgBMpnxqgT4lkSAEyV2VvJlwWa0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/O1Qc58Y8ZoDoCSyTMy0cxjicF0L4uRy8MQT4/cGw0w=; b=V1OqGkI3XAHrr5bscBz4/SUA2UoGu1MZloIolUeWx9v1wSfMK5OFjPvF/jq7dlItec 1Z5BQ7KnzUd6dzzKlNLQQnlYUOAEuESjyjZ6ms6Kp4z2nIGpNj90I+9pDbdS9rpL9jj7 zYO1/Vr8HmL1Mfu6iQ+KLCJuhfKls+pxXJqgfA5g/zkJTrcxxrdIWPSp8NuU3tcC9qiG Ktb6Fyj9SBoXbMg8R+TuMrdhkIVorrgnRdemGcu0uk/BG7JovmJtArOgmP5bJ5D6AAEz 3lT/BQzObNJgrj8tbvyRPi4Xv/WUnskYWHaaY7X2J+2+8eWRK2/9JiUH9eY+YrgRtIZH SKYg== X-Gm-Message-State: ABuFfohEIpTvrJj2GXlRrKq2dujzFQczCWgdtV7knVT8QD0Y9z5VzUMP gO+mPxZ1lFg+jyEUXrxQsHB59lUhzYw= X-Received: by 2002:a1c:1d58:: with SMTP id d85-v6mr4411454wmd.142.1538220492035; Sat, 29 Sep 2018 04:28:12 -0700 (PDT) Received: from localhost.localdomain ([2a02:587:4603:8d00:2e56:dcff:fe9a:8f06]) by smtp.gmail.com with ESMTPSA id q5-v6sm6841080wmd.29.2018.09.29.04.28.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 29 Sep 2018 04:28:11 -0700 (PDT) From: Ilias Apalodimas To: netdev@vger.kernel.org, jaswinder.singh@linaro.org Cc: ard.biesheuvel@linaro.org, masami.hiramatsu@linaro.org, arnd@arndb.de, bjorn.topel@intel.com, magnus.karlsson@intel.com, brouer@redhat.com, daniel@iogearbox.net, ast@kernel.org, jesus.sanchez-palencia@intel.com, vinicius.gomes@intel.com, makita.toshiaki@lab.ntt.co.jp, Ilias Apalodimas Subject: [net-next, PATCH 1/2, v3] net: socionext: different approach on DMA Date: Sat, 29 Sep 2018 14:28:01 +0300 Message-Id: <1538220482-16129-2-git-send-email-ilias.apalodimas@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538220482-16129-1-git-send-email-ilias.apalodimas@linaro.org> References: <1538220482-16129-1-git-send-email-ilias.apalodimas@linaro.org> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Current driver dynamically allocates an skb and maps it as DMA rx buffer. A following patch introduces XDP functionality, so we need a different allocation scheme. Buffers are allocated dynamically and mapped into hardware. During the Rx operation the driver uses build_skb() to produce the necessary buffers for the network stack Signed-off-by: Ilias Apalodimas --- drivers/net/ethernet/socionext/netsec.c | 238 +++++++++++++++++--------------- 1 file changed, 129 insertions(+), 109 deletions(-) -- 2.7.4 diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index 7aa5ebb..8f788a1 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -296,6 +296,11 @@ struct netsec_rx_pkt_info { bool err_flag; }; +static void netsec_rx_fill(struct netsec_priv *priv, u16 from, u16 num); + +static void *netsec_alloc_rx_data(struct netsec_priv *priv, + dma_addr_t *dma_addr, u16 *len); + static void netsec_write(struct netsec_priv *priv, u32 reg_addr, u32 val) { writel(val, priv->ioaddr + reg_addr); @@ -556,34 +561,10 @@ static const struct ethtool_ops netsec_ethtool_ops = { /************* NETDEV_OPS FOLLOW *************/ -static struct sk_buff *netsec_alloc_skb(struct netsec_priv *priv, - struct netsec_desc *desc) -{ - struct sk_buff *skb; - - if (device_get_dma_attr(priv->dev) == DEV_DMA_COHERENT) { - skb = netdev_alloc_skb_ip_align(priv->ndev, desc->len); - } else { - desc->len = L1_CACHE_ALIGN(desc->len); - skb = netdev_alloc_skb(priv->ndev, desc->len); - } - if (!skb) - return NULL; - - desc->addr = skb->data; - desc->dma_addr = dma_map_single(priv->dev, desc->addr, desc->len, - DMA_FROM_DEVICE); - if (dma_mapping_error(priv->dev, desc->dma_addr)) { - dev_kfree_skb_any(skb); - return NULL; - } - return skb; -} static void netsec_set_rx_de(struct netsec_priv *priv, struct netsec_desc_ring *dring, u16 idx, - const struct netsec_desc *desc, - struct sk_buff *skb) + const struct netsec_desc *desc) { struct netsec_de *de = dring->vaddr + DESC_SZ * idx; u32 attr = (1 << NETSEC_RX_PKT_OWN_FIELD) | @@ -602,59 +583,6 @@ static void netsec_set_rx_de(struct netsec_priv *priv, dring->desc[idx].dma_addr = desc->dma_addr; dring->desc[idx].addr = desc->addr; dring->desc[idx].len = desc->len; - dring->desc[idx].skb = skb; -} - -static struct sk_buff *netsec_get_rx_de(struct netsec_priv *priv, - struct netsec_desc_ring *dring, - u16 idx, - struct netsec_rx_pkt_info *rxpi, - struct netsec_desc *desc, u16 *len) -{ - struct netsec_de de = {}; - - memcpy(&de, dring->vaddr + DESC_SZ * idx, DESC_SZ); - - *len = de.buf_len_info >> 16; - - rxpi->err_flag = (de.attr >> NETSEC_RX_PKT_ER_FIELD) & 1; - rxpi->rx_cksum_result = (de.attr >> NETSEC_RX_PKT_CO_FIELD) & 3; - rxpi->err_code = (de.attr >> NETSEC_RX_PKT_ERR_FIELD) & - NETSEC_RX_PKT_ERR_MASK; - *desc = dring->desc[idx]; - return desc->skb; -} - -static struct sk_buff *netsec_get_rx_pkt_data(struct netsec_priv *priv, - struct netsec_rx_pkt_info *rxpi, - struct netsec_desc *desc, - u16 *len) -{ - struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; - struct sk_buff *tmp_skb, *skb = NULL; - struct netsec_desc td; - int tail; - - *rxpi = (struct netsec_rx_pkt_info){}; - - td.len = priv->ndev->mtu + 22; - - tmp_skb = netsec_alloc_skb(priv, &td); - - tail = dring->tail; - - if (!tmp_skb) { - netsec_set_rx_de(priv, dring, tail, &dring->desc[tail], - dring->desc[tail].skb); - } else { - skb = netsec_get_rx_de(priv, dring, tail, rxpi, desc, len); - netsec_set_rx_de(priv, dring, tail, &td, tmp_skb); - } - - /* move tail ahead */ - dring->tail = (dring->tail + 1) % DESC_NUM; - - return skb; } static int netsec_clean_tx_dring(struct netsec_priv *priv, int budget) @@ -721,19 +649,28 @@ static int netsec_process_tx(struct netsec_priv *priv, int budget) return done; } +static void netsec_adv_desc(u16 *idx) +{ + *idx = *idx + 1; + if (unlikely(*idx >= DESC_NUM)) + *idx = 0; +} + static int netsec_process_rx(struct netsec_priv *priv, int budget) { struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; struct net_device *ndev = priv->ndev; - struct netsec_rx_pkt_info rx_info; - int done = 0; - struct netsec_desc desc; struct sk_buff *skb; - u16 len; + int done = 0; while (done < budget) { u16 idx = dring->tail; struct netsec_de *de = dring->vaddr + (DESC_SZ * idx); + struct netsec_desc *desc = &dring->desc[idx]; + struct netsec_rx_pkt_info rpi; + u16 pkt_len, desc_len; + dma_addr_t dma_handle; + void *buf_addr; if (de->attr & (1U << NETSEC_RX_PKT_OWN_FIELD)) break; @@ -744,28 +681,62 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) */ dma_rmb(); done++; - skb = netsec_get_rx_pkt_data(priv, &rx_info, &desc, &len); - if (unlikely(!skb) || rx_info.err_flag) { + + pkt_len = de->buf_len_info >> 16; + rpi.err_code = (de->attr >> NETSEC_RX_PKT_ERR_FIELD) & + NETSEC_RX_PKT_ERR_MASK; + rpi.err_flag = (de->attr >> NETSEC_RX_PKT_ER_FIELD) & 1; + if (rpi.err_flag) { netif_err(priv, drv, priv->ndev, - "%s: rx fail err(%d)\n", - __func__, rx_info.err_code); + "%s: rx fail err(%d)\n", __func__, + rpi.err_code); ndev->stats.rx_dropped++; + netsec_adv_desc(&dring->tail); + /* reuse buffer page frag */ + netsec_rx_fill(priv, idx, 1); continue; } + rpi.rx_cksum_result = (de->attr >> NETSEC_RX_PKT_CO_FIELD) & 3; - dma_unmap_single(priv->dev, desc.dma_addr, desc.len, - DMA_FROM_DEVICE); - skb_put(skb, len); + buf_addr = netsec_alloc_rx_data(priv, &dma_handle, &desc_len); + if (unlikely(!buf_addr)) + break; + + dma_sync_single_for_cpu(priv->dev, desc->dma_addr, pkt_len, + DMA_FROM_DEVICE); + prefetch(desc->addr); + + skb = build_skb(desc->addr, desc->len); + if (unlikely(!skb)) { + dma_unmap_single(priv->dev, dma_handle, desc_len, + DMA_TO_DEVICE); + skb_free_frag(buf_addr); + netif_err(priv, drv, priv->ndev, + "rx failed to alloc skb\n"); + break; + } + dma_unmap_single_attrs(priv->dev, desc->dma_addr, desc->len, + DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); + + /* Update the descriptor with fresh buffers */ + desc->len = desc_len; + desc->dma_addr = dma_handle; + desc->addr = buf_addr; + + skb_put(skb, pkt_len); skb->protocol = eth_type_trans(skb, priv->ndev); if (priv->rx_cksum_offload_flag && - rx_info.rx_cksum_result == NETSEC_RX_CKSUM_OK) + rpi.rx_cksum_result == NETSEC_RX_CKSUM_OK) skb->ip_summed = CHECKSUM_UNNECESSARY; if (napi_gro_receive(&priv->napi, skb) != GRO_DROP) { ndev->stats.rx_packets++; - ndev->stats.rx_bytes += len; + ndev->stats.rx_bytes += pkt_len; } + + netsec_rx_fill(priv, idx, 1); + netsec_adv_desc(&dring->tail); } return done; @@ -928,7 +899,10 @@ static void netsec_uninit_pkt_dring(struct netsec_priv *priv, int id) dma_unmap_single(priv->dev, desc->dma_addr, desc->len, id == NETSEC_RING_RX ? DMA_FROM_DEVICE : DMA_TO_DEVICE); - dev_kfree_skb(desc->skb); + if (id == NETSEC_RING_RX) + skb_free_frag(desc->addr); + else if (id == NETSEC_RING_TX) + dev_kfree_skb(desc->skb); } memset(dring->desc, 0, sizeof(struct netsec_desc) * DESC_NUM); @@ -953,50 +927,96 @@ static void netsec_free_dring(struct netsec_priv *priv, int id) dring->desc = NULL; } +static void *netsec_alloc_rx_data(struct netsec_priv *priv, + dma_addr_t *dma_handle, u16 *desc_len) +{ + size_t len = priv->ndev->mtu + ETH_HLEN + 2 * VLAN_HLEN + NET_SKB_PAD + + NET_IP_ALIGN; + dma_addr_t mapping; + void *buf; + + len += SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + len = SKB_DATA_ALIGN(len); + + buf = napi_alloc_frag(len); + if (!buf) + return NULL; + + mapping = dma_map_single(priv->dev, buf, len, DMA_FROM_DEVICE); + if (unlikely(dma_mapping_error(priv->dev, mapping))) + goto err_out; + + *dma_handle = mapping; + *desc_len = len; + + return buf; + +err_out: + skb_free_frag(buf); + return NULL; +} + +static void netsec_rx_fill(struct netsec_priv *priv, u16 from, u16 num) +{ + struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; + u16 idx = from; + + while (num) { + netsec_set_rx_de(priv, dring, idx, &dring->desc[idx]); + idx++; + if (idx >= DESC_NUM) + idx = 0; + num--; + } +} + static int netsec_alloc_dring(struct netsec_priv *priv, enum ring_id id) { struct netsec_desc_ring *dring = &priv->desc_ring[id]; - int ret = 0; dring->vaddr = dma_zalloc_coherent(priv->dev, DESC_SZ * DESC_NUM, &dring->desc_dma, GFP_KERNEL); - if (!dring->vaddr) { - ret = -ENOMEM; + if (!dring->vaddr) goto err; - } dring->desc = kcalloc(DESC_NUM, sizeof(*dring->desc), GFP_KERNEL); - if (!dring->desc) { - ret = -ENOMEM; + if (!dring->desc) goto err; - } return 0; err: netsec_free_dring(priv, id); - return ret; + return -ENOMEM; } static int netsec_setup_rx_dring(struct netsec_priv *priv) { struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; - struct netsec_desc desc; - struct sk_buff *skb; - int n; + int i; - desc.len = priv->ndev->mtu + 22; + for (i = 0; i < DESC_NUM; i++) { + struct netsec_desc *desc = &dring->desc[i]; + dma_addr_t dma_handle; + void *buf; + u16 len; - for (n = 0; n < DESC_NUM; n++) { - skb = netsec_alloc_skb(priv, &desc); - if (!skb) { + buf = netsec_alloc_rx_data(priv, &dma_handle, &len); + if (!buf) { netsec_uninit_pkt_dring(priv, NETSEC_RING_RX); - return -ENOMEM; + goto err_out; } - netsec_set_rx_de(priv, dring, n, &desc, skb); + desc->dma_addr = dma_handle; + desc->addr = buf; + desc->len = len; } + netsec_rx_fill(priv, 0, DESC_NUM); + return 0; + +err_out: + return -ENOMEM; } static int netsec_netdev_load_ucode_region(struct netsec_priv *priv, u32 reg, From patchwork Sat Sep 29 11:28:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilias Apalodimas X-Patchwork-Id: 147837 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1815194lji; Sat, 29 Sep 2018 04:28:18 -0700 (PDT) X-Google-Smtp-Source: ACcGV604ya8CbXbVz/AbQG1lMSSqN0AGophyFu1aPuk/7q/f66yauQ9W5vXFESLwSV3EaClVm9Vi X-Received: by 2002:a17:902:6f10:: with SMTP id w16-v6mr2775363plk.216.1538220498505; Sat, 29 Sep 2018 04:28:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538220498; cv=none; d=google.com; s=arc-20160816; b=yHeSE4XoQU0q/kU7jiOKq3vY0K9zbwmfQsiaFnn08yHJ1KcPEKtJ/W5YRc6yYvXAuu qMOs0W4630Ubo0Ugd6XdTv5JIyE65cm+h//3EPQk3Glo5Y1GXM5kQfeN7+Q4MT2F0iYA 9ftjwRyqFrs0YFiCOdM2klXfVeTp7CbrYfMdsxQK0H+fW9kkEHPdQPqFZYlVNo62Dz4d SmXAMvHoKb8NtqYPxBCgjBqKvo5iNFZEBSAW2Xz61QkqePs4V46SuYlp5rPQv7RQMm9u dLtj9aVvza33QFqm5Ed9WC8tbN5AV0qhmMs8tBftAHiymyeUQ6S3L4arxLMHbWWO+YrW i09w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=u4kb3b4V0oHf/X7KdGrKJQIHSb1Js6cM4FsOqDdTERw=; b=U5/FUxzX8EaMR9F+bh5zENmKp7gXLfk3I5KjSBEtQAwwdXRGSFPQ4+6aiWn2dmyzE+ 9nl4MiS8oc65Fg0piOMC5ajpcm9OxvVRL8tAnnrTodgjdSyxfNmQSacv80bu9RFAQxQY LTBtdaYgVDtClmtcbGRr+TCvsmyBF0ePNw0hxu2dP07AT64m2078lGSYcmGHM9aqdLtG wVNN+a1+8CbigeXJnoE17AU32elxTgcCstVkHpV66QChvucmnIWhYNB1+wFP967xAE0u MDE0kv5onnG0cADRo1RoINYNJ80sB4Yss21olFZrhLFM3nEBe2KZqUnIuDHucIvpMDRK 4jVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N9lr2Zz0; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m76-v6si7494789pfg.323.2018.09.29.04.28.18; Sat, 29 Sep 2018 04:28:18 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N9lr2Zz0; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728192AbeI2R4W (ORCPT + 10 others); Sat, 29 Sep 2018 13:56:22 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:42029 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727865AbeI2R4W (ORCPT ); Sat, 29 Sep 2018 13:56:22 -0400 Received: by mail-wr1-f68.google.com with SMTP id b11-v6so8895755wru.9 for ; Sat, 29 Sep 2018 04:28:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u4kb3b4V0oHf/X7KdGrKJQIHSb1Js6cM4FsOqDdTERw=; b=N9lr2Zz0hGjnxFChvRaLkc4P9+EurFV+PmvcM/s1wRo8tAXx5YrFR9aTZ++hMzckO9 5ZJMDkFv8GxfaolaVdDFOPC2fa9vBgdUSC3v5K0rBfFrTvKfcEmVdfAWHVTU5oXUJJac aTGuO/vy3UbhnxED237ZPdfPNKmy3m9Q8aM2s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=u4kb3b4V0oHf/X7KdGrKJQIHSb1Js6cM4FsOqDdTERw=; b=mM1hjwauxAUtESMxoqCqAPsAhtM4sc155rKIBmmcx+OJf/GrRzygpmvuL082T6nG/p PMZYIPLvuuCaeqDnsBDFlSv+WuEbkfjN7riTuX6cF/uG9m6CH+M0uHzSLUbHM5Logmyp QAp1KTfBXsnzPBXzzTS5vK/eNnqwjNyqK0E16E5NxPg4uHNkkMsCSYaYEuDn/aNH3EmA DGBku5FDOugBz2OOVtMYBE2XM9pPrpcu2Qx4Hs7HqGG7k8OEnU9EHaBu+Ip2yp/thxnZ TGaB9U09QrClAf2YHrZlP/JIUjuVYhCvd3PjCgIGkyyLkHHZdcmtSec4v4rSuFhO5AWs 34pQ== X-Gm-Message-State: ABuFfojOvE7nZcVSJVJZQNMElaTOi1OaxNdY7wjMkUyg9Wv4CC5QaeMW 9qUdR6FqG5J0q0iGHnPZ6ccaClABnhU= X-Received: by 2002:adf:a41c:: with SMTP id d28-v6mr1678987wra.121.1538220494106; Sat, 29 Sep 2018 04:28:14 -0700 (PDT) Received: from localhost.localdomain ([2a02:587:4603:8d00:2e56:dcff:fe9a:8f06]) by smtp.gmail.com with ESMTPSA id q5-v6sm6841080wmd.29.2018.09.29.04.28.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 29 Sep 2018 04:28:13 -0700 (PDT) From: Ilias Apalodimas To: netdev@vger.kernel.org, jaswinder.singh@linaro.org Cc: ard.biesheuvel@linaro.org, masami.hiramatsu@linaro.org, arnd@arndb.de, bjorn.topel@intel.com, magnus.karlsson@intel.com, brouer@redhat.com, daniel@iogearbox.net, ast@kernel.org, jesus.sanchez-palencia@intel.com, vinicius.gomes@intel.com, makita.toshiaki@lab.ntt.co.jp, Ilias Apalodimas Subject: [net-next, PATCH 2/2, v3] net: socionext: add XDP support Date: Sat, 29 Sep 2018 14:28:02 +0300 Message-Id: <1538220482-16129-3-git-send-email-ilias.apalodimas@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538220482-16129-1-git-send-email-ilias.apalodimas@linaro.org> References: <1538220482-16129-1-git-send-email-ilias.apalodimas@linaro.org> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add basic XDP support. The interface only supports 1 Tx queue for now so locking is introduced on the Tx queue if XDP is enabled to make sure .ndo_start_xmit and .ndo_xdp_xmit won't corrupt Tx ring Signed-off-by: Ilias Apalodimas --- drivers/net/ethernet/socionext/netsec.c | 345 +++++++++++++++++++++++++++++--- 1 file changed, 318 insertions(+), 27 deletions(-) -- 2.7.4 diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index 8f788a1..2b29363 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -9,6 +9,9 @@ #include #include #include +#include +#include +#include #include #include @@ -238,6 +241,15 @@ #define NETSEC_F_NETSEC_VER_MAJOR_NUM(x) ((x) & 0xffff0000) +#define NETSEC_XDP_PASS 0 +#define NETSEC_XDP_CONSUMED BIT(0) +#define NETSEC_XDP_TX BIT(1) +#define NETSEC_XDP_REDIR BIT(2) +#define NETSEC_XDP_RX_OK (NETSEC_XDP_PASS | NETSEC_XDP_TX | NETSEC_XDP_REDIR) + +#define NETSEC_RXBUF_HEADROOM (max(XDP_PACKET_HEADROOM, NET_SKB_PAD) + \ + NET_IP_ALIGN) + enum ring_id { NETSEC_RING_TX = 0, NETSEC_RING_RX @@ -256,11 +268,16 @@ struct netsec_desc_ring { void *vaddr; u16 pkt_cnt; u16 head, tail; + u16 xdp_xmit; /* netsec_xdp_xmit packets */ + bool is_xdp; + struct xdp_rxq_info xdp_rxq; + spinlock_t lock; /* XDP tx queue locking */ }; struct netsec_priv { struct netsec_desc_ring desc_ring[NETSEC_RING_MAX]; struct ethtool_coalesce et_coalesce; + struct bpf_prog *xdp_prog; spinlock_t reglock; /* protect reg access */ struct napi_struct napi; phy_interface_t phy_interface; @@ -297,6 +314,8 @@ struct netsec_rx_pkt_info { }; static void netsec_rx_fill(struct netsec_priv *priv, u16 from, u16 num); +static u32 netsec_run_xdp(struct netsec_priv *priv, struct bpf_prog *prog, + struct xdp_buff *xdp); static void *netsec_alloc_rx_data(struct netsec_priv *priv, dma_addr_t *dma_addr, u16 *len); @@ -590,6 +609,8 @@ static int netsec_clean_tx_dring(struct netsec_priv *priv, int budget) struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_TX]; unsigned int pkts, bytes; + if (dring->is_xdp) + spin_lock(&dring->lock); dring->pkt_cnt += netsec_read(priv, NETSEC_REG_NRM_TX_DONE_PKTCNT); if (dring->pkt_cnt < budget) @@ -615,13 +636,23 @@ static int netsec_clean_tx_dring(struct netsec_priv *priv, int budget) dma_unmap_single(priv->dev, desc->dma_addr, desc->len, DMA_TO_DEVICE); - if (eop) { - pkts++; + + if (!eop) { + *desc = (struct netsec_desc){}; + continue; + } + + if (!desc->skb) { + skb_free_frag(desc->addr); + } else { bytes += desc->skb->len; dev_kfree_skb(desc->skb); } + pkts++; *desc = (struct netsec_desc){}; } + if (dring->is_xdp) + spin_unlock(&dring->lock); dring->pkt_cnt -= budget; priv->ndev->stats.tx_packets += budget; @@ -656,11 +687,30 @@ static void netsec_adv_desc(u16 *idx) *idx = 0; } +static void netsec_xdp_ring_tx_db(struct netsec_priv *priv, u16 pkts) +{ + if (likely(pkts)) + netsec_write(priv, NETSEC_REG_NRM_TX_PKTCNT, pkts); +} + +static void netsec_finalize_xdp_rx(struct netsec_priv *priv, u32 xdp_res, + u16 pkts) +{ + if (xdp_res & NETSEC_XDP_REDIR) + xdp_do_flush_map(); + + if (xdp_res & NETSEC_XDP_TX) + netsec_xdp_ring_tx_db(priv, pkts); +} + static int netsec_process_rx(struct netsec_priv *priv, int budget) { struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; + struct bpf_prog *xdp_prog = READ_ONCE(priv->xdp_prog); struct net_device *ndev = priv->ndev; - struct sk_buff *skb; + struct sk_buff *skb = NULL; + u16 xdp_xmit = 0; + u32 xdp_act = 0; int done = 0; while (done < budget) { @@ -668,8 +718,10 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) struct netsec_de *de = dring->vaddr + (DESC_SZ * idx); struct netsec_desc *desc = &dring->desc[idx]; struct netsec_rx_pkt_info rpi; - u16 pkt_len, desc_len; + u32 xdp_result = XDP_PASS; dma_addr_t dma_handle; + u16 pkt_len, desc_len; + struct xdp_buff xdp; void *buf_addr; if (de->attr & (1U << NETSEC_RX_PKT_OWN_FIELD)) @@ -706,7 +758,23 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) DMA_FROM_DEVICE); prefetch(desc->addr); - skb = build_skb(desc->addr, desc->len); + xdp.data_hard_start = desc->addr; + xdp.data = desc->addr + NETSEC_RXBUF_HEADROOM; + xdp_set_data_meta_invalid(&xdp); + xdp.data_end = xdp.data + pkt_len; + xdp.rxq = &dring->xdp_rxq; + + if (xdp_prog) { + xdp_result = netsec_run_xdp(priv, xdp_prog, &xdp); + if (xdp_result != NETSEC_XDP_PASS) { + xdp_act |= xdp_result; + if (xdp_result == NETSEC_XDP_TX) + xdp_xmit++; + goto next; + } + } + + skb = build_skb(xdp.data_hard_start, desc->len); if (unlikely(!skb)) { dma_unmap_single(priv->dev, dma_handle, desc_len, DMA_TO_DEVICE); @@ -715,30 +783,35 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) "rx failed to alloc skb\n"); break; } - dma_unmap_single_attrs(priv->dev, desc->dma_addr, desc->len, - DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); - /* Update the descriptor with fresh buffers */ - desc->len = desc_len; - desc->dma_addr = dma_handle; - desc->addr = buf_addr; - - skb_put(skb, pkt_len); + skb_reserve(skb, xdp.data - xdp.data_hard_start); + skb_put(skb, xdp.data_end - xdp.data); skb->protocol = eth_type_trans(skb, priv->ndev); if (priv->rx_cksum_offload_flag && rpi.rx_cksum_result == NETSEC_RX_CKSUM_OK) skb->ip_summed = CHECKSUM_UNNECESSARY; - if (napi_gro_receive(&priv->napi, skb) != GRO_DROP) { +next: + if ((skb && napi_gro_receive(&priv->napi, skb) != GRO_DROP) || + xdp_result & NETSEC_XDP_RX_OK) { ndev->stats.rx_packets++; - ndev->stats.rx_bytes += pkt_len; + ndev->stats.rx_bytes += xdp.data_end - xdp.data; } + dma_unmap_single_attrs(priv->dev, desc->dma_addr, desc->len, + DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); + + /* Update the descriptor with fresh buffers */ + desc->len = desc_len; + desc->dma_addr = dma_handle; + desc->addr = buf_addr; netsec_rx_fill(priv, idx, 1); netsec_adv_desc(&dring->tail); } + netsec_finalize_xdp_rx(priv, xdp_act, xdp_xmit); + return done; } @@ -805,7 +878,9 @@ static void netsec_set_tx_de(struct netsec_priv *priv, de->data_buf_addr_lw = lower_32_bits(desc->dma_addr); de->buf_len_info = (tx_ctrl->tcp_seg_len << 16) | desc->len; de->attr = attr; - dma_wmb(); + /* under spin_lock if using XDP */ + if (!dring->is_xdp) + dma_wmb(); dring->desc[idx] = *desc; dring->desc[idx].skb = skb; @@ -824,6 +899,8 @@ static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, u16 tso_seg_len = 0; int filled; + if (dring->is_xdp) + spin_lock_bh(&dring->lock); /* differentiate between full/emtpy ring */ if (dring->head >= dring->tail) filled = dring->head - dring->tail; @@ -831,6 +908,8 @@ static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, filled = dring->head + DESC_NUM - dring->tail; if (DESC_NUM - filled < 2) { /* if less than 2 available */ + if (dring->is_xdp) + spin_unlock_bh(&dring->lock); netif_err(priv, drv, priv->ndev, "%s: TxQFull!\n", __func__); netif_stop_queue(priv->ndev); dma_wmb(); @@ -864,6 +943,8 @@ static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, tx_desc.dma_addr = dma_map_single(priv->dev, skb->data, skb_headlen(skb), DMA_TO_DEVICE); if (dma_mapping_error(priv->dev, tx_desc.dma_addr)) { + if (dring->is_xdp) + spin_unlock_bh(&dring->lock); netif_err(priv, drv, priv->ndev, "%s: DMA mapping failed\n", __func__); ndev->stats.tx_dropped++; @@ -877,6 +958,8 @@ static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, netdev_sent_queue(priv->ndev, skb->len); netsec_set_tx_de(priv, dring, &tx_ctrl, &tx_desc, skb); + if (dring->is_xdp) + spin_unlock_bh(&dring->lock); netsec_write(priv, NETSEC_REG_NRM_TX_PKTCNT, 1); /* submit another tx */ return NETDEV_TX_OK; @@ -891,6 +974,9 @@ static void netsec_uninit_pkt_dring(struct netsec_priv *priv, int id) if (!dring->vaddr || !dring->desc) return; + if (xdp_rxq_info_is_reg(&dring->xdp_rxq)) + xdp_rxq_info_unreg(&dring->xdp_rxq); + for (idx = 0; idx < DESC_NUM; idx++) { desc = &dring->desc[idx]; if (!desc->addr) @@ -930,24 +1016,24 @@ static void netsec_free_dring(struct netsec_priv *priv, int id) static void *netsec_alloc_rx_data(struct netsec_priv *priv, dma_addr_t *dma_handle, u16 *desc_len) { - size_t len = priv->ndev->mtu + ETH_HLEN + 2 * VLAN_HLEN + NET_SKB_PAD + - NET_IP_ALIGN; + size_t total_len = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + size_t payload_len = priv->ndev->mtu + ETH_HLEN + 2 * VLAN_HLEN; dma_addr_t mapping; void *buf; - len += SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); - len = SKB_DATA_ALIGN(len); + total_len += SKB_DATA_ALIGN(payload_len + NETSEC_RXBUF_HEADROOM); - buf = napi_alloc_frag(len); + buf = napi_alloc_frag(total_len); if (!buf) return NULL; - mapping = dma_map_single(priv->dev, buf, len, DMA_FROM_DEVICE); + mapping = dma_map_single(priv->dev, buf + NETSEC_RXBUF_HEADROOM, + payload_len, DMA_FROM_DEVICE); if (unlikely(dma_mapping_error(priv->dev, mapping))) goto err_out; *dma_handle = mapping; - *desc_len = len; + *desc_len = total_len; return buf; @@ -990,10 +1076,27 @@ static int netsec_alloc_dring(struct netsec_priv *priv, enum ring_id id) return -ENOMEM; } +static void netsec_setup_tx_dring(struct netsec_priv *priv) +{ + struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_TX]; + struct bpf_prog *xdp_prog = READ_ONCE(priv->xdp_prog); + + if (xdp_prog) + dring->is_xdp = true; + else + dring->is_xdp = false; +} + static int netsec_setup_rx_dring(struct netsec_priv *priv) { struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX]; - int i; + struct bpf_prog *xdp_prog = READ_ONCE(priv->xdp_prog); + int i, err; + + if (xdp_prog) + dring->is_xdp = true; + else + dring->is_xdp = false; for (i = 0; i < DESC_NUM; i++) { struct netsec_desc *desc = &dring->desc[i]; @@ -1002,20 +1105,29 @@ static int netsec_setup_rx_dring(struct netsec_priv *priv) u16 len; buf = netsec_alloc_rx_data(priv, &dma_handle, &len); - if (!buf) { - netsec_uninit_pkt_dring(priv, NETSEC_RING_RX); + if (!buf) goto err_out; - } desc->dma_addr = dma_handle; desc->addr = buf; desc->len = len; } netsec_rx_fill(priv, 0, DESC_NUM); + err = xdp_rxq_info_reg(&dring->xdp_rxq, priv->ndev, 0); + if (err) + goto err_out; + + err = xdp_rxq_info_reg_mem_model(&dring->xdp_rxq, MEM_TYPE_PAGE_SHARED, + NULL); + if (err) { + xdp_rxq_info_unreg(&dring->xdp_rxq); + goto err_out; + } return 0; err_out: + netsec_uninit_pkt_dring(priv, NETSEC_RING_RX); return -ENOMEM; } @@ -1290,6 +1402,7 @@ static int netsec_netdev_open(struct net_device *ndev) pm_runtime_get_sync(priv->dev); + netsec_setup_tx_dring(priv); ret = netsec_setup_rx_dring(priv); if (ret) { netif_err(priv, probe, priv->ndev, @@ -1387,6 +1500,9 @@ static int netsec_netdev_init(struct net_device *ndev) if (ret) goto err2; + spin_lock_init(&priv->desc_ring[NETSEC_RING_TX].lock); + spin_lock_init(&priv->desc_ring[NETSEC_RING_RX].lock); + return 0; err2: netsec_free_dring(priv, NETSEC_RING_RX); @@ -1419,6 +1535,179 @@ static int netsec_netdev_ioctl(struct net_device *ndev, struct ifreq *ifr, return phy_mii_ioctl(ndev->phydev, ifr, cmd); } +/* The current driver only supports 1 Txq, this should run under spin_lock() */ +static u32 netsec_xdp_queue_one(struct netsec_priv *priv, + struct xdp_frame *xdpf) + +{ + struct netsec_desc_ring *tx_ring = &priv->desc_ring[NETSEC_RING_TX]; + struct netsec_tx_pkt_ctrl tx_ctrl = {}; + struct netsec_desc tx_desc; + dma_addr_t dma_handle; + u16 filled; + + if (tx_ring->head >= tx_ring->tail) + filled = tx_ring->head - tx_ring->tail; + else + filled = tx_ring->head + DESC_NUM - tx_ring->tail; + + if (DESC_NUM - filled <= 1) + return NETSEC_XDP_CONSUMED; + + dma_handle = dma_map_single(priv->dev, xdpf->data, xdpf->len, + DMA_TO_DEVICE); + if (dma_mapping_error(priv->dev, dma_handle)) + return NETSEC_XDP_CONSUMED; + + tx_ctrl.cksum_offload_flag = false; + tx_ctrl.tcp_seg_offload_flag = false; + tx_ctrl.tcp_seg_len = 0; + + tx_desc.dma_addr = dma_handle; + tx_desc.addr = xdpf->data; + tx_desc.len = xdpf->len; + + netsec_set_tx_de(priv, tx_ring, &tx_ctrl, &tx_desc, NULL); + + return NETSEC_XDP_TX; +} + +static int netsec_xdp_xmit(struct net_device *ndev, int n, + struct xdp_frame **frames, u32 flags) +{ + struct netsec_priv *priv = netdev_priv(ndev); + struct netsec_desc_ring *tx_ring = &priv->desc_ring[NETSEC_RING_TX]; + int drops = 0; + int i; + + if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) + return -EINVAL; + + spin_lock(&tx_ring->lock); + for (i = 0; i < n; i++) { + struct xdp_frame *xdpf = frames[i]; + int err; + + err = netsec_xdp_queue_one(priv, xdpf); + if (err != NETSEC_XDP_TX) { + xdp_return_frame_rx_napi(xdpf); + drops++; + } else { + tx_ring->xdp_xmit++; + } + } + spin_unlock(&tx_ring->lock); + + if (unlikely(flags & XDP_XMIT_FLUSH)) { + netsec_xdp_ring_tx_db(priv, tx_ring->xdp_xmit); + tx_ring->xdp_xmit = 0; + } + + return n - drops; +} + +static u32 netsec_xdp_xmit_back(struct netsec_priv *priv, struct xdp_buff *xdp) +{ + struct netsec_desc_ring *tx_ring = &priv->desc_ring[NETSEC_RING_TX]; + struct xdp_frame *xdpf = convert_to_xdp_frame(xdp); + u32 ret; + + if (unlikely(!xdpf)) + return NETSEC_XDP_CONSUMED; + + spin_lock(&tx_ring->lock); + ret = netsec_xdp_queue_one(priv, xdpf); + spin_unlock(&tx_ring->lock); + + return ret; +} + +static u32 netsec_run_xdp(struct netsec_priv *priv, struct bpf_prog *prog, + struct xdp_buff *xdp) +{ + u32 ret = NETSEC_XDP_PASS; + int err; + u32 act; + + rcu_read_lock(); + act = bpf_prog_run_xdp(prog, xdp); + + switch (act) { + case XDP_PASS: + ret = NETSEC_XDP_PASS; + break; + case XDP_TX: + ret = netsec_xdp_xmit_back(priv, xdp); + if (ret != NETSEC_XDP_TX) + xdp_return_buff(xdp); + break; + case XDP_REDIRECT: + err = xdp_do_redirect(priv->ndev, xdp, prog); + if (!err) { + ret = NETSEC_XDP_REDIR; + } else { + ret = NETSEC_XDP_CONSUMED; + xdp_return_buff(xdp); + } + break; + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(priv->ndev, prog, act); + /* fall through -- handle aborts by dropping packet */ + case XDP_DROP: + ret = NETSEC_XDP_CONSUMED; + xdp_return_buff(xdp); + break; + } + + rcu_read_unlock(); + + return ret; +} + +static int netsec_xdp_setup(struct netsec_priv *priv, struct bpf_prog *prog, + struct netlink_ext_ack *extack) +{ + struct net_device *dev = priv->ndev; + struct bpf_prog *old_prog; + + /* For now just support only the usual MTU sized frames */ + if (prog && dev->mtu > 1500) { + NL_SET_ERR_MSG_MOD(extack, "Jumbo frames not supported on XDP"); + return -EOPNOTSUPP; + } + + if (netif_running(dev)) + netsec_netdev_stop(dev); + + /* Detach old prog, if any */ + old_prog = xchg(&priv->xdp_prog, prog); + if (old_prog) + bpf_prog_put(old_prog); + + if (netif_running(dev)) + netsec_netdev_open(dev); + + return 0; +} + +static int netsec_xdp(struct net_device *ndev, struct netdev_bpf *xdp) +{ + struct netsec_priv *priv = netdev_priv(ndev); + + switch (xdp->command) { + case XDP_SETUP_PROG: + return netsec_xdp_setup(priv, xdp->prog, xdp->extack); + case XDP_QUERY_PROG: + xdp->prog_id = priv->xdp_prog ? priv->xdp_prog->aux->id : 0; + return 0; + default: + return -EINVAL; + } +} + static const struct net_device_ops netsec_netdev_ops = { .ndo_init = netsec_netdev_init, .ndo_uninit = netsec_netdev_uninit, @@ -1429,6 +1718,8 @@ static const struct net_device_ops netsec_netdev_ops = { .ndo_set_mac_address = eth_mac_addr, .ndo_validate_addr = eth_validate_addr, .ndo_do_ioctl = netsec_netdev_ioctl, + .ndo_xdp_xmit = netsec_xdp_xmit, + .ndo_bpf = netsec_xdp, }; static int netsec_of_probe(struct platform_device *pdev,