From patchwork Tue Jul 13 16:07:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 475052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E9B5C11F67 for ; Tue, 13 Jul 2021 16:08:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF75A61183 for ; Tue, 13 Jul 2021 16:08:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230039AbhGMQKt (ORCPT ); Tue, 13 Jul 2021 12:10:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229643AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D19B4C061788; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=TXXpZdQabLRguffEGhge1V0w3b5CtBHrtxITPCZI2HY=; b=cExJpzdZSfOMhpsbaNzyxINh0w dsqUzuXNAV484LKNZzMPTQiyHQXhllP2Pryzjj2lR3cYsB5/raCx18LeTc2dGQ9pXK2uKeJWU6HWo /HwxVgn3KYuvdp8wDiOypVfLEgycnRrx55IwDE0lVlJVgfPEUuQkTJMsIusYYjvO1rz0=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwm-0008TX-5J; Tue, 13 Jul 2021 18:07:48 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 1/7] mac80211: add support for .ndo_fill_forward_path Date: Tue, 13 Jul 2021 18:07:39 +0200 Message-Id: <20210713160745.59707-2-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org This allows drivers to provide a destination device + info for flow offload Only supported in combination with 802.3 encap offload Signed-off-by: Felix Fietkau --- include/net/mac80211.h | 5 ++++ net/mac80211/driver-ops.h | 22 +++++++++++++++ net/mac80211/ieee80211_i.h | 2 +- net/mac80211/iface.c | 58 ++++++++++++++++++++++++++++++++++++++ net/mac80211/trace.h | 7 +++++ 5 files changed, 93 insertions(+), 1 deletion(-) diff --git a/include/net/mac80211.h b/include/net/mac80211.h index d8a1d09a2141..c09ec24f8348 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -4242,6 +4242,11 @@ struct ieee80211_ops { void (*sta_set_decap_offload)(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_sta *sta, bool enabled); + int (*net_fill_forward_path)(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_sta *sta, + struct net_device_path_ctx *ctx, + struct net_device_path *path); }; /** diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h index bcb7cc06db3d..f487002660e5 100644 --- a/net/mac80211/driver-ops.h +++ b/net/mac80211/driver-ops.h @@ -1447,4 +1447,26 @@ static inline void drv_sta_set_decap_offload(struct ieee80211_local *local, trace_drv_return_void(local); } +static inline int drv_net_fill_forward_path(struct ieee80211_local *local, + struct ieee80211_sub_if_data *sdata, + struct ieee80211_sta *sta, + struct net_device_path_ctx *ctx, + struct net_device_path *path) +{ + int ret = -EOPNOTSUPP; + + sdata = get_bss_sdata(sdata); + if (!check_sdata_in_driver(sdata)) + return -EIO; + + trace_drv_net_fill_forward_path(local, sdata, sta); + if (local->ops->net_fill_forward_path) + ret = local->ops->net_fill_forward_path(&local->hw, + &sdata->vif, sta, + ctx, path); + trace_drv_return_int(local, ret); + + return ret; +} + #endif /* __MAC80211_DRIVER_OPS */ diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 22549b95d1aa..5b96e79402a1 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -1451,7 +1451,7 @@ struct ieee80211_local { }; static inline struct ieee80211_sub_if_data * -IEEE80211_DEV_TO_SUB_IF(struct net_device *dev) +IEEE80211_DEV_TO_SUB_IF(const struct net_device *dev) { return netdev_priv(dev); } diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c index 1e5e9fc45523..fe093da63243 100644 --- a/net/mac80211/iface.c +++ b/net/mac80211/iface.c @@ -717,6 +717,7 @@ static const struct net_device_ops ieee80211_dataif_ops = { .ndo_set_mac_address = ieee80211_change_mac, .ndo_select_queue = ieee80211_netdev_select_queue, .ndo_get_stats64 = ieee80211_get_stats64, + .ndo_fill_forward_path = ieee80211_netdev_fill_forward_path, }; static u16 ieee80211_monitor_select_queue(struct net_device *dev, @@ -758,6 +759,63 @@ static const struct net_device_ops ieee80211_monitorif_ops = { .ndo_get_stats64 = ieee80211_get_stats64, }; + +static int ieee80211_netdev_fill_forward_path(struct net_device_path_ctx *ctx, + struct net_device_path *path) +{ + struct ieee80211_sub_if_data *sdata; + struct ieee80211_local *local; + struct sta_info *sta; + int ret = -ENOENT; + + sdata = IEEE80211_DEV_TO_SUB_IF(ctx->dev); + local = sdata->local; + + if (!local->ops->net_fill_forward_path) + return -EOPNOTSUPP; + + rcu_read_lock(); + switch (sdata->vif.type) { + case NL80211_IFTYPE_AP_VLAN: + sta = rcu_dereference(sdata->u.vlan.sta); + if (sta) + break; + if (!sdata->wdev.use_4addr) + goto out; + fallthrough; + case NL80211_IFTYPE_AP: + if (is_multicast_ether_addr(ctx->daddr)) + goto out; + sta = sta_info_get_bss(sdata, ctx->daddr); + break; + case NL80211_IFTYPE_STATION: + if (sdata->wdev.wiphy->flags & WIPHY_FLAG_SUPPORTS_TDLS) { + sta = sta_info_get(sdata, ctx->daddr); + if (sta && test_sta_flag(sta, WLAN_STA_TDLS_PEER)) { + if (!test_sta_flag(sta, WLAN_STA_TDLS_PEER_AUTH)) + goto out; + + break; + } + } + + sta = sta_info_get(sdata, sdata->u.mgd.bssid); + break; + default: + goto out; + } + + if (!sta) + goto out; + + ret = drv_net_fill_forward_path(local, sdata, &sta->sta, ctx, path); +out: + rcu_read_unlock(); + + return ret; +} + + static const struct net_device_ops ieee80211_dataif_8023_ops = { .ndo_open = ieee80211_open, .ndo_stop = ieee80211_stop, diff --git a/net/mac80211/trace.h b/net/mac80211/trace.h index f6ef15366938..611bff1e1cd8 100644 --- a/net/mac80211/trace.h +++ b/net/mac80211/trace.h @@ -2825,6 +2825,13 @@ DEFINE_EVENT(sta_flag_evt, drv_sta_set_decap_offload, TP_ARGS(local, sdata, sta, enabled) ); +DEFINE_EVENT(sta_event, drv_net_fill_forward_path, + TP_PROTO(struct ieee80211_local *local, + struct ieee80211_sub_if_data *sdata, + struct ieee80211_sta *sta), + TP_ARGS(local, sdata, sta) +); + #endif /* !__MAC80211_DRIVER_TRACE || TRACE_HEADER_MULTI_READ */ #undef TRACE_INCLUDE_PATH From patchwork Tue Jul 13 16:07:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 477271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 651BEC07E95 for ; Tue, 13 Jul 2021 16:07:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 42CB561183 for ; Tue, 13 Jul 2021 16:07:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229449AbhGMQKq (ORCPT ); Tue, 13 Jul 2021 12:10:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229604AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA52FC0613EE; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=joNkWmn5qVMu9KtQ2EOffOtK6v9TM5gMwo+EaCtDiIQ=; b=XM7J6H5a2Zb+DYwp1pfshmukqY 1p9Asd/6R1H+ApyRm6I2o47rvVWz6r5rWISBOY9Ot9JfsCASipqQvYftlXKYjWvJyh6zy+Tqri67l 52CWDOd5w4kPDrdL9/hrjDz+9JExBh7xCq/co3SNCoo81sGlKOVfJmzxkE/03EqFSFm8=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwm-0008TX-F9; Tue, 13 Jul 2021 18:07:48 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 2/7] net: ethernet: mtk_eth_soc: add support for Wireless Ethernet Dispatch (WED) Date: Tue, 13 Jul 2021 18:07:40 +0200 Message-Id: <20210713160745.59707-3-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org The Wireless Ethernet Dispatch subsystem on the MT7622 SoC can be configured to intercept and handle access to the DMA queues and PCIe interrupts for a MT7615/MT7915 wireless card. It can manage the internal WDMA (Wireless DMA) controller, which allows ethernet packets to be passed from the packet switch engine (PSE) to the wireless card, bypassing the CPU entirely. This can be used to implement hardware flow offloading from ethernet to WLAN. Signed-off-by: Felix Fietkau --- arch/arm64/boot/dts/mediatek/mt7622.dtsi | 22 + drivers/net/ethernet/mediatek/Kconfig | 4 + drivers/net/ethernet/mediatek/Makefile | 5 + drivers/net/ethernet/mediatek/mtk_eth_soc.c | 23 + drivers/net/ethernet/mediatek/mtk_eth_soc.h | 3 + drivers/net/ethernet/mediatek/mtk_wed.c | 837 ++++++++++++++++++ drivers/net/ethernet/mediatek/mtk_wed.h | 123 +++ .../net/ethernet/mediatek/mtk_wed_debugfs.c | 175 ++++ drivers/net/ethernet/mediatek/mtk_wed_ops.c | 8 + drivers/net/ethernet/mediatek/mtk_wed_regs.h | 248 ++++++ include/linux/soc/mediatek/mtk_wed.h | 127 +++ 11 files changed, 1575 insertions(+) create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.h create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_debugfs.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_ops.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_regs.h create mode 100644 include/linux/soc/mediatek/mtk_wed.h diff --git a/arch/arm64/boot/dts/mediatek/mt7622.dtsi b/arch/arm64/boot/dts/mediatek/mt7622.dtsi index 890a942ec608..a5232a15bc98 100644 --- a/arch/arm64/boot/dts/mediatek/mt7622.dtsi +++ b/arch/arm64/boot/dts/mediatek/mt7622.dtsi @@ -911,6 +911,26 @@ hsdma: dma-controller@1b007000 { #dma-cells = <1>; }; + pcie_mirror: pcie-mirror@10000400 { + compatible = "mediatek,mt7622-pcie-mirror", + "syscon"; + reg = <0 0x10000400 0 0x10>; + }; + + wed0: wed@1020a000 { + compatible = "mediatek,mt7622-wed", + "syscon"; + reg = <0 0x1020a000 0 0x1000>; + interrupts = ; + }; + + wed1: wed@1020b000 { + compatible = "mediatek,mt7622-wed", + "syscon"; + reg = <0 0x1020b000 0 0x1000>; + interrupts = ; + }; + eth: ethernet@1b100000 { compatible = "mediatek,mt7622-eth", "mediatek,mt2701-eth", @@ -937,6 +957,8 @@ eth: ethernet@1b100000 { power-domains = <&scpsys MT7622_POWER_DOMAIN_ETHSYS>; mediatek,ethsys = <ðsys>; mediatek,sgmiisys = <&sgmiisys>; + mediatek,wed = <&wed0>, <&wed1>; + mediatek,pcie-mirror = <&pcie_mirror>; #address-cells = <1>; #size-cells = <0>; status = "disabled"; diff --git a/drivers/net/ethernet/mediatek/Kconfig b/drivers/net/ethernet/mediatek/Kconfig index c357c193378e..03173b69b128 100644 --- a/drivers/net/ethernet/mediatek/Kconfig +++ b/drivers/net/ethernet/mediatek/Kconfig @@ -7,6 +7,10 @@ config NET_VENDOR_MEDIATEK if NET_VENDOR_MEDIATEK +config NET_MEDIATEK_SOC_WED + depends on ARCH_MEDIATEK + def_tristate NET_MEDIATEK_SOC + config NET_MEDIATEK_SOC tristate "MediaTek SoC Gigabit Ethernet support" depends on NET_DSA || !NET_DSA diff --git a/drivers/net/ethernet/mediatek/Makefile b/drivers/net/ethernet/mediatek/Makefile index 79d4cdbbcbf5..45ba0970504a 100644 --- a/drivers/net/ethernet/mediatek/Makefile +++ b/drivers/net/ethernet/mediatek/Makefile @@ -5,4 +5,9 @@ obj-$(CONFIG_NET_MEDIATEK_SOC) += mtk_eth.o mtk_eth-y := mtk_eth_soc.o mtk_sgmii.o mtk_eth_path.o mtk_ppe.o mtk_ppe_debugfs.o mtk_ppe_offload.o +mtk_eth-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed.o +ifdef CONFIG_DEBUG_FS +mtk_eth-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed_debugfs.o +endif +obj-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed_ops.o obj-$(CONFIG_NET_MEDIATEK_STAR_EMAC) += mtk_star_emac.o diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c index 64adfd24e134..337ae6268fae 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c @@ -23,6 +23,7 @@ #include #include "mtk_eth_soc.h" +#include "mtk_wed.h" static int mtk_msg_level = -1; module_param_named(msg_level, mtk_msg_level, int, 0); @@ -3132,6 +3133,28 @@ static int mtk_probe(struct platform_device *pdev) } } + for (i = 0;; i++) { + struct device_node *np = of_parse_phandle(pdev->dev.of_node, + "mediatek,wed", i); + static const u32 wdma_regs[] = { + MTK_WDMA0_BASE, + MTK_WDMA1_BASE + }; + struct regmap *mirror; + void __iomem *wdma; + + if (!np || i >= ARRAY_SIZE(wdma_regs)) + break; + + mirror = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, + "mediatek,pcie-mirror"); + if (IS_ERR(mirror)) + break; + + wdma = eth->base + wdma_regs[i]; + mtk_wed_add_hw(np, wdma, mirror, i); + } + for (i = 0; i < 3; i++) { if (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_INT) && i > 0) eth->irq[i] = eth->irq[0]; diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h index 5ef70dd8b49c..1bde9aa219ba 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h @@ -295,6 +295,9 @@ #define MTK_GDM1_TX_GPCNT 0x2438 #define MTK_STAT_OFFSET 0x40 +#define MTK_WDMA0_BASE 0x2800 +#define MTK_WDMA1_BASE 0x2c00 + /* QDMA descriptor txd4 */ #define TX_DMA_CHKSUM (0x7 << 29) #define TX_DMA_TSO BIT(28) diff --git a/drivers/net/ethernet/mediatek/mtk_wed.c b/drivers/net/ethernet/mediatek/mtk_wed.c new file mode 100644 index 000000000000..402afeb7e4c2 --- /dev/null +++ b/drivers/net/ethernet/mediatek/mtk_wed.c @@ -0,0 +1,837 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2021 Felix Fietkau */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "mtk_wed_regs.h" +#include "mtk_wed.h" +#include "mtk_ppe.h" + +#define MTK_PCIE_BASE(n) (0x1a143000 + (n) * 0x2000) + +#define MTK_WED_PKT_SIZE 1900 +#define MTK_WED_BUF_SIZE 2048 +#define MTK_WED_BUF_PER_PAGE (PAGE_SIZE / 2048) + +#define MTK_WED_TX_RING_SIZE 2048 +#define MTK_WED_WDMA_RING_SIZE 1024 + +static struct mtk_wed_hw *hw_list[2]; +static DEFINE_MUTEX(hw_lock); + +static inline void +wed_m32(struct mtk_wed_device *dev, u32 reg, u32 mask, u32 val) +{ + regmap_update_bits(dev->hw->regs, reg, mask | val, val); +} + +static inline void +wed_set(struct mtk_wed_device *dev, u32 reg, u32 mask) +{ + return wed_m32(dev, reg, 0, mask); +} + +static inline void +wed_clr(struct mtk_wed_device *dev, u32 reg, u32 mask) +{ + return wed_m32(dev, reg, mask, 0); +} + +static inline void +wdma_m32(struct mtk_wed_device *dev, u32 reg, u32 mask, u32 val) +{ + wdma_w32(dev, reg, (wdma_r32(dev, reg) & ~mask) | val); +} + +static inline void +wdma_clr(struct mtk_wed_device *dev, u32 reg, u32 mask) +{ + wdma_m32(dev, reg, mask, 0); +} + +static inline void +wdma_set(struct mtk_wed_device *dev, u32 reg, u32 mask) +{ + wdma_m32(dev, reg, 0, mask); +} + +static void +mtk_wed_reset(struct mtk_wed_device *dev, u32 mask) +{ + int i; + + wed_w32(dev, MTK_WED_RESET, mask); + for (i = 0; i < 100; i++) { + if (wed_r32(dev, MTK_WED_RESET) & mask) + continue; + + return; + } + + WARN_ON_ONCE(1); +} + +static struct mtk_wed_hw * +mtk_wed_assign(struct mtk_wed_device *dev) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(hw_list); i++) { + struct mtk_wed_hw *hw = hw_list[i]; + + if (!hw || hw->wed_dev) + continue; + + hw->wed_dev = dev; + return hw; + } + + return NULL; +} + +static int +mtk_wed_buffer_alloc(struct mtk_wed_device *dev) +{ + struct mtk_wdma_desc *desc; + dma_addr_t desc_phys; + void **page_list; + int token = dev->wlan.token_start; + int ring_size; + int n_pages; + int i, page_idx; + + ring_size = dev->wlan.nbuf & ~(MTK_WED_BUF_PER_PAGE - 1); + n_pages = ring_size / MTK_WED_BUF_PER_PAGE; + + page_list = kcalloc(n_pages, sizeof(*page_list), GFP_KERNEL); + if (!page_list) + return -ENOMEM; + + dev->buf_ring.size = ring_size; + dev->buf_ring.pages = page_list; + + desc = dma_alloc_coherent(dev->hw->dev, ring_size * sizeof(*desc), + &desc_phys, GFP_KERNEL); + if (!desc) + return -ENOMEM; + + dev->buf_ring.desc = desc; + dev->buf_ring.desc_phys = desc_phys; + + for (i = 0, page_idx = 0; i < ring_size; i += MTK_WED_BUF_PER_PAGE) { + dma_addr_t page_phys, buf_phys; + struct page *page; + void *buf; + int s; + + page = __dev_alloc_pages(GFP_KERNEL, 0); + if (!page) + return -ENOMEM; + + page_phys = dma_map_page(dev->hw->dev, page, 0, PAGE_SIZE, + DMA_BIDIRECTIONAL); + if (dma_mapping_error(dev->hw->dev, page_phys)) { + __free_page(page); + return -ENOMEM; + } + + page_list[page_idx++] = page; + dma_sync_single_for_cpu(dev->hw->dev, page_phys, PAGE_SIZE, + DMA_BIDIRECTIONAL); + + buf = page_to_virt(page); + buf_phys = page_phys; + + for (s = 0; s < MTK_WED_BUF_PER_PAGE; s++) { + u32 txd_size; + + txd_size = dev->wlan.init_buf(buf, buf_phys, token++); + + desc->buf0 = buf_phys; + desc->buf1 = buf_phys + txd_size; + desc->ctrl = FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN0, + txd_size) | + FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN1, + MTK_WED_BUF_SIZE - txd_size) | + MTK_WDMA_DESC_CTRL_LAST_SEG1; + desc->info = 0; + desc++; + + buf += MTK_WED_BUF_SIZE; + buf_phys += MTK_WED_BUF_SIZE; + } + + dma_sync_single_for_device(dev->hw->dev, page_phys, PAGE_SIZE, + DMA_BIDIRECTIONAL); + } + + return 0; +} + +static void +mtk_wed_free_buffer(struct mtk_wed_device *dev) +{ + struct mtk_wdma_desc *desc = dev->buf_ring.desc; + void **page_list = dev->buf_ring.pages; + int page_idx; + int i; + + if (!page_list) + return; + + if (!desc) + goto free_pagelist; + + for (i = 0, page_idx = 0; i < dev->buf_ring.size; i += MTK_WED_BUF_PER_PAGE) { + void *page = page_list[page_idx++]; + + if (!page) + break; + + dma_unmap_page(dev->hw->dev, desc[i].buf0, + PAGE_SIZE, DMA_BIDIRECTIONAL); + __free_page(page); + } + + dma_free_coherent(dev->hw->dev, dev->buf_ring.size * sizeof(*desc), + desc, dev->buf_ring.desc_phys); + +free_pagelist: + kfree(page_list); +} + +static void +mtk_wed_free_ring(struct mtk_wed_device *dev, struct mtk_wed_ring *ring) +{ + if (!ring->desc) + return; + + dma_free_coherent(dev->hw->dev, ring->size * sizeof(*ring->desc), + ring->desc, ring->desc_phys); +} + +static void +mtk_wed_free_tx_rings(struct mtk_wed_device *dev) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dev->tx_ring); i++) + mtk_wed_free_ring(dev, &dev->tx_ring[i]); + for (i = 0; i < ARRAY_SIZE(dev->tx_wdma); i++) + mtk_wed_free_ring(dev, &dev->tx_wdma[i]); +} + +static void +mtk_wed_set_ext_int(struct mtk_wed_device *dev, bool en) +{ + wed_w32(dev, MTK_WED_EXT_INT_MASK, + en ? MTK_WED_EXT_INT_STATUS_ERROR_MASK : 0); + wed_r32(dev, MTK_WED_EXT_INT_MASK); +} + +static void +mtk_wed_stop(struct mtk_wed_device *dev) +{ + int pcie_idx = !!pci_domain_nr(dev->wlan.pci_dev->bus); + + regmap_write(dev->hw->mirror, pcie_idx * 4, 0); + mtk_wed_set_ext_int(dev, false); + + wed_clr(dev, MTK_WED_CTRL, + MTK_WED_CTRL_WDMA_INT_AGENT_EN | + MTK_WED_CTRL_WPDMA_INT_AGENT_EN | + MTK_WED_CTRL_WED_TX_BM_EN | + MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); + wed_w32(dev, MTK_WED_WPDMA_INT_TRIGGER, 0); + wed_w32(dev, MTK_WED_WDMA_INT_TRIGGER, 0); + wdma_w32(dev, MTK_WDMA_INT_MASK, 0); + wdma_w32(dev, MTK_WDMA_INT_GRP2, 0); + wed_w32(dev, MTK_WED_WPDMA_INT_MASK, 0); + + wed_clr(dev, MTK_WED_GLO_CFG, + MTK_WED_GLO_CFG_TX_DMA_EN | + MTK_WED_GLO_CFG_RX_DMA_EN); + wed_clr(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_TX_DRV_EN | + MTK_WED_WPDMA_GLO_CFG_RX_DRV_EN); + wed_clr(dev, MTK_WED_WDMA_GLO_CFG, + MTK_WED_WDMA_GLO_CFG_RX_DRV_EN); +} + +static void +mtk_wed_detach(struct mtk_wed_device *dev) +{ + struct mtk_wed_hw *hw = dev->hw; + + mutex_lock(&hw_lock); + + mtk_wed_stop(dev); + + wdma_w32(dev, MTK_WDMA_RESET_IDX, MTK_WDMA_RESET_IDX_RX); + wdma_w32(dev, MTK_WDMA_RESET_IDX, 0); + + mtk_wed_reset(dev, MTK_WED_RESET_WED); + + mtk_wed_free_buffer(dev); + mtk_wed_free_tx_rings(dev); + + memset(dev, 0, sizeof(*dev)); + module_put(THIS_MODULE); + + hw->wed_dev = NULL; + mutex_unlock(&hw_lock); +} + +static void +mtk_wed_hw_init_early(struct mtk_wed_device *dev) +{ + int pcie_idx = !!pci_domain_nr(dev->wlan.pci_dev->bus); + u32 mask, set; + u32 offset; + + mtk_wed_stop(dev); + mtk_wed_reset(dev, MTK_WED_RESET_WED); + + mask = MTK_WED_WDMA_GLO_CFG_BT_SIZE | + MTK_WED_WDMA_GLO_CFG_DYNAMIC_DMAD_RECYCLE | + MTK_WED_WDMA_GLO_CFG_RX_DIS_FSM_AUTO_IDLE; + set = FIELD_PREP(MTK_WED_WDMA_GLO_CFG_BT_SIZE, 2) | + MTK_WED_WDMA_GLO_CFG_DYNAMIC_SKIP_DMAD_PREP | + MTK_WED_WDMA_GLO_CFG_IDLE_DMAD_SUPPLY; + wed_m32(dev, MTK_WED_WDMA_GLO_CFG, mask, set); + + wdma_set(dev, MTK_WDMA_GLO_CFG, MTK_WDMA_GLO_CFG_RX_INFO_PRERES); + + offset = dev->hw->index ? 0x04000400 : 0; + wed_w32(dev, MTK_WED_WDMA_OFFSET0, 0x2a042a20 + offset); + wed_w32(dev, MTK_WED_WDMA_OFFSET1, 0x29002800 + offset); + + wed_w32(dev, MTK_WED_PCIE_CFG_BASE, MTK_PCIE_BASE(pcie_idx)); + wed_w32(dev, MTK_WED_WPDMA_CFG_BASE, dev->wlan.wpdma_phys); +} + +static void +mtk_wed_hw_init(struct mtk_wed_device *dev) +{ + if (dev->init_done) + return; + + dev->init_done = true; + mtk_wed_set_ext_int(dev, false); + wed_w32(dev, MTK_WED_TX_BM_CTRL, + MTK_WED_TX_BM_CTRL_PAUSE | + FIELD_PREP(MTK_WED_TX_BM_CTRL_VLD_GRP_NUM, + dev->buf_ring.size / 128) | + FIELD_PREP(MTK_WED_TX_BM_CTRL_RSV_GRP_NUM, + MTK_WED_TX_RING_SIZE / 256)); + + wed_w32(dev, MTK_WED_TX_BM_BASE, dev->buf_ring.desc_phys); + + wed_w32(dev, MTK_WED_TX_BM_TKID, + FIELD_PREP(MTK_WED_TX_BM_TKID_START, + dev->wlan.token_start) | + FIELD_PREP(MTK_WED_TX_BM_TKID_END, + dev->wlan.token_start + dev->wlan.nbuf)); + + wed_w32(dev, MTK_WED_TX_BM_BUF_LEN, MTK_WED_PKT_SIZE); + + wed_w32(dev, MTK_WED_TX_BM_DYN_THR, + FIELD_PREP(MTK_WED_TX_BM_DYN_THR_LO, 1) | + MTK_WED_TX_BM_DYN_THR_HI); + + mtk_wed_reset(dev, MTK_WED_RESET_TX_BM); + + wed_set(dev, MTK_WED_CTRL, + MTK_WED_CTRL_WED_TX_BM_EN | + MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); + + wed_clr(dev, MTK_WED_TX_BM_CTRL, MTK_WED_TX_BM_CTRL_PAUSE); +} + +static void +mtk_wed_ring_reset(struct mtk_wdma_desc *desc, int size) +{ + int i; + + for (i = 0; i < size; i++) { + desc[i].buf0 = 0; + desc[i].ctrl = cpu_to_le32(MTK_WDMA_DESC_CTRL_DMA_DONE); + desc[i].buf1 = 0; + desc[i].info = 0; + } +} + +static u32 +mtk_wed_check_busy(struct mtk_wed_device *dev) +{ + if (wed_r32(dev, MTK_WED_GLO_CFG) & MTK_WED_GLO_CFG_TX_DMA_BUSY) + return true; + + if (wed_r32(dev, MTK_WED_WPDMA_GLO_CFG) & + MTK_WED_WPDMA_GLO_CFG_TX_DRV_BUSY) + return true; + + if (wed_r32(dev, MTK_WED_CTRL) & MTK_WED_CTRL_WDMA_INT_AGENT_BUSY) + return true; + + if (wed_r32(dev, MTK_WED_WDMA_GLO_CFG) & + MTK_WED_WDMA_GLO_CFG_RX_DRV_BUSY) + return true; + + if (wdma_r32(dev, MTK_WDMA_GLO_CFG) & + MTK_WED_WDMA_GLO_CFG_RX_DRV_BUSY) + return true; + + if (wed_r32(dev, MTK_WED_CTRL) & + (MTK_WED_CTRL_WED_TX_BM_BUSY | MTK_WED_CTRL_WED_TX_FREE_AGENT_BUSY)) + return true; + + return false; +} + +static int +mtk_wed_poll_busy(struct mtk_wed_device *dev) +{ + int sleep = 15000; + int timeout = 100 * sleep; + u32 val; + + return read_poll_timeout(mtk_wed_check_busy, val, !val, sleep, + timeout, false, dev); +} + +static void +mtk_wed_reset_dma(struct mtk_wed_device *dev) +{ + bool busy = false; + u32 val; + int i; + + for (i = 0; i < ARRAY_SIZE(dev->tx_ring); i++) { + struct mtk_wdma_desc *desc = dev->tx_ring[i].desc; + + if (!desc) + continue; + + mtk_wed_ring_reset(desc, MTK_WED_TX_RING_SIZE); + } + + if (mtk_wed_poll_busy(dev)) + busy = mtk_wed_check_busy(dev); + + if (busy) { + mtk_wed_reset(dev, MTK_WED_RESET_WED_TX_DMA); + } else { + wed_w32(dev, MTK_WED_RESET_IDX, + MTK_WED_RESET_IDX_TX | + MTK_WED_RESET_IDX_RX); + wed_w32(dev, MTK_WED_RESET_IDX, 0); + } + + wdma_w32(dev, MTK_WDMA_RESET_IDX, MTK_WDMA_RESET_IDX_RX); + wdma_w32(dev, MTK_WDMA_RESET_IDX, 0); + + if (busy) { + mtk_wed_reset(dev, MTK_WED_RESET_WDMA_INT_AGENT); + mtk_wed_reset(dev, MTK_WED_RESET_WDMA_RX_DRV); + } else { + wed_w32(dev, MTK_WED_WDMA_RESET_IDX, + MTK_WED_WDMA_RESET_IDX_RX | MTK_WED_WDMA_RESET_IDX_DRV); + wed_w32(dev, MTK_WED_WDMA_RESET_IDX, 0); + + wed_set(dev, MTK_WED_WDMA_GLO_CFG, + MTK_WED_WDMA_GLO_CFG_RST_INIT_COMPLETE); + + wed_clr(dev, MTK_WED_WDMA_GLO_CFG, + MTK_WED_WDMA_GLO_CFG_RST_INIT_COMPLETE); + } + + for (i = 0; i < 100; i++) { + val = wed_r32(dev, MTK_WED_TX_BM_INTF); + if (FIELD_GET(MTK_WED_TX_BM_INTF_TKFIFO_FDEP, val) == 0x40) + break; + } + + mtk_wed_reset(dev, MTK_WED_RESET_TX_FREE_AGENT); + mtk_wed_reset(dev, MTK_WED_RESET_TX_BM); + + if (busy) { + mtk_wed_reset(dev, MTK_WED_RESET_WPDMA_INT_AGENT); + mtk_wed_reset(dev, MTK_WED_RESET_WPDMA_TX_DRV); + mtk_wed_reset(dev, MTK_WED_RESET_WPDMA_RX_DRV); + } else { + wed_w32(dev, MTK_WED_WPDMA_RESET_IDX, + MTK_WED_WPDMA_RESET_IDX_TX | + MTK_WED_WPDMA_RESET_IDX_RX); + wed_w32(dev, MTK_WED_WPDMA_RESET_IDX, 0); + } + +} + +static int +mtk_wed_ring_alloc(struct mtk_wed_device *dev, struct mtk_wed_ring *ring, + int size) +{ + ring->desc = dma_alloc_coherent(dev->hw->dev, + size * sizeof(*ring->desc), + &ring->desc_phys, GFP_KERNEL); + if (!ring->desc) + return -ENOMEM; + + ring->size = size; + mtk_wed_ring_reset(ring->desc, size); + + return 0; +} + +static int +mtk_wed_wdma_ring_setup(struct mtk_wed_device *dev, int idx, int size) +{ + struct mtk_wed_ring *wdma = &dev->tx_wdma[idx]; + + if (mtk_wed_ring_alloc(dev, wdma, MTK_WED_WDMA_RING_SIZE)) + return -ENOMEM; + + wdma_w32(dev, MTK_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_BASE, + wdma->desc_phys); + wdma_w32(dev, MTK_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_COUNT, + size); + wdma_w32(dev, MTK_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_CPU_IDX, 0); + + wed_w32(dev, MTK_WED_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_BASE, + wdma->desc_phys); + wed_w32(dev, MTK_WED_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_COUNT, + size); + + return 0; +} + +static void +mtk_wed_start(struct mtk_wed_device *dev, u32 irq_mask) +{ + int pcie_idx = !!pci_domain_nr(dev->wlan.pci_dev->bus); + u32 wdma_mask; + u32 val; + int i; + + for (i = 0; i < ARRAY_SIZE(dev->tx_wdma); i++) + if (!dev->tx_wdma[i].desc) + mtk_wed_wdma_ring_setup(dev, i, 16); + + wdma_mask = FIELD_PREP(MTK_WDMA_INT_MASK_RX_DONE, GENMASK(1, 0)); + + mtk_wed_hw_init(dev); + + wed_set(dev, MTK_WED_CTRL, + MTK_WED_CTRL_WDMA_INT_AGENT_EN | + MTK_WED_CTRL_WPDMA_INT_AGENT_EN | + MTK_WED_CTRL_WED_TX_BM_EN | + MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); + + wed_w32(dev, MTK_WED_PCIE_INT_TRIGGER, MTK_WED_PCIE_INT_TRIGGER_STATUS); + + wed_w32(dev, MTK_WED_WPDMA_INT_TRIGGER, + MTK_WED_WPDMA_INT_TRIGGER_RX_DONE | + MTK_WED_WPDMA_INT_TRIGGER_TX_DONE); + + wed_set(dev, MTK_WED_WPDMA_INT_CTRL, + MTK_WED_WPDMA_INT_CTRL_SUBRT_ADV); + + wed_w32(dev, MTK_WED_WDMA_INT_TRIGGER, wdma_mask); + wed_clr(dev, MTK_WED_WDMA_INT_CTRL, wdma_mask); + + wdma_w32(dev, MTK_WDMA_INT_MASK, wdma_mask); + wdma_w32(dev, MTK_WDMA_INT_GRP2, wdma_mask); + + wed_w32(dev, MTK_WED_WPDMA_INT_MASK, irq_mask); + wed_w32(dev, MTK_WED_INT_MASK, irq_mask); + + wed_set(dev, MTK_WED_GLO_CFG, + MTK_WED_GLO_CFG_TX_DMA_EN | + MTK_WED_GLO_CFG_RX_DMA_EN); + wed_set(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_TX_DRV_EN | + MTK_WED_WPDMA_GLO_CFG_RX_DRV_EN); + wed_set(dev, MTK_WED_WDMA_GLO_CFG, + MTK_WED_WDMA_GLO_CFG_RX_DRV_EN); + + mtk_wed_set_ext_int(dev, true); + val = dev->wlan.wpdma_phys | + MTK_PCIE_MIRROR_MAP_EN | + FIELD_PREP(MTK_PCIE_MIRROR_MAP_WED_ID, dev->hw->index); + + if (dev->hw->index) + val |= BIT(1); + val |= BIT(0); + regmap_write(dev->hw->mirror, pcie_idx * 4, val); + + dev->running = true; +} + +static int +mtk_wed_attach(struct mtk_wed_device *dev) +{ + struct mtk_wed_hw *hw; + int ret = 0; + + if (pci_domain_nr(dev->wlan.pci_dev->bus) > 1) + return -ENODEV; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + mutex_lock(&hw_lock); + + hw = mtk_wed_assign(dev); + if (!hw) { + module_put(THIS_MODULE); + ret = -ENODEV; + goto out; + } + + dev->hw = hw; + dev->irq = hw->irq; + dev->wdma_idx = hw->index; + + ret = mtk_wed_buffer_alloc(dev); + if (ret) { + mtk_wed_detach(dev); + goto out; + } + + mtk_wed_hw_init_early(dev); + +out: + mutex_unlock(&hw_lock); + + return ret; +} + +static int +mtk_wed_tx_ring_setup(struct mtk_wed_device *dev, int idx, void __iomem *regs) +{ + struct mtk_wed_ring *ring = &dev->tx_ring[idx]; + + /* + * Tx ring redirection: + * Instead of configuring the WLAN PDMA TX ring directly, the WLAN + * driver allocated DMA ring gets configured into WED MTK_WED_RING_TX(n) + * registers. + * + * WED driver posts its own DMA ring as WLAN PDMA TX and configures it + * into MTK_WED_WPDMA_RING_TX(n) registers. + * It gets filled with packets picked up from WED TX ring and from + * WDMA RX. + */ + + BUG_ON(idx > ARRAY_SIZE(dev->tx_ring)); + + if (mtk_wed_ring_alloc(dev, ring, MTK_WED_TX_RING_SIZE)) + return -ENOMEM; + + if (mtk_wed_wdma_ring_setup(dev, idx, MTK_WED_WDMA_RING_SIZE)) + return -ENOMEM; + + ring->reg_base = MTK_WED_RING_TX(idx); + ring->wpdma = regs; + + /* WED -> WPDMA */ + wpdma_tx_w32(dev, idx, MTK_WED_RING_OFS_BASE, ring->desc_phys); + wpdma_tx_w32(dev, idx, MTK_WED_RING_OFS_COUNT, MTK_WED_TX_RING_SIZE); + wpdma_tx_w32(dev, idx, MTK_WED_RING_OFS_CPU_IDX, 0); + + wed_w32(dev, MTK_WED_WPDMA_RING_TX(idx) + MTK_WED_RING_OFS_BASE, + ring->desc_phys); + wed_w32(dev, MTK_WED_WPDMA_RING_TX(idx) + MTK_WED_RING_OFS_COUNT, + MTK_WED_TX_RING_SIZE); + wed_w32(dev, MTK_WED_WPDMA_RING_TX(idx) + MTK_WED_RING_OFS_CPU_IDX, 0); + + return 0; +} + +static int +mtk_wed_txfree_ring_setup(struct mtk_wed_device *dev, void __iomem *regs) +{ + struct mtk_wed_ring *ring = &dev->txfree_ring; + int i; + + /* + * For txfree event handling, the same DMA ring is shared between WED + * and WLAN. The WLAN driver accesses the ring index registers through + * WED + */ + ring->reg_base = MTK_WED_RING_RX(1); + ring->wpdma = regs; + + for (i = 0; i < 12; i += 4) { + u32 val = readl(regs + i); + + wed_w32(dev, MTK_WED_RING_RX(1) + i, val); + wed_w32(dev, MTK_WED_WPDMA_RING_RX(1) + i, val); + } + + return 0; +} + +static u32 +mtk_wed_irq_get(struct mtk_wed_device *dev, u32 mask) +{ + u32 val; + + val = wed_r32(dev, MTK_WED_EXT_INT_STATUS); + wed_w32(dev, MTK_WED_EXT_INT_STATUS, val); + val &= MTK_WED_EXT_INT_STATUS_ERROR_MASK; + WARN_RATELIMIT(val, "mtk_wed%d: error status=%08x\n", + dev->hw->index, val); + + val = wed_r32(dev, MTK_WED_INT_STATUS); + val &= mask; + wed_w32(dev, MTK_WED_INT_STATUS, val); /* ACK */ + + return val; +} + +static void +mtk_wed_irq_set_mask(struct mtk_wed_device *dev, u32 mask) +{ + if (!dev->running) + return; + + mtk_wed_set_ext_int(dev, !!mask); + wed_w32(dev, MTK_WED_INT_MASK, mask); +} + +int mtk_wed_flow_add(int index) +{ + struct mtk_wed_hw *hw = hw_list[index]; + int ret; + + if (!hw || !hw->wed_dev) + return -ENODEV; + + if (hw->num_flows) { + hw->num_flows++; + return 0; + } + + mutex_lock(&hw_lock); + if (!hw->wed_dev) + ret = -ENODEV; + else + ret = hw->wed_dev->wlan.offload_enable(hw->wed_dev); + if (!ret) + hw->num_flows++; + mutex_unlock(&hw_lock); + + return ret; +} + +void mtk_wed_flow_remove(int index) +{ + struct mtk_wed_hw *hw = hw_list[index]; + + if (!hw) + return; + + if (--hw->num_flows) + return; + + mutex_lock(&hw_lock); + if (hw->wed_dev) + hw->wed_dev->wlan.offload_disable(hw->wed_dev); + mutex_unlock(&hw_lock); +} + +void mtk_wed_add_hw(struct device_node *np, void __iomem *wdma, + void __iomem *mirror, int index) +{ + static const struct mtk_wed_ops wed_ops = { + .attach = mtk_wed_attach, + .tx_ring_setup = mtk_wed_tx_ring_setup, + .txfree_ring_setup = mtk_wed_txfree_ring_setup, + .start = mtk_wed_start, + .stop = mtk_wed_stop, + .reset_dma = mtk_wed_reset_dma, + .reg_read = wed_r32, + .reg_write = wed_w32, + .irq_get = mtk_wed_irq_get, + .irq_set_mask = mtk_wed_irq_set_mask, + .detach = mtk_wed_detach, + }; + struct platform_device *pdev; + struct mtk_wed_hw *hw; + struct regmap *regs; + int irq; + + if (!np) + return; + + pdev = of_find_device_by_node(np); + if (!pdev) + return; + + get_device(&pdev->dev); + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return; + + regs = syscon_regmap_lookup_by_phandle(np, NULL); + if (!regs) + return; + + rcu_assign_pointer(mtk_soc_wed_ops, &wed_ops); + + mutex_lock(&hw_lock); + + if (WARN_ON(hw_list[index])) + goto unlock; + + hw = kzalloc(sizeof(*hw), GFP_KERNEL); + hw->node = np; + hw->regs = regs; + hw->dev = &pdev->dev; + hw->wdma = wdma; + hw->index = index; + hw->irq = irq; + hw->mirror = mirror; + if (!index) { + regmap_write(mirror, 0, 0); + regmap_write(mirror, 4, 0); + } + mtk_wed_hw_add_debugfs(hw); + + hw_list[index] = hw; + +unlock: + mutex_unlock(&hw_lock); +} + +void mtk_wed_exit(void) +{ + int i; + + rcu_assign_pointer(mtk_soc_wed_ops, NULL); + + synchronize_rcu(); + + for (i = 0; i < ARRAY_SIZE(hw_list); i++) { + struct mtk_wed_hw *hw; + + hw = hw_list[i]; + if (!hw) + continue; + + hw_list[i] = NULL; + debugfs_remove(hw->debugfs_dir); + put_device(hw->dev); + kfree(hw); + } +} diff --git a/drivers/net/ethernet/mediatek/mtk_wed.h b/drivers/net/ethernet/mediatek/mtk_wed.h new file mode 100644 index 000000000000..6a424dc4c249 --- /dev/null +++ b/drivers/net/ethernet/mediatek/mtk_wed.h @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2021 Felix Fietkau */ + +#ifndef __MTK_WED_PRIV_H +#define __MTK_WED_PRIV_H + +#include +#include +#include + +struct mtk_wed_hw { + struct device_node *node; + struct regmap *regs; + struct device *dev; + void __iomem *wdma; + struct regmap *mirror; + struct dentry *debugfs_dir; + struct mtk_wed_device *wed_dev; + u32 debugfs_reg; + u32 num_flows; + char dirname[5]; + int irq; + int index; +}; + +static inline void +wed_w32(struct mtk_wed_device *dev, u32 reg, u32 val) +{ + regmap_write(dev->hw->regs, reg, val); +} + +static inline u32 +wed_r32(struct mtk_wed_device *dev, u32 reg) +{ + unsigned int val; + + regmap_read(dev->hw->regs, reg, &val); + + return val; +} + +static inline void +wdma_w32(struct mtk_wed_device *dev, u32 reg, u32 val) +{ + writel(val, dev->hw->wdma + reg); +} + +static inline u32 +wdma_r32(struct mtk_wed_device *dev, u32 reg) +{ + return readl(dev->hw->wdma + reg); +} + +static inline u32 +wpdma_tx_r32(struct mtk_wed_device *dev, int ring, u32 reg) +{ + if (!dev->tx_ring[ring].wpdma) + return 0; + + return readl(dev->tx_ring[ring].wpdma + reg); +} + +static inline void +wpdma_tx_w32(struct mtk_wed_device *dev, int ring, u32 reg, u32 val) +{ + if (!dev->tx_ring[ring].wpdma) + return; + + writel(val, dev->tx_ring[ring].wpdma + reg); +} + +static inline u32 +wpdma_txfree_r32(struct mtk_wed_device *dev, u32 reg) +{ + if (!dev->txfree_ring.wpdma) + return 0; + + return readl(dev->txfree_ring.wpdma + reg); +} + +static inline void +wpdma_txfree_w32(struct mtk_wed_device *dev, u32 reg, u32 val) +{ + if (!dev->txfree_ring.wpdma) + return; + + writel(val, dev->txfree_ring.wpdma + reg); +} + +#ifdef CONFIG_NET_MEDIATEK_SOC_WED +void mtk_wed_add_hw(struct device_node *np, void __iomem *wdma, + void __iomem *mirror, int index); +void mtk_wed_exit(void); +int mtk_wed_flow_add(int index); +void mtk_wed_flow_remove(int index); +#else +static inline void +mtk_wed_add_hw(struct device_node *np, void __iomem *wdma, + void __iomem *mirror, int index) +{ +} +static inline void +mtk_wed_exit(void) +{ +} +static inline int mtk_wed_flow_add(int index) +{ + return -EINVAL; +} +static inline void mtk_wed_flow_remove(int index) +{ +} +#endif + +#ifdef CONFIG_DEBUG_FS +void mtk_wed_hw_add_debugfs(struct mtk_wed_hw *hw); +#else +static inline void mtk_wed_hw_add_debugfs(struct mtk_wed_hw *hw) +{ +} +#endif + +#endif diff --git a/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c new file mode 100644 index 000000000000..a81d3fd1a439 --- /dev/null +++ b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2021 Felix Fietkau */ + +#include +#include "mtk_wed.h" +#include "mtk_wed_regs.h" + +struct reg_dump { + const char *name; + u16 offset; + u8 type; + u8 base; +}; + +enum { + DUMP_TYPE_STRING, + DUMP_TYPE_WED, + DUMP_TYPE_WDMA, + DUMP_TYPE_WPDMA_TX, + DUMP_TYPE_WPDMA_TXFREE, +}; + +#define DUMP_STR(_str) { _str, 0, DUMP_TYPE_STRING } +#define DUMP_REG(_reg, ...) { #_reg, MTK_##_reg, __VA_ARGS__ } +#define DUMP_RING(_prefix, _base, ...) \ + { _prefix " BASE", _base, __VA_ARGS__ }, \ + { _prefix " CNT", _base + 0x4, __VA_ARGS__ }, \ + { _prefix " CIDX", _base + 0x8, __VA_ARGS__ }, \ + { _prefix " DIDX", _base + 0xc, __VA_ARGS__ } + +#define DUMP_WED(_reg) DUMP_REG(_reg, DUMP_TYPE_WED) +#define DUMP_WED_RING(_base) DUMP_RING(#_base, MTK_##_base, DUMP_TYPE_WED) + +#define DUMP_WDMA(_reg) DUMP_REG(_reg, DUMP_TYPE_WDMA) +#define DUMP_WDMA_RING(_base) DUMP_RING(#_base, MTK_##_base, DUMP_TYPE_WDMA) + +#define DUMP_WPDMA_TX_RING(_n) DUMP_RING("WPDMA_TX" #_n, 0, DUMP_TYPE_WPDMA_TX, _n) +#define DUMP_WPDMA_TXFREE_RING DUMP_RING("WPDMA_RX1", 0, DUMP_TYPE_WPDMA_TXFREE) + +static void +print_reg_val(struct seq_file *s, const char *name, u32 val) +{ + seq_printf(s, "%-32s %08x\n", name, val); +} + +static void +dump_wed_regs(struct seq_file *s, struct mtk_wed_device *dev, + const struct reg_dump *regs, int n_regs) +{ + const struct reg_dump *cur; + u32 val; + + for (cur = regs; cur < ®s[n_regs]; cur++) { + switch (cur->type) { + case DUMP_TYPE_STRING: + seq_printf(s, "%s======== %s:\n", + cur > regs ? "\n" : "", + cur->name); + continue; + case DUMP_TYPE_WED: + val = wed_r32(dev, cur->offset); + break; + case DUMP_TYPE_WDMA: + val = wdma_r32(dev, cur->offset); + break; + case DUMP_TYPE_WPDMA_TX: + val = wpdma_tx_r32(dev, cur->base, cur->offset); + break; + case DUMP_TYPE_WPDMA_TXFREE: + val = wpdma_txfree_r32(dev, cur->offset); + break; + } + print_reg_val(s, cur->name, val); + } +} + + +static int +wed_txinfo_show(struct seq_file *s, void *data) +{ + static const struct reg_dump regs[] = { + DUMP_STR("WED TX"), + DUMP_WED(WED_TX_MIB(0)), + DUMP_WED_RING(WED_RING_TX(0)), + + DUMP_WED(WED_TX_MIB(1)), + DUMP_WED_RING(WED_RING_TX(1)), + + DUMP_STR("WPDMA TX"), + DUMP_WED(WED_WPDMA_TX_MIB(0)), + DUMP_WED_RING(WED_WPDMA_RING_TX(0)), + DUMP_WED(WED_WPDMA_TX_COHERENT_MIB(0)), + + DUMP_WED(WED_WPDMA_TX_MIB(1)), + DUMP_WED_RING(WED_WPDMA_RING_TX(1)), + DUMP_WED(WED_WPDMA_TX_COHERENT_MIB(1)), + + DUMP_STR("WPDMA TX"), + DUMP_WPDMA_TX_RING(0), + DUMP_WPDMA_TX_RING(1), + + DUMP_STR("WED WDMA RX"), + DUMP_WED(WED_WDMA_RX_MIB(0)), + DUMP_WED_RING(WED_WDMA_RING_RX(0)), + DUMP_WED(WED_WDMA_RX_THRES(0)), + DUMP_WED(WED_WDMA_RX_RECYCLE_MIB(0)), + DUMP_WED(WED_WDMA_RX_PROCESSED_MIB(0)), + + DUMP_WED(WED_WDMA_RX_MIB(1)), + DUMP_WED_RING(WED_WDMA_RING_RX(1)), + DUMP_WED(WED_WDMA_RX_THRES(1)), + DUMP_WED(WED_WDMA_RX_RECYCLE_MIB(1)), + DUMP_WED(WED_WDMA_RX_PROCESSED_MIB(1)), + + DUMP_STR("WDMA RX"), + DUMP_WDMA(WDMA_GLO_CFG), + DUMP_WDMA_RING(WDMA_RING_RX(0)), + DUMP_WDMA_RING(WDMA_RING_RX(1)), + }; + struct mtk_wed_hw *hw = s->private; + struct mtk_wed_device *dev = hw->wed_dev; + + if (!dev) + return 0; + + dump_wed_regs(s, dev, regs, ARRAY_SIZE(regs)); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(wed_txinfo); + + +static int +mtk_wed_reg_set(void *data, u64 val) +{ + struct mtk_wed_hw *hw = data; + + regmap_write(hw->regs, hw->debugfs_reg, val); + + return 0; +} + +static int +mtk_wed_reg_get(void *data, u64 *val) +{ + struct mtk_wed_hw *hw = data; + unsigned int regval; + int ret; + + ret = regmap_read(hw->regs, hw->debugfs_reg, ®val); + if (ret) + return ret; + + *val = regval; + + return 0; +} + +DEFINE_DEBUGFS_ATTRIBUTE(fops_regval, mtk_wed_reg_get, mtk_wed_reg_set, + "0x%08llx\n"); + +void mtk_wed_hw_add_debugfs(struct mtk_wed_hw *hw) +{ + struct dentry *dir; + + snprintf(hw->dirname, sizeof(hw->dirname), "wed%d", hw->index); + dir = debugfs_create_dir(hw->dirname, NULL); + if (!dir) + return; + + hw->debugfs_dir = dir; + debugfs_create_u32("regidx", 0600, dir, &hw->debugfs_reg); + debugfs_create_file_unsafe("regval", 0600, dir, hw, &fops_regval); + debugfs_create_file_unsafe("txinfo", 0400, dir, hw, &wed_txinfo_fops); +} diff --git a/drivers/net/ethernet/mediatek/mtk_wed_ops.c b/drivers/net/ethernet/mediatek/mtk_wed_ops.c new file mode 100644 index 000000000000..a5d9d8a5bce2 --- /dev/null +++ b/drivers/net/ethernet/mediatek/mtk_wed_ops.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2020 Felix Fietkau */ + +#include +#include + +const struct mtk_wed_ops __rcu *mtk_soc_wed_ops; +EXPORT_SYMBOL_GPL(mtk_soc_wed_ops); diff --git a/drivers/net/ethernet/mediatek/mtk_wed_regs.h b/drivers/net/ethernet/mediatek/mtk_wed_regs.h new file mode 100644 index 000000000000..cbeaa0876ff4 --- /dev/null +++ b/drivers/net/ethernet/mediatek/mtk_wed_regs.h @@ -0,0 +1,248 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2020 Felix Fietkau */ + +#ifndef __MTK_WED_REGS_H +#define __MTK_WED_REGS_H + +#define MTK_WDMA_DESC_CTRL_LEN1 GENMASK(14, 0) +#define MTK_WDMA_DESC_CTRL_LAST_SEG1 BIT(15) +#define MTK_WDMA_DESC_CTRL_BURST BIT(16) +#define MTK_WDMA_DESC_CTRL_LEN0 GENMASK(29, 16) +#define MTK_WDMA_DESC_CTRL_LAST_SEG0 BIT(30) +#define MTK_WDMA_DESC_CTRL_DMA_DONE BIT(31) + +struct mtk_wdma_desc { + __le32 buf0; + __le32 ctrl; + __le32 buf1; + __le32 info; +} __packed __aligned(4); + +#define MTK_WED_RESET 0x008 +#define MTK_WED_RESET_TX_BM BIT(0) +#define MTK_WED_RESET_TX_FREE_AGENT BIT(4) +#define MTK_WED_RESET_WPDMA_TX_DRV BIT(8) +#define MTK_WED_RESET_WPDMA_RX_DRV BIT(9) +#define MTK_WED_RESET_WPDMA_INT_AGENT BIT(11) +#define MTK_WED_RESET_WED_TX_DMA BIT(12) +#define MTK_WED_RESET_WDMA_RX_DRV BIT(17) +#define MTK_WED_RESET_WDMA_INT_AGENT BIT(19) +#define MTK_WED_RESET_WED BIT(31) + +#define MTK_WED_CTRL 0x00c +#define MTK_WED_CTRL_WPDMA_INT_AGENT_EN BIT(0) +#define MTK_WED_CTRL_WPDMA_INT_AGENT_BUSY BIT(1) +#define MTK_WED_CTRL_WDMA_INT_AGENT_EN BIT(2) +#define MTK_WED_CTRL_WDMA_INT_AGENT_BUSY BIT(3) +#define MTK_WED_CTRL_WED_TX_BM_EN BIT(8) +#define MTK_WED_CTRL_WED_TX_BM_BUSY BIT(9) +#define MTK_WED_CTRL_WED_TX_FREE_AGENT_EN BIT(10) +#define MTK_WED_CTRL_WED_TX_FREE_AGENT_BUSY BIT(11) +#define MTK_WED_CTRL_RESERVE_EN BIT(12) +#define MTK_WED_CTRL_RESERVE_BUSY BIT(13) +#define MTK_WED_CTRL_FINAL_DIDX_READ BIT(24) +#define MTK_WED_CTRL_MIB_READ_CLEAR BIT(28) + +#define MTK_WED_EXT_INT_STATUS 0x020 +#define MTK_WED_EXT_INT_STATUS_TF_LEN_ERR BIT(0) +#define MTK_WED_EXT_INT_STATUS_TKID_WO_PYLD BIT(1) +#define MTK_WED_EXT_INT_STATUS_TKID_TITO_INVALID BIT(4) +#define MTK_WED_EXT_INT_STATUS_TX_FBUF_LO_TH BIT(8) +#define MTK_WED_EXT_INT_STATUS_TX_FBUF_HI_TH BIT(9) +#define MTK_WED_EXT_INT_STATUS_RX_FBUF_LO_TH BIT(12) +#define MTK_WED_EXT_INT_STATUS_RX_FBUF_HI_TH BIT(13) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_R_RESP_ERR BIT(16) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_W_RESP_ERR BIT(17) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_COHERENT BIT(18) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_INIT_WDMA_EN BIT(19) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_BM_DMAD_COHERENT BIT(20) +#define MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR BIT(21) +#define MTK_WED_EXT_INT_STATUS_TX_DRV_W_RESP_ERR BIT(22) +#define MTK_WED_EXT_INT_STATUS_RX_DRV_DMA_RECYCLE BIT(24) +#define MTK_WED_EXT_INT_STATUS_ERROR_MASK (MTK_WED_EXT_INT_STATUS_TF_LEN_ERR | \ + MTK_WED_EXT_INT_STATUS_TKID_WO_PYLD | \ + MTK_WED_EXT_INT_STATUS_TKID_TITO_INVALID | \ + MTK_WED_EXT_INT_STATUS_RX_DRV_R_RESP_ERR | \ + MTK_WED_EXT_INT_STATUS_RX_DRV_W_RESP_ERR | \ + MTK_WED_EXT_INT_STATUS_RX_DRV_INIT_WDMA_EN | \ + MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR | \ + MTK_WED_EXT_INT_STATUS_TX_DRV_W_RESP_ERR) + +#define MTK_WED_EXT_INT_MASK 0x028 + +#define MTK_WED_STATUS 0x060 +#define MTK_WED_STATUS_TX GENMASK(15, 8) + +#define MTK_WED_TX_BM_CTRL 0x080 +#define MTK_WED_TX_BM_CTRL_VLD_GRP_NUM GENMASK(6, 0) +#define MTK_WED_TX_BM_CTRL_RSV_GRP_NUM GENMASK(22, 16) +#define MTK_WED_TX_BM_CTRL_PAUSE BIT(28) + +#define MTK_WED_TX_BM_BASE 0x084 + +#define MTK_WED_TX_BM_TKID 0x088 +#define MTK_WED_TX_BM_TKID_START GENMASK(15, 0) +#define MTK_WED_TX_BM_TKID_END GENMASK(31, 16) + +#define MTK_WED_TX_BM_BUF_LEN 0x08c + +#define MTK_WED_TX_BM_INTF 0x09c +#define MTK_WED_TX_BM_INTF_TKID GENMASK(15, 0) +#define MTK_WED_TX_BM_INTF_TKFIFO_FDEP GENMASK(23, 16) +#define MTK_WED_TX_BM_INTF_TKID_VALID BIT(28) +#define MTK_WED_TX_BM_INTF_TKID_READ BIT(29) + +#define MTK_WED_TX_BM_DYN_THR 0x0a0 +#define MTK_WED_TX_BM_DYN_THR_LO GENMASK(6, 0) +#define MTK_WED_TX_BM_DYN_THR_HI GENMASK(22, 16) + +#define MTK_WED_INT_STATUS 0x200 +#define MTK_WED_INT_MASK 0x204 + +#define MTK_WED_GLO_CFG 0x208 +#define MTK_WED_GLO_CFG_TX_DMA_EN BIT(0) +#define MTK_WED_GLO_CFG_TX_DMA_BUSY BIT(1) +#define MTK_WED_GLO_CFG_RX_DMA_EN BIT(2) +#define MTK_WED_GLO_CFG_RX_DMA_BUSY BIT(3) +#define MTK_WED_GLO_CFG_RX_BT_SIZE GENMASK(5, 4) +#define MTK_WED_GLO_CFG_TX_WB_DDONE BIT(6) +#define MTK_WED_GLO_CFG_BIG_ENDIAN BIT(7) +#define MTK_WED_GLO_CFG_DIS_BT_SIZE_ALIGN BIT(8) +#define MTK_WED_GLO_CFG_TX_BT_SIZE_LO BIT(9) +#define MTK_WED_GLO_CFG_MULTI_DMA_EN GENMASK(11, 10) +#define MTK_WED_GLO_CFG_FIFO_LITTLE_ENDIAN BIT(12) +#define MTK_WED_GLO_CFG_MI_DEPTH_RD GENMASK(21, 13) +#define MTK_WED_GLO_CFG_TX_BT_SIZE_HI GENMASK(23, 22) +#define MTK_WED_GLO_CFG_SW_RESET BIT(24) +#define MTK_WED_GLO_CFG_FIRST_TOKEN_ONLY BIT(26) +#define MTK_WED_GLO_CFG_OMIT_RX_INFO BIT(27) +#define MTK_WED_GLO_CFG_OMIT_TX_INFO BIT(28) +#define MTK_WED_GLO_CFG_BYTE_SWAP BIT(29) +#define MTK_WED_GLO_CFG_RX_2B_OFFSET BIT(31) + +#define MTK_WED_RESET_IDX 0x20c +#define MTK_WED_RESET_IDX_TX GENMASK(3, 0) +#define MTK_WED_RESET_IDX_RX GENMASK(17, 16) + +#define MTK_WED_TX_MIB(_n) (0x2a0 + (_n) * 4) + +#define MTK_WED_RING_TX(_n) (0x300 + (_n) * 0x10) + +#define MTK_WED_RING_RX(_n) (0x400 + (_n) * 0x10) + +#define MTK_WED_WPDMA_INT_TRIGGER 0x504 +#define MTK_WED_WPDMA_INT_TRIGGER_RX_DONE BIT(1) +#define MTK_WED_WPDMA_INT_TRIGGER_TX_DONE GENMASK(5, 4) + +#define MTK_WED_WPDMA_GLO_CFG 0x508 +#define MTK_WED_WPDMA_GLO_CFG_TX_DRV_EN BIT(0) +#define MTK_WED_WPDMA_GLO_CFG_TX_DRV_BUSY BIT(1) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_EN BIT(2) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_BUSY BIT(3) +#define MTK_WED_WPDMA_GLO_CFG_RX_BT_SIZE GENMASK(5, 4) +#define MTK_WED_WPDMA_GLO_CFG_TX_WB_DDONE BIT(6) +#define MTK_WED_WPDMA_GLO_CFG_BIG_ENDIAN BIT(7) +#define MTK_WED_WPDMA_GLO_CFG_DIS_BT_SIZE_ALIGN BIT(8) +#define MTK_WED_WPDMA_GLO_CFG_TX_BT_SIZE_LO BIT(9) +#define MTK_WED_WPDMA_GLO_CFG_MULTI_DMA_EN GENMASK(11, 10) +#define MTK_WED_WPDMA_GLO_CFG_FIFO_LITTLE_ENDIAN BIT(12) +#define MTK_WED_WPDMA_GLO_CFG_MI_DEPTH_RD GENMASK(21, 13) +#define MTK_WED_WPDMA_GLO_CFG_TX_BT_SIZE_HI GENMASK(23, 22) +#define MTK_WED_WPDMA_GLO_CFG_SW_RESET BIT(24) +#define MTK_WED_WPDMA_GLO_CFG_FIRST_TOKEN_ONLY BIT(26) +#define MTK_WED_WPDMA_GLO_CFG_OMIT_RX_INFO BIT(27) +#define MTK_WED_WPDMA_GLO_CFG_OMIT_TX_INFO BIT(28) +#define MTK_WED_WPDMA_GLO_CFG_BYTE_SWAP BIT(29) +#define MTK_WED_WPDMA_GLO_CFG_RX_2B_OFFSET BIT(31) + +#define MTK_WED_WPDMA_RESET_IDX 0x50c +#define MTK_WED_WPDMA_RESET_IDX_TX GENMASK(3, 0) +#define MTK_WED_WPDMA_RESET_IDX_RX GENMASK(17, 16) + +#define MTK_WED_WPDMA_INT_CTRL 0x520 +#define MTK_WED_WPDMA_INT_CTRL_SUBRT_ADV BIT(21) + +#define MTK_WED_WPDMA_INT_MASK 0x524 + +#define MTK_WED_PCIE_CFG_BASE 0x560 + +#define MTK_WED_PCIE_INT_TRIGGER 0x570 +#define MTK_WED_PCIE_INT_TRIGGER_STATUS BIT(16) + +#define MTK_WED_WPDMA_CFG_BASE 0x580 + +#define MTK_WED_WPDMA_TX_MIB(_n) (0x5a0 + (_n) * 4) +#define MTK_WED_WPDMA_TX_COHERENT_MIB(_n) (0x5d0 + (_n) * 4) + +#define MTK_WED_WPDMA_RING_TX(_n) (0x600 + (_n) * 0x10) +#define MTK_WED_WPDMA_RING_RX(_n) (0x700 + (_n) * 0x10) +#define MTK_WED_WDMA_RING_RX(_n) (0x900 + (_n) * 0x10) +#define MTK_WED_WDMA_RX_THRES(_n) (0x940 + (_n) * 0x4) + +#define MTK_WED_WDMA_GLO_CFG 0xa04 +#define MTK_WED_WDMA_GLO_CFG_TX_DRV_EN BIT(0) +#define MTK_WED_WDMA_GLO_CFG_RX_DRV_EN BIT(2) +#define MTK_WED_WDMA_GLO_CFG_RX_DRV_BUSY BIT(3) +#define MTK_WED_WDMA_GLO_CFG_BT_SIZE GENMASK(5, 4) +#define MTK_WED_WDMA_GLO_CFG_TX_WB_DDONE BIT(6) +#define MTK_WED_WDMA_GLO_CFG_RX_DIS_FSM_AUTO_IDLE BIT(13) +#define MTK_WED_WDMA_GLO_CFG_WCOMPLETE_SEL BIT(16) +#define MTK_WED_WDMA_GLO_CFG_INIT_PHASE_RXDMA_BYPASS BIT(17) +#define MTK_WED_WDMA_GLO_CFG_INIT_PHASE_BYPASS BIT(18) +#define MTK_WED_WDMA_GLO_CFG_FSM_RETURN_IDLE BIT(19) +#define MTK_WED_WDMA_GLO_CFG_WAIT_COHERENT BIT(20) +#define MTK_WED_WDMA_GLO_CFG_AXI_W_AFTER_AW BIT(21) +#define MTK_WED_WDMA_GLO_CFG_IDLE_DMAD_SUPPLY_SINGLE_W BIT(22) +#define MTK_WED_WDMA_GLO_CFG_IDLE_DMAD_SUPPLY BIT(23) +#define MTK_WED_WDMA_GLO_CFG_DYNAMIC_SKIP_DMAD_PREP BIT(24) +#define MTK_WED_WDMA_GLO_CFG_DYNAMIC_DMAD_RECYCLE BIT(25) +#define MTK_WED_WDMA_GLO_CFG_RST_INIT_COMPLETE BIT(26) +#define MTK_WED_WDMA_GLO_CFG_RXDRV_CLKGATE_BYPASS BIT(30) + +#define MTK_WED_WDMA_RESET_IDX 0xa08 +#define MTK_WED_WDMA_RESET_IDX_RX GENMASK(17, 16) +#define MTK_WED_WDMA_RESET_IDX_DRV GENMASK(25, 24) + +#define MTK_WED_WDMA_INT_TRIGGER 0xa28 +#define MTK_WED_WDMA_INT_TRIGGER_RX_DONE GENMASK(17, 16) + +#define MTK_WED_WDMA_INT_CTRL 0xa2c +#define MTK_WED_WDMA_INT_CTRL_POLL_SRC_SEL GENMASK(17, 16) + +#define MTK_WED_WDMA_OFFSET0 0xaa4 +#define MTK_WED_WDMA_OFFSET1 0xaa8 + +#define MTK_WED_WDMA_RX_MIB(_n) (0xae0 + (_n) * 4) +#define MTK_WED_WDMA_RX_RECYCLE_MIB(_n) (0xae8 + (_n) * 4) +#define MTK_WED_WDMA_RX_PROCESSED_MIB(_n) (0xaf0 + (_n) * 4) + +#define MTK_WED_RING_OFS_BASE 0x00 +#define MTK_WED_RING_OFS_COUNT 0x04 +#define MTK_WED_RING_OFS_CPU_IDX 0x08 +#define MTK_WED_RING_OFS_DMA_IDX 0x0c + +#define MTK_WDMA_RING_RX(_n) (0x100 + (_n) * 0x10) + +#define MTK_WDMA_GLO_CFG 0x204 +#define MTK_WDMA_GLO_CFG_RX_INFO_PRERES GENMASK(28, 26) + +#define MTK_WDMA_RESET_IDX 0x208 +#define MTK_WDMA_RESET_IDX_TX GENMASK(3, 0) +#define MTK_WDMA_RESET_IDX_RX GENMASK(17, 16) + +#define MTK_WDMA_INT_MASK 0x228 +#define MTK_WDMA_INT_MASK_TX_DONE GENMASK(3, 0) +#define MTK_WDMA_INT_MASK_RX_DONE GENMASK(17, 16) +#define MTK_WDMA_INT_MASK_TX_DELAY BIT(28) +#define MTK_WDMA_INT_MASK_TX_COHERENT BIT(29) +#define MTK_WDMA_INT_MASK_RX_DELAY BIT(30) +#define MTK_WDMA_INT_MASK_RX_COHERENT BIT(31) + +#define MTK_WDMA_INT_GRP1 0x250 +#define MTK_WDMA_INT_GRP2 0x254 + +#define MTK_PCIE_MIRROR_MAP(n) ((n) ? 0x4 : 0x0) +#define MTK_PCIE_MIRROR_MAP_EN BIT(0) +#define MTK_PCIE_MIRROR_MAP_WED_ID BIT(1) + +#endif diff --git a/include/linux/soc/mediatek/mtk_wed.h b/include/linux/soc/mediatek/mtk_wed.h new file mode 100644 index 000000000000..96689861630a --- /dev/null +++ b/include/linux/soc/mediatek/mtk_wed.h @@ -0,0 +1,127 @@ +#ifndef __MTK_WED_H +#define __MTK_WED_H + +#include +#include +#include +#include + +#define MTK_WED_TX_QUEUES 2 + +struct mtk_wed_hw; +struct mtk_wdma_desc; + +struct mtk_wed_ring { + struct mtk_wdma_desc *desc; + dma_addr_t desc_phys; + int size; + + u32 reg_base; + void __iomem *wpdma; +}; + +struct mtk_wed_device { +#ifdef CONFIG_NET_MEDIATEK_SOC_WED + const struct mtk_wed_ops *ops; + struct mtk_wed_hw *hw; + bool init_done, running; + int wdma_idx; + int irq; + + struct mtk_wed_ring tx_ring[MTK_WED_TX_QUEUES]; + struct mtk_wed_ring txfree_ring; + struct mtk_wed_ring tx_wdma[MTK_WED_TX_QUEUES]; + + struct { + int size; + void **pages; + struct mtk_wdma_desc *desc; + dma_addr_t desc_phys; + } buf_ring; + + /* filled by driver: */ + struct { + struct pci_dev *pci_dev; + + u32 wpdma_phys; + + u16 token_start; + unsigned int nbuf; + + u32 (*init_buf)(void *ptr, dma_addr_t phys, int token_id); + int (*offload_enable)(struct mtk_wed_device *wed); + void (*offload_disable)(struct mtk_wed_device *wed); + } wlan; +#endif +}; + +struct mtk_wed_ops { + int (*attach)(struct mtk_wed_device *dev); + int (*tx_ring_setup)(struct mtk_wed_device *dev, int ring, + void __iomem *regs); + int (*txfree_ring_setup)(struct mtk_wed_device *dev, + void __iomem *regs); + void (*detach)(struct mtk_wed_device *dev); + + void (*stop)(struct mtk_wed_device *dev); + void (*start)(struct mtk_wed_device *dev, u32 irq_mask); + void (*reset_dma)(struct mtk_wed_device *dev); + + u32 (*reg_read)(struct mtk_wed_device *dev, u32 reg); + void (*reg_write)(struct mtk_wed_device *dev, u32 reg, u32 val); + + u32 (*irq_get)(struct mtk_wed_device *dev, u32 mask); + void (*irq_set_mask)(struct mtk_wed_device *dev, u32 mask); +}; + +extern const struct mtk_wed_ops __rcu *mtk_soc_wed_ops; + +static inline int +mtk_wed_device_attach(struct mtk_wed_device *dev) +{ + int ret = -ENODEV; + +#ifdef CONFIG_NET_MEDIATEK_SOC_WED + rcu_read_lock(); + dev->ops = rcu_dereference(mtk_soc_wed_ops); + if (dev->ops) + ret = dev->ops->attach(dev); + rcu_read_unlock(); + + if (ret) + dev->ops = NULL; +#endif + + return ret; +} + +#ifdef CONFIG_NET_MEDIATEK_SOC_WED +#define mtk_wed_device_active(_dev) !!(_dev)->ops +#define mtk_wed_device_detach(_dev) (_dev)->ops->detach(_dev) +#define mtk_wed_device_start(_dev, _mask) (_dev)->ops->start(_dev, _mask) +#define mtk_wed_device_tx_ring_setup(_dev, _ring, _regs) \ + (_dev)->ops->tx_ring_setup(_dev, _ring, _regs) +#define mtk_wed_device_txfree_ring_setup(_dev, _regs) \ + (_dev)->ops->txfree_ring_setup(_dev, _regs) +#define mtk_wed_device_reg_read(_dev, _reg) \ + (_dev)->ops->reg_read(_dev, _reg) +#define mtk_wed_device_reg_write(_dev, _reg, _val) \ + (_dev)->ops->reg_write(_dev, _reg, _val) +#define mtk_wed_device_irq_get(_dev, _mask) \ + (_dev)->ops->irq_get(_dev, _mask) +#define mtk_wed_device_irq_set_mask(_dev, _mask) \ + (_dev)->ops->irq_set_mask(_dev, _mask) +#else +#define mtk_wed_device_active(_dev) false +#define mtk_wed_device_attach(_dev) -ENODEV +#define mtk_wed_device_detach(_dev) do {} while (0) +#define mtk_wed_device_start(_dev, _mask) do {} while (0) +#define mtk_wed_device_tx_ring_setup(_dev, _ring, _regs) -ENODEV +#define mtk_wed_device_txfree_ring_setup(_dev, _ring, _regs) -ENODEV +#define mtk_wed_device_reg_read(_dev, _reg) 0 +#define mtk_wed_device_reg_write(_dev, _reg, _val) do {} while (0) +#define mtk_wed_device_irq_get(_dev, _mask) 0 +#define mtk_wed_device_irq_set_mask(_dev, _mask) do {} while (0) +#endif + +#endif From patchwork Tue Jul 13 16:07:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 475054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34CA9C11F68 for ; Tue, 13 Jul 2021 16:07:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 167C961164 for ; Tue, 13 Jul 2021 16:07:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229437AbhGMQKo (ORCPT ); Tue, 13 Jul 2021 12:10:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B13B7C0613EF; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=sFwfQ9Uw9O52hL6lZwNWeKhI6xCaEO5BECLZyahskzE=; b=WIXyCvBWmlGG7bhrM1hmyXMGev rbXCcSGxJxF7nkCV1A4F9+YO3/3I2VVWrgqq7cgJFG3d2G9Tig4XT0XA9JGGJKrgNMB9S3I6Y89JL TIycZopmt7OdCFV1753Ufg39eu2Wa0dVJp+1hTxeQYlEEszOmTIa5LnEsK4Cb479JK7Q=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwm-0008TX-Rd; Tue, 13 Jul 2021 18:07:48 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 3/7] net: ethernet: mtk_eth_soc: implement flow offloading to WED devices Date: Tue, 13 Jul 2021 18:07:41 +0200 Message-Id: <20210713160745.59707-4-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org This allows hardware flow offloading from Ethernet to WLAN on MT7622 SoC Signed-off-by: Felix Fietkau --- drivers/net/ethernet/mediatek/mtk_ppe.c | 18 +++++ drivers/net/ethernet/mediatek/mtk_ppe.h | 14 ++-- .../net/ethernet/mediatek/mtk_ppe_offload.c | 67 ++++++++++++------- include/linux/netdevice.h | 7 ++ net/core/dev.c | 4 ++ 5 files changed, 78 insertions(+), 32 deletions(-) diff --git a/drivers/net/ethernet/mediatek/mtk_ppe.c b/drivers/net/ethernet/mediatek/mtk_ppe.c index 3ad10c793308..472bcd3269a7 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe.c @@ -329,6 +329,24 @@ int mtk_foe_entry_set_pppoe(struct mtk_foe_entry *entry, int sid) return 0; } +int mtk_foe_entry_set_wdma(struct mtk_foe_entry *entry, int wdma_idx, int txq, + int bss, int wcid) +{ + struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(entry); + u32 *ib2 = mtk_foe_entry_ib2(entry); + + *ib2 &= ~MTK_FOE_IB2_PORT_MG; + *ib2 |= MTK_FOE_IB2_WDMA_WINFO; + if (wdma_idx) + *ib2 |= MTK_FOE_IB2_WDMA_DEVIDX; + + l2->vlan2 = FIELD_PREP(MTK_FOE_VLAN2_WINFO_BSS, bss) | + FIELD_PREP(MTK_FOE_VLAN2_WINFO_WCID, wcid) | + FIELD_PREP(MTK_FOE_VLAN2_WINFO_RING, txq); + + return 0; +} + static inline bool mtk_foe_entry_usable(struct mtk_foe_entry *entry) { return !(entry->ib1 & MTK_FOE_IB1_STATIC) && diff --git a/drivers/net/ethernet/mediatek/mtk_ppe.h b/drivers/net/ethernet/mediatek/mtk_ppe.h index 242fb8f2ae65..df8ccaf48171 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe.h +++ b/drivers/net/ethernet/mediatek/mtk_ppe.h @@ -48,9 +48,9 @@ enum { #define MTK_FOE_IB2_DEST_PORT GENMASK(7, 5) #define MTK_FOE_IB2_MULTICAST BIT(8) -#define MTK_FOE_IB2_WHNAT_QID2 GENMASK(13, 12) -#define MTK_FOE_IB2_WHNAT_DEVIDX BIT(16) -#define MTK_FOE_IB2_WHNAT_NAT BIT(17) +#define MTK_FOE_IB2_WDMA_QID2 GENMASK(13, 12) +#define MTK_FOE_IB2_WDMA_DEVIDX BIT(16) +#define MTK_FOE_IB2_WDMA_WINFO BIT(17) #define MTK_FOE_IB2_PORT_MG GENMASK(17, 12) @@ -58,9 +58,9 @@ enum { #define MTK_FOE_IB2_DSCP GENMASK(31, 24) -#define MTK_FOE_VLAN2_WHNAT_BSS GEMMASK(5, 0) -#define MTK_FOE_VLAN2_WHNAT_WCID GENMASK(13, 6) -#define MTK_FOE_VLAN2_WHNAT_RING GENMASK(15, 14) +#define MTK_FOE_VLAN2_WINFO_BSS GENMASK(5, 0) +#define MTK_FOE_VLAN2_WINFO_WCID GENMASK(13, 6) +#define MTK_FOE_VLAN2_WINFO_RING GENMASK(15, 14) enum { MTK_FOE_STATE_INVALID, @@ -281,6 +281,8 @@ int mtk_foe_entry_set_ipv6_tuple(struct mtk_foe_entry *entry, int mtk_foe_entry_set_dsa(struct mtk_foe_entry *entry, int port); int mtk_foe_entry_set_vlan(struct mtk_foe_entry *entry, int vid); int mtk_foe_entry_set_pppoe(struct mtk_foe_entry *entry, int sid); +int mtk_foe_entry_set_wdma(struct mtk_foe_entry *entry, int wdma_idx, int txq, + int bss, int wcid); int mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_foe_entry *entry, u16 timestamp); int mtk_ppe_debugfs_init(struct mtk_ppe *ppe); diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c index b5f68f66d42a..00b1d06f60d1 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c @@ -10,6 +10,7 @@ #include #include #include "mtk_eth_soc.h" +#include "mtk_wed.h" struct mtk_flow_data { struct ethhdr eth; @@ -39,6 +40,7 @@ struct mtk_flow_entry { struct rhash_head node; unsigned long cookie; u16 hash; + s8 wed_index; }; static const struct rhashtable_params mtk_flow_ht_params = { @@ -127,35 +129,38 @@ mtk_flow_mangle_ipv4(const struct flow_action_entry *act, } static int -mtk_flow_get_dsa_port(struct net_device **dev) +mtk_flow_set_output_device(struct mtk_eth *eth, struct mtk_foe_entry *foe, + struct net_device *dev, const u8 *dest_mac, + int *wed_index) { -#if IS_ENABLED(CONFIG_NET_DSA) - struct dsa_port *dp; - - dp = dsa_port_from_netdev(*dev); - if (IS_ERR(dp)) - return -ENODEV; - - if (dp->cpu_dp->tag_ops->proto != DSA_TAG_PROTO_MTK) - return -ENODEV; + struct net_device_path_ctx ctx = { + .dev = dev, + .daddr = dest_mac, + }; + struct net_device_path path = {}; + int pse_port; - *dev = dp->cpu_dp->master; + if (!dev->netdev_ops->ndo_fill_forward_path || + dev->netdev_ops->ndo_fill_forward_path(&ctx, &path) < 0) + path.type = DEV_PATH_ETHERNET; - return dp->index; -#else - return -ENODEV; -#endif -} - -static int -mtk_flow_set_output_device(struct mtk_eth *eth, struct mtk_foe_entry *foe, - struct net_device *dev) -{ - int pse_port, dsa_port; + switch (path.type) { + case DEV_PATH_DSA: + if (path.dsa.proto != DSA_TAG_PROTO_MTK) + break; - dsa_port = mtk_flow_get_dsa_port(&dev); - if (dsa_port >= 0) - mtk_foe_entry_set_dsa(foe, dsa_port); + mtk_foe_entry_set_dsa(foe, path.dsa.port); + break; + case DEV_PATH_MTK_WDMA: + mtk_foe_entry_set_wdma(foe, path.mtk_wdma.wdma_idx, + path.mtk_wdma.queue, path.mtk_wdma.bss, + path.mtk_wdma.wcid); + mtk_foe_entry_set_pse_port(foe, 3); + *wed_index = path.mtk_wdma.wdma_idx; + return 0; + default: + break; + } if (dev == eth->netdev[0]) pse_port = 1; @@ -179,6 +184,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) struct net_device *odev = NULL; struct mtk_flow_entry *entry; int offload_type = 0; + int wed_index = -1; u16 addr_type = 0; u32 timestamp; u8 l4proto = 0; @@ -323,10 +329,14 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) if (data.pppoe.num == 1) mtk_foe_entry_set_pppoe(&foe, data.pppoe.sid); - err = mtk_flow_set_output_device(eth, &foe, odev); + err = mtk_flow_set_output_device(eth, &foe, odev, data.eth.h_dest, + &wed_index); if (err) return err; + if (wed_index >= 0 && (err = mtk_wed_flow_add(wed_index)) < 0) + return err; + entry = kzalloc(sizeof(*entry), GFP_KERNEL); if (!entry) return -ENOMEM; @@ -340,6 +350,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) } entry->hash = hash; + entry->wed_index = wed_index; err = rhashtable_insert_fast(ð->flow_table, &entry->node, mtk_flow_ht_params); if (err < 0) @@ -350,6 +361,8 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) mtk_foe_entry_clear(ð->ppe, hash); free: kfree(entry); + if (wed_index >= 0) + mtk_wed_flow_remove(wed_index); return err; } @@ -366,6 +379,8 @@ mtk_flow_offload_destroy(struct mtk_eth *eth, struct flow_cls_offload *f) mtk_foe_entry_clear(ð->ppe, entry->hash); rhashtable_remove_fast(ð->flow_table, &entry->node, mtk_flow_ht_params); + if (entry->wed_index >= 0) + mtk_wed_flow_remove(entry->wed_index); kfree(entry); return 0; diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index eaf5bb008aa9..67bba2826bc4 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -856,6 +856,7 @@ enum net_device_path_type { DEV_PATH_BRIDGE, DEV_PATH_PPPOE, DEV_PATH_DSA, + DEV_PATH_MTK_WDMA, }; struct net_device_path { @@ -881,6 +882,12 @@ struct net_device_path { int port; u16 proto; } dsa; + struct { + u8 wdma_idx; + u8 queue; + u8 bss; + u8 wcid; + } mtk_wdma; }; }; diff --git a/net/core/dev.c b/net/core/dev.c index c253c2aafe97..7ea6a1db0338 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -885,6 +885,10 @@ int dev_fill_forward_path(const struct net_device *dev, const u8 *daddr, if (WARN_ON_ONCE(last_dev == ctx.dev)) return -1; } + + if (!ctx.dev) + return ret; + path = dev_fwd_path(stack); if (!path) return -1; From patchwork Tue Jul 13 16:07:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 477274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25DBCC11F67 for ; Tue, 13 Jul 2021 16:07:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F1078611C0 for ; Tue, 13 Jul 2021 16:07:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229771AbhGMQKn (ORCPT ); Tue, 13 Jul 2021 12:10:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229437AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2DD0C0613E9; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=AdrJCYtnw/cIlb0o00U3MkZlWlJ5tSgHOHXdIcxONhs=; b=j/b1T3CdFPMK2xsbs5/1md+78s e/e5X2jWg0dXxZI5CgvlaEj8qElOQy7tFWP60zGncg4wtpQtmyO5cmvnaaZvC3Ar8h8fDXhuOzmjD hxBhHS8ahSD8NxBuDivB2LbTidLqP0RAJd8fc+qY1sbpOitEVTfnccBdCpFZik76PJhY=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwn-0008TX-5I; Tue, 13 Jul 2021 18:07:49 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 4/7] mt76: dma: add wrapper macro for accessing queue registers Date: Tue, 13 Jul 2021 18:07:42 +0200 Message-Id: <20210713160745.59707-5-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Preparation for adding indirection used for Wireless Ethernet Dispatch support Signed-off-by: Felix Fietkau --- drivers/net/wireless/mediatek/mt76/dma.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c index 5e1c1506a4c6..ccdc020f72e5 100644 --- a/drivers/net/wireless/mediatek/mt76/dma.c +++ b/drivers/net/wireless/mediatek/mt76/dma.c @@ -7,6 +7,10 @@ #include "mt76.h" #include "dma.h" +#define Q_READ(_dev, _q, _field) readl(&(_q)->regs->_field) +#define Q_WRITE(_dev, _q, _field, _val) writel(_val, &(_q)->regs->_field) + + static struct mt76_txwi_cache * mt76_alloc_txwi(struct mt76_dev *dev) { @@ -82,9 +86,9 @@ mt76_free_pending_txwi(struct mt76_dev *dev) static void mt76_dma_sync_idx(struct mt76_dev *dev, struct mt76_queue *q) { - writel(q->desc_dma, &q->regs->desc_base); - writel(q->ndesc, &q->regs->ring_size); - q->head = readl(&q->regs->dma_idx); + Q_WRITE(dev, q, desc_base, q->desc_dma); + Q_WRITE(dev, q, ring_size, q->ndesc); + q->head = Q_READ(dev, q, dma_idx); q->tail = q->head; } @@ -100,8 +104,8 @@ mt76_dma_queue_reset(struct mt76_dev *dev, struct mt76_queue *q) for (i = 0; i < q->ndesc; i++) q->desc[i].ctrl = cpu_to_le32(MT_DMA_CTL_DMA_DONE); - writel(0, &q->regs->cpu_idx); - writel(0, &q->regs->dma_idx); + Q_WRITE(dev, q, cpu_idx, 0); + Q_WRITE(dev, q, dma_idx, 0); mt76_dma_sync_idx(dev, q); } @@ -224,7 +228,7 @@ static void mt76_dma_kick_queue(struct mt76_dev *dev, struct mt76_queue *q) { wmb(); - writel(q->head, &q->regs->cpu_idx); + Q_WRITE(dev, q, cpu_idx, q->head); } static void @@ -240,7 +244,7 @@ mt76_dma_tx_cleanup(struct mt76_dev *dev, struct mt76_queue *q, bool flush) if (flush) last = -1; else - last = readl(&q->regs->dma_idx); + last = Q_READ(dev, q, dma_idx); while (q->queued > 0 && q->tail != last) { mt76_dma_tx_cleanup_idx(dev, q, q->tail, &entry); @@ -252,7 +256,7 @@ mt76_dma_tx_cleanup(struct mt76_dev *dev, struct mt76_queue *q, bool flush) } if (!flush && q->tail == last) - last = readl(&q->regs->dma_idx); + last = Q_READ(dev, q, dma_idx); } spin_unlock_bh(&q->cleanup_lock); From patchwork Tue Jul 13 16:07:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 477273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07442C11F6B for ; Tue, 13 Jul 2021 16:07:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC87061164 for ; Tue, 13 Jul 2021 16:07:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229953AbhGMQKp (ORCPT ); Tue, 13 Jul 2021 12:10:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B774EC0613F0; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=osryDm8MY41tftsoKlAz3j0I5w5bStb1c+qiIRfmEwk=; b=L+70IKRfJxdlDf80TyXdwJB3m0 YK1DvnqiPIvynpPkHxFJnFjwPxJkKA3pAkWaJL4M10AAKnPcTJ53MZX8EHApkWxQTay0vaOydG/7Z YcGaFFMhy2ZZrAyJVbO4CMswut3lgbL61NRL03+cIe3WwSVxqP2kijNMCo6OMfJh0H9Q=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwn-0008TX-F0; Tue, 13 Jul 2021 18:07:49 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 5/7] mt76: make number of tokens configurable dynamically Date: Tue, 13 Jul 2021 18:07:43 +0200 Message-Id: <20210713160745.59707-6-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Preparation for adding Wireless Ethernet Dispatch support Signed-off-by: Felix Fietkau --- drivers/net/wireless/mediatek/mt76/mac80211.c | 1 + drivers/net/wireless/mediatek/mt76/mt76.h | 6 +++--- drivers/net/wireless/mediatek/mt76/tx.c | 7 +++---- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c index d03aedc3286b..ba8b06acace3 100644 --- a/drivers/net/wireless/mediatek/mt76/mac80211.c +++ b/drivers/net/wireless/mediatek/mt76/mac80211.c @@ -459,6 +459,7 @@ mt76_alloc_device(struct device *pdev, unsigned int size, idr_init(&dev->token); INIT_LIST_HEAD(&dev->txwi_cache); + dev->token_size = dev->drv->token_size; for (i = 0; i < ARRAY_SIZE(dev->q_rx); i++) skb_queue_head_init(&dev->rx_skb[i]); diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h index 25c5ceef5257..840778ffe8ea 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76.h +++ b/drivers/net/wireless/mediatek/mt76/mt76.h @@ -680,7 +680,8 @@ struct mt76_dev { spinlock_t token_lock; struct idr token; - int token_count; + u16 token_count; + u16 token_size; wait_queue_head_t tx_wait; struct sk_buff_head status_list; @@ -1274,8 +1275,7 @@ mt76_token_get(struct mt76_dev *dev, struct mt76_txwi_cache **ptxwi) int token; spin_lock_bh(&dev->token_lock); - token = idr_alloc(&dev->token, *ptxwi, 0, dev->drv->token_size, - GFP_ATOMIC); + token = idr_alloc(&dev->token, *ptxwi, 0, dev->token_size, GFP_ATOMIC); spin_unlock_bh(&dev->token_lock); return token; diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c index f0f7a913eaab..9c6d26f4c795 100644 --- a/drivers/net/wireless/mediatek/mt76/tx.c +++ b/drivers/net/wireless/mediatek/mt76/tx.c @@ -691,12 +691,11 @@ int mt76_token_consume(struct mt76_dev *dev, struct mt76_txwi_cache **ptxwi) spin_lock_bh(&dev->token_lock); - token = idr_alloc(&dev->token, *ptxwi, 0, dev->drv->token_size, - GFP_ATOMIC); + token = idr_alloc(&dev->token, *ptxwi, 0, dev->token_size, GFP_ATOMIC); if (token >= 0) dev->token_count++; - if (dev->token_count >= dev->drv->token_size - MT76_TOKEN_FREE_THR) + if (dev->token_count >= dev->token_size - MT76_TOKEN_FREE_THR) __mt76_set_tx_blocked(dev, true); spin_unlock_bh(&dev->token_lock); @@ -716,7 +715,7 @@ mt76_token_release(struct mt76_dev *dev, int token, bool *wake) if (txwi) dev->token_count--; - if (dev->token_count < dev->drv->token_size - MT76_TOKEN_FREE_THR && + if (dev->token_count < dev->token_size - MT76_TOKEN_FREE_THR && dev->phy.q_tx[0]->blocked) *wake = true; From patchwork Tue Jul 13 16:07:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 475053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 998F5C07E96 for ; Tue, 13 Jul 2021 16:07:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7BFB161183 for ; Tue, 13 Jul 2021 16:07:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230156AbhGMQKs (ORCPT ); Tue, 13 Jul 2021 12:10:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229615AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4BABC061787; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=kALmQlVD1mXKtMl6SNLC4u0N8cLUAa7AgdTZMgUMrik=; b=HO7dvPiFO7OIlF4z8ZZnmKEnme xCwubAKsnJBXSBeu6kuwOxtRWa3LwtkKTocdOszdXp+zKXYvfSEQ+Qpd7cAI8BLCTVVUyT4DAsgDz fmsTF5GD8hi4LHqx1ZC4UyVL0lAb+nlO284OY0cONHa+624Y8fnm70+Fr60sJ2/vHKLw=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwn-0008TX-Ox; Tue, 13 Jul 2021 18:07:49 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 6/7] mt76: mt7915: remove irq parameter from mt7915_mmio_init Date: Tue, 13 Jul 2021 18:07:44 +0200 Message-Id: <20210713160745.59707-7-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org It is unused Signed-off-by: Felix Fietkau --- drivers/net/wireless/mediatek/mt76/mt7915/mmio.c | 2 +- drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h | 2 +- drivers/net/wireless/mediatek/mt76/mt7915/pci.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c index af712a936ef6..45b277618ef6 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c @@ -123,7 +123,7 @@ static u32 mt7915_rmw(struct mt76_dev *mdev, u32 offset, u32 mask, u32 val) return dev->bus_ops->rmw(mdev, addr, mask, val); } -int mt7915_mmio_init(struct mt76_dev *mdev, void __iomem *mem_base, int irq) +int mt7915_mmio_init(struct mt76_dev *mdev, void __iomem *mem_base) { struct mt76_bus_ops *bus_ops; struct mt7915_dev *dev; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h index 3f613fae6218..0652b711ae81 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h @@ -418,7 +418,7 @@ void mt7915_mac_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif, void mt7915_mac_work(struct work_struct *work); void mt7915_mac_reset_work(struct work_struct *work); void mt7915_mac_sta_rc_work(struct work_struct *work); -int mt7915_mmio_init(struct mt76_dev *mdev, void __iomem *mem_base, int irq); +int mt7915_mmio_init(struct mt76_dev *mdev, void __iomem *mem_base); int mt7915_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, enum mt76_txq_id qid, struct mt76_wcid *wcid, struct ieee80211_sta *sta, diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c index 340b364da5f0..5e93620195da 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c @@ -271,7 +271,7 @@ static int mt7915_pci_probe(struct pci_dev *pdev, if (ret < 0) goto free; - ret = mt7915_mmio_init(mdev, pcim_iomap_table(pdev)[0], pdev->irq); + ret = mt7915_mmio_init(mdev, pcim_iomap_table(pdev)[0]); if (ret) goto error; From patchwork Tue Jul 13 16:07:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Fietkau X-Patchwork-Id: 477272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2055C11F69 for ; Tue, 13 Jul 2021 16:07:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B22061183 for ; Tue, 13 Jul 2021 16:07:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230000AbhGMQKq (ORCPT ); Tue, 13 Jul 2021 12:10:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229500AbhGMQKm (ORCPT ); Tue, 13 Jul 2021 12:10:42 -0400 Received: from nbd.name (nbd.name [IPv6:2a01:4f8:221:3d45::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE422C061786; Tue, 13 Jul 2021 09:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nbd.name; s=20160729; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=wKmXz6DWd7BpdvLNDrNuZdtlhNtIRqIOTmMrLF8x1dA=; b=WhYhbZ6g5DR32kA3lB9GWaRmr+ DKWPL8hqVMTqbeXumn/TWAzdP4ztgwt/m37xjUhJ1P4nX1uTt/Jz1tavnOpH5VsMUXvdftSwKNDaK fuHEkL16NBTkB7Zqf664ZMt6TqHjVPOKi5DCLSJK+j1rdBWktwGCX3DVcPUUCaY4WVm4=; Received: from p54ae93f7.dip0.t-ipconnect.de ([84.174.147.247] helo=localhost.localdomain) by ds12 with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1m3Kwo-0008TX-27; Tue, 13 Jul 2021 18:07:50 +0200 From: Felix Fietkau To: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org, pablo@netfilter.org, ryder.lee@mediatek.com Subject: [RFC 7/7] mt76: mt7915: add Wireless Ethernet Dispatch support Date: Tue, 13 Jul 2021 18:07:45 +0200 Message-Id: <20210713160745.59707-8-nbd@nbd.name> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210713160745.59707-1-nbd@nbd.name> References: <20210713160745.59707-1-nbd@nbd.name> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org This is used to support hardware flow offloading from Ethernet to WLAN Signed-off-by: Felix Fietkau --- drivers/net/wireless/mediatek/mt76/dma.c | 97 +++++++++++++- drivers/net/wireless/mediatek/mt76/mac80211.c | 4 +- drivers/net/wireless/mediatek/mt76/mmio.c | 9 +- drivers/net/wireless/mediatek/mt76/mt76.h | 30 ++++- .../net/wireless/mediatek/mt76/mt7603/dma.c | 8 +- .../net/wireless/mediatek/mt76/mt7615/dma.c | 6 +- .../net/wireless/mediatek/mt76/mt76x02_mmio.c | 4 +- .../net/wireless/mediatek/mt76/mt7915/dma.c | 38 +++++- .../net/wireless/mediatek/mt76/mt7915/mac.c | 123 +++++++++++++++--- .../net/wireless/mediatek/mt76/mt7915/mac.h | 2 + .../net/wireless/mediatek/mt76/mt7915/main.c | 32 +++++ .../net/wireless/mediatek/mt76/mt7915/mcu.c | 4 + .../wireless/mediatek/mt76/mt7915/mt7915.h | 1 + .../net/wireless/mediatek/mt76/mt7915/pci.c | 96 +++++++++++--- .../net/wireless/mediatek/mt76/mt7915/regs.h | 19 ++- .../net/wireless/mediatek/mt76/mt7921/dma.c | 2 +- drivers/net/wireless/mediatek/mt76/tx.c | 14 +- 17 files changed, 422 insertions(+), 67 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c index ccdc020f72e5..10b924217a6c 100644 --- a/drivers/net/wireless/mediatek/mt76/dma.c +++ b/drivers/net/wireless/mediatek/mt76/dma.c @@ -7,9 +7,31 @@ #include "mt76.h" #include "dma.h" -#define Q_READ(_dev, _q, _field) readl(&(_q)->regs->_field) -#define Q_WRITE(_dev, _q, _field, _val) writel(_val, &(_q)->regs->_field) - +#define Q_READ(_dev, _q, _field) ({ \ + u32 _offset = offsetof(struct mt76_queue_regs, _field); \ + u32 _val; \ + if (IS_ENABLED(CONFIG_NET_MEDIATEK_SOC_WED) && \ + (_q)->flags & MT_QFLAG_WED) \ + _val = mtk_wed_device_reg_read(&(_dev)->mmio.wed, \ + ((_q)->wed_regs + \ + _offset)); \ + else \ + _val = readl(&(_q)->regs->_field); \ + _val; \ +}) + +#define Q_WRITE(_dev, _q, _field, _val) do { \ + u32 _offset = offsetof(struct mt76_queue_regs, _field); \ + if (IS_ENABLED(CONFIG_NET_MEDIATEK_SOC_WED) && \ + (_q)->flags & MT_QFLAG_WED) \ + mtk_wed_device_reg_write(&(_dev)->mmio.wed, \ + ((_q)->wed_regs + _offset), \ + _val); \ + else \ + writel(_val, &(_q)->regs->_field); \ +} while (0) + +static int mt76_dma_rx_fill(struct mt76_dev *dev, struct mt76_queue *q); static struct mt76_txwi_cache * mt76_alloc_txwi(struct mt76_dev *dev) @@ -109,12 +131,52 @@ mt76_dma_queue_reset(struct mt76_dev *dev, struct mt76_queue *q) mt76_dma_sync_idx(dev, q); } +static int +mt76_dma_wed_setup(struct mt76_dev *dev, struct mt76_queue *q) +{ + struct mtk_wed_device *wed = &dev->mmio.wed; + int ret, type, ring; + u8 flags = q->flags; + + if (!mtk_wed_device_active(wed)) + q->flags &= ~MT_QFLAG_WED; + + if (!(q->flags & MT_QFLAG_WED)) + return 0; + + type = FIELD_GET(MT_QFLAG_WED_TYPE, q->flags); + ring = FIELD_GET(MT_QFLAG_WED_RING, q->flags); + + switch (type) { + case MT76_WED_Q_TX: + ret = mtk_wed_device_tx_ring_setup(wed, ring, q->regs); + if (!ret) + q->wed_regs = wed->tx_ring[ring].reg_base; + break; + case MT76_WED_Q_TXFREE: + /* WED txfree queue needs ring to be initialized before setup */ + q->flags = 0; + mt76_dma_queue_reset(dev, q); + mt76_dma_rx_fill(dev, q); + q->flags = flags; + + ret = mtk_wed_device_txfree_ring_setup(wed, q->regs); + if (!ret) + q->wed_regs = wed->txfree_ring.reg_base; + break; + default: + ret = -EINVAL; + } + + return ret; +} + static int mt76_dma_alloc_queue(struct mt76_dev *dev, struct mt76_queue *q, int idx, int n_desc, int bufsize, u32 ring_base) { - int size; + int ret, size; spin_lock_init(&q->lock); spin_lock_init(&q->cleanup_lock); @@ -134,7 +196,12 @@ mt76_dma_alloc_queue(struct mt76_dev *dev, struct mt76_queue *q, if (!q->entry) return -ENOMEM; - mt76_dma_queue_reset(dev, q); + ret = mt76_dma_wed_setup(dev, q); + if (ret) + return ret; + + if (q->flags != MT_WED_Q_TXFREE) + mt76_dma_queue_reset(dev, q); return 0; } @@ -556,14 +623,29 @@ mt76_add_fragment(struct mt76_dev *dev, struct mt76_queue *q, void *data, static int mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget) { - int len, data_len, done = 0; + int len, data_len, done = 0, dma_idx; struct sk_buff *skb; unsigned char *data; + bool check_ddone = false; bool more; + if (IS_ENABLED(CONFIG_NET_MEDIATEK_SOC_WED) && + q->flags == MT_WED_Q_TXFREE) { + dma_idx = Q_READ(dev, q, dma_idx); + check_ddone = true; + } + while (done < budget) { u32 info; + if (check_ddone) { + if (q->tail == dma_idx) + dma_idx = Q_READ(dev, q, dma_idx); + + if (q->tail == dma_idx) + break; + } + data = mt76_dma_dequeue(dev, q, false, &len, &info, &more); if (!data) break; @@ -699,5 +781,8 @@ void mt76_dma_cleanup(struct mt76_dev *dev) } mt76_free_pending_txwi(dev); + + if (mtk_wed_device_active(&dev->mmio.wed)) + mtk_wed_device_detach(&dev->mmio.wed); } EXPORT_SYMBOL_GPL(mt76_dma_cleanup); diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c index ba8b06acace3..bc77bff047a3 100644 --- a/drivers/net/wireless/mediatek/mt76/mac80211.c +++ b/drivers/net/wireless/mediatek/mt76/mac80211.c @@ -1321,7 +1321,7 @@ EXPORT_SYMBOL_GPL(mt76_get_antenna); struct mt76_queue * mt76_init_queue(struct mt76_dev *dev, int qid, int idx, int n_desc, - int ring_base) + int ring_base, u32 flags) { struct mt76_queue *hwq; int err; @@ -1330,6 +1330,8 @@ mt76_init_queue(struct mt76_dev *dev, int qid, int idx, int n_desc, if (!hwq) return ERR_PTR(-ENOMEM); + hwq->flags = flags; + err = dev->queue_ops->alloc(dev, hwq, idx, n_desc, 0, ring_base); if (err < 0) return ERR_PTR(err); diff --git a/drivers/net/wireless/mediatek/mt76/mmio.c b/drivers/net/wireless/mediatek/mt76/mmio.c index 26353b6bce97..86e3d2ac4d0d 100644 --- a/drivers/net/wireless/mediatek/mt76/mmio.c +++ b/drivers/net/wireless/mediatek/mt76/mmio.c @@ -73,8 +73,13 @@ void mt76_set_irq_mask(struct mt76_dev *dev, u32 addr, spin_lock_irqsave(&dev->mmio.irq_lock, flags); dev->mmio.irqmask &= ~clear; dev->mmio.irqmask |= set; - if (addr) - mt76_mmio_wr(dev, addr, dev->mmio.irqmask); + if (addr) { + if (mtk_wed_device_active(&dev->mmio.wed)) + mtk_wed_device_irq_set_mask(&dev->mmio.wed, + dev->mmio.irqmask); + else + mt76_mmio_wr(dev, addr, dev->mmio.irqmask); + } spin_unlock_irqrestore(&dev->mmio.irq_lock, flags); } EXPORT_SYMBOL_GPL(mt76_set_irq_mask); diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h index 840778ffe8ea..f100c32cf33f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76.h +++ b/drivers/net/wireless/mediatek/mt76/mt76.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include "util.h" #include "testmode.h" @@ -26,6 +27,16 @@ #define MT76_TOKEN_FREE_THR 64 +#define MT_QFLAG_WED_RING GENMASK(1, 0) +#define MT_QFLAG_WED_TYPE GENMASK(3, 2) +#define MT_QFLAG_WED BIT(4) + +#define __MT_WED_Q(_type, _n) (MT_QFLAG_WED | \ + FIELD_PREP(MT_QFLAG_WED_TYPE, _type) | \ + FIELD_PREP(MT_QFLAG_WED_RING, _n)) +#define MT_WED_Q_TX(_n) __MT_WED_Q(MT76_WED_Q_TX, _n) +#define MT_WED_Q_TXFREE __MT_WED_Q(MT76_WED_Q_TXFREE, 0) + struct mt76_dev; struct mt76_phy; struct mt76_wcid; @@ -41,6 +52,11 @@ enum mt76_bus_type { MT76_BUS_SDIO, }; +enum mt76_wed_type { + MT76_WED_Q_TX, + MT76_WED_Q_TXFREE, +}; + struct mt76_bus_ops { u32 (*rr)(struct mt76_dev *dev, u32 offset); void (*wr)(struct mt76_dev *dev, u32 offset, u32 val); @@ -161,6 +177,9 @@ struct mt76_queue { u8 buf_offset; u8 hw_idx; u8 qid; + u8 flags; + + u32 wed_regs; dma_addr_t desc_dma; struct sk_buff *rx_head; @@ -511,6 +530,8 @@ struct mt76_mmio { void __iomem *regs; spinlock_t irq_lock; u32 irqmask; + + struct mtk_wed_device wed; }; struct mt76_rx_status { @@ -680,6 +701,7 @@ struct mt76_dev { spinlock_t token_lock; struct idr token; + u16 wed_token_count; u16 token_count; u16 token_size; @@ -881,13 +903,13 @@ int mt76_get_of_eeprom(struct mt76_dev *dev, void *data, int offset, int len); struct mt76_queue * mt76_init_queue(struct mt76_dev *dev, int qid, int idx, int n_desc, - int ring_base); + int ring_base, u32 flags); static inline int mt76_init_tx_queue(struct mt76_phy *phy, int qid, int idx, - int n_desc, int ring_base) + int n_desc, int ring_base, u32 flags) { struct mt76_queue *q; - q = mt76_init_queue(phy->dev, qid, idx, n_desc, ring_base); + q = mt76_init_queue(phy->dev, qid, idx, n_desc, ring_base, flags); if (IS_ERR(q)) return PTR_ERR(q); @@ -902,7 +924,7 @@ static inline int mt76_init_mcu_queue(struct mt76_dev *dev, int qid, int idx, { struct mt76_queue *q; - q = mt76_init_queue(dev, qid, idx, n_desc, ring_base); + q = mt76_init_queue(dev, qid, idx, n_desc, ring_base, 0); if (IS_ERR(q)) return PTR_ERR(q); diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/dma.c b/drivers/net/wireless/mediatek/mt76/mt7603/dma.c index 415ea17b9be6..89acebcfe61c 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7603/dma.c +++ b/drivers/net/wireless/mediatek/mt76/mt7603/dma.c @@ -173,13 +173,13 @@ int mt7603_dma_init(struct mt7603_dev *dev) for (i = 0; i < ARRAY_SIZE(wmm_queue_map); i++) { ret = mt76_init_tx_queue(&dev->mphy, i, wmm_queue_map[i], - MT7603_TX_RING_SIZE, MT_TX_RING_BASE); + MT7603_TX_RING_SIZE, MT_TX_RING_BASE, 0); if (ret) return ret; } ret = mt76_init_tx_queue(&dev->mphy, MT_TXQ_PSD, MT_TX_HW_QUEUE_MGMT, - MT7603_PSD_RING_SIZE, MT_TX_RING_BASE); + MT7603_PSD_RING_SIZE, MT_TX_RING_BASE, 0); if (ret) return ret; @@ -189,12 +189,12 @@ int mt7603_dma_init(struct mt7603_dev *dev) return ret; ret = mt76_init_tx_queue(&dev->mphy, MT_TXQ_BEACON, MT_TX_HW_QUEUE_BCN, - MT_MCU_RING_SIZE, MT_TX_RING_BASE); + MT_MCU_RING_SIZE, MT_TX_RING_BASE, 0); if (ret) return ret; ret = mt76_init_tx_queue(&dev->mphy, MT_TXQ_CAB, MT_TX_HW_QUEUE_BMC, - MT_MCU_RING_SIZE, MT_TX_RING_BASE); + MT_MCU_RING_SIZE, MT_TX_RING_BASE, 0); if (ret) return ret; diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c index 00aefea1bf61..3a79a2d4f288 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c @@ -26,14 +26,14 @@ mt7622_init_tx_queues_multi(struct mt7615_dev *dev) for (i = 0; i < ARRAY_SIZE(wmm_queue_map); i++) { ret = mt76_init_tx_queue(&dev->mphy, i, wmm_queue_map[i], MT7615_TX_RING_SIZE / 2, - MT_TX_RING_BASE); + MT_TX_RING_BASE, 0); if (ret) return ret; } ret = mt76_init_tx_queue(&dev->mphy, MT_TXQ_PSD, MT7622_TXQ_MGMT, MT7615_TX_MGMT_RING_SIZE, - MT_TX_RING_BASE); + MT_TX_RING_BASE, 0); if (ret) return ret; @@ -55,7 +55,7 @@ mt7615_init_tx_queues(struct mt7615_dev *dev) return mt7622_init_tx_queues_multi(dev); ret = mt76_init_tx_queue(&dev->mphy, 0, 0, MT7615_TX_RING_SIZE, - MT_TX_RING_BASE); + MT_TX_RING_BASE, 0); if (ret) return ret; diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c index b50084bbe83d..7de91f8db9dd 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c @@ -191,13 +191,13 @@ int mt76x02_dma_init(struct mt76x02_dev *dev) for (i = 0; i < IEEE80211_NUM_ACS; i++) { ret = mt76_init_tx_queue(&dev->mphy, i, mt76_ac_to_hwq(i), MT76x02_TX_RING_SIZE, - MT_TX_RING_BASE); + MT_TX_RING_BASE, 0); if (ret) return ret; } ret = mt76_init_tx_queue(&dev->mphy, MT_TXQ_PSD, MT_TX_HW_QUEUE_MGMT, - MT76x02_PSD_RING_SIZE, MT_TX_RING_BASE); + MT76x02_PSD_RING_SIZE, MT_TX_RING_BASE, 0); if (ret) return ret; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/dma.c b/drivers/net/wireless/mediatek/mt76/mt7915/dma.c index 9182568f95c7..ae271968b60c 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/dma.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/dma.c @@ -7,9 +7,16 @@ int mt7915_init_tx_queues(struct mt7915_phy *phy, int idx, int n_desc) { + u32 base = MT_TX_RING_BASE; int i, err; - err = mt76_init_tx_queue(phy->mt76, 0, idx, n_desc, MT_TX_RING_BASE); + if (mtk_wed_device_active(&phy->dev->mt76.mmio.wed)) { + base = MT_WED_TX_RING_BASE; + idx -= MT7915_TXQ_BAND0; + } + + err = mt76_init_tx_queue(phy->mt76, 0, idx, n_desc, base, + MT_WED_Q_TX(idx)); if (err < 0) return err; @@ -79,6 +86,9 @@ void mt7915_dma_prefetch(struct mt7915_dev *dev) int mt7915_dma_init(struct mt7915_dev *dev) { + u32 irq_mask = MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_MCU | + MT_INT_MCU_CMD; + u32 wa_rx_base = MT_RX_EVENT_RING_BASE; u32 hif1_ofs = 0; int ret; @@ -112,6 +122,15 @@ int mt7915_dma_init(struct mt7915_dev *dev) mt76_wr(dev, MT_WFDMA1_PRI_DLY_INT_CFG0 + hif1_ofs, 0); } + if (mtk_wed_device_active(&dev->mt76.mmio.wed)) { + mt76_set(dev, MT_WFDMA_HOST_CONFIG, MT_WFDMA_HOST_CONFIG_WED); + + mt76_wr(dev, MT_WFDMA_WED_RING_CONTROL, + FIELD_PREP(MT_WFDMA_WED_RING_CONTROL_TX0, 18) | + FIELD_PREP(MT_WFDMA_WED_RING_CONTROL_TX1, 19) | + FIELD_PREP(MT_WFDMA_WED_RING_CONTROL_RX1, 1)); + } + /* configure perfetch settings */ mt7915_dma_prefetch(dev); @@ -147,9 +166,13 @@ int mt7915_dma_init(struct mt7915_dev *dev) return ret; /* event from WA */ + if (mtk_wed_device_active(&dev->mt76.mmio.wed)) { + wa_rx_base = MT_WED_RX_RING_BASE; + dev->mt76.q_rx[MT_RXQ_MCU_WA].flags = MT_WED_Q_TXFREE; + } ret = mt76_queue_alloc(dev, &dev->mt76.q_rx[MT_RXQ_MCU_WA], MT7915_RXQ_MCU_WA, MT7915_RX_MCU_RING_SIZE, - MT_RX_BUF_SIZE, MT_RX_EVENT_RING_BASE); + MT_RX_BUF_SIZE, wa_rx_base); if (ret) return ret; @@ -227,9 +250,16 @@ int mt7915_dma_init(struct mt7915_dev *dev) MT_WFDMA_HOST_CONFIG_PDMA_BAND); } + if (mtk_wed_device_active(&dev->mt76.mmio.wed)) { + u32 wed_irq_mask = irq_mask; + + wed_irq_mask |= MT_INT_TX_DONE_BAND0 | MT_INT_TX_DONE_BAND1; + mt76_wr(dev, MT_INT_WED_MASK_CSR, wed_irq_mask); + mtk_wed_device_start(&dev->mt76.mmio.wed, wed_irq_mask); + } + /* enable interrupts for TX/RX rings */ - mt7915_irq_enable(dev, MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_MCU | - MT_INT_MCU_CMD); + mt7915_irq_enable(dev, irq_mask); return 0; } diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c index 2462704094b0..a216c68fe132 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c @@ -1065,6 +1065,29 @@ int mt7915_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, return 0; } +u32 mt7915_wed_init_buf(void *ptr, dma_addr_t phys, int token_id) +{ + struct mt7915_txp *txp = ptr + MT_TXD_SIZE; + __le32 *txwi = ptr; + u32 val; + + memset(ptr, 0, MT_TXD_SIZE + sizeof(*txp)); + + val = FIELD_PREP(MT_TXD0_TX_BYTES, MT_TXD_SIZE) | + FIELD_PREP(MT_TXD0_PKT_FMT, MT_TX_TYPE_CT); + txwi[0] = cpu_to_le32(val); + + val = MT_TXD1_LONG_FORMAT | + FIELD_PREP(MT_TXD1_HDR_FORMAT, MT_HDR_FORMAT_802_3); + txwi[1] = cpu_to_le32(val); + + txp->token = cpu_to_le16(token_id); + txp->nbuf = 1; + txp->buf[0] = cpu_to_le32(phys + MT_TXD_SIZE + sizeof(*txp)); + + return MT_TXD_SIZE + sizeof(*txp); +} + static void mt7915_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi) { @@ -1107,6 +1130,8 @@ mt7915_txwi_free(struct mt7915_dev *dev, struct mt76_txwi_cache *t, struct ieee80211_sta *sta, struct list_head *free_list) { struct mt76_dev *mdev = &dev->mt76; + struct mt7915_sta *msta; + struct mt7915_phy *phy; struct mt76_wcid *wcid; __le32 *txwi; u16 wcid_idx; @@ -1119,13 +1144,27 @@ mt7915_txwi_free(struct mt7915_dev *dev, struct mt76_txwi_cache *t, if (sta) { wcid = (struct mt76_wcid *)sta->drv_priv; wcid_idx = wcid->idx; - - if (likely(t->skb->protocol != cpu_to_be16(ETH_P_PAE))) - mt7915_tx_check_aggr(sta, txwi); } else { wcid_idx = FIELD_GET(MT_TXD1_WLAN_IDX, le32_to_cpu(txwi[1])); + wcid = rcu_dereference(dev->mt76.wcid[wcid_idx]); + + if (wcid && wcid->sta) { + msta = container_of(wcid, struct mt7915_sta, wcid); + sta = container_of((void *)msta, struct ieee80211_sta, + drv_priv); + phy = msta->vif->phy; + spin_lock_bh(&dev->sta_poll_lock); + if (list_empty(&msta->stats_list)) + list_add_tail(&msta->stats_list, &phy->stats_list); + if (list_empty(&msta->poll_list)) + list_add_tail(&msta->poll_list, &dev->sta_poll_list); + spin_unlock_bh(&dev->sta_poll_lock); + } } + if (sta && likely(t->skb->protocol != cpu_to_be16(ETH_P_PAE))) + mt7915_tx_check_aggr(sta, txwi); + __mt76_tx_complete_skb(mdev, wcid_idx, t->skb, free_list); out: @@ -1134,17 +1173,10 @@ mt7915_txwi_free(struct mt7915_dev *dev, struct mt76_txwi_cache *t, } static void -mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb) +mt7915_mac_tx_free_prepare(struct mt7915_dev *dev) { - struct mt7915_tx_free *free = (struct mt7915_tx_free *)skb->data; struct mt76_dev *mdev = &dev->mt76; struct mt76_phy *mphy_ext = mdev->phy2; - struct mt76_txwi_cache *txwi; - struct ieee80211_sta *sta = NULL; - LIST_HEAD(free_list); - struct sk_buff *tmp; - u8 i, count; - bool wake = false; /* clean DMA queues and unmap buffers first */ mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[MT_TXQ_PSD], false); @@ -1153,6 +1185,41 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb) mt76_queue_tx_cleanup(dev, mphy_ext->q_tx[MT_TXQ_PSD], false); mt76_queue_tx_cleanup(dev, mphy_ext->q_tx[MT_TXQ_BE], false); } +} + +static void +mt7915_mac_tx_free_done(struct mt7915_dev *dev, struct sk_buff *skb, + struct list_head *free_list, bool wake) +{ + struct sk_buff *tmp; + + mt7915_mac_sta_poll(dev); + + if (wake) + mt76_set_tx_blocked(&dev->mt76, false); + + mt76_worker_schedule(&dev->mt76.tx_worker); + + napi_consume_skb(skb, 1); + + list_for_each_entry_safe(skb, tmp, free_list, list) { + skb_list_del_init(skb); + napi_consume_skb(skb, 1); + } +} + +static void +mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb) +{ + struct mt7915_tx_free *free = (struct mt7915_tx_free *)skb->data; + struct mt76_dev *mdev = &dev->mt76; + struct mt76_txwi_cache *txwi; + struct ieee80211_sta *sta = NULL; + LIST_HEAD(free_list); + u8 i, count; + bool wake = false; + + mt7915_mac_tx_free_prepare(dev); /* * TODO: MT_TX_FREE_LATENCY is msdu time from the TXD is queued into PLE, @@ -1202,19 +1269,34 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, struct sk_buff *skb) mt7915_txwi_free(dev, txwi, sta, &free_list); } - mt7915_mac_sta_poll(dev); + mt7915_mac_tx_free_done(dev, skb, &free_list, wake); +} - if (wake) - mt76_set_tx_blocked(&dev->mt76, false); +static void +mt7915_mac_tx_free_v0(struct mt7915_dev *dev, struct sk_buff *skb) +{ + struct mt7915_tx_free *free = (struct mt7915_tx_free *)skb->data; + struct mt76_dev *mdev = &dev->mt76; + __le16 *info = (__le16 *)free->info; + LIST_HEAD(free_list); + bool wake = false; + u8 i, count; - mt76_worker_schedule(&dev->mt76.tx_worker); + mt7915_mac_tx_free_prepare(dev); - napi_consume_skb(skb, 1); + count = FIELD_GET(MT_TX_FREE_MSDU_CNT_V0, le16_to_cpu(free->ctrl)); + for (i = 0; i < count; i++) { + struct mt76_txwi_cache *txwi; + u16 msdu = le16_to_cpu(info[i]); - list_for_each_entry_safe(skb, tmp, &free_list, list) { - skb_list_del_init(skb); - napi_consume_skb(skb, 1); + txwi = mt76_token_release(mdev, msdu, &wake); + if (!txwi) + continue; + + mt7915_txwi_free(dev, txwi, NULL, &free_list); } + + mt7915_mac_tx_free_done(dev, skb, &free_list, wake); } static bool @@ -1308,6 +1390,9 @@ void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, case PKT_TYPE_TXRX_NOTIFY: mt7915_mac_tx_free(dev, skb); break; + case PKT_TYPE_TXRX_NOTIFY_V0: + mt7915_mac_tx_free_v0(dev, skb); + break; case PKT_TYPE_RX_EVENT: mt7915_mcu_rx_event(dev, skb); break; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.h b/drivers/net/wireless/mediatek/mt76/mt7915/mac.h index eb1885f4bd8e..83c457eb5cc4 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.h @@ -23,6 +23,7 @@ enum rx_pkt_type { PKT_TYPE_RETRIEVE, PKT_TYPE_TXRX_NOTIFY, PKT_TYPE_RX_EVENT, + PKT_TYPE_TXRX_NOTIFY_V0 = 0x18, }; /* RXD DW1 */ @@ -295,6 +296,7 @@ struct mt7915_tx_free { } __packed __aligned(4); #define MT_TX_FREE_MSDU_CNT GENMASK(9, 0) +#define MT_TX_FREE_MSDU_CNT_V0 GENMASK(6, 0) #define MT_TX_FREE_WLAN_ID GENMASK(23, 14) #define MT_TX_FREE_LATENCY GENMASK(12, 0) /* 0: success, others: dropped */ diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c index c25f8da590dd..7d29e23e823a 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c @@ -1032,6 +1032,37 @@ static void mt7915_sta_set_decap_offload(struct ieee80211_hw *hw, mt7915_mcu_sta_update_hdr_trans(dev, vif, sta); } +static int +mt7915_net_fill_forward_path(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_sta *sta, + struct net_device_path_ctx *ctx, + struct net_device_path *path) +{ + struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv; + struct mt7915_sta *msta = (struct mt7915_sta *)sta->drv_priv; + struct mt7915_dev *dev = mt7915_hw_dev(hw); + struct mt7915_phy *phy = mt7915_hw_phy(hw); + struct mtk_wed_device *wed = &dev->mt76.mmio.wed; + + if (!mtk_wed_device_active(wed)) + return -ENODEV; + + if (msta->wcid.idx > 0xff) + return -EIO; + + path->type = DEV_PATH_MTK_WDMA; + path->dev = ctx->dev; + path->mtk_wdma.wdma_idx = wed->wdma_idx; + path->mtk_wdma.bss = mvif->omac_idx; + path->mtk_wdma.wcid = msta->wcid.idx; + path->mtk_wdma.queue = phy != &dev->phy; + + ctx->dev = NULL; + + return 0; +} + const struct ieee80211_ops mt7915_ops = { .tx = mt7915_tx, .start = mt7915_start, @@ -1072,4 +1103,5 @@ const struct ieee80211_ops mt7915_ops = { #ifdef CONFIG_MAC80211_DEBUGFS .sta_add_debugfs = mt7915_sta_add_debugfs, #endif + .net_fill_forward_path = mt7915_net_fill_forward_path, }; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c index 863aa18b3024..444db639e9a0 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c @@ -3011,6 +3011,10 @@ int mt7915_mcu_init(struct mt7915_dev *dev) return ret; set_bit(MT76_STATE_MCU_RUNNING, &dev->mphy.state); + + if (mtk_wed_device_active(&dev->mt76.mmio.wed)) + mt7915_mcu_wa_cmd(dev, MCU_WA_PARAM_CMD(CAPABILITY), 0, 0, 0); + mt7915_mcu_fw_log_2_host(dev, 0); mt7915_mcu_set_mwds(dev, 1); mt7915_mcu_wa_cmd(dev, MCU_WA_PARAM_CMD(SET), MCU_WA_PARAM_RED, 0, 0); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h index 0652b711ae81..2dc5a4154155 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h @@ -294,6 +294,7 @@ extern const struct ieee80211_ops mt7915_ops; extern const struct mt76_testmode_ops mt7915_testmode_ops; u32 mt7915_reg_map(struct mt7915_dev *dev, u32 addr); +u32 mt7915_wed_init_buf(void *ptr, dma_addr_t phys, int token_id); int mt7915_register_device(struct mt7915_dev *dev); void mt7915_unregister_device(struct mt7915_dev *dev); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c index 5e93620195da..164ccab556f9 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c @@ -97,15 +97,21 @@ mt7915_rx_poll_complete(struct mt76_dev *mdev, enum mt76_rxq_id q) static void mt7915_irq_tasklet(struct tasklet_struct *t) { struct mt7915_dev *dev = from_tasklet(dev, t, irq_tasklet); + struct mtk_wed_device *wed = &dev->mt76.mmio.wed; u32 intr, intr1, mask; - mt76_wr(dev, MT_INT_MASK_CSR, 0); - if (dev->hif2) - mt76_wr(dev, MT_INT1_MASK_CSR, 0); - - intr = mt76_rr(dev, MT_INT_SOURCE_CSR); - intr &= dev->mt76.mmio.irqmask; - mt76_wr(dev, MT_INT_SOURCE_CSR, intr); + if (mtk_wed_device_active(wed)) { + mtk_wed_device_irq_set_mask(wed, 0); + intr = mtk_wed_device_irq_get(wed, dev->mt76.mmio.irqmask); + } else { + mt76_wr(dev, MT_INT_MASK_CSR, 0); + if (dev->hif2) + mt76_wr(dev, MT_INT1_MASK_CSR, 0); + + intr = mt76_rr(dev, MT_INT_SOURCE_CSR); + intr &= dev->mt76.mmio.irqmask; + mt76_wr(dev, MT_INT_SOURCE_CSR, intr); + } if (dev->hif2) { intr1 = mt76_rr(dev, MT_INT1_SOURCE_CSR); @@ -156,10 +162,15 @@ static void mt7915_irq_tasklet(struct tasklet_struct *t) static irqreturn_t mt7915_irq_handler(int irq, void *dev_instance) { struct mt7915_dev *dev = dev_instance; - - mt76_wr(dev, MT_INT_MASK_CSR, 0); - if (dev->hif2) - mt76_wr(dev, MT_INT1_MASK_CSR, 0); + struct mtk_wed_device *wed = &dev->mt76.mmio.wed; + + if (mtk_wed_device_active(wed)) { + mtk_wed_device_irq_set_mask(wed, 0); + } else { + mt76_wr(dev, MT_INT_MASK_CSR, 0); + if (dev->hif2) + mt76_wr(dev, MT_INT1_MASK_CSR, 0); + } if (!test_bit(MT76_STATE_INITIALIZED, &dev->mphy.state)) return IRQ_NONE; @@ -216,6 +227,36 @@ static int mt7915_pci_hif2_probe(struct pci_dev *pdev) return 0; } +static int mt7915_wed_offload_enable(struct mtk_wed_device *wed) +{ + struct mt7915_dev *dev; + int ret; + + dev = container_of(wed, struct mt7915_dev, mt76.mmio.wed); + + spin_lock_bh(&dev->mt76.token_lock); + dev->mt76.token_size = wed->wlan.token_start; + spin_unlock_bh(&dev->mt76.token_lock); + + ret = wait_event_timeout(dev->mt76.tx_wait, + !dev->mt76.wed_token_count, HZ); + if (!ret) + return -EAGAIN; + + return 0; +} + +static void mt7915_wed_offload_disable(struct mtk_wed_device *wed) +{ + struct mt7915_dev *dev; + + dev = container_of(wed, struct mt7915_dev, mt76.mmio.wed); + + spin_lock_bh(&dev->mt76.token_lock); + dev->mt76.token_size = MT7915_TOKEN_SIZE; + spin_unlock_bh(&dev->mt76.token_lock); +} + static int mt7915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { @@ -237,8 +278,10 @@ static int mt7915_pci_probe(struct pci_dev *pdev, .sta_remove = mt7915_mac_sta_remove, .update_survey = mt7915_update_channel, }; + struct mtk_wed_device *wed; struct mt7915_dev *dev; struct mt76_dev *mdev; + int irq; int ret; ret = pcim_enable_device(pdev); @@ -267,10 +310,6 @@ static int mt7915_pci_probe(struct pci_dev *pdev, dev = container_of(mdev, struct mt7915_dev, mt76); - ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES); - if (ret < 0) - goto free; - ret = mt7915_mmio_init(mdev, pcim_iomap_table(pdev)[0]); if (ret) goto error; @@ -279,10 +318,30 @@ static int mt7915_pci_probe(struct pci_dev *pdev, mt76_wr(dev, MT_INT_MASK_CSR, 0); + wed = &mdev->mmio.wed; + wed->wlan.pci_dev = pdev; + wed->wlan.wpdma_phys = pci_resource_start(pdev, 0) + + MT_WFDMA_EXT_CSR_BASE; + wed->wlan.nbuf = 4096; + wed->wlan.token_start = MT7915_TOKEN_SIZE - wed->wlan.nbuf - 1; + wed->wlan.init_buf = mt7915_wed_init_buf; + wed->wlan.offload_enable = mt7915_wed_offload_enable; + wed->wlan.offload_disable = mt7915_wed_offload_disable; + + if (mtk_wed_device_attach(wed) == 0) { + irq = wed->irq; + } else { + ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES); + if (ret < 0) + goto free; + + irq = pdev->irq; + } + /* master switch of PCIe tnterrupt enable */ mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff); - ret = devm_request_irq(mdev->dev, pdev->irq, mt7915_irq_handler, + ret = devm_request_irq(mdev->dev, irq, mt7915_irq_handler, IRQF_SHARED, KBUILD_MODNAME, dev); if (ret) goto error; @@ -297,7 +356,10 @@ static int mt7915_pci_probe(struct pci_dev *pdev, free_irq: devm_free_irq(mdev->dev, pdev->irq, dev); error: - pci_free_irq_vectors(pdev); + if (mtk_wed_device_active(wed)) + mtk_wed_device_detach(wed); + else + pci_free_irq_vectors(pdev); free: mt76_free_device(&dev->mt76); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/regs.h b/drivers/net/wireless/mediatek/mt76/mt7915/regs.h index a213b5cb82f8..633a63bef433 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/regs.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/regs.h @@ -334,7 +334,9 @@ /* WFDMA CSR */ #define MT_WFDMA_EXT_CSR_BASE 0xd7000 +#define MT_WFDMA_EXT_CSR_PHYS_BASE 0x18027000 #define MT_WFDMA_EXT_CSR(ofs) (MT_WFDMA_EXT_CSR_BASE + (ofs)) +#define MT_WFDMA_EXT_CSR_PHYS(ofs) (MT_WFDMA_EXT_CSR_PHYS_BASE + (ofs)) #define MT_INT_SOURCE_CSR MT_WFDMA_EXT_CSR(0x10) #define MT_INT_MASK_CSR MT_WFDMA_EXT_CSR(0x14) @@ -359,19 +361,30 @@ MT_INT_TX_DONE_MCU_WM | \ MT_INT_TX_DONE_FWDL) -#define MT_WFDMA_HOST_CONFIG MT_WFDMA_EXT_CSR(0x30) +#define MT_WFDMA_HOST_CONFIG MT_WFDMA_EXT_CSR_PHYS(0x30) #define MT_WFDMA_HOST_CONFIG_PDMA_BAND BIT(0) +#define MT_WFDMA_HOST_CONFIG_WED BIT(1) -#define MT_WFDMA_EXT_CSR_HIF_MISC MT_WFDMA_EXT_CSR(0x44) +#define MT_WFDMA_WED_RING_CONTROL MT_WFDMA_EXT_CSR_PHYS(0x34) +#define MT_WFDMA_WED_RING_CONTROL_TX0 GENMASK(4, 0) +#define MT_WFDMA_WED_RING_CONTROL_TX1 GENMASK(12, 8) +#define MT_WFDMA_WED_RING_CONTROL_RX1 GENMASK(20, 16) + +#define MT_WFDMA_EXT_CSR_HIF_MISC MT_WFDMA_EXT_CSR_PHYS(0x44) #define MT_WFDMA_EXT_CSR_HIF_MISC_BUSY BIT(0) #define MT_INT1_SOURCE_CSR MT_WFDMA_EXT_CSR(0x88) #define MT_INT1_MASK_CSR MT_WFDMA_EXT_CSR(0x8c) -#define MT_PCIE_RECOG_ID MT_WFDMA_EXT_CSR(0x90) +#define MT_PCIE_RECOG_ID MT_WFDMA_EXT_CSR_PHYS(0x90) #define MT_PCIE_RECOG_ID_MASK GENMASK(30, 0) #define MT_PCIE_RECOG_ID_SEM BIT(31) +#define MT_INT_WED_MASK_CSR MT_WFDMA_EXT_CSR(0x204) + +#define MT_WED_TX_RING_BASE MT_WFDMA_EXT_CSR(0x300) +#define MT_WED_RX_RING_BASE MT_WFDMA_EXT_CSR(0x400) + /* WFDMA0 PCIE1 */ #define MT_WFDMA0_PCIE1_BASE 0xd8000 #define MT_WFDMA0_PCIE1(ofs) (MT_WFDMA0_PCIE1_BASE + (ofs)) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/dma.c b/drivers/net/wireless/mediatek/mt76/mt7921/dma.c index 7d7d43a5422f..b99be1cbb866 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/dma.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/dma.c @@ -9,7 +9,7 @@ int mt7921_init_tx_queues(struct mt7921_phy *phy, int idx, int n_desc) { int i, err; - err = mt76_init_tx_queue(phy->mt76, 0, idx, n_desc, MT_TX_RING_BASE); + err = mt76_init_tx_queue(phy->mt76, 0, idx, n_desc, MT_TX_RING_BASE, 0); if (err < 0) return err; diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c index 9c6d26f4c795..86eb22711956 100644 --- a/drivers/net/wireless/mediatek/mt76/tx.c +++ b/drivers/net/wireless/mediatek/mt76/tx.c @@ -687,6 +687,7 @@ EXPORT_SYMBOL_GPL(__mt76_set_tx_blocked); int mt76_token_consume(struct mt76_dev *dev, struct mt76_txwi_cache **ptxwi) { + struct mtk_wed_device *wed = &dev->mmio.wed; int token; spin_lock_bh(&dev->token_lock); @@ -695,6 +696,10 @@ int mt76_token_consume(struct mt76_dev *dev, struct mt76_txwi_cache **ptxwi) if (token >= 0) dev->token_count++; + if (mtk_wed_device_active(wed) && + token >= wed->wlan.token_start) + dev->wed_token_count++; + if (dev->token_count >= dev->token_size - MT76_TOKEN_FREE_THR) __mt76_set_tx_blocked(dev, true); @@ -707,14 +712,21 @@ EXPORT_SYMBOL_GPL(mt76_token_consume); struct mt76_txwi_cache * mt76_token_release(struct mt76_dev *dev, int token, bool *wake) { + struct mtk_wed_device *wed = &dev->mmio.wed; struct mt76_txwi_cache *txwi; spin_lock_bh(&dev->token_lock); txwi = idr_remove(&dev->token, token); - if (txwi) + if (txwi) { dev->token_count--; + if (mtk_wed_device_active(wed) && + token >= wed->wlan.token_start && + --dev->wed_token_count == 0) + wake_up(&dev->tx_wait); + } + if (dev->token_count < dev->token_size - MT76_TOKEN_FREE_THR && dev->phy.q_tx[0]->blocked) *wake = true;