From patchwork Mon Nov 4 17:16:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Issam Hamdi X-Patchwork-Id: 840750 Received: from mail.simonwunderlich.de (mail.simonwunderlich.de [23.88.38.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D06A417583 for ; Mon, 4 Nov 2024 17:25:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=23.88.38.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730741121; cv=none; b=nV74sKbAijGLGJxFiqjwSt1QI19lyQIqcMwgjhu3m/gjGdzc8P6BXzlyPS3AKNFNZ99xqXaRbnUXiRVrz03UkOB4YFwBVxwnfXYWnwfiIIkETP4hhgzqGcKjQNEIuS1378p5cR0zZQhKJSadl/lRUnwXwOxcb2MG9DOgXMqCaEI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730741121; c=relaxed/simple; bh=frT+i/m43tebJ7LPnRV9z6cD+yVa4szXmoyDX/DM734=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=EBI7NAbJcaNLI5qEGKUu6oHe/+d92g8F1kXbN0kF056YcWjdhwzCqgppA+xituFkNT6L6Tr/SsUwRnqegg0+f+d+8cuTAxrZgGrc8PXyoWF3jj3PhnZ/50wXY5B9NCOkAzeC8kET+B4WcGs1DZwQeCYEwGD6hgrBmUQfSGhwHEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=simonwunderlich.de; spf=pass smtp.mailfrom=simonwunderlich.de; arc=none smtp.client-ip=23.88.38.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=simonwunderlich.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=simonwunderlich.de Received: from localhost (p200300c59740c190000000000000032b.dip0.t-ipconnect.de [IPv6:2003:c5:9740:c190::32b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.simonwunderlich.de (Postfix) with ESMTPSA id BE8D8FA132; Mon, 4 Nov 2024 18:16:44 +0100 (CET) From: Issam Hamdi To: johannes@sipsolutions.net Cc: linux-wireless@vger.kernel.org, mathias.kretschmer@fit.fraunhofer.de, Simon Wunderlich , Simon Wunderlich , Sven Eckelmann , Issam Hamdi Subject: [PATCH 1/2] wifi: ath9k: work around AR_CFG 0xdeadbeef chip hang Date: Mon, 4 Nov 2024 18:16:26 +0100 Message-Id: <20241104171627.3789199-1-ih@simonwunderlich.de> X-Mailer: git-send-email 2.39.2 Precedence: bulk X-Mailing-List: linux-wireless@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Simon Wunderlich QCA 802.11n chips (especially AR9330/AR9340) sometimes end up in a state in which a read of AR_CFG always returns 0xdeadbeef. This should not happen when when the power_mode of the device is ATH9K_PM_AWAKE. This problem is not yet detected by any other workaround in ath9k. No way is known to reproduce the problem easily. This patch originally developed by "Simon Wunderlich " and "Sven Eckelmann " Co-developed-by: Simon Wunderlich Co-developed-by: Sven Eckelmann Signed-off-by: Issam Hamdi --- drivers/net/wireless/ath/ath9k/ath9k.h | 3 +++ drivers/net/wireless/ath/ath9k/debug.c | 1 + drivers/net/wireless/ath/ath9k/debug.h | 1 + drivers/net/wireless/ath/ath9k/init.c | 1 + drivers/net/wireless/ath/ath9k/link.c | 31 ++++++++++++++++++++++++++ drivers/net/wireless/ath/ath9k/main.c | 4 ++++ 6 files changed, 41 insertions(+) base-commit: 2b94751626a6d49bbe42a19cc1503bd391016bd5 diff --git a/drivers/net/wireless/ath/ath9k/ath9k.h b/drivers/net/wireless/ath/ath9k/ath9k.h index 29ca65a732a6..c1ce081445a9 100644 --- a/drivers/net/wireless/ath/ath9k/ath9k.h +++ b/drivers/net/wireless/ath/ath9k/ath9k.h @@ -739,11 +739,13 @@ void ath9k_csa_update(struct ath_softc *sc); #define ATH_ANI_MAX_SKIP_COUNT 10 #define ATH_PAPRD_TIMEOUT 100 /* msecs */ #define ATH_PLL_WORK_INTERVAL 100 +#define ATH_HANG_WORK_INTERVAL 4000 void ath_hw_check_work(struct work_struct *work); void ath_reset_work(struct work_struct *work); bool ath_hw_check(struct ath_softc *sc); void ath_hw_pll_work(struct work_struct *work); +void ath_hw_hang_work(struct work_struct *work); void ath_paprd_calibrate(struct work_struct *work); void ath_ani_calibrate(struct timer_list *t); void ath_start_ani(struct ath_softc *sc); @@ -1044,6 +1046,7 @@ struct ath_softc { #endif struct delayed_work hw_check_work; struct delayed_work hw_pll_work; + struct delayed_work hw_hang_work; struct timer_list sleep_timer; #ifdef CONFIG_ATH9K_BTCOEX_SUPPORT diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c index eff894958a73..6b2469a01f17 100644 --- a/drivers/net/wireless/ath/ath9k/debug.c +++ b/drivers/net/wireless/ath/ath9k/debug.c @@ -750,6 +750,7 @@ static int read_file_reset(struct seq_file *file, void *data) [RESET_TYPE_CALIBRATION] = "Calibration error", [RESET_TX_DMA_ERROR] = "Tx DMA stop error", [RESET_RX_DMA_ERROR] = "Rx DMA stop error", + [RESET_TYPE_DEADBEEF] = "deadbeef hang", }; int i; diff --git a/drivers/net/wireless/ath/ath9k/debug.h b/drivers/net/wireless/ath/ath9k/debug.h index 389459c04d14..6ebb6053a8c1 100644 --- a/drivers/net/wireless/ath/ath9k/debug.h +++ b/drivers/net/wireless/ath/ath9k/debug.h @@ -53,6 +53,7 @@ enum ath_reset_type { RESET_TYPE_CALIBRATION, RESET_TX_DMA_ERROR, RESET_RX_DMA_ERROR, + RESET_TYPE_DEADBEEF, __RESET_TYPE_MAX }; diff --git a/drivers/net/wireless/ath/ath9k/init.c b/drivers/net/wireless/ath/ath9k/init.c index f9e77c4624d9..833474d7281f 100644 --- a/drivers/net/wireless/ath/ath9k/init.c +++ b/drivers/net/wireless/ath/ath9k/init.c @@ -740,6 +740,7 @@ static int ath9k_init_softc(u16 devid, struct ath_softc *sc, INIT_WORK(&sc->paprd_work, ath_paprd_calibrate); INIT_DELAYED_WORK(&sc->hw_pll_work, ath_hw_pll_work); INIT_DELAYED_WORK(&sc->hw_check_work, ath_hw_check_work); + INIT_DELAYED_WORK(&sc->hw_hang_work, ath_hw_hang_work); ath9k_init_channel_context(sc); diff --git a/drivers/net/wireless/ath/ath9k/link.c b/drivers/net/wireless/ath/ath9k/link.c index d1e5767aab3c..37438960c278 100644 --- a/drivers/net/wireless/ath/ath9k/link.c +++ b/drivers/net/wireless/ath/ath9k/link.c @@ -142,6 +142,37 @@ void ath_hw_pll_work(struct work_struct *work) msecs_to_jiffies(ATH_PLL_WORK_INTERVAL)); } +static bool ath_hw_hang_deadbeef(struct ath_softc *sc) +{ + struct ath_common *common = ath9k_hw_common(sc->sc_ah); + u32 reg; + + /* check for stucked MAC */ + ath9k_ps_wakeup(sc); + reg = REG_READ(sc->sc_ah, AR_CFG); + ath9k_ps_restore(sc); + + if (reg != 0xdeadbeef) + return false; + + ath_dbg(common, RESET, + "0xdeadbeef hang is detected. Schedule chip reset\n"); + ath9k_queue_reset(sc, RESET_TYPE_DEADBEEF); + + return true; +} + +void ath_hw_hang_work(struct work_struct *work) +{ + struct ath_softc *sc = container_of(work, struct ath_softc, + hw_hang_work.work); + + ath_hw_hang_deadbeef(sc); + + ieee80211_queue_delayed_work(sc->hw, &sc->hw_hang_work, + msecs_to_jiffies(ATH_HANG_WORK_INTERVAL)); +} + /* * PA Pre-distortion. */ diff --git a/drivers/net/wireless/ath/ath9k/main.c b/drivers/net/wireless/ath/ath9k/main.c index b92c89dad8de..024028ce8417 100644 --- a/drivers/net/wireless/ath/ath9k/main.c +++ b/drivers/net/wireless/ath/ath9k/main.c @@ -186,6 +186,7 @@ static void __ath_cancel_work(struct ath_softc *sc) cancel_work_sync(&sc->paprd_work); cancel_delayed_work_sync(&sc->hw_check_work); cancel_delayed_work_sync(&sc->hw_pll_work); + cancel_delayed_work_sync(&sc->hw_hang_work); #ifdef CONFIG_ATH9K_BTCOEX_SUPPORT if (ath9k_hw_mci_is_enabled(sc->sc_ah)) @@ -208,6 +209,9 @@ void ath_restart_work(struct ath_softc *sc) ieee80211_queue_delayed_work(sc->hw, &sc->hw_pll_work, msecs_to_jiffies(ATH_PLL_WORK_INTERVAL)); + ieee80211_queue_delayed_work(sc->hw, &sc->hw_hang_work, + msecs_to_jiffies(ATH_HANG_WORK_INTERVAL)); + ath_start_ani(sc); } From patchwork Mon Nov 4 17:16:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Issam Hamdi X-Patchwork-Id: 841048 Received: from mail.simonwunderlich.de (mail.simonwunderlich.de [23.88.38.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D07071C07D3 for ; Mon, 4 Nov 2024 17:25:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=23.88.38.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730741121; cv=none; b=byUXjFzn2L49UVgaeYpbHAi+nXG97RAvapFI7T3wTiIHrBvoQMWx1q1D8QIqfTuasIkOLNLx6HositoDGbRKEvCQ8Rejv/xxZ8qWjZPL3pepMskwo645pknYvz978lFbk6lbTxzeb+rx2OTHjEzmLBQWW0g2DBC75KHBko+Hgvw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730741121; c=relaxed/simple; bh=d5Bwf+oxJdh1n6qGZ6GW1P26x5b+o/yGrzoeXE2lG+4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IVV86JilNqX4Z1F4SNtlF5l2HAN+fzxHl2o2u7ZkdWXtKUSVftpq61MI5GB6SKmlrmR2Sai/FcEuUT3MAU3iVcjFpcfyoVJT0ewml/er0y6LtSLajL2KzSTv2/JdLdfN0osA1rjj2mZJ4JQ1GdcNasDEgSeahxeD80n3kzVGLyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=simonwunderlich.de; spf=pass smtp.mailfrom=simonwunderlich.de; arc=none smtp.client-ip=23.88.38.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=simonwunderlich.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=simonwunderlich.de Received: from localhost (p200300c59740C190000000000000032B.dip0.t-ipconnect.de [IPv6:2003:c5:9740:c190::32b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.simonwunderlich.de (Postfix) with ESMTPSA id D5B01FA181; Mon, 4 Nov 2024 18:16:47 +0100 (CET) From: Issam Hamdi To: johannes@sipsolutions.net Cc: linux-wireless@vger.kernel.org, mathias.kretschmer@fit.fraunhofer.de, Simon Wunderlich , Simon Wunderlich , Sven Eckelmann , Issam Hamdi Subject: [PATCH 2/2] wifi: ath9k: Reset chip on potential deaf state Date: Mon, 4 Nov 2024 18:16:27 +0100 Message-Id: <20241104171627.3789199-2-ih@simonwunderlich.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241104171627.3789199-1-ih@simonwunderlich.de> References: <20241104171627.3789199-1-ih@simonwunderlich.de> Precedence: bulk X-Mailing-List: linux-wireless@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Simon Wunderlich The chip is switching seemingly random into a state which can be described as "deaf". No or nearly no interrupts are generated anymore for incoming packets. Existing links either break down after a while and new links will not be established. The driver doesn't know if there is no other device available or if it ended up in an "deaf" state. Resetting the chip proactively avoids permanent problems in case the chip really was in its "deaf" state but maybe causes unnecessary resets in case it wasn't "deaf". This patch originally developed by "Simon Wunderlich " and "Sven Eckelmann " Co-developed-by: Simon Wunderlich Co-developed-by: Sven Eckelmann Signed-off-by: Issam Hamdi --- drivers/net/wireless/ath/ath9k/ath9k.h | 3 ++ drivers/net/wireless/ath/ath9k/debug.c | 1 + drivers/net/wireless/ath/ath9k/debug.h | 1 + drivers/net/wireless/ath/ath9k/link.c | 48 +++++++++++++++++++++++++- 4 files changed, 52 insertions(+), 1 deletion(-) diff --git a/drivers/net/wireless/ath/ath9k/ath9k.h b/drivers/net/wireless/ath/ath9k/ath9k.h index c1ce081445a9..2b98c69fa37f 100644 --- a/drivers/net/wireless/ath/ath9k/ath9k.h +++ b/drivers/net/wireless/ath/ath9k/ath9k.h @@ -1026,6 +1026,9 @@ struct ath_softc { short nbcnvifs; unsigned long ps_usecount; + unsigned long last_check_time; + u32 last_check_interrupts; + struct ath_rx rx; struct ath_tx tx; struct ath_beacon beacon; diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c index 6b2469a01f17..4128cf691166 100644 --- a/drivers/net/wireless/ath/ath9k/debug.c +++ b/drivers/net/wireless/ath/ath9k/debug.c @@ -751,6 +751,7 @@ static int read_file_reset(struct seq_file *file, void *data) [RESET_TX_DMA_ERROR] = "Tx DMA stop error", [RESET_RX_DMA_ERROR] = "Rx DMA stop error", [RESET_TYPE_DEADBEEF] = "deadbeef hang", + [RESET_TYPE_DEAF] = "deaf hang", }; int i; diff --git a/drivers/net/wireless/ath/ath9k/debug.h b/drivers/net/wireless/ath/ath9k/debug.h index 6ebb6053a8c1..76e27860455c 100644 --- a/drivers/net/wireless/ath/ath9k/debug.h +++ b/drivers/net/wireless/ath/ath9k/debug.h @@ -54,6 +54,7 @@ enum ath_reset_type { RESET_TX_DMA_ERROR, RESET_RX_DMA_ERROR, RESET_TYPE_DEADBEEF, + RESET_TYPE_DEAF, __RESET_TYPE_MAX }; diff --git a/drivers/net/wireless/ath/ath9k/link.c b/drivers/net/wireless/ath/ath9k/link.c index 37438960c278..d1762cc3129d 100644 --- a/drivers/net/wireless/ath/ath9k/link.c +++ b/drivers/net/wireless/ath/ath9k/link.c @@ -162,13 +162,59 @@ static bool ath_hw_hang_deadbeef(struct ath_softc *sc) return true; } +static bool ath_hw_hang_deaf(struct ath_softc *sc) +{ +#if !defined(CPTCFG_ATH9K_DEBUGFS) || defined(CPTCFG_ATH9K_TX99) + return false; +#else + struct ath_common *common = ath9k_hw_common(sc->sc_ah); + u32 interrupts, interrupt_per_s; + unsigned int interval; + + /* get historic data */ + interval = jiffies_to_msecs(jiffies - sc->last_check_time); + if (sc->sc_ah->caps.hw_caps & ATH9K_HW_CAP_EDMA) + interrupts = sc->debug.stats.istats.rxlp; + else + interrupts = sc->debug.stats.istats.rxok; + + interrupts -= sc->last_check_interrupts; + + /* save current data */ + sc->last_check_time = jiffies; + if (sc->sc_ah->caps.hw_caps & ATH9K_HW_CAP_EDMA) + sc->last_check_interrupts = sc->debug.stats.istats.rxlp; + else + sc->last_check_interrupts = sc->debug.stats.istats.rxok; + + /* sanity check, should be 4 seconds */ + if (interval > 10000 || interval < 1000) + return false; + + /* should be at least one interrupt per second */ + interrupt_per_s = interrupts / (interval / 1000); + if (interrupt_per_s >= 1) + return false; + + ath_dbg(common, RESET, + "RX deaf hang is detected. Schedule chip reset\n"); + ath9k_queue_reset(sc, RESET_TYPE_DEAF); + + return true; +#endif +} + void ath_hw_hang_work(struct work_struct *work) { struct ath_softc *sc = container_of(work, struct ath_softc, hw_hang_work.work); - ath_hw_hang_deadbeef(sc); + if (ath_hw_hang_deadbeef(sc)) + goto requeue_worker; + ath_hw_hang_deaf(sc); + +requeue_worker: ieee80211_queue_delayed_work(sc->hw, &sc->hw_hang_work, msecs_to_jiffies(ATH_HANG_WORK_INTERVAL)); }