ath9k: Add RX inactivity detection and reset chip when it occurs

Some ath9k chips can, seemingly at random, end up in a state which can
be described as "deaf". No or nearly no interrupts are generated anymore
for incoming packets. Existing links either break down after a while and
new links will not be established.

The circumstances leading to this "deafness" is still unclear, but some
particular chips (especially 2-stream 11n SoCs, but also others) can go
'deaf' when running AP or mesh (or both) after some time. It's probably
a hardware issue, and doing a channel scan to trigger a chip
reset (which one normally can't do on an AP interface) recovers the
hardware.

The only way the driver can detect this state, is by detecting if there
has been no RX activity for a while. In this case we can proactively
reset the chip (which only takes a small number of milliseconds, so
shouldn't interrupt things too much if it has been idle for several
seconds), which functions as a workaround.

OpenWrt, and various derivatives, have been carrying versions of this
workaround for years, that were never upstreamed. One version[0],
written by Felix Fietkau, used a simple counter and only reset if there
was precisely zero RX activity for a long period of time. This had the
problem that in some cases a small number of interrupts would appear
even if the device was otherwise not responsive. For this reason,
another version[1], written by Simon Wunderlich and Sven Eckelmann, used
a time-based approach to calculate the average number of RX interrupts
over a longer (four-second) interval, and reset the chip when seeing
less than one interrupt per second over this period. However, that
version relied on debugfs counters to keep track of the number of
interrupts, which means it didn't work at all if debugfs was not
enabled.

This patch unifies the two versions: it uses the same approach as Felix'
patch to count the number of RX handler invocations, but uses the same
time-based windowing approach as Simon and Sven's patch to still handle
the case where occasional interrupts appear but the device is otherwise
deaf.

Since this is based on ideas by all three people, but not actually
directly derived from any of the patches, I'm including Suggested-by
tags from Simon, Sven and Felix below, which should hopefully serve as
proper credit.

[0] https://patchwork.kernel.org/project/linux-wireless/patch/20170125163654.66431-3-nbd@nbd.name/
[1] https://patchwork.kernel.org/project/linux-wireless/patch/20161117083614.19188-2-sven.eckelmann@open-mesh.com/

Suggested-by: Simon Wunderlich <sw@simonwunderlich.de>
Suggested-by: Sven Eckelmann <se@simonwunderlich.de>
Suggested-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
---
 drivers/net/wireless/ath/ath9k/ath9k.h |  2 ++
 drivers/net/wireless/ath/ath9k/debug.c |  1 +
 drivers/net/wireless/ath/ath9k/debug.h |  1 +
 drivers/net/wireless/ath/ath9k/link.c  | 33 +++++++++++++++++++++++++++++++--
 drivers/net/wireless/ath/ath9k/main.c  |  1 +
 5 files changed, 36 insertions(+), 2 deletions(-)


---
base-commit: c33f9c2728d0ccc7472e6239346c0fb3de556e0f
change-id: 20241105-ath9k-deaf-detection-0e26bb3243a6

Hi,

Thank you for submitting the patch.

On Wednesday, 6 November 2024 13:41:44 CET Toke Høiland-Jørgensen wrote:
> Since this is based on ideas by all three people, but not actually
> directly derived from any of the patches, I'm including Suggested-by
> tags from Simon, Sven and Felix below, which should hopefully serve as
> proper credit.

At least for me, this is more than enough. Thanks.

I don't have the setup at the moment to test it again - maybe Issam can do 
this. One concern I would have (because I don't find the notes regarding this 
problem), is whether this check is now breaking because we count more things. 
In the past, rxlp/rxok was used for the check. And now I don't know whether 
the count for the other ones were still increasing.

* RXHP (rather sure that "high priority frame" wasn't increasing)
* RXEOL ("no RX descriptors available" - I would guess no, but I can't say for
  sure)
* RXORN ("FIFO overrun" I would guess no, but I can't say for sure)

Reviewed-by: Sven Eckelmann <se@simonwunderlich.de>

Kind regards,
	Sven

Sven Eckelmann <se@simonwunderlich.de> writes:

> Hi,
>
> Thank you for submitting the patch.
>
> On Wednesday, 6 November 2024 13:41:44 CET Toke Høiland-Jørgensen wrote:
>> Since this is based on ideas by all three people, but not actually
>> directly derived from any of the patches, I'm including Suggested-by
>> tags from Simon, Sven and Felix below, which should hopefully serve as
>> proper credit.
>
> At least for me, this is more than enough. Thanks.
>
> I don't have the setup at the moment to test it again - maybe Issam can do 
> this. One concern I would have (because I don't find the notes regarding this 
> problem), is whether this check is now breaking because we count more things. 
> In the past, rxlp/rxok was used for the check. And now I don't know whether 
> the count for the other ones were still increasing.
>
> * RXHP (rather sure that "high priority frame" wasn't increasing)
> * RXEOL ("no RX descriptors available" - I would guess no, but I can't say for
>   sure)
> * RXORN ("FIFO overrun" I would guess no, but I can't say for sure)
>
> Reviewed-by: Sven Eckelmann <se@simonwunderlich.de>

Great, thanks for the review! I'll let it sit in patchwork for a little
while to give people a chance to test it out before sending it over to
Kalle to be applied :)

-Toke

On Wednesday, November 6, 2024 3:12:59 PM CET Toke Høiland-Jørgensen wrote:
> Sven Eckelmann <se@simonwunderlich.de> writes:
> > Hi,
> > 
> > Thank you for submitting the patch.
> > 
> > On Wednesday, 6 November 2024 13:41:44 CET Toke Høiland-Jørgensen wrote:
> >> Since this is based on ideas by all three people, but not actually
> >> directly derived from any of the patches, I'm including Suggested-by
> >> tags from Simon, Sven and Felix below, which should hopefully serve as
> >> proper credit.
> > 
> > At least for me, this is more than enough. Thanks.
> > 
> > I don't have the setup at the moment to test it again - maybe Issam can do
> > this. One concern I would have (because I don't find the notes regarding
> > this problem), is whether this check is now breaking because we count
> > more things. In the past, rxlp/rxok was used for the check. And now I
> > don't know whether the count for the other ones were still increasing.
> > 
> > * RXHP (rather sure that "high priority frame" wasn't increasing)
> > * RXEOL ("no RX descriptors available" - I would guess no, but I can't say
> > for> 
> >   sure)
> > 
> > * RXORN ("FIFO overrun" I would guess no, but I can't say for sure)
> > 
> > Reviewed-by: Sven Eckelmann <se@simonwunderlich.de>
> 
> Great, thanks for the review! I'll let it sit in patchwork for a little
> while to give people a chance to test it out before sending it over to
> Kalle to be applied :)
> 
> -Toke

Hi Toke,

this looks good to me in general. I'm not sure either about the particular RX 
interrupts. We can test this by putting the AP in a shield box and verify that 
the counters are actually increasing, and that should be good enough.

Acked-by: Simon Wunderlich <sw@simonwunderlich.de>

Thank you!
      Simon

Simon Wunderlich <sw@simonwunderlich.de> writes:

> On Wednesday, November 6, 2024 3:12:59 PM CET Toke Høiland-Jørgensen wrote:
>> Sven Eckelmann <se@simonwunderlich.de> writes:
>> > Hi,
>> > 
>> > Thank you for submitting the patch.
>> > 
>> > On Wednesday, 6 November 2024 13:41:44 CET Toke Høiland-Jørgensen wrote:
>> >> Since this is based on ideas by all three people, but not actually
>> >> directly derived from any of the patches, I'm including Suggested-by
>> >> tags from Simon, Sven and Felix below, which should hopefully serve as
>> >> proper credit.
>> > 
>> > At least for me, this is more than enough. Thanks.
>> > 
>> > I don't have the setup at the moment to test it again - maybe Issam can do
>> > this. One concern I would have (because I don't find the notes regarding
>> > this problem), is whether this check is now breaking because we count
>> > more things. In the past, rxlp/rxok was used for the check. And now I
>> > don't know whether the count for the other ones were still increasing.
>> > 
>> > * RXHP (rather sure that "high priority frame" wasn't increasing)
>> > * RXEOL ("no RX descriptors available" - I would guess no, but I can't say
>> > for> 
>> >   sure)
>> > 
>> > * RXORN ("FIFO overrun" I would guess no, but I can't say for sure)
>> > 
>> > Reviewed-by: Sven Eckelmann <se@simonwunderlich.de>
>> 
>> Great, thanks for the review! I'll let it sit in patchwork for a little
>> while to give people a chance to test it out before sending it over to
>> Kalle to be applied :)
>> 
>> -Toke
>
> Hi Toke,
>
> this looks good to me in general. I'm not sure either about the particular RX 
> interrupts. We can test this by putting the AP in a shield box and verify that 
> the counters are actually increasing, and that should be good enough.
>
> Acked-by: Simon Wunderlich <sw@simonwunderlich.de>

Great, thanks! Would be awesome if you could test it out an report back! :)

-Toke