(Octeon) MMC performance degradation due to too many requests

Hi Linus,

let me appologize for walking into dark past :) Here's some more context
for a start:
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1691602.html
(note that "OK phew then I'm safe" ;-))

After moving a few Octeon 70XX based boards from vendor based 4.9 kernel
to 6.1 mainline a huge drop of MMC performance was observed.

While running badblocks on 2G partition takes 1min (±5s) and about 35k
interrupts on 4.9, recent kernels need over 3mins and 500k+ interrupts.

How do we get there?
ff4143ccff31 ("MIPS: Octeon: cavium_octeon_defconfig: Enable Octeon MMC")
enabled MMC driver, but left MMC_BLOCK_BOUNCE disabled, although driver
performace depends on it.
c3dccb74be28 ("mmc: core: Delete bounce buffer Kconfig option")
Added MMC_CAP_NO_BOUNCE_BUFF to the caps, based on assumption it should
be there as MMC_BLOCK_BOUNCE is disabled in defconfig
de3ee99b097d ("mmc: Delete bounce buffer handling")
finally removed all bounce buffer handling as almost nothing needs that.

Sadly, 70XX SoC cannot do SG, so it suffers a lot. Strangely enough,
above patches are either authored or suggested by Cavium's employees.

So, given the number of affected SoC and before cooking driver specific
solution, are we sure we indeed do not want some generic one?

While there, you might consider following patch:
-- >8 --
From: Ladislav Michl <ladis@linux-mips.org>
Subject: [PATCH] mmc: cavium: Remove misleading comment

Comment about disabling bounce buffers was added with c3dccb74be28
("mmc: core: Delete bounce buffer Kconfig option") and should be
deleted with de3ee99b097d ("mmc: Delete bounce buffer handling").
For the record, this driver should have never been used
MMC_CAP_NO_BOUNCE_BUFF as it hits performance badly for non SG
capable SoC.

Signed-off-by: Ladislav Michl <ladis@linux-mips.org>
Fixes: de3ee99b097d ("mmc: Delete bounce buffer handling")
---
 drivers/mmc/host/cavium.c | 2 --
 1 file changed, 2 deletions(-)

On Sun, Aug 6, 2023 at 1:48 PM Ladislav Michl <oss-lists@triops.cz> wrote:

> How do we get there?
> ff4143ccff31 ("MIPS: Octeon: cavium_octeon_defconfig: Enable Octeon MMC")
> enabled MMC driver, but left MMC_BLOCK_BOUNCE disabled, although driver
> performace depends on it.

Ooops.

> c3dccb74be28 ("mmc: core: Delete bounce buffer Kconfig option")
> Added MMC_CAP_NO_BOUNCE_BUFF to the caps, based on assumption it should
> be there as MMC_BLOCK_BOUNCE is disabled in defconfig
> de3ee99b097d ("mmc: Delete bounce buffer handling")
> finally removed all bounce buffer handling as almost nothing needs that.
>
> Sadly, 70XX SoC cannot do SG, so it suffers a lot. Strangely enough,
> above patches are either authored or suggested by Cavium's employees.
>
> So, given the number of affected SoC and before cooking driver specific
> solution, are we sure we indeed do not want some generic one?

So you are talking about something along the lines of:

commit bd9b902798ab14d19ca116b10bde581ddff8f905
Author: Linus Walleij <linus.walleij@linaro.org>
Date:   Mon Jan 29 00:44:53 2018 +0100

    mmc: sdhci: Implement an SDHCI-specific bounce buffer

?

Yeah I guess that if this is needed by more than one driver it
should be made into a library, or say a piece of code turned on by
a config option that the dependent drivers select.

Interested in the job? :D

Yours,
Linus Walleij

On Sun, Aug 06, 2023 at 10:44:21PM +0200, Linus Walleij wrote:
> On Sun, Aug 6, 2023 at 1:48 PM Ladislav Michl <oss-lists@triops.cz> wrote:
> 
> > How do we get there?
> > ff4143ccff31 ("MIPS: Octeon: cavium_octeon_defconfig: Enable Octeon MMC")
> > enabled MMC driver, but left MMC_BLOCK_BOUNCE disabled, although driver
> > performace depends on it.
> 
> Ooops.
> 
> > c3dccb74be28 ("mmc: core: Delete bounce buffer Kconfig option")
> > Added MMC_CAP_NO_BOUNCE_BUFF to the caps, based on assumption it should
> > be there as MMC_BLOCK_BOUNCE is disabled in defconfig
> > de3ee99b097d ("mmc: Delete bounce buffer handling")
> > finally removed all bounce buffer handling as almost nothing needs that.
> >
> > Sadly, 70XX SoC cannot do SG, so it suffers a lot. Strangely enough,
> > above patches are either authored or suggested by Cavium's employees.
> >
> > So, given the number of affected SoC and before cooking driver specific
> > solution, are we sure we indeed do not want some generic one?
> 
> So you are talking about something along the lines of:
> 
> commit bd9b902798ab14d19ca116b10bde581ddff8f905
> Author: Linus Walleij <linus.walleij@linaro.org>
> Date:   Mon Jan 29 00:44:53 2018 +0100
> 
>     mmc: sdhci: Implement an SDHCI-specific bounce buffer
> 
> ?

Yes, this is the exact commit I had in mind :)

> Yeah I guess that if this is needed by more than one driver it
> should be made into a library, or say a piece of code turned on by
> a config option that the dependent drivers select.
> 
> Interested in the job? :D

Interested is not the word I'd use, but yes, I'll give it a try making it
a little more generic solution.

> Yours,
> Linus Walleij

Best reards,
	ladis