| Message ID | 20230702203429.237615-1-prabhakar.mahadev-lad.rj@bp.renesas.com |
|---|---|
| Headers | show |
| Series | Add non-coherent DMA support for AX45MP | expand |
Hi Emil, Thank you for the review. On Mon, Jul 24, 2023 at 11:18 AM Emil Renner Berthing <emil.renner.berthing@canonical.com> wrote: > > On Sun, 2 Jul 2023 at 22:36, Prabhakar <prabhakar.csengg@gmail.com> wrote: > > > > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > > > > Introduce support for nonstandard noncoherent systems in the RISC-V > > architecture. It enables function pointer support to handle cache > > management in such systems. > > > > This patch adds a new configuration option called > > "RISCV_NONSTANDARD_CACHE_OPS." This option is a boolean flag that > > depends on "RISCV_DMA_NONCOHERENT" and enables the function pointer > > support for cache management in nonstandard noncoherent systems. > > > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> > > Reviewed-by: Conor Dooley <conor.dooley@microchip.com> > > Tested-by: Conor Dooley <conor.dooley@microchip.com> # tyre-kicking on a d1 > > --- > > v9 -> v10 > > * Added __ro_after_init compiler attribute for noncoherent_cache_ops > > * Renamed clean -> wback > > * Renamed inval -> inv > > * Renamed flush -> wback_inv > > > > v8 -> v9 > > * New patch > > --- > > arch/riscv/Kconfig | 7 ++++ > > arch/riscv/include/asm/dma-noncoherent.h | 28 +++++++++++++++ > > arch/riscv/mm/dma-noncoherent.c | 43 ++++++++++++++++++++++++ > > arch/riscv/mm/pmem.c | 13 +++++++ > > 4 files changed, 91 insertions(+) > > create mode 100644 arch/riscv/include/asm/dma-noncoherent.h > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > index d9e451ac862a..42c86b13c5e1 100644 > > --- a/arch/riscv/Kconfig > > +++ b/arch/riscv/Kconfig > > @@ -265,6 +265,13 @@ config RISCV_DMA_NONCOHERENT > > select ARCH_HAS_SYNC_DMA_FOR_DEVICE > > select DMA_DIRECT_REMAP > > > > +config RISCV_NONSTANDARD_CACHE_OPS > > + bool > > + depends on RISCV_DMA_NONCOHERENT > > + help > > + This enables function pointer support for non-standard noncoherent > > + systems to handle cache management. > > + > > config AS_HAS_INSN > > def_bool $(as-instr,.insn r 51$(comma) 0$(comma) 0$(comma) t0$(comma) t0$(comma) zero) > > > > diff --git a/arch/riscv/include/asm/dma-noncoherent.h b/arch/riscv/include/asm/dma-noncoherent.h > > new file mode 100644 > > index 000000000000..969cf1f1363a > > --- /dev/null > > +++ b/arch/riscv/include/asm/dma-noncoherent.h > > @@ -0,0 +1,28 @@ > > +/* SPDX-License-Identifier: GPL-2.0-only */ > > +/* > > + * Copyright (C) 2023 Renesas Electronics Corp. > > + */ > > + > > +#ifndef __ASM_DMA_NONCOHERENT_H > > +#define __ASM_DMA_NONCOHERENT_H > > + > > +#include <linux/dma-direct.h> > > + > > +/* > > + * struct riscv_cache_ops - Structure for CMO function pointers > > + * > > + * @wback: Function pointer for cache writeback > > + * @inv: Function pointer for invalidating cache > > + * @wback_inv: Function pointer for flushing the cache (writeback + invalidating) > > + */ > > +struct riscv_cache_ops { > > + void (*wback)(phys_addr_t paddr, unsigned long size); > > + void (*inv)(phys_addr_t paddr, unsigned long size); > > + void (*wback_inv)(phys_addr_t paddr, unsigned long size); > > Hi Prabhakar > > Just a quick question. After Arnd's patchset the > arch_dma_cache{inv,wback,wback_inv} functions take a phys_addr_t and > size_t, but here you want these callbacks to take a phys_addr_t and > unsigned long instead. Why not keep them using size_t? > Agreed, I will update it to use size_t instead. Cheers, Prabhakar
From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Hi All, non-coherent DMA support for AX45MP ==================================== On the Andes AX45MP core, cache coherency is a specification option so it may not be supported. In this case DMA will fail. To get around with this issue this patch series does the below: 1] Andes alternative ports is implemented as errata which checks if the IOCP is missing and only then applies to CMO errata. One vendor specific SBI EXT (ANDES_SBI_EXT_IOCP_SW_WORKAROUND) is implemented as part of errata. Below are the configs which Andes port provides (and are selected by RZ/Five): - ERRATA_ANDES - ERRATA_ANDES_CMO OpenSBI patch supporting ANDES_SBI_EXT_IOCP_SW_WORKAROUND SBI is now part v1.3 release. 2] Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) block that allows dynamic adjustment of memory attributes in the runtime. It contains a configurable amount of PMA entries implemented as CSR registers to control the attributes of memory locations in interest. OpenSBI configures the PMA regions as required and creates a reserve memory node and propagates it to the higher boot stack. Currently OpenSBI (upstream) configures the required PMA region and passes this a shared DMA pool to Linux. reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; pma_resv0@58000000 { compatible = "shared-dma-pool"; reg = <0x0 0x58000000 0x0 0x08000000>; no-map; linux,dma-default; }; }; The above shared DMA pool gets appended to Linux DTB so the DMA memory requests go through this region. 3] We provide callbacks to synchronize specific content between memory and cache. 4] RZ/Five SoC selects the below configs - AX45MP_L2_CACHE - DMA_GLOBAL_POOL - ERRATA_ANDES - ERRATA_ANDES_CMO ----------x---------------------x--------------------x---------------x---- Note, - Ive used GCC 12.2.0 for compilation - Tested all the IP blocks on RZ/Five which use DMA - Patch series is dependent on the series from Arnd, https://patchwork.kernel.org/project/linux-riscv/cover/20230327121317.4081816-1-arnd@kernel.org/. (Ive rebased Arnd's series on v6.4-rc-1) - Patches applies on top of palmer/for-next (669ee914f515) - Ive pushed the complete tree here https://github.com/prabhakarlad/linux/commits/rzfive-cmo-v10 - Previously the function pointer approach was NAKed by Christoph Hellwig but based on the discussion on #riscv Ive implemented this approach. v9 -> v10 * Included RB/TB tags from Conor * Fixed review comments pointed by Geert to rename cache callbacks * Dropped preserving the cache lines in ax45mp_dma_cache_inv() callback v8 -> v9 * Dropped adding ALTERNATIVE_3 * Implemented function pointer support for nonstandard noncoherent systems * Added a new config option CONFIG_RISCV_NONSTANDARD_CACHE_OPS * Updated Andes errata code to drop patching the calls as we no more use ALTERNATIVE_X() macro. * Updated Andes CMO code to use function pointer for doing cache management. v7 -> v8 * Dropped using function pointers and switched to ALTERNATIVE_X() * Added new patches (#1, #2) v6 -> v7 * Reworked the code based on Arnd's work * Fixed review comments pointed by Arnd * Fixed review comments pointed by Conor v5.1 -> v6 * Dropped use of ALTERNATIVE_x() macro * Now switched to used function pointers for CMO * Moved driver to drivers/cache folder v5 -> v5.1 * https://patchwork.kernel.org/project/linux-riscv/list/?series=708610&state=%2A&archive=both v4 -> v5 * Rebased ALTERNATIVE_3() macro on top of Andrew's patches * Rebased the changes on top of Heiko's alternative call patches * Dropped configuring the PMA from Linux * Dropped configuring the L2 cache from Linux and dropped the binding for same * Now using runtime patching mechanism instead of compile time config RFC v3 -> v4 * Implemented ALTERNATIVE_3() macro * Now using runtime patching mechanism instead of compile time config * Added Andes CMO as and errata * Fixed comments pointed by Geert RFC v2-> RFC v3 * Fixed review comments pointed by Conor * Move DT binding into cache folder * Fixed DT binding check issue * Added andestech,ax45mp-cache.h header file * Now passing the flags for the PMA setup as part of andestech,pma-regions property. * Added andestech,inst/data-prefetch and andestech,tag/data-ram-ctl properties to configure the L2 cache. * Registered the cache driver as platform driver RFC v1-> RFC v2 * Moved out the code from arc/riscv to drivers/soc/renesas * Now handling the PMA setup as part of the L2 cache * Now making use of dma-noncoherent.c instead SoC specific implementation. * Dropped arch_dma_alloc() and arch_dma_free() * Switched to RISCV_DMA_NONCOHERENT * Included DT binding doc RFC v2: https://patchwork.kernel.org/project/linux-renesas-soc/cover/20221003223222.448551-1-prabhakar.mahadev-lad.rj@bp.renesas.com/ RFC v1: https://patchwork.kernel.org/project/linux-renesas-soc/cover/20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com/ Cheers, Prabhakar Lad Prabhakar (6): riscv: asm: vendorid_list: Add Andes Technology to the vendors list riscv: errata: Add Andes alternative ports riscv: mm: dma-noncoherent: nonstandard cache operations support dt-bindings: cache: andestech,ax45mp-cache: Add DT binding documentation for L2 cache controller cache: Add L2 cache management for Andes AX45MP RISC-V core soc: renesas: Kconfig: Select the required configs for RZ/Five SoC .../cache/andestech,ax45mp-cache.yaml | 81 +++++++ MAINTAINERS | 7 + arch/riscv/Kconfig | 7 + arch/riscv/Kconfig.errata | 21 ++ arch/riscv/errata/Makefile | 1 + arch/riscv/errata/andes/Makefile | 1 + arch/riscv/errata/andes/errata.c | 66 ++++++ arch/riscv/include/asm/alternative.h | 3 + arch/riscv/include/asm/dma-noncoherent.h | 28 +++ arch/riscv/include/asm/errata_list.h | 5 + arch/riscv/include/asm/vendorid_list.h | 1 + arch/riscv/kernel/alternative.c | 5 + arch/riscv/mm/dma-noncoherent.c | 43 ++++ arch/riscv/mm/pmem.c | 13 ++ drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/cache/Kconfig | 11 + drivers/cache/Makefile | 3 + drivers/cache/ax45mp_cache.c | 213 ++++++++++++++++++ drivers/soc/renesas/Kconfig | 4 + 20 files changed, 516 insertions(+) create mode 100644 Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml create mode 100644 arch/riscv/errata/andes/Makefile create mode 100644 arch/riscv/errata/andes/errata.c create mode 100644 arch/riscv/include/asm/dma-noncoherent.h create mode 100644 drivers/cache/Kconfig create mode 100644 drivers/cache/Makefile create mode 100644 drivers/cache/ax45mp_cache.c