mbox series

[v5,0/6] AX45MP: Add support to non-coherent DMA

Message ID 20221212115505.36770-1-prabhakar.mahadev-lad.rj@bp.renesas.com
Headers show
Series AX45MP: Add support to non-coherent DMA | expand

Message

Prabhakar Dec. 12, 2022, 11:54 a.m. UTC 
From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Hi All,

On the Andes AX45MP core, cache coherency is a specification option so it
may not be supported. In this case DMA will fail. To get around with this
issue this patch series does the below:

1] Andes alternative ports is implemented as errata which checks if the IOCP
is missing and only then applies to CMO errata. One vendor specific SBI EXT
(RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND) is implemented as part of errata. If we could
access the DTB in errata I can get rid of this EXT ID from OpenSBI. Is there any
approach we can access the DTB in patch callback?

Below are the configs which Andes port provides (and are selected by RZ/Five):
      - ERRATA_ANDES
      - ERRATA_ANDES_CMO

2] Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
block that allows dynamic adjustment of memory attributes in the runtime.
It contains a configurable amount of PMA entries implemented as CSR
registers to control the attributes of memory locations in interest.
OpenSBI configures the PMA regions as required and creates a reserve memory
node and propagates it to the higher boot stack.

An RFC patch for OpenSBI is posted here:
https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj@bp.renesas.com/

    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;

        pma_resv0@58000000 {
            compatible = "shared-dma-pool";
            reg = <0x0 0x58000000 0x0 0x08000000>;
            no-map;
            linux,dma-default;
        };
    };

The above shared DMA pool gets appended to Linux DTB so the DMA memory
requests go through this region.

3] We provide callbacks to synchronize specific content between memory and
cache.

        - arch_sync_dma_for_device()
        - arch_sync_dma_for_cpu()

4] RZ/Five SoC selects the below configs
        - AX45MP_L2_CACHE
        - DMA_GLOBAL_POOL
        - ERRATA_ANDES
        - ERRATA_ANDES_CMO

OpenSBI implementation patches can be found here:
1] https://patchwork.ozlabs.org/project/opensbi/cover/20221210103011.7814-1-prabhakar.mahadev-lad.rj@bp.renesas.com/
2]https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj@bp.renesas.com/

Note,
- This series requires testing on Cores with zibcom and T-Head SoCs
- Ive used GCC 12.2.0 for compilation (tested with allmodconfig)
- Tested all the IP blocks on RZ/Five which use DMA
- Series is dependant on https://patchwork.kernel.org/project/linux-riscv/cover/20221207180821.2479987-1-heiko@sntech.de/

v4 -> v5
* Rebased ALTERNATIVE_3() macro on top of Andrew's patches
* Rebased the changes on top of Heiko's alternative call patches
* Dropped configuring the PMA from Linux
* Dropped configuring the L2 cache from Linux and dropped the binding for same
* Now using runtime patching mechanism instead of compile time config

RFC v3 -> v4
* Implemented ALTERNATIVE_3() macro 
* Now using runtime patching mechanism instead of compile time config
* Added Andes CMO as and errata
* Fixed comments pointed by Geert

RFC v2-> RFC v3
* Fixed review comments pointed by Conor
* Move DT binding into cache folder
* Fixed DT binding check issue
* Added andestech,ax45mp-cache.h header file
* Now passing the flags for the PMA setup as part of andestech,pma-regions
  property.
* Added andestech,inst/data-prefetch and andestech,tag/data-ram-ctl
  properties to configure the L2 cache.
* Registered the cache driver as platform driver

RFC v1-> RFC v2
* Moved out the code from arc/riscv to drivers/soc/renesas
* Now handling the PMA setup as part of the L2 cache
* Now making use of dma-noncoherent.c instead SoC specific implementation.
* Dropped arch_dma_alloc() and arch_dma_free()
* Switched to RISCV_DMA_NONCOHERENT
* Included DT binding doc

RFC v2: https://patchwork.kernel.org/project/linux-renesas-soc/cover/20221003223222.448551-1-prabhakar.mahadev-lad.rj@bp.renesas.com/
RFC v1: https://patchwork.kernel.org/project/linux-renesas-soc/cover/20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com/

Cheers,
Prabhakar

Lad Prabhakar (6):
  riscv: asm: alternative-macros: Introduce ALTERNATIVE_3() macro
  riscv: asm: vendorid_list: Add Andes Technology to the vendors list
  riscv: errata: Add Andes alternative ports
  riscv: mm: dma-noncoherent: Pass direction and operation to
    ALT_CMO_OP()
  dt-bindings: cache: r9a07g043f-l2-cache: Add DT binding documentation
    for L2 cache controller
  soc: renesas: Add L2 cache management for RZ/Five SoC

 .../cache/andestech,ax45mp-cache.yaml         |  81 ++++++
 arch/riscv/Kconfig.erratas                    |  22 ++
 arch/riscv/errata/Makefile                    |   1 +
 arch/riscv/errata/andes/Makefile              |   1 +
 arch/riscv/errata/andes/errata.c              |  93 +++++++
 arch/riscv/include/asm/alternative-macros.h   |  46 +++-
 arch/riscv/include/asm/alternative.h          |   3 +
 arch/riscv/include/asm/cacheflush.h           |  12 +
 arch/riscv/include/asm/errata_list.h          |  41 ++-
 arch/riscv/include/asm/vendorid_list.h        |   1 +
 arch/riscv/kernel/alternative.c               |   5 +
 arch/riscv/mm/dma-noncoherent.c               |  15 +-
 drivers/soc/renesas/Kconfig                   |   6 +
 drivers/soc/renesas/Makefile                  |   2 +
 drivers/soc/renesas/rzfive/Kconfig            |   6 +
 drivers/soc/renesas/rzfive/Makefile           |   3 +
 drivers/soc/renesas/rzfive/ax45mp_cache.c     | 256 ++++++++++++++++++
 17 files changed, 576 insertions(+), 18 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml
 create mode 100644 arch/riscv/errata/andes/Makefile
 create mode 100644 arch/riscv/errata/andes/errata.c
 create mode 100644 drivers/soc/renesas/rzfive/Kconfig
 create mode 100644 drivers/soc/renesas/rzfive/Makefile
 create mode 100644 drivers/soc/renesas/rzfive/ax45mp_cache.c

Comments

Conor Dooley Dec. 17, 2022, 9:19 p.m. UTC | #1 
On Mon, Dec 12, 2022 at 11:55:02AM +0000, Prabhakar wrote:
> From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> 
> Add required ports of the Alternative scheme for Andes CPU cores.
> 
> I/O Coherence Port (IOCP) provides an AXI interface for connecting external
> non-caching masters, such as DMA controllers. IOCP is a specification
> option and is disabled on the Renesas RZ/Five SoC due to this reason cache
> management needs a software workaround.
> 
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
> v4 -> v5
> * Sorted the Kconfig/Makefile/Switch based on Core name
> * Added a comments
> * Introduced RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT ID to check if
>   CMO needs to be applied. Is there a way we can access the DTB while patching
>   as we can drop this SBI EXT ID and add a DT property instead for cmo?
> 
> RFC v3 -> v4
> * New patch
> ---
>  arch/riscv/Kconfig.erratas           | 22 +++++++
>  arch/riscv/errata/Makefile           |  1 +
>  arch/riscv/errata/andes/Makefile     |  1 +
>  arch/riscv/errata/andes/errata.c     | 93 ++++++++++++++++++++++++++++
>  arch/riscv/include/asm/alternative.h |  3 +
>  arch/riscv/include/asm/errata_list.h |  5 ++
>  arch/riscv/kernel/alternative.c      |  5 ++
>  7 files changed, 130 insertions(+)
>  create mode 100644 arch/riscv/errata/andes/Makefile
>  create mode 100644 arch/riscv/errata/andes/errata.c
> 
> diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
> index 69621ae6d647..f0f0c1abd52b 100644
> --- a/arch/riscv/Kconfig.erratas
> +++ b/arch/riscv/Kconfig.erratas
> @@ -1,5 +1,27 @@
>  menu "CPU errata selection"
>  
> +config ERRATA_ANDES
> +	bool "Andes AX45MP errata"
> +	depends on !XIP_KERNEL
> +	select RISCV_ALTERNATIVE
> +	help
> +	  All Andes errata Kconfig depend on this Kconfig. Disabling
> +	  this Kconfig will disable all Andes errata. Please say "Y"
> +	  here if your platform uses Andes CPU cores.
> +
> +	  Otherwise, please say "N" here to avoid unnecessary overhead.
> +
> +config ERRATA_ANDES_CMO
> +	bool "Apply Andes cache management errata"
> +	depends on ERRATA_ANDES && MMU && ARCH_R9A07G043
> +	select RISCV_DMA_NONCOHERENT
> +	default y
> +	help
> +	  This will apply the cache management errata to handle the
> +	  non-standard handling on non-coherent operations on Andes cores.
> +
> +	  If you don't know what to do here, say "Y".
> +
>  config ERRATA_SIFIVE
>  	bool "SiFive errata"
>  	depends on !XIP_KERNEL
> diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
> index a1055965fbee..6f1c693af92d 100644
> --- a/arch/riscv/errata/Makefile
> +++ b/arch/riscv/errata/Makefile
> @@ -1,2 +1,3 @@
> +obj-$(CONFIG_ERRATA_ANDES) += andes/
>  obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
>  obj-$(CONFIG_ERRATA_THEAD) += thead/
> diff --git a/arch/riscv/errata/andes/Makefile b/arch/riscv/errata/andes/Makefile
> new file mode 100644
> index 000000000000..2d644e19caef
> --- /dev/null
> +++ b/arch/riscv/errata/andes/Makefile
> @@ -0,0 +1 @@
> +obj-y += errata.o
> diff --git a/arch/riscv/errata/andes/errata.c b/arch/riscv/errata/andes/errata.c
> new file mode 100644
> index 000000000000..3d04f15df8d5
> --- /dev/null
> +++ b/arch/riscv/errata/andes/errata.c
> @@ -0,0 +1,93 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Erratas to be applied for Andes CPU cores
> + *
> + *  Copyright (C) 2022 Renesas Electronics Corporation.
> + *
> + * Author: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +
> +#include <asm/alternative.h>
> +#include <asm/cacheflush.h>
> +#include <asm/errata_list.h>
> +#include <asm/patch.h>
> +#include <asm/sbi.h>
> +#include <asm/vendorid_list.h>
> +
> +#define ANDESTECH_AX45MP_MARCHID	0x8000000000008a45UL
> +#define ANDESTECH_AX45MP_MIMPID		0x500UL
> +#define ANDESTECH_SBI_EXT_ANDES		0x0900031E
> +
> +#define RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND	0
> +
> +static long ax45mp_iocp_sw_workaround(void)
> +{
> +	struct sbiret ret;
> +
> +	ret = sbi_ecall(ANDESTECH_SBI_EXT_ANDES, RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND,
> +			0, 0, 0, 0, 0, 0);

Seeing as you need a new version for some of the other bits, I think it
would be good to add a minor comment here somewhere (be it here or the
commit message) that links to the SBI specs for this.
I think this looks pretty good though.
Thanks,
Conor.

> +
> +	return ret.error ? 0 : ret.value;
> +}
> +
> +static bool errata_probe_iocp(unsigned int stage, unsigned long arch_id, unsigned long impid)
> +{
> +	if (!IS_ENABLED(CONFIG_ERRATA_ANDES_CMO))
> +		return false;
> +
> +	if (arch_id != ANDESTECH_AX45MP_MARCHID || impid != ANDESTECH_AX45MP_MIMPID)
> +		return false;
> +
> +	if (!ax45mp_iocp_sw_workaround())
> +		return false;
> +
> +	/* Set this just to make core cbo code happy */
> +	riscv_cbom_block_size = 1;
> +	riscv_noncoherent_supported();
> +
> +	return true;
> +}
> +
> +static u32 andes_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid)
> +{
> +	u32 cpu_req_errata = 0;
> +
> +	/*
> +	 * In the absence of the I/O Coherency Port, access to certain peripherals
> +	 * requires vendor specific DMA handling.
> +	 */
> +	if (errata_probe_iocp(stage, archid, impid))
> +		cpu_req_errata |= BIT(ERRATA_ANDESTECH_NO_IOCP);
> +
> +	return cpu_req_errata;
> +}
> +
> +void __init_or_module andes_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> +					      unsigned long archid, unsigned long impid,
> +					      unsigned int stage)
> +{
> +	u32 cpu_req_errata = andes_errata_probe(stage, archid, impid);
> +	struct alt_entry *alt;
> +	u32 tmp;
> +
> +	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
> +		return;
> +
> +	for (alt = begin; alt < end; alt++) {
> +		if (alt->vendor_id != ANDESTECH_VENDOR_ID)
> +			continue;
> +		if (alt->errata_id >= ERRATA_ANDESTECH_NUMBER)
> +			continue;
> +
> +		tmp = BIT(alt->errata_id);
> +		if (cpu_req_errata & tmp) {
> +			patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
> +
> +			riscv_alternative_fix_offsets(alt->old_ptr, alt->alt_len,
> +						      alt->old_ptr - alt->alt_ptr);
> +		}
> +	}
> +}
> diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
> index 1bd4027d34ca..e3a8e603eb5a 100644
> --- a/arch/riscv/include/asm/alternative.h
> +++ b/arch/riscv/include/asm/alternative.h
> @@ -43,6 +43,9 @@ struct errata_checkfunc_id {
>  	bool (*func)(struct alt_entry *alt);
>  };
>  
> +void andes_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> +			     unsigned long archid, unsigned long impid,
> +			     unsigned int stage);
>  void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
>  			      unsigned long archid, unsigned long impid,
>  			      unsigned int stage);
> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> index 4180312d2a70..2ba7e6e74540 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -9,6 +9,11 @@
>  #include <asm/csr.h>
>  #include <asm/vendorid_list.h>
>  
> +#ifdef CONFIG_ERRATA_ANDES
> +#define ERRATA_ANDESTECH_NO_IOCP	0
> +#define ERRATA_ANDESTECH_NUMBER		1
> +#endif
> +
>  #ifdef CONFIG_ERRATA_SIFIVE
>  #define	ERRATA_SIFIVE_CIP_453 0
>  #define	ERRATA_SIFIVE_CIP_1200 1
> diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
> index e12b06940154..0a09cbbc2593 100644
> --- a/arch/riscv/kernel/alternative.c
> +++ b/arch/riscv/kernel/alternative.c
> @@ -40,6 +40,11 @@ static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_inf
>  #endif
>  
>  	switch (cpu_mfr_info->vendor_id) {
> +#ifdef CONFIG_ERRATA_ANDES
> +	case ANDESTECH_VENDOR_ID:
> +		cpu_mfr_info->patch_func = andes_errata_patch_func;
> +		break;
> +#endif
>  #ifdef CONFIG_ERRATA_SIFIVE
>  	case SIFIVE_VENDOR_ID:
>  		cpu_mfr_info->patch_func = sifive_errata_patch_func;
> -- 
> 2.25.1
>
Prabhakar Dec. 19, 2022, 11:19 a.m. UTC | #2 
Hi Conor,

Thank you for the review.

On Sat, Dec 17, 2022 at 9:19 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Mon, Dec 12, 2022 at 11:55:02AM +0000, Prabhakar wrote:
> > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> >
> > Add required ports of the Alternative scheme for Andes CPU cores.
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting external
> > non-caching masters, such as DMA controllers. IOCP is a specification
> > option and is disabled on the Renesas RZ/Five SoC due to this reason cache
> > management needs a software workaround.
> >
> > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > ---
> > v4 -> v5
> > * Sorted the Kconfig/Makefile/Switch based on Core name
> > * Added a comments
> > * Introduced RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT ID to check if
> >   CMO needs to be applied. Is there a way we can access the DTB while patching
> >   as we can drop this SBI EXT ID and add a DT property instead for cmo?
> >
> > RFC v3 -> v4
> > * New patch
> > ---
> >  arch/riscv/Kconfig.erratas           | 22 +++++++
> >  arch/riscv/errata/Makefile           |  1 +
> >  arch/riscv/errata/andes/Makefile     |  1 +
> >  arch/riscv/errata/andes/errata.c     | 93 ++++++++++++++++++++++++++++
> >  arch/riscv/include/asm/alternative.h |  3 +
> >  arch/riscv/include/asm/errata_list.h |  5 ++
> >  arch/riscv/kernel/alternative.c      |  5 ++
> >  7 files changed, 130 insertions(+)
> >  create mode 100644 arch/riscv/errata/andes/Makefile
> >  create mode 100644 arch/riscv/errata/andes/errata.c
> >
> > diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
> > index 69621ae6d647..f0f0c1abd52b 100644
> > --- a/arch/riscv/Kconfig.erratas
> > +++ b/arch/riscv/Kconfig.erratas
> > @@ -1,5 +1,27 @@
> >  menu "CPU errata selection"
> >
> > +config ERRATA_ANDES
> > +     bool "Andes AX45MP errata"
> > +     depends on !XIP_KERNEL
> > +     select RISCV_ALTERNATIVE
> > +     help
> > +       All Andes errata Kconfig depend on this Kconfig. Disabling
> > +       this Kconfig will disable all Andes errata. Please say "Y"
> > +       here if your platform uses Andes CPU cores.
> > +
> > +       Otherwise, please say "N" here to avoid unnecessary overhead.
> > +
> > +config ERRATA_ANDES_CMO
> > +     bool "Apply Andes cache management errata"
> > +     depends on ERRATA_ANDES && MMU && ARCH_R9A07G043
> > +     select RISCV_DMA_NONCOHERENT
> > +     default y
> > +     help
> > +       This will apply the cache management errata to handle the
> > +       non-standard handling on non-coherent operations on Andes cores.
> > +
> > +       If you don't know what to do here, say "Y".
> > +
> >  config ERRATA_SIFIVE
> >       bool "SiFive errata"
> >       depends on !XIP_KERNEL
> > diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
> > index a1055965fbee..6f1c693af92d 100644
> > --- a/arch/riscv/errata/Makefile
> > +++ b/arch/riscv/errata/Makefile
> > @@ -1,2 +1,3 @@
> > +obj-$(CONFIG_ERRATA_ANDES) += andes/
> >  obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
> >  obj-$(CONFIG_ERRATA_THEAD) += thead/
> > diff --git a/arch/riscv/errata/andes/Makefile b/arch/riscv/errata/andes/Makefile
> > new file mode 100644
> > index 000000000000..2d644e19caef
> > --- /dev/null
> > +++ b/arch/riscv/errata/andes/Makefile
> > @@ -0,0 +1 @@
> > +obj-y += errata.o
> > diff --git a/arch/riscv/errata/andes/errata.c b/arch/riscv/errata/andes/errata.c
> > new file mode 100644
> > index 000000000000..3d04f15df8d5
> > --- /dev/null
> > +++ b/arch/riscv/errata/andes/errata.c
> > @@ -0,0 +1,93 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * Erratas to be applied for Andes CPU cores
> > + *
> > + *  Copyright (C) 2022 Renesas Electronics Corporation.
> > + *
> > + * Author: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > + */
> > +
> > +#include <linux/kernel.h>
> > +#include <linux/module.h>
> > +
> > +#include <asm/alternative.h>
> > +#include <asm/cacheflush.h>
> > +#include <asm/errata_list.h>
> > +#include <asm/patch.h>
> > +#include <asm/sbi.h>
> > +#include <asm/vendorid_list.h>
> > +
> > +#define ANDESTECH_AX45MP_MARCHID     0x8000000000008a45UL
> > +#define ANDESTECH_AX45MP_MIMPID              0x500UL
> > +#define ANDESTECH_SBI_EXT_ANDES              0x0900031E
> > +
> > +#define RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND    0
> > +
> > +static long ax45mp_iocp_sw_workaround(void)
> > +{
> > +     struct sbiret ret;
> > +
> > +     ret = sbi_ecall(ANDESTECH_SBI_EXT_ANDES, RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND,
> > +                     0, 0, 0, 0, 0, 0);
>
> Seeing as you need a new version for some of the other bits, I think it
> would be good to add a minor comment here somewhere (be it here or the
> commit message) that links to the SBI specs for this.
> I think this looks pretty good though.
Sure I'll add a comment here.

I was wondering if we can get rid of this vendor specific extension
here if we get access to the DT here (for example having a DT property
which would indicate if IOCP CMO should be applied or not). Do you
think that would be good approach?  ATM we dont have a pointer here
for FDT whie early patching.

Cheers,
Prabhakar
Conor Dooley Dec. 19, 2022, 4:20 p.m. UTC | #3 
On Mon, Dec 19, 2022 at 11:19:13AM +0000, Lad, Prabhakar wrote:
> Hi Conor,
> 
> Thank you for the review.
> 
> On Sat, Dec 17, 2022 at 9:19 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Mon, Dec 12, 2022 at 11:55:02AM +0000, Prabhakar wrote:
> > > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > >
> > > Add required ports of the Alternative scheme for Andes CPU cores.
> > >
> > > I/O Coherence Port (IOCP) provides an AXI interface for connecting external
> > > non-caching masters, such as DMA controllers. IOCP is a specification
> > > option and is disabled on the Renesas RZ/Five SoC due to this reason cache
> > > management needs a software workaround.
> > >
> > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > > ---
> > > v4 -> v5
> > > * Sorted the Kconfig/Makefile/Switch based on Core name
> > > * Added a comments
> > > * Introduced RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT ID to check if
> > >   CMO needs to be applied. Is there a way we can access the DTB while patching
> > >   as we can drop this SBI EXT ID and add a DT property instead for cmo?
> > >
> > > RFC v3 -> v4
> > > * New patch
> > > ---
> > >  arch/riscv/Kconfig.erratas           | 22 +++++++
> > >  arch/riscv/errata/Makefile           |  1 +
> > >  arch/riscv/errata/andes/Makefile     |  1 +
> > >  arch/riscv/errata/andes/errata.c     | 93 ++++++++++++++++++++++++++++
> > >  arch/riscv/include/asm/alternative.h |  3 +
> > >  arch/riscv/include/asm/errata_list.h |  5 ++
> > >  arch/riscv/kernel/alternative.c      |  5 ++
> > >  7 files changed, 130 insertions(+)
> > >  create mode 100644 arch/riscv/errata/andes/Makefile
> > >  create mode 100644 arch/riscv/errata/andes/errata.c
> > >
> > > diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
> > > index 69621ae6d647..f0f0c1abd52b 100644
> > > --- a/arch/riscv/Kconfig.erratas
> > > +++ b/arch/riscv/Kconfig.erratas
> > > @@ -1,5 +1,27 @@
> > >  menu "CPU errata selection"
> > >
> > > +config ERRATA_ANDES
> > > +     bool "Andes AX45MP errata"
> > > +     depends on !XIP_KERNEL
> > > +     select RISCV_ALTERNATIVE
> > > +     help
> > > +       All Andes errata Kconfig depend on this Kconfig. Disabling
> > > +       this Kconfig will disable all Andes errata. Please say "Y"
> > > +       here if your platform uses Andes CPU cores.
> > > +
> > > +       Otherwise, please say "N" here to avoid unnecessary overhead.
> > > +
> > > +config ERRATA_ANDES_CMO
> > > +     bool "Apply Andes cache management errata"
> > > +     depends on ERRATA_ANDES && MMU && ARCH_R9A07G043
> > > +     select RISCV_DMA_NONCOHERENT
> > > +     default y
> > > +     help
> > > +       This will apply the cache management errata to handle the
> > > +       non-standard handling on non-coherent operations on Andes cores.
> > > +
> > > +       If you don't know what to do here, say "Y".
> > > +
> > >  config ERRATA_SIFIVE
> > >       bool "SiFive errata"
> > >       depends on !XIP_KERNEL
> > > diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
> > > index a1055965fbee..6f1c693af92d 100644
> > > --- a/arch/riscv/errata/Makefile
> > > +++ b/arch/riscv/errata/Makefile
> > > @@ -1,2 +1,3 @@
> > > +obj-$(CONFIG_ERRATA_ANDES) += andes/
> > >  obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
> > >  obj-$(CONFIG_ERRATA_THEAD) += thead/
> > > diff --git a/arch/riscv/errata/andes/Makefile b/arch/riscv/errata/andes/Makefile
> > > new file mode 100644
> > > index 000000000000..2d644e19caef
> > > --- /dev/null
> > > +++ b/arch/riscv/errata/andes/Makefile
> > > @@ -0,0 +1 @@
> > > +obj-y += errata.o
> > > diff --git a/arch/riscv/errata/andes/errata.c b/arch/riscv/errata/andes/errata.c
> > > new file mode 100644
> > > index 000000000000..3d04f15df8d5
> > > --- /dev/null
> > > +++ b/arch/riscv/errata/andes/errata.c
> > > @@ -0,0 +1,93 @@
> > > +// SPDX-License-Identifier: GPL-2.0-only
> > > +/*
> > > + * Erratas to be applied for Andes CPU cores
> > > + *
> > > + *  Copyright (C) 2022 Renesas Electronics Corporation.
> > > + *
> > > + * Author: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > > + */
> > > +
> > > +#include <linux/kernel.h>
> > > +#include <linux/module.h>
> > > +
> > > +#include <asm/alternative.h>
> > > +#include <asm/cacheflush.h>
> > > +#include <asm/errata_list.h>
> > > +#include <asm/patch.h>
> > > +#include <asm/sbi.h>
> > > +#include <asm/vendorid_list.h>
> > > +
> > > +#define ANDESTECH_AX45MP_MARCHID     0x8000000000008a45UL
> > > +#define ANDESTECH_AX45MP_MIMPID              0x500UL
> > > +#define ANDESTECH_SBI_EXT_ANDES              0x0900031E
> > > +
> > > +#define RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND    0
> > > +
> > > +static long ax45mp_iocp_sw_workaround(void)
> > > +{
> > > +     struct sbiret ret;
> > > +
> > > +     ret = sbi_ecall(ANDESTECH_SBI_EXT_ANDES, RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND,
> > > +                     0, 0, 0, 0, 0, 0);
> >
> > Seeing as you need a new version for some of the other bits, I think it
> > would be good to add a minor comment here somewhere (be it here or the
> > commit message) that links to the SBI specs for this.
> > I think this looks pretty good though.
> Sure I'll add a comment here.
> 
> I was wondering if we can get rid of this vendor specific extension
> here if we get access to the DT here (for example having a DT property
> which would indicate if IOCP CMO should be applied or not). Do you
> think that would be good approach?  ATM we dont have a pointer here
> for FDT whie early patching.

I dunno. I think it is fine to use the ECALL to be honest - I'd rather
that than a property that someone may omit.

That said, for the cache management stuff we are gonna need for
PolarFire SoC, we will need to have info from the DT AFAICT - marchid
etc are all set to zero on our platform so cannot be used.

I was thinking about using the compatible instead, but...
we've not tried to "forward"-port our stuff from 5.15 yet as we have
not yet completed testing testing on our vendor tree (and need some PCI
changes accepted upstream first anyway), as a result I have not looked
into what's needed there for use with alternatives. We've been using a
pre-alternatives version of that patchset from around the 5.15
development point in time instead.

Thanks,
Conor.
Prabhakar Dec. 21, 2022, 12:31 a.m. UTC | #4 
Hi Conor,

On Mon, Dec 19, 2022 at 4:20 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Mon, Dec 19, 2022 at 11:19:13AM +0000, Lad, Prabhakar wrote:
> > Hi Conor,
> >
> > Thank you for the review.
> >
> > On Sat, Dec 17, 2022 at 9:19 PM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Mon, Dec 12, 2022 at 11:55:02AM +0000, Prabhakar wrote:
> > > > From: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > > >
> > > > Add required ports of the Alternative scheme for Andes CPU cores.
> > > >
> > > > I/O Coherence Port (IOCP) provides an AXI interface for connecting external
> > > > non-caching masters, such as DMA controllers. IOCP is a specification
> > > > option and is disabled on the Renesas RZ/Five SoC due to this reason cache
> > > > management needs a software workaround.
> > > >
> > > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > > > ---
> > > > v4 -> v5
> > > > * Sorted the Kconfig/Makefile/Switch based on Core name
> > > > * Added a comments
> > > > * Introduced RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT ID to check if
> > > >   CMO needs to be applied. Is there a way we can access the DTB while patching
> > > >   as we can drop this SBI EXT ID and add a DT property instead for cmo?
> > > >
<snip>
> > > Seeing as you need a new version for some of the other bits, I think it
> > > would be good to add a minor comment here somewhere (be it here or the
> > > commit message) that links to the SBI specs for this.
> > > I think this looks pretty good though.
> > Sure I'll add a comment here.
> >
> > I was wondering if we can get rid of this vendor specific extension
> > here if we get access to the DT here (for example having a DT property
> > which would indicate if IOCP CMO should be applied or not). Do you
> > think that would be good approach?  ATM we dont have a pointer here
> > for FDT whie early patching.
>
> I dunno. I think it is fine to use the ECALL to be honest - I'd rather
> that than a property that someone may omit.
>
Ok, I was so I will stick with the current implementation.

> That said, for the cache management stuff we are gonna need for
> PolarFire SoC, we will need to have info from the DT AFAICT - marchid
> etc are all set to zero on our platform so cannot be used.
>
Aha so while patching you will need a pointer to FDT node.

> I was thinking about using the compatible instead, but...
> we've not tried to "forward"-port our stuff from 5.15 yet as we have
> not yet completed testing testing on our vendor tree (and need some PCI
> changes accepted upstream first anyway), as a result I have not looked
> into what's needed there for use with alternatives. We've been using a
> pre-alternatives version of that patchset from around the 5.15
> development point in time instead.
>
Good to know. Let me know if you plan to implement the patching
mechanism based on FDT soon. I can give it a test.

Cheers,
Prabhakar