[v2,5/9] arm64: dts: ls1046a: make dma-coherent global to the SoC

These SoCs are really completely dma coherent in their entirety so add
the dma-coherent property at the soc level in the device tree and drop
the instances where it's specifically added to a few select devices.

Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
Signed-off-by: Li Yang <leoyang.li@nxp.com>
---
 arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

Hi All,

On 9/15/22 7:34 PM, Li Yang wrote:
> These SoCs are really completely dma coherent in their entirety so add
> the dma-coherent property at the soc level in the device tree and drop
> the instances where it's specifically added to a few select devices.
> 
> Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
> Signed-off-by: Li Yang <leoyang.li@nxp.com>
> ---
>  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> index 27033c558e3e..e406499a26b4 100644
> --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> @@ -273,6 +273,7 @@ soc: soc {
>  		#size-cells = <2>;
>  		ranges;
>  		dma-ranges = <0x0 0x0 0x0 0x0 0x10000 0x00000000>;
> +		dma-coherent;
>  
>  		ddr: memory-controller@1080000 {
>  			compatible = "fsl,qoriq-memory-controller";
> @@ -355,7 +356,6 @@ crypto: crypto@1700000 {
>  			ranges = <0x0 0x00 0x1700000 0x100000>;
>  			reg = <0x00 0x1700000 0x0 0x100000>;
>  			interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
> -			dma-coherent;
>  
>  			sec_jr0: jr@10000 {
>  				compatible = "fsl,sec-v5.4-job-ring",
> @@ -794,7 +794,6 @@ pcie1: pcie@3400000 {
>  			#address-cells = <3>;
>  			#size-cells = <2>;
>  			device_type = "pci";
> -			dma-coherent;
>  			num-viewport = <8>;
>  			bus-range = <0x0 0xff>;
>  			ranges = <0x81000000 0x0 0x00000000 0x40 0x00010000 0x0 0x00010000   /* downstream I/O */
> @@ -834,7 +833,6 @@ pcie2: pcie@3500000 {
>  			#address-cells = <3>;
>  			#size-cells = <2>;
>  			device_type = "pci";
> -			dma-coherent;
>  			num-viewport = <8>;
>  			bus-range = <0x0 0xff>;
>  			ranges = <0x81000000 0x0 0x00000000 0x48 0x00010000 0x0 0x00010000   /* downstream I/O */
> @@ -874,7 +872,6 @@ pcie3: pcie@3600000 {
>  			#address-cells = <3>;
>  			#size-cells = <2>;
>  			device_type = "pci";
> -			dma-coherent;
>  			num-viewport = <8>;
>  			bus-range = <0x0 0xff>;
>  			ranges = <0x81000000 0x0 0x00000000 0x50 0x00010000 0x0 0x00010000   /* downstream I/O */
> 

I'd like to summarize the conclusions of [1] below. This patch breaks
I2C0, which is the only user of eDMA at the moment. eDMA is noncoherent
because snooping is not enabled for it. I have submitted a patch [2] to
U-Boot to enable snooping for eDMA. For now, this patch must add
dma-noncoherent to the i2c0 node.

--Sean

[1] https://lore.kernel.org/linux-arm-kernel/29a09f45-24f9-68be-bdc2-8691af41b5db@seco.com/T/#t
[2] https://lore.kernel.org/u-boot/20220923160551.2200873-1-sean.anderson@seco.com/T/#u

> -----Original Message-----
> From: Sean Anderson <sean.anderson@seco.com>
> Sent: Friday, September 23, 2022 11:11 AM
> To: Leo Li <leoyang.li@nxp.com>; shawnguo@kernel.org;
> devicetree@vger.kernel.org
> Cc: robh+dt@kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> kernel@vger.kernel.org; Laurentiu Tudor <laurentiu.tudor@nxp.com>
> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent global
> to the SoC
> 
> 
> Hi All,
> 
> On 9/15/22 7:34 PM, Li Yang wrote:
> > These SoCs are really completely dma coherent in their entirety so add
> > the dma-coherent property at the soc level in the device tree and drop
> > the instances where it's specifically added to a few select devices.
> >
> > Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
> > Signed-off-by: Li Yang <leoyang.li@nxp.com>
> > ---
> >  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
> >  1 file changed, 1 insertion(+), 4 deletions(-)
> >
> > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> > b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> > index 27033c558e3e..e406499a26b4 100644
> > --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> > @@ -273,6 +273,7 @@ soc: soc {
> >  		#size-cells = <2>;
> >  		ranges;
> >  		dma-ranges = <0x0 0x0 0x0 0x0 0x10000 0x00000000>;
> > +		dma-coherent;
> >
> >  		ddr: memory-controller@1080000 {
> >  			compatible = "fsl,qoriq-memory-controller"; @@ -
> 355,7 +356,6 @@
> > crypto: crypto@1700000 {
> >  			ranges = <0x0 0x00 0x1700000 0x100000>;
> >  			reg = <0x00 0x1700000 0x0 0x100000>;
> >  			interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
> > -			dma-coherent;
> >
> >  			sec_jr0: jr@10000 {
> >  				compatible = "fsl,sec-v5.4-job-ring", @@ -
> 794,7 +794,6 @@ pcie1:
> > pcie@3400000 {
> >  			#address-cells = <3>;
> >  			#size-cells = <2>;
> >  			device_type = "pci";
> > -			dma-coherent;
> >  			num-viewport = <8>;
> >  			bus-range = <0x0 0xff>;
> >  			ranges = <0x81000000 0x0 0x00000000 0x40
> 0x00010000 0x0 0x00010000   /* downstream I/O */
> > @@ -834,7 +833,6 @@ pcie2: pcie@3500000 {
> >  			#address-cells = <3>;
> >  			#size-cells = <2>;
> >  			device_type = "pci";
> > -			dma-coherent;
> >  			num-viewport = <8>;
> >  			bus-range = <0x0 0xff>;
> >  			ranges = <0x81000000 0x0 0x00000000 0x48
> 0x00010000 0x0 0x00010000   /* downstream I/O */
> > @@ -874,7 +872,6 @@ pcie3: pcie@3600000 {
> >  			#address-cells = <3>;
> >  			#size-cells = <2>;
> >  			device_type = "pci";
> > -			dma-coherent;
> >  			num-viewport = <8>;
> >  			bus-range = <0x0 0xff>;
> >  			ranges = <0x81000000 0x0 0x00000000 0x50
> 0x00010000 0x0 0x00010000   /* downstream I/O */
> >
> 
> I'd like to summarize the conclusions of [1] below. This patch breaks I2C0,
> which is the only user of eDMA at the moment. eDMA is noncoherent
> because snooping is not enabled for it. I have submitted a patch [2] to U-
> Boot to enable snooping for eDMA. For now, this patch must add dma-
> noncoherent to the i2c0 node.

I have sent a V3 yesterday to set dma-noncoherent on edma node.  But are you saying that the dma-noncoherent need to be added to the i2c node to make it work?

For the u-boot patch, I will check with the hardware team to see if it is safe to set the reserved bit for edma snooping.  There is a problem with this is that it breaks the i2c for older u-boot.  Probably the best way is to make the default to be non-coherent in dts and update it in u-boot when snooping is enabled?

Regards,
Leo

On 9/23/22 12:26 PM, Leo Li wrote:
> 
> 
>> -----Original Message-----
>> From: Sean Anderson <sean.anderson@seco.com>
>> Sent: Friday, September 23, 2022 11:11 AM
>> To: Leo Li <leoyang.li@nxp.com>; shawnguo@kernel.org;
>> devicetree@vger.kernel.org
>> Cc: robh+dt@kernel.org; linux-arm-kernel@lists.infradead.org; linux-
>> kernel@vger.kernel.org; Laurentiu Tudor <laurentiu.tudor@nxp.com>
>> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent global
>> to the SoC
>> 
>> 
>> Hi All,
>> 
>> On 9/15/22 7:34 PM, Li Yang wrote:
>> > These SoCs are really completely dma coherent in their entirety so add
>> > the dma-coherent property at the soc level in the device tree and drop
>> > the instances where it's specifically added to a few select devices.
>> >
>> > Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
>> > Signed-off-by: Li Yang <leoyang.li@nxp.com>
>> > ---
>> >  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
>> >  1 file changed, 1 insertion(+), 4 deletions(-)
>> >
>> > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> > b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> > index 27033c558e3e..e406499a26b4 100644
>> > --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> > @@ -273,6 +273,7 @@ soc: soc {
>> >  		#size-cells = <2>;
>> >  		ranges;
>> >  		dma-ranges = <0x0 0x0 0x0 0x0 0x10000 0x00000000>;
>> > +		dma-coherent;
>> >
>> >  		ddr: memory-controller@1080000 {
>> >  			compatible = "fsl,qoriq-memory-controller"; @@ -
>> 355,7 +356,6 @@
>> > crypto: crypto@1700000 {
>> >  			ranges = <0x0 0x00 0x1700000 0x100000>;
>> >  			reg = <0x00 0x1700000 0x0 0x100000>;
>> >  			interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
>> > -			dma-coherent;
>> >
>> >  			sec_jr0: jr@10000 {
>> >  				compatible = "fsl,sec-v5.4-job-ring", @@ -
>> 794,7 +794,6 @@ pcie1:
>> > pcie@3400000 {
>> >  			#address-cells = <3>;
>> >  			#size-cells = <2>;
>> >  			device_type = "pci";
>> > -			dma-coherent;
>> >  			num-viewport = <8>;
>> >  			bus-range = <0x0 0xff>;
>> >  			ranges = <0x81000000 0x0 0x00000000 0x40
>> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> > @@ -834,7 +833,6 @@ pcie2: pcie@3500000 {
>> >  			#address-cells = <3>;
>> >  			#size-cells = <2>;
>> >  			device_type = "pci";
>> > -			dma-coherent;
>> >  			num-viewport = <8>;
>> >  			bus-range = <0x0 0xff>;
>> >  			ranges = <0x81000000 0x0 0x00000000 0x48
>> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> > @@ -874,7 +872,6 @@ pcie3: pcie@3600000 {
>> >  			#address-cells = <3>;
>> >  			#size-cells = <2>;
>> >  			device_type = "pci";
>> > -			dma-coherent;
>> >  			num-viewport = <8>;
>> >  			bus-range = <0x0 0xff>;
>> >  			ranges = <0x81000000 0x0 0x00000000 0x50
>> 0x00010000 0x0 0x00010000   /* downstream I/O */
>> >
>> 
>> I'd like to summarize the conclusions of [1] below. This patch breaks I2C0,
>> which is the only user of eDMA at the moment. eDMA is noncoherent
>> because snooping is not enabled for it. I have submitted a patch [2] to U-
>> Boot to enable snooping for eDMA. For now, this patch must add dma-
>> noncoherent to the i2c0 node.
> 
> I have sent a V3 yesterday to set dma-noncoherent on edma node.  But are you saying that the dma-noncoherent need to be added to the i2c node to make it work?

I believe dma coherency is a property of the consumer, not the provider. See
e.g. really_probe/platform_dma_configure/of_dma_configure/of_dma_is_coherent.

> For the u-boot patch, I will check with the hardware team to see if it is safe to set the reserved bit for edma snooping.

Thanks. I'm curious as to whether this omission is intentional or not.

> There is a problem with this is that it breaks the i2c for older u-boot.  Probably the best way is to make the default to be non-coherent in dts and update it in u-boot when snooping is enabled?

Yes, that is what I propose.

--Sean

> -----Original Message-----
> From: Sean Anderson <sean.anderson@seco.com>
> Sent: Friday, September 23, 2022 11:35 AM
> To: Leo Li <leoyang.li@nxp.com>; shawnguo@kernel.org;
> devicetree@vger.kernel.org
> Cc: robh+dt@kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> kernel@vger.kernel.org; Laurentiu Tudor <laurentiu.tudor@nxp.com>
> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent global
> to the SoC
> 
> 
> 
> On 9/23/22 12:26 PM, Leo Li wrote:
> >
> >
> >> -----Original Message-----
> >> From: Sean Anderson <sean.anderson@seco.com>
> >> Sent: Friday, September 23, 2022 11:11 AM
> >> To: Leo Li <leoyang.li@nxp.com>; shawnguo@kernel.org;
> >> devicetree@vger.kernel.org
> >> Cc: robh+dt@kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> >> kernel@vger.kernel.org; Laurentiu Tudor <laurentiu.tudor@nxp.com>
> >> Subject: Re: [PATCH v2 5/9] arm64: dts: ls1046a: make dma-coherent
> >> global to the SoC
> >>
> >>
> >> Hi All,
> >>
> >> On 9/15/22 7:34 PM, Li Yang wrote:
> >> > These SoCs are really completely dma coherent in their entirety so
> >> > add the dma-coherent property at the soc level in the device tree
> >> > and drop the instances where it's specifically added to a few select
> devices.
> >> >
> >> > Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
> >> > Signed-off-by: Li Yang <leoyang.li@nxp.com>
> >> > ---
> >> >  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +----
> >> >  1 file changed, 1 insertion(+), 4 deletions(-)
> >> >
> >> > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> >> > b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> >> > index 27033c558e3e..e406499a26b4 100644
> >> > --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> >> > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> >> > @@ -273,6 +273,7 @@ soc: soc {
> >> >  		#size-cells = <2>;
> >> >  		ranges;
> >> >  		dma-ranges = <0x0 0x0 0x0 0x0 0x10000 0x00000000>;
> >> > +		dma-coherent;
> >> >
> >> >  		ddr: memory-controller@1080000 {
> >> >  			compatible = "fsl,qoriq-memory-controller"; @@ -
> >> 355,7 +356,6 @@
> >> > crypto: crypto@1700000 {
> >> >  			ranges = <0x0 0x00 0x1700000 0x100000>;
> >> >  			reg = <0x00 0x1700000 0x0 0x100000>;
> >> >  			interrupts = <GIC_SPI 75 IRQ_TYPE_LEVEL_HIGH>;
> >> > -			dma-coherent;
> >> >
> >> >  			sec_jr0: jr@10000 {
> >> >  				compatible = "fsl,sec-v5.4-job-ring", @@ -
> >> 794,7 +794,6 @@ pcie1:
> >> > pcie@3400000 {
> >> >  			#address-cells = <3>;
> >> >  			#size-cells = <2>;
> >> >  			device_type = "pci";
> >> > -			dma-coherent;
> >> >  			num-viewport = <8>;
> >> >  			bus-range = <0x0 0xff>;
> >> >  			ranges = <0x81000000 0x0 0x00000000 0x40
> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
> >> > @@ -834,7 +833,6 @@ pcie2: pcie@3500000 {
> >> >  			#address-cells = <3>;
> >> >  			#size-cells = <2>;
> >> >  			device_type = "pci";
> >> > -			dma-coherent;
> >> >  			num-viewport = <8>;
> >> >  			bus-range = <0x0 0xff>;
> >> >  			ranges = <0x81000000 0x0 0x00000000 0x48
> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
> >> > @@ -874,7 +872,6 @@ pcie3: pcie@3600000 {
> >> >  			#address-cells = <3>;
> >> >  			#size-cells = <2>;
> >> >  			device_type = "pci";
> >> > -			dma-coherent;
> >> >  			num-viewport = <8>;
> >> >  			bus-range = <0x0 0xff>;
> >> >  			ranges = <0x81000000 0x0 0x00000000 0x50
> >> 0x00010000 0x0 0x00010000   /* downstream I/O */
> >> >
> >>
> >> I'd like to summarize the conclusions of [1] below. This patch breaks
> >> I2C0, which is the only user of eDMA at the moment. eDMA is
> >> noncoherent because snooping is not enabled for it. I have submitted
> >> a patch [2] to U- Boot to enable snooping for eDMA. For now, this
> >> patch must add dma- noncoherent to the i2c0 node.
> >
> > I have sent a V3 yesterday to set dma-noncoherent on edma node.  But are
> you saying that the dma-noncoherent need to be added to the i2c node to
> make it work?
> 
> I believe dma coherency is a property of the consumer, not the provider. See
> e.g.
> really_probe/platform_dma_configure/of_dma_configure/of_dma_is_cohe
> rent.

You are probably right.  The provider dma driver only maps the MMIO register while the consumer i2c driver maps the data buffer in memory which needs correct coherency setting.

> 
> > For the u-boot patch, I will check with the hardware team to see if it is safe
> to set the reserved bit for edma snooping.
> 
> Thanks. I'm curious as to whether this omission is intentional or not.

I have asked around but no one seems to know why LS1046a is different.  So from your experiment, changing the reserved bit does make a impact on the final result?  And setting it together with i2c nodes having dma-coherent enabled make it work.  Right?  If that's the case, probably it is just a documentation issue that we should fix.

- Leo