mbox series

[RFC,0/6] spi: Add OSPI PHY calibration support for spi-cadence-quadspi

Message ID 20210311191216.7363-1-p.yadav@ti.com
Headers show
Series spi: Add OSPI PHY calibration support for spi-cadence-quadspi | expand

Message

Pratyush Yadav March 11, 2021, 7:12 p.m. UTC
Hi,

This series adds support for OSPI PHY calibration on the Cadence OSPI
controller. This calibration procedure is needed to allow high clock
speeds in 8D-8D-8D mode. The procedure reads some pre-determined pattern
data from the flash and runs a sequence of test reads to find out the
optimal delays for high speed transfer. More details on the calibration
procedure in patch 5/6.

The main problem here is telling the controller where to find the
pattern and how to read it. This RFC uses nvmem cells which point to a
fixed partition containing the data to do the reads. It depends on [0]
and [1].

The obvious problem with this is it won't work when the partitions are
defined via command line. I don't see any good way to add nvmem cells to
command line partitions. I would like some help or ideas here. We don't
necessarily have to use nvmem either. Any way that can cleanly and
consistently let the controller find out where the pattern is stored is
good.

The dts patch depends on [2].

Tested on TI's J721E EVM.

[0] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210302190012.1255-1-zajec5@gmail.com/
[1] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210308011853.19360-1-ansuelsmth@gmail.com/
[2] https://patchwork.kernel.org/project/linux-arm-kernel/patch/20210305153926.3479-2-p.yadav@ti.com/

Pratyush Yadav (6):
  spi: spi-mem: Tell controller when device is ready for calibration
  mtd: spi-nor: core: consolidate read op creation
  mtd: spi-nor: core: run calibration when initialization is done
  spi: cadence-qspi: Use PHY for DAC reads if possible
  spi: cadence-qspi: Tune PHY to allow running at higher frequencies
  arm64: dts: ti: k3-j721e-som-p0: Enable PHY calibration

 arch/arm64/boot/dts/ti/k3-j721e-som-p0.dtsi |  55 ++
 drivers/mtd/spi-nor/core.c                  |  74 +-
 drivers/spi/spi-cadence-quadspi.c           | 820 +++++++++++++++++++-
 drivers/spi/spi-mem.c                       |  12 +
 include/linux/spi/spi-mem.h                 |   8 +
 5 files changed, 916 insertions(+), 53 deletions(-)

--
2.30.0

Comments

Tudor Ambarus March 12, 2021, 9:09 a.m. UTC | #1
On 3/11/21 9:12 PM, Pratyush Yadav wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

> 

> Hi,

> 

> This series adds support for OSPI PHY calibration on the Cadence OSPI

> controller. This calibration procedure is needed to allow high clock

> speeds in 8D-8D-8D mode. The procedure reads some pre-determined pattern

> data from the flash and runs a sequence of test reads to find out the

> optimal delays for high speed transfer. More details on the calibration

> procedure in patch 5/6.


Can the calibration sequence be avoided if the controller is informed
about the frequency on which the flash operates?

Can you add more details about the optimal delays? Are we talking about
flash's AC characteristics? Is the calibration still needed if the upper
layer informs the QSPI controller about the needed delays?

Cheers,
ta

> 

> The main problem here is telling the controller where to find the

> pattern and how to read it. This RFC uses nvmem cells which point to a

> fixed partition containing the data to do the reads. It depends on [0]

> and [1].

> 

> The obvious problem with this is it won't work when the partitions are

> defined via command line. I don't see any good way to add nvmem cells to

> command line partitions. I would like some help or ideas here. We don't

> necessarily have to use nvmem either. Any way that can cleanly and

> consistently let the controller find out where the pattern is stored is

> good.

> 

> The dts patch depends on [2].

> 

> Tested on TI's J721E EVM.

> 

> [0] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210302190012.1255-1-zajec5@gmail.com/

> [1] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210308011853.19360-1-ansuelsmth@gmail.com/

> [2] https://patchwork.kernel.org/project/linux-arm-kernel/patch/20210305153926.3479-2-p.yadav@ti.com/

> 

> Pratyush Yadav (6):

>   spi: spi-mem: Tell controller when device is ready for calibration

>   mtd: spi-nor: core: consolidate read op creation

>   mtd: spi-nor: core: run calibration when initialization is done

>   spi: cadence-qspi: Use PHY for DAC reads if possible

>   spi: cadence-qspi: Tune PHY to allow running at higher frequencies

>   arm64: dts: ti: k3-j721e-som-p0: Enable PHY calibration

> 

>  arch/arm64/boot/dts/ti/k3-j721e-som-p0.dtsi |  55 ++

>  drivers/mtd/spi-nor/core.c                  |  74 +-

>  drivers/spi/spi-cadence-quadspi.c           | 820 +++++++++++++++++++-

>  drivers/spi/spi-mem.c                       |  12 +

>  include/linux/spi/spi-mem.h                 |   8 +

>  5 files changed, 916 insertions(+), 53 deletions(-)

> 

> --

> 2.30.0

> 

> 

> _______________________________________________

> linux-arm-kernel mailing list

> linux-arm-kernel@lists.infradead.org

> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

>
Pratyush Yadav March 12, 2021, 10:10 a.m. UTC | #2
On 12/03/21 09:09AM, Tudor.Ambarus@microchip.com wrote:
> On 3/11/21 9:12 PM, Pratyush Yadav wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > 
> > Hi,
> > 
> > This series adds support for OSPI PHY calibration on the Cadence OSPI
> > controller. This calibration procedure is needed to allow high clock
> > speeds in 8D-8D-8D mode. The procedure reads some pre-determined pattern
> > data from the flash and runs a sequence of test reads to find out the
> > optimal delays for high speed transfer. More details on the calibration
> > procedure in patch 5/6.
> 
> Can the calibration sequence be avoided if the controller is informed
> about the frequency on which the flash operates?

Maybe I don't understand this correctly, but there should not be any 
frequency on which the flash operates. The controller drives the SPI 
clock so the frequency is decided by the controller. Sure, there is a 
max supported frequency for the flash but the controller can run it 
slower than that if it wishes. The flash has no say in that.

Anyway, the exact frequency at which the flash is running is not it is 
looking for. More details below.

> 
> Can you add more details about the optimal delays? Are we talking about
> flash's AC characteristics? Is the calibration still needed if the upper
> layer informs the QSPI controller about the needed delays?

There is usually a delay from when the flash drives the data line (IOW, 
puts a data bit on it) and when the signal reaches the controller. This 
delay can vary by the flash, board, silicon characteristics, 
temperature, etc.

At lower speeds (25 MHz for example) this delay is not a problem because 
the clock period is longer so there is much more time to sample the data 
line. It is very likely the controller will sample at a time when the 
data line is valid. At high speeds (166 MHz for example), especially in 
DDR mode, this delay starts to play a larger role because the time to 
sample the data line is much smaller. Now unless the delay is accounted 
for, it is possible that the controller samples the data line too late 
or too early and sees invalid data.

These delays depend on physical characteristics so it is not possible 
for any upper layer to inform the controller about it. How will they 
even know what the required delay is?

In summary, no, there is no way an upper layer can inform the controller 
about this delay.

> 
> Cheers,
> ta
> 
> > 
> > The main problem here is telling the controller where to find the
> > pattern and how to read it. This RFC uses nvmem cells which point to a
> > fixed partition containing the data to do the reads. It depends on [0]
> > and [1].
> > 
> > The obvious problem with this is it won't work when the partitions are
> > defined via command line. I don't see any good way to add nvmem cells to
> > command line partitions. I would like some help or ideas here. We don't
> > necessarily have to use nvmem either. Any way that can cleanly and
> > consistently let the controller find out where the pattern is stored is
> > good.
> > 
> > The dts patch depends on [2].
> > 
> > Tested on TI's J721E EVM.
> > 
> > [0] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210302190012.1255-1-zajec5@gmail.com/
> > [1] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210308011853.19360-1-ansuelsmth@gmail.com/
> > [2] https://patchwork.kernel.org/project/linux-arm-kernel/patch/20210305153926.3479-2-p.yadav@ti.com/
> > 
> > Pratyush Yadav (6):
> >   spi: spi-mem: Tell controller when device is ready for calibration
> >   mtd: spi-nor: core: consolidate read op creation
> >   mtd: spi-nor: core: run calibration when initialization is done
> >   spi: cadence-qspi: Use PHY for DAC reads if possible
> >   spi: cadence-qspi: Tune PHY to allow running at higher frequencies
> >   arm64: dts: ti: k3-j721e-som-p0: Enable PHY calibration
> > 
> >  arch/arm64/boot/dts/ti/k3-j721e-som-p0.dtsi |  55 ++
> >  drivers/mtd/spi-nor/core.c                  |  74 +-
> >  drivers/spi/spi-cadence-quadspi.c           | 820 +++++++++++++++++++-
> >  drivers/spi/spi-mem.c                       |  12 +
> >  include/linux/spi/spi-mem.h                 |   8 +
> >  5 files changed, 916 insertions(+), 53 deletions(-)
> > 
> > --
> > 2.30.0
> > 
> > 
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> > 
>
Michael Walle March 12, 2021, 10:20 a.m. UTC | #3
Am 2021-03-12 11:10, schrieb Pratyush Yadav:
> There is usually a delay from when the flash drives the data line (IOW,
> puts a data bit on it) and when the signal reaches the controller. This
> delay can vary by the flash, board, silicon characteristics,
> temperature, etc.

Temperature might change over time, but the calibration is only done
once. I don't know how much influence the temperature actually has, but
our boards are usually operating from -40°C to +85°C. So there might be
a possible temperature difference of 125K between actual calibration and
when the flash is accessed.

-michael
Pratyush Yadav March 12, 2021, 11:07 a.m. UTC | #4
On 12/03/21 11:20AM, Michael Walle wrote:
> Am 2021-03-12 11:10, schrieb Pratyush Yadav:
> > There is usually a delay from when the flash drives the data line (IOW,
> > puts a data bit on it) and when the signal reaches the controller. This
> > delay can vary by the flash, board, silicon characteristics,
> > temperature, etc.
> 
> Temperature might change over time, but the calibration is only done
> once. I don't know how much influence the temperature actually has, but
> our boards are usually operating from -40°C to +85°C. So there might be
> a possible temperature difference of 125K between actual calibration and
> when the flash is accessed.

The algorithm supports a temperature range of -45 C to +130 C. The 
temperature is checked at calibration time and adjustments are made to 
make sure the reads work over the entire temperature range [0].

[0] The current implementation does not have a way to query the 
temperature from the sensor (see cqspi_get_temp()), so it always assumes 
the temperature at calibration time is 45 C. But that can be added later 
once the temperature sensor driver is implemented.
Michael Walle March 12, 2021, 1:26 p.m. UTC | #5
Am 2021-03-12 12:07, schrieb Pratyush Yadav:
> On 12/03/21 11:20AM, Michael Walle wrote:
>> Am 2021-03-12 11:10, schrieb Pratyush Yadav:
>> > There is usually a delay from when the flash drives the data line (IOW,
>> > puts a data bit on it) and when the signal reaches the controller. This
>> > delay can vary by the flash, board, silicon characteristics,
>> > temperature, etc.
>> 
>> Temperature might change over time, but the calibration is only done
>> once. I don't know how much influence the temperature actually has, 
>> but
>> our boards are usually operating from -40°C to +85°C. So there might 
>> be
>> a possible temperature difference of 125K between actual calibration 
>> and
>> when the flash is accessed.
> 
> The algorithm supports a temperature range of -45 C to +130 C. The
> temperature is checked at calibration time and adjustments are made to
> make sure the reads work over the entire temperature range [0].

Ah, nice. And you need the current temperature to correlate it to the
meassured timings, right?

-michael
Tudor Ambarus March 12, 2021, 2:59 p.m. UTC | #6
On 3/12/21 3:32 PM, Michael Walle wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

> 

> Am 2021-03-11 20:12, schrieb Pratyush Yadav:

>> The main problem here is telling the controller where to find the

>> pattern and how to read it. This RFC uses nvmem cells which point to a

>> fixed partition containing the data to do the reads. It depends on [0]

>> and [1].

>>

>> The obvious problem with this is it won't work when the partitions are

>> defined via command line. I don't see any good way to add nvmem cells

>> to

>> command line partitions. I would like some help or ideas here. We don't

>> necessarily have to use nvmem either. Any way that can cleanly and

>> consistently let the controller find out where the pattern is stored is

>> good.

> 

> The NXP LS1028A SoC has a similar calibration (although there its done

> in hardware it seems) and there the datasheet mentions there are flash

> devices which supports a preamble before a read function. The preamble

> is then some kind of learning pattern. Did you see a flash which

> actually

> supports that in the wild? I can't find any publicly available


MX66LM1G45G is an example.

> datasheets> of 8bit I/O SPI NOR flashes
Pratyush Yadav March 12, 2021, 5 p.m. UTC | #7
On 12/03/21 02:32PM, Michael Walle wrote:
> Am 2021-03-11 20:12, schrieb Pratyush Yadav:

> > The main problem here is telling the controller where to find the

> > pattern and how to read it. This RFC uses nvmem cells which point to a

> > fixed partition containing the data to do the reads. It depends on [0]

> > and [1].

> > 

> > The obvious problem with this is it won't work when the partitions are

> > defined via command line. I don't see any good way to add nvmem cells to

> > command line partitions. I would like some help or ideas here. We don't

> > necessarily have to use nvmem either. Any way that can cleanly and

> > consistently let the controller find out where the pattern is stored is

> > good.

> 

> The NXP LS1028A SoC has a similar calibration (although there its done

> in hardware it seems) and there the datasheet mentions there are flash

> devices which supports a preamble before a read function. The preamble

> is then some kind of learning pattern. Did you see a flash which actually

> supports that in the wild? I can't find any publicly available datasheets

> of 8bit I/O SPI NOR flashes.


I haven't seen any such flash but it looks like Tudor has.

> 

> -michael


-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.
Pratyush Yadav March 12, 2021, 6:14 p.m. UTC | #8
On 12/03/21 11:23AM, Tudor.Ambarus@microchip.com wrote:
> On 3/12/21 12:10 PM, Pratyush Yadav wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > 
> > On 12/03/21 09:09AM, Tudor.Ambarus@microchip.com wrote:
> >> On 3/11/21 9:12 PM, Pratyush Yadav wrote:
> >>> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> >>>
> >>> Hi,
> >>>
> >>> This series adds support for OSPI PHY calibration on the Cadence OSPI
> >>> controller. This calibration procedure is needed to allow high clock
> >>> speeds in 8D-8D-8D mode. The procedure reads some pre-determined pattern
> >>> data from the flash and runs a sequence of test reads to find out the
> >>> optimal delays for high speed transfer. More details on the calibration
> >>> procedure in patch 5/6.
> >>
> >> Can the calibration sequence be avoided if the controller is informed
> >> about the frequency on which the flash operates?
> 
> s/frequency/maximum supported frequency by the flash

Again, the max frequency does not matter. If it is fast enough PHY 
calibration is needed to make sure lines are sampled at the correct 
time.

> 
> > 
> > Maybe I don't understand this correctly, but there should not be any
> > frequency on which the flash operates. The controller drives the SPI
> > clock so the frequency is decided by the controller. Sure, there is a
> > max supported frequency for the flash but the controller can run it
> > slower than that if it wishes. The flash has no say in that.
> > 
> > Anyway, the exact frequency at which the flash is running is not it is
> > looking for. More details below.
> 
> I thought about choosing at the controller side:
> min(max_frequency_controller, max_frequency_flash)
> 
> And there is also the need of changing the frequency on which an op
> runs, like the READ SFDP cmd, for which it is recommended to be run at
> 50 MHz, but maybe this is another topic, let's see.

Right. This is not directly related to the need for having the 
calibration.

Right now calibration is run only after the flash is fully initialized, 
so there should not be any SFDP commands from that point on. But if we 
do want to be conservative about it, a field can be added in spi_mem_op 
that mentions the maximum speed for the op so the controller can decide 
accordingly.

> 
> > 
> >>
> >> Can you add more details about the optimal delays? Are we talking about
> >> flash's AC characteristics? Is the calibration still needed if the upper
> >> layer informs the QSPI controller about the needed delays?
> > 
> > There is usually a delay from when the flash drives the data line (IOW,
> > puts a data bit on it) and when the signal reaches the controller. This
> > delay can vary by the flash, board, silicon characteristics,
> > temperature, etc.
> 
> I wonder whether the delay advertised by the flash matters the most, while
> all the other are negligible.

The delay advertised by the flash does matter. Specifically, the clock 
to data output delay. But the IO delay in the host (the delay from the 
pin to the internal FIFO) matters equally as much. Both are accounted 
for by the tuning.

> When I talk about delay, I'm thinking for example at the delay required
> between two consecutive transfers without removing the chip select, or about
> the minimum delay needed between the activation or the deactivation of the
> chip select. These are all described by the flash. Does your controller have
> such fields in its registers, to set such delays? If yes, is the calibration 
> sequence still needed if all the delays are set correctly?

The CS related delays are indeed accounted for by a register in the 
controller. But this is not the delay the calibration is concerned with. 
The delays the calibration is concerned with are the clock edge to data 
transition delay (TX delay), and the DS edge to data transition delay 
(RX delay). The two are totally unrelated.

> 
> When I hear about "board delays", I think about the impedance of the lines,
> which should correspond to the impedance of the Flash's IOs (which depends on
> the frequency on which the flash runs). A mechanism to choose the best
> frequency and impedance level can be added.

Board delays in this case are caused by the length of the wires/lines. 
Even if the lines are perfectly impedance matched to the flash's IOs, 
there will be a small time delay from when the data signal is launched 
by the flash and when it is received by the device. This causes a small 
but noticeable difference in the timing and consequently the final 
calibration values. For example, this is observed when comparing J721E 
EVM (evaluation module) and SVB (silicon validation board) platforms.

> 
> Flashes have an interval of temperature on which they are guaranteed to
> work (I would expect in the same conditions). Information about temperature
> ranges and associated delays (if measured?) can be passed too.

This would not be sufficient for placing TX and RX delays unless we have 
a perfect model for both the flash and the host device IO delays. Such a 
model would have to account for variations in timing caused by 
variations in the manufacturing process, voltage, and temperature. This 
is not practically feasible.

> 
> Cheers,
> ta
> > 
> > At lower speeds (25 MHz for example) this delay is not a problem because
> > the clock period is longer so there is much more time to sample the data
> > line. It is very likely the controller will sample at a time when the
> > data line is valid. At high speeds (166 MHz for example), especially in
> > DDR mode, this delay starts to play a larger role because the time to
> > sample the data line is much smaller. Now unless the delay is accounted
> > for, it is possible that the controller samples the data line too late
> > or too early and sees invalid data.
> > 
> > These delays depend on physical characteristics so it is not possible
> > for any upper layer to inform the controller about it. How will they
> > even know what the required delay is?
> > 
> > In summary, no, there is no way an upper layer can inform the controller
> > about this delay.
> > 
> >>
> >> Cheers,
> >> ta
> >>
> >>>
> >>> The main problem here is telling the controller where to find the
> >>> pattern and how to read it. This RFC uses nvmem cells which point to a
> >>> fixed partition containing the data to do the reads. It depends on [0]
> >>> and [1].
> >>>
> >>> The obvious problem with this is it won't work when the partitions are
> >>> defined via command line. I don't see any good way to add nvmem cells to
> >>> command line partitions. I would like some help or ideas here. We don't
> >>> necessarily have to use nvmem either. Any way that can cleanly and
> >>> consistently let the controller find out where the pattern is stored is
> >>> good.
> >>>
> >>> The dts patch depends on [2].
> >>>
> >>> Tested on TI's J721E EVM.
> >>>
> >>> [0] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210302190012.1255-1-zajec5@gmail.com/
> >>> [1] https://patchwork.ozlabs.org/project/linux-mtd/patch/20210308011853.19360-1-ansuelsmth@gmail.com/
> >>> [2] https://patchwork.kernel.org/project/linux-arm-kernel/patch/20210305153926.3479-2-p.yadav@ti.com/
> >>>
> >>> Pratyush Yadav (6):
> >>>   spi: spi-mem: Tell controller when device is ready for calibration
> >>>   mtd: spi-nor: core: consolidate read op creation
> >>>   mtd: spi-nor: core: run calibration when initialization is done
> >>>   spi: cadence-qspi: Use PHY for DAC reads if possible
> >>>   spi: cadence-qspi: Tune PHY to allow running at higher frequencies
> >>>   arm64: dts: ti: k3-j721e-som-p0: Enable PHY calibration
> >>>
> >>>  arch/arm64/boot/dts/ti/k3-j721e-som-p0.dtsi |  55 ++
> >>>  drivers/mtd/spi-nor/core.c                  |  74 +-
> >>>  drivers/spi/spi-cadence-quadspi.c           | 820 +++++++++++++++++++-
> >>>  drivers/spi/spi-mem.c                       |  12 +
> >>>  include/linux/spi/spi-mem.h                 |   8 +
> >>>  5 files changed, 916 insertions(+), 53 deletions(-)
> >>>
Michael Walle March 23, 2021, 11:07 p.m. UTC | #9
Hi Pratyush,

Am 2021-03-11 20:12, schrieb Pratyush Yadav:
> Some controllers like the Cadence OSPI controller need to perform a

> calibration sequence to operate at high clock speeds. This calibration

> should happen after the flash is fully initialized otherwise the

> calibration might happen in a different SPI mode from the one the flash

> is finally set to. Add a hook that can be used to tell the controller

> when the flash is ready for calibration. Whether calibration is needed

> depends on the controller.

> 

> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>

> ---

>  drivers/spi/spi-mem.c       | 12 ++++++++++++

>  include/linux/spi/spi-mem.h |  8 ++++++++

>  2 files changed, 20 insertions(+)

> 

> diff --git a/drivers/spi/spi-mem.c b/drivers/spi/spi-mem.c

> index dc713b0c3c4d..e2f05ad3f4dc 100644

> --- a/drivers/spi/spi-mem.c

> +++ b/drivers/spi/spi-mem.c

> @@ -464,6 +464,18 @@ int spi_mem_adjust_op_size(struct spi_mem *mem,

> struct spi_mem_op *op)

>  }

>  EXPORT_SYMBOL_GPL(spi_mem_adjust_op_size);

> 

> +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op)

> +{

> +	struct spi_controller *ctlr = mem->spi->controller;

> +

> +	if (!ctlr->mem_ops || !ctlr->mem_ops->do_calibration)

> +		return -EOPNOTSUPP;

> +

> +	ctlr->mem_ops->do_calibration(mem, op);


Can't a calibration fail?

> +	return 0;

> +}

> +EXPORT_SYMBOL_GPL(spi_mem_do_calibration);

> +

>  static ssize_t spi_mem_no_dirmap_read(struct spi_mem_dirmap_desc 

> *desc,

>  				      u64 offs, size_t len, void *buf)

>  {

> diff --git a/include/linux/spi/spi-mem.h b/include/linux/spi/spi-mem.h

> index 2b65c9edc34e..97a2d280f2d0 100644

> --- a/include/linux/spi/spi-mem.h

> +++ b/include/linux/spi/spi-mem.h

> @@ -250,6 +250,12 @@ static inline void *spi_mem_get_drvdata(struct

> spi_mem *mem)

>   *		  the currently mapped area), and the caller of

>   *		  spi_mem_dirmap_write() is responsible for calling it again in

>   *		  this case.

> + * @do_calibration: perform calibration needed for high SPI clock 

> speed

> + *		    operation. Should be called after the SPI memory device has

> + *		    been completely initialized. The op passed should contain

> + *		    a template for the read operation used for the device so

> + *		    the controller can decide what type of calibration is

> + *		    required for this type of read.

>   *

>   * This interface should be implemented by SPI controllers providing 

> an

>   * high-level interface to execute SPI memory operation, which is 

> usually the

> @@ -274,6 +280,7 @@ struct spi_controller_mem_ops {

>  			       u64 offs, size_t len, void *buf);

>  	ssize_t (*dirmap_write)(struct spi_mem_dirmap_desc *desc,

>  				u64 offs, size_t len, const void *buf);

> +	void (*do_calibration)(struct spi_mem *mem, struct spi_mem_op *op);

>  };

> 

>  /**

> @@ -346,6 +353,7 @@ bool spi_mem_dtr_supports_op(struct spi_mem *mem,

>  #endif /* CONFIG_SPI_MEM */

> 

>  int spi_mem_adjust_op_size(struct spi_mem *mem, struct spi_mem_op 

> *op);

> +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op 

> *op);

> 

>  bool spi_mem_supports_op(struct spi_mem *mem,

>  			 const struct spi_mem_op *op);


-- 
-michael
Pratyush Yadav March 24, 2021, 8:08 a.m. UTC | #10
On 24/03/21 12:07AM, Michael Walle wrote:
> Hi Pratyush,

> 

> Am 2021-03-11 20:12, schrieb Pratyush Yadav:

> > Some controllers like the Cadence OSPI controller need to perform a

> > calibration sequence to operate at high clock speeds. This calibration

> > should happen after the flash is fully initialized otherwise the

> > calibration might happen in a different SPI mode from the one the flash

> > is finally set to. Add a hook that can be used to tell the controller

> > when the flash is ready for calibration. Whether calibration is needed

> > depends on the controller.

> > 

> > Signed-off-by: Pratyush Yadav <p.yadav@ti.com>

> > ---

> >  drivers/spi/spi-mem.c       | 12 ++++++++++++

> >  include/linux/spi/spi-mem.h |  8 ++++++++

> >  2 files changed, 20 insertions(+)

> > 

> > diff --git a/drivers/spi/spi-mem.c b/drivers/spi/spi-mem.c

> > index dc713b0c3c4d..e2f05ad3f4dc 100644

> > --- a/drivers/spi/spi-mem.c

> > +++ b/drivers/spi/spi-mem.c

> > @@ -464,6 +464,18 @@ int spi_mem_adjust_op_size(struct spi_mem *mem,

> > struct spi_mem_op *op)

> >  }

> >  EXPORT_SYMBOL_GPL(spi_mem_adjust_op_size);

> > 

> > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op)

> > +{

> > +	struct spi_controller *ctlr = mem->spi->controller;

> > +

> > +	if (!ctlr->mem_ops || !ctlr->mem_ops->do_calibration)

> > +		return -EOPNOTSUPP;

> > +

> > +	ctlr->mem_ops->do_calibration(mem, op);

> 

> Can't a calibration fail?


It can. If it does, the controller falls back to lower speed transfers. 
There is not much the upper layer can do about this. That's why it is 
not informed whether it succeeded or not.

> 

> > +	return 0;

> > +}

> > +EXPORT_SYMBOL_GPL(spi_mem_do_calibration);

> > +

> >  static ssize_t spi_mem_no_dirmap_read(struct spi_mem_dirmap_desc *desc,

> >  				      u64 offs, size_t len, void *buf)

> >  {

> > diff --git a/include/linux/spi/spi-mem.h b/include/linux/spi/spi-mem.h

> > index 2b65c9edc34e..97a2d280f2d0 100644

> > --- a/include/linux/spi/spi-mem.h

> > +++ b/include/linux/spi/spi-mem.h

> > @@ -250,6 +250,12 @@ static inline void *spi_mem_get_drvdata(struct

> > spi_mem *mem)

> >   *		  the currently mapped area), and the caller of

> >   *		  spi_mem_dirmap_write() is responsible for calling it again in

> >   *		  this case.

> > + * @do_calibration: perform calibration needed for high SPI clock speed

> > + *		    operation. Should be called after the SPI memory device has

> > + *		    been completely initialized. The op passed should contain

> > + *		    a template for the read operation used for the device so

> > + *		    the controller can decide what type of calibration is

> > + *		    required for this type of read.

> >   *

> >   * This interface should be implemented by SPI controllers providing an

> >   * high-level interface to execute SPI memory operation, which is

> > usually the

> > @@ -274,6 +280,7 @@ struct spi_controller_mem_ops {

> >  			       u64 offs, size_t len, void *buf);

> >  	ssize_t (*dirmap_write)(struct spi_mem_dirmap_desc *desc,

> >  				u64 offs, size_t len, const void *buf);

> > +	void (*do_calibration)(struct spi_mem *mem, struct spi_mem_op *op);

> >  };

> > 

> >  /**

> > @@ -346,6 +353,7 @@ bool spi_mem_dtr_supports_op(struct spi_mem *mem,

> >  #endif /* CONFIG_SPI_MEM */

> > 

> >  int spi_mem_adjust_op_size(struct spi_mem *mem, struct spi_mem_op *op);

> > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op);

> > 

> >  bool spi_mem_supports_op(struct spi_mem *mem,

> >  			 const struct spi_mem_op *op);

> 

> -- 

> -michael


-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.
Michael Walle April 29, 2021, 4:23 p.m. UTC | #11
Hi Pratyush,

I've had a look at the LS1028A FlexSPI calibration feature. The
reference manual is very sparse on details, though. What you need to
do there is to program a special read command sequence (the whole
controller is made of these lookup table entries, where you can
have a short sequence of operations for read/write/program and so
on). Therefore, for data learning you'll take the read operation
and insert a LEARN op in between and read a specific data pattern.
Then the hardware will automatically figure out the correct sample
phase for the read data pins.

Unfortunately, it does not mention how often you have to do it. It
might be the case that is has to be calibrated more than once.

I'm just mentioning this so it won't be lost. If needed, it can
be added later.

Am 2021-03-24 09:08, schrieb Pratyush Yadav:
> On 24/03/21 12:07AM, Michael Walle wrote:

>> Am 2021-03-11 20:12, schrieb Pratyush Yadav:

>> > Some controllers like the Cadence OSPI controller need to perform a

>> > calibration sequence to operate at high clock speeds. This calibration

>> > should happen after the flash is fully initialized otherwise the

>> > calibration might happen in a different SPI mode from the one the flash

>> > is finally set to. Add a hook that can be used to tell the controller

>> > when the flash is ready for calibration. Whether calibration is needed

>> > depends on the controller.

>> >

>> > Signed-off-by: Pratyush Yadav <p.yadav@ti.com>

>> > ---

>> >  drivers/spi/spi-mem.c       | 12 ++++++++++++

>> >  include/linux/spi/spi-mem.h |  8 ++++++++

>> >  2 files changed, 20 insertions(+)

>> >

>> > diff --git a/drivers/spi/spi-mem.c b/drivers/spi/spi-mem.c

>> > index dc713b0c3c4d..e2f05ad3f4dc 100644

>> > --- a/drivers/spi/spi-mem.c

>> > +++ b/drivers/spi/spi-mem.c

>> > @@ -464,6 +464,18 @@ int spi_mem_adjust_op_size(struct spi_mem *mem,

>> > struct spi_mem_op *op)

>> >  }

>> >  EXPORT_SYMBOL_GPL(spi_mem_adjust_op_size);

>> >

>> > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op)

>> > +{

>> > +	struct spi_controller *ctlr = mem->spi->controller;

>> > +

>> > +	if (!ctlr->mem_ops || !ctlr->mem_ops->do_calibration)

>> > +		return -EOPNOTSUPP;

>> > +

>> > +	ctlr->mem_ops->do_calibration(mem, op);

>> 

>> Can't a calibration fail?

> 

> It can. If it does, the controller falls back to lower speed transfers.

> There is not much the upper layer can do about this. That's why it is

> not informed whether it succeeded or not.


Ok, if needed, that should be an easy change.

op is there to decide if we need a calibration at all, correct?
What if there are different factors, like frequency? For example
on the LS1028A its just a matter of the SCK frequency. It seems
that this parameter is tailored to the OPHY.

-michael

>> > +	return 0;

>> > +}

>> > +EXPORT_SYMBOL_GPL(spi_mem_do_calibration);

>> > +

>> >  static ssize_t spi_mem_no_dirmap_read(struct spi_mem_dirmap_desc *desc,

>> >  				      u64 offs, size_t len, void *buf)

>> >  {

>> > diff --git a/include/linux/spi/spi-mem.h b/include/linux/spi/spi-mem.h

>> > index 2b65c9edc34e..97a2d280f2d0 100644

>> > --- a/include/linux/spi/spi-mem.h

>> > +++ b/include/linux/spi/spi-mem.h

>> > @@ -250,6 +250,12 @@ static inline void *spi_mem_get_drvdata(struct

>> > spi_mem *mem)

>> >   *		  the currently mapped area), and the caller of

>> >   *		  spi_mem_dirmap_write() is responsible for calling it again in

>> >   *		  this case.

>> > + * @do_calibration: perform calibration needed for high SPI clock speed

>> > + *		    operation. Should be called after the SPI memory device has

>> > + *		    been completely initialized. The op passed should contain

>> > + *		    a template for the read operation used for the device so

>> > + *		    the controller can decide what type of calibration is

>> > + *		    required for this type of read.

>> >   *

>> >   * This interface should be implemented by SPI controllers providing an

>> >   * high-level interface to execute SPI memory operation, which is

>> > usually the

>> > @@ -274,6 +280,7 @@ struct spi_controller_mem_ops {

>> >  			       u64 offs, size_t len, void *buf);

>> >  	ssize_t (*dirmap_write)(struct spi_mem_dirmap_desc *desc,

>> >  				u64 offs, size_t len, const void *buf);

>> > +	void (*do_calibration)(struct spi_mem *mem, struct spi_mem_op *op);

>> >  };

>> >

>> >  /**

>> > @@ -346,6 +353,7 @@ bool spi_mem_dtr_supports_op(struct spi_mem *mem,

>> >  #endif /* CONFIG_SPI_MEM */

>> >

>> >  int spi_mem_adjust_op_size(struct spi_mem *mem, struct spi_mem_op *op);

>> > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op);

>> >

>> >  bool spi_mem_supports_op(struct spi_mem *mem,

>> >  			 const struct spi_mem_op *op);

>>
Pratyush Yadav April 29, 2021, 6:19 p.m. UTC | #12
On 29/04/21 06:28PM, Michael Walle wrote:
> Am 2021-03-12 11:17, schrieb Pratyush Yadav:
> > On 12/03/21 09:13AM, Tudor.Ambarus@microchip.com wrote:
> > > On 3/11/21 9:12 PM, Pratyush Yadav wrote:
> > > > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > > >
> > > > Check if a read is eligible for PHY and if it is, enable PHY and DQS.
> > > 
> > > DQS as in data strobe? Shouldn't the upper layer inform the QSPI
> > > controller
> > > whether DS is required or not?
> > 
> > Yes, DQS as in data strobe. I need to check this again, but IIRC the
> > controller cannot run in PHY mode unless DS is used. Ideally the upper
> > layer should indeed inform the controller whether DS is supported/in-use
> > or not. That can be used to decide whether PHY mode (and consequently
> > the DS line) is to be used or not.
> > 
> > Currently there are only two flashes that use 8D-8D-8D mode (S28HS512T
> > and MT35XU512ABA), and both of them drive the DS line.
> 
> The LS1028A datasheet explicitly states that the calibration is only
> used for non-DQS flashes. Which makes sense, because it just determine at
> which point the input data is sampled. And if the flash provides a data
> strobe, it already know when to sample it. What I am missing here?

If there was 0 delay in transferring the signals from flash to 
SoC/controller, you would be right. But in practice there is a small but 
noticeable delay from when the flash launches the signal and when it is 
received by the device. So by the time the DQS signal reaches the SoC it 
might already be too late and the data lines might not be valid any 
more. The calibration accounts for these (and some others) delays.

See [0] for a somewhat similar discussion I had with Tudor.

[0] https://lore.kernel.org/linux-mtd/20210312181447.dlecnw2oed7jtxe7@ti.com/
Pratyush Yadav April 29, 2021, 6:41 p.m. UTC | #13
On 29/04/21 06:23PM, Michael Walle wrote:
> Hi Pratyush,

> 

> I've had a look at the LS1028A FlexSPI calibration feature. The

> reference manual is very sparse on details, though. What you need to

> do there is to program a special read command sequence (the whole

> controller is made of these lookup table entries, where you can

> have a short sequence of operations for read/write/program and so

> on). Therefore, for data learning you'll take the read operation

> and insert a LEARN op in between and read a specific data pattern.

> Then the hardware will automatically figure out the correct sample

> phase for the read data pins.

> 

> Unfortunately, it does not mention how often you have to do it. It

> might be the case that is has to be calibrated more than once.


I haven't read the datasheet, I wonder how long this calibration takes. 
If it is too long then the overhead might not even be worth the extra 
read throughput. Especially when using a file system on top which 
generally don't do very large reads in one go.

Anyway, when the do_calibration() is called the controller can save the 
calibration op and use it later as needed. It knows when an exec_op() 
will result in a read since it has access to the whole op.

> 

> I'm just mentioning this so it won't be lost. If needed, it can

> be added later.

> 

> Am 2021-03-24 09:08, schrieb Pratyush Yadav:

> > On 24/03/21 12:07AM, Michael Walle wrote:

> > > Am 2021-03-11 20:12, schrieb Pratyush Yadav:

> > > > Some controllers like the Cadence OSPI controller need to perform a

> > > > calibration sequence to operate at high clock speeds. This calibration

> > > > should happen after the flash is fully initialized otherwise the

> > > > calibration might happen in a different SPI mode from the one the flash

> > > > is finally set to. Add a hook that can be used to tell the controller

> > > > when the flash is ready for calibration. Whether calibration is needed

> > > > depends on the controller.

> > > >

> > > > Signed-off-by: Pratyush Yadav <p.yadav@ti.com>

> > > > ---

> > > >  drivers/spi/spi-mem.c       | 12 ++++++++++++

> > > >  include/linux/spi/spi-mem.h |  8 ++++++++

> > > >  2 files changed, 20 insertions(+)

> > > >

> > > > diff --git a/drivers/spi/spi-mem.c b/drivers/spi/spi-mem.c

> > > > index dc713b0c3c4d..e2f05ad3f4dc 100644

> > > > --- a/drivers/spi/spi-mem.c

> > > > +++ b/drivers/spi/spi-mem.c

> > > > @@ -464,6 +464,18 @@ int spi_mem_adjust_op_size(struct spi_mem *mem,

> > > > struct spi_mem_op *op)

> > > >  }

> > > >  EXPORT_SYMBOL_GPL(spi_mem_adjust_op_size);

> > > >

> > > > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op)

> > > > +{

> > > > +	struct spi_controller *ctlr = mem->spi->controller;

> > > > +

> > > > +	if (!ctlr->mem_ops || !ctlr->mem_ops->do_calibration)

> > > > +		return -EOPNOTSUPP;

> > > > +

> > > > +	ctlr->mem_ops->do_calibration(mem, op);

> > > 

> > > Can't a calibration fail?

> > 

> > It can. If it does, the controller falls back to lower speed transfers.

> > There is not much the upper layer can do about this. That's why it is

> > not informed whether it succeeded or not.

> 

> Ok, if needed, that should be an easy change.

> 

> op is there to decide if we need a calibration at all, correct?


Yes. It can also be used to choose which calibration algorithm to use. 
For example on the Cadence controller, there are different algorithms 
for 8S and 8D operations.

> What if there are different factors, like frequency? For example

> on the LS1028A its just a matter of the SCK frequency. It seems

> that this parameter is tailored to the OPHY.


As of now there is no way in SPI MEM to tell the controller the expected 
speed of the operation. AFAIK most controllers get the speed via device 
tree. So in the current case, the controller already knows the speed it 
should run at, and can decide if calibration is needed or not.

But if operation speed is eventually added to SPI MEM, I would assume it 
would be part of struct spi_mem_op. The op passed in would have this 
information filled, and the controller can use that information to 
decide if it needs to perform the calibration or not.

I am all for making this API flexible, but with very few controllers 
supporting this feature in the wild, it is difficult to predict all the 
information that might be needed. In the current state, I think the API 
provides a fair bit of information to the controller about how a read 
operation would look like.

> 

> -michael

> 

> > > > +	return 0;

> > > > +}

> > > > +EXPORT_SYMBOL_GPL(spi_mem_do_calibration);

> > > > +

> > > >  static ssize_t spi_mem_no_dirmap_read(struct spi_mem_dirmap_desc *desc,

> > > >  				      u64 offs, size_t len, void *buf)

> > > >  {

> > > > diff --git a/include/linux/spi/spi-mem.h b/include/linux/spi/spi-mem.h

> > > > index 2b65c9edc34e..97a2d280f2d0 100644

> > > > --- a/include/linux/spi/spi-mem.h

> > > > +++ b/include/linux/spi/spi-mem.h

> > > > @@ -250,6 +250,12 @@ static inline void *spi_mem_get_drvdata(struct

> > > > spi_mem *mem)

> > > >   *		  the currently mapped area), and the caller of

> > > >   *		  spi_mem_dirmap_write() is responsible for calling it again in

> > > >   *		  this case.

> > > > + * @do_calibration: perform calibration needed for high SPI clock speed

> > > > + *		    operation. Should be called after the SPI memory device has

> > > > + *		    been completely initialized. The op passed should contain

> > > > + *		    a template for the read operation used for the device so

> > > > + *		    the controller can decide what type of calibration is

> > > > + *		    required for this type of read.

> > > >   *

> > > >   * This interface should be implemented by SPI controllers providing an

> > > >   * high-level interface to execute SPI memory operation, which is

> > > > usually the

> > > > @@ -274,6 +280,7 @@ struct spi_controller_mem_ops {

> > > >  			       u64 offs, size_t len, void *buf);

> > > >  	ssize_t (*dirmap_write)(struct spi_mem_dirmap_desc *desc,

> > > >  				u64 offs, size_t len, const void *buf);

> > > > +	void (*do_calibration)(struct spi_mem *mem, struct spi_mem_op *op);

> > > >  };

> > > >

> > > >  /**

> > > > @@ -346,6 +353,7 @@ bool spi_mem_dtr_supports_op(struct spi_mem *mem,

> > > >  #endif /* CONFIG_SPI_MEM */

> > > >

> > > >  int spi_mem_adjust_op_size(struct spi_mem *mem, struct spi_mem_op *op);

> > > > +int spi_mem_do_calibration(struct spi_mem *mem, struct spi_mem_op *op);

> > > >

> > > >  bool spi_mem_supports_op(struct spi_mem *mem,

> > > >  			 const struct spi_mem_op *op);

> > > 

> 


-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.
Michael Walle April 29, 2021, 10:20 p.m. UTC | #14
Am 2021-04-29 20:19, schrieb Pratyush Yadav:
> On 29/04/21 06:28PM, Michael Walle wrote:
>> Am 2021-03-12 11:17, schrieb Pratyush Yadav:
>> > On 12/03/21 09:13AM, Tudor.Ambarus@microchip.com wrote:
>> > > On 3/11/21 9:12 PM, Pratyush Yadav wrote:
>> > > > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>> > > >
>> > > > Check if a read is eligible for PHY and if it is, enable PHY and DQS.
>> > >
>> > > DQS as in data strobe? Shouldn't the upper layer inform the QSPI
>> > > controller
>> > > whether DS is required or not?
>> >
>> > Yes, DQS as in data strobe. I need to check this again, but IIRC the
>> > controller cannot run in PHY mode unless DS is used. Ideally the upper
>> > layer should indeed inform the controller whether DS is supported/in-use
>> > or not. That can be used to decide whether PHY mode (and consequently
>> > the DS line) is to be used or not.
>> >
>> > Currently there are only two flashes that use 8D-8D-8D mode (S28HS512T
>> > and MT35XU512ABA), and both of them drive the DS line.
>> 
>> The LS1028A datasheet explicitly states that the calibration is only
>> used for non-DQS flashes. Which makes sense, because it just determine 
>> at
>> which point the input data is sampled. And if the flash provides a 
>> data
>> strobe, it already know when to sample it. What I am missing here?
> 
> If there was 0 delay in transferring the signals from flash to
> SoC/controller, you would be right. But in practice there is a small 
> but
> noticeable delay from when the flash launches the signal and when it is
> received by the device. So by the time the DQS signal reaches the SoC 
> it
> might already be too late and the data lines might not be valid any
> more. The calibration accounts for these (and some others) delays.

DQS and the data signals are trace length matched, so for data reads
they will end up on the IO pad of the SoC at the same time. This is
also mentioned in [1] (Fig 1.1, point 4 and 5). So while there needs
to be a delay on the clock line for the receiving FF, the best value
for this should be half the SCK clock period.

Does this work without DQS? That should be the main purpose for a
calibration, no? Because in this case, you'll have to determine
the delay between SCK and the data signals (for reads).

Btw. I can't get my head around how the TX delay search would work.
Basically you shift the SCK to the command / data to the flash. So
the flash will either recognize a valid read command or if the delay
is too short/too long the flash will (hopefully) ignore the wrong
command, correct? Might there be any misinterpreted commands which
might be harmful? Are there any flashes which actually need a delay
between data out and SCK?

Of course, the calibration might help with broken hardware where the
SCK/DQ/DQS traces are not length matched.

-michael

> 
> See [0] for a somewhat similar discussion I had with Tudor.
> 
> [0] 
> https://lore.kernel.org/linux-mtd/20210312181447.dlecnw2oed7jtxe7@ti.com/

[1] https://www.ti.com/lit/an/spract2/spract2.pdf
Michael Walle April 29, 2021, 10:48 p.m. UTC | #15
Am 2021-03-11 20:12, schrieb Pratyush Yadav:
> +	if (of_property_read_u32(np, "cdns,phy-tx-start", 

> &f_pdata->phy_tx_start))

> +		f_pdata->phy_tx_start = 16;

> +

> +	if (of_property_read_u32(np, "cdns,phy-tx-end", 

> &f_pdata->phy_tx_end))

> +		f_pdata->phy_tx_end = 48;

> +


I didn't see a dt-bindings patch for these.

-michael
Pratyush Yadav April 30, 2021, 5:42 a.m. UTC | #16
On 30/04/21 12:48AM, Michael Walle wrote:
> Am 2021-03-11 20:12, schrieb Pratyush Yadav:

> > +	if (of_property_read_u32(np, "cdns,phy-tx-start",

> > &f_pdata->phy_tx_start))

> > +		f_pdata->phy_tx_start = 16;

> > +

> > +	if (of_property_read_u32(np, "cdns,phy-tx-end", &f_pdata->phy_tx_end))

> > +		f_pdata->phy_tx_end = 48;

> > +

> 

> I didn't see a dt-bindings patch for these.


Right. Will add them in the next re-roll.
 
> -michael


-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.
Miquel Raynal May 17, 2022, 2:02 p.m. UTC | #17
Hi Pratyush,

p.yadav@ti.com wrote on Fri, 12 Mar 2021 00:42:13 +0530:

> Once the flash is initialized tell the controller it can run
> calibration procedures if needed. This can be useful when calibration is
> needed to run at higher clock speeds.
> 
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> ---
>  drivers/mtd/spi-nor/core.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
> index 88888df009f0..e0cbcaf1be89 100644
> --- a/drivers/mtd/spi-nor/core.c
> +++ b/drivers/mtd/spi-nor/core.c
> @@ -3650,6 +3650,7 @@ static int spi_nor_probe(struct spi_mem *spimem)
>  	 * checking what's really supported using spi_mem_supports_op().
>  	 */
>  	const struct spi_nor_hwcaps hwcaps = { .mask = SNOR_HWCAPS_ALL };
> +	struct spi_mem_op op;
>  	char *flash_name;
>  	int ret;
>  
> @@ -3709,8 +3710,15 @@ static int spi_nor_probe(struct spi_mem *spimem)
>  	if (ret)
>  		return ret;
>  
> -	return mtd_device_register(&nor->mtd, data ? data->parts : NULL,
> -				   data ? data->nr_parts : 0);
> +	ret = mtd_device_register(&nor->mtd, data ? data->parts : NULL,
> +				  data ? data->nr_parts : 0);
> +	if (ret)
> +		return ret;
> +
> +	op = spi_nor_spimem_get_read_op(nor);

Isn't this too specific? I really don't know much about spi-nors, but I
find odd to have this op being created here, why not moving this into
the _do_calibration() helper?

> +	spi_mem_do_calibration(nor->spimem, &op);

A warning/info upon calibration error (not on the absence of the hook)
would be nice?

> +
> +	return 0;
>  }
>  
>  static int spi_nor_remove(struct spi_mem *spimem)

Otherwise I like the overall idea.

Thanks,
Miquèl
Miquel Raynal May 18, 2022, 7:19 a.m. UTC | #18
Hi Pratyush,

p.yadav@ti.com wrote on Wed, 18 May 2022 11:37:05 +0530:

> +Cedric
> 
> On 17/05/22 04:02PM, Miquel Raynal wrote:
> > Hi Pratyush,
> > 
> > p.yadav@ti.com wrote on Fri, 12 Mar 2021 00:42:13 +0530:
> >   
> > > Once the flash is initialized tell the controller it can run
> > > calibration procedures if needed. This can be useful when calibration is
> > > needed to run at higher clock speeds.
> > > 
> > > Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> > > ---
> > >  drivers/mtd/spi-nor/core.c | 12 ++++++++++--
> > >  1 file changed, 10 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
> > > index 88888df009f0..e0cbcaf1be89 100644
> > > --- a/drivers/mtd/spi-nor/core.c
> > > +++ b/drivers/mtd/spi-nor/core.c
> > > @@ -3650,6 +3650,7 @@ static int spi_nor_probe(struct spi_mem *spimem)
> > >  	 * checking what's really supported using spi_mem_supports_op().
> > >  	 */
> > >  	const struct spi_nor_hwcaps hwcaps = { .mask = SNOR_HWCAPS_ALL };
> > > +	struct spi_mem_op op;
> > >  	char *flash_name;
> > >  	int ret;
> > >  
> > > @@ -3709,8 +3710,15 @@ static int spi_nor_probe(struct spi_mem *spimem)
> > >  	if (ret)
> > >  		return ret;
> > >  
> > > -	return mtd_device_register(&nor->mtd, data ? data->parts : NULL,
> > > -				   data ? data->nr_parts : 0);
> > > +	ret = mtd_device_register(&nor->mtd, data ? data->parts : NULL,
> > > +				  data ? data->nr_parts : 0);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	op = spi_nor_spimem_get_read_op(nor);  
> > 
> > Isn't this too specific? I really don't know much about spi-nors, but I
> > find odd to have this op being created here, why not moving this into
> > the _do_calibration() helper?  
> 
> Maybe the naming confused you but this is a function in the SPI NOR 
> core, not in SPI MEM. SPI NOR supports both SPI MEM based controllers 
> and "legacy" controllers, so the convention is to add the "spimem" 
> prefix before SPI MEM specific functions. So I don't get the comment 
> about it being too specific. It is too specific to what?

Mmh right, it's fine then.

> 
> And how can spi_mem_do_calibration() know what op the flash uses to read 
> data? SPI NOR or SPI NAND would know it, but not SPI MEM. That is why we 
> pass in that information to spi_mem_do_calibration().

But here the op is "spi-nor wide", I would have expected a
per-device op. But that is not a big deal, that is something that can
also be updated later if needed I guess.

One last question, is there something that mtd_device_register() does
that is really needed for the calibration to work? Otherwise I would
rather prefer to have that calibration happening before the user gets
access to the device.

> 
> >   
> > > +	spi_mem_do_calibration(nor->spimem, &op);  
> > 
> > A warning/info upon calibration error (not on the absence of the hook)
> > would be nice?  
> 
> Yes, agreed.
> 
> >   
> > > +
> > > +	return 0;
> > >  }
> > >  
> > >  static int spi_nor_remove(struct spi_mem *spimem)  
> > 
> > Otherwise I like the overall idea.  
> 
> Thanks for reviewing.
> 
> > 
> > Thanks,
> > Miquèl  
> 


Thanks,
Miquèl
Cédric Le Goater May 18, 2022, 8:51 a.m. UTC | #19
Hello,

On 5/18/22 09:56, Pratyush Yadav wrote:
> On 18/05/22 09:19AM, Miquel Raynal wrote:
>> Hi Pratyush,
>>
>> p.yadav@ti.com wrote on Wed, 18 May 2022 11:37:05 +0530:
>>
>>> +Cedric
>>>
>>> On 17/05/22 04:02PM, Miquel Raynal wrote:
>>>> Hi Pratyush,
>>>>
>>>> p.yadav@ti.com wrote on Fri, 12 Mar 2021 00:42:13 +0530:
>>>>    
>>>>> Once the flash is initialized tell the controller it can run
>>>>> calibration procedures if needed. This can be useful when calibration is
>>>>> needed to run at higher clock speeds.
>>>>>
>>>>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>>>>> ---
>>>>>   drivers/mtd/spi-nor/core.c | 12 ++++++++++--
>>>>>   1 file changed, 10 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
>>>>> index 88888df009f0..e0cbcaf1be89 100644
>>>>> --- a/drivers/mtd/spi-nor/core.c
>>>>> +++ b/drivers/mtd/spi-nor/core.c
>>>>> @@ -3650,6 +3650,7 @@ static int spi_nor_probe(struct spi_mem *spimem)
>>>>>   	 * checking what's really supported using spi_mem_supports_op().
>>>>>   	 */
>>>>>   	const struct spi_nor_hwcaps hwcaps = { .mask = SNOR_HWCAPS_ALL };
>>>>> +	struct spi_mem_op op;
>>>>>   	char *flash_name;
>>>>>   	int ret;
>>>>>   
>>>>> @@ -3709,8 +3710,15 @@ static int spi_nor_probe(struct spi_mem *spimem)
>>>>>   	if (ret)
>>>>>   		return ret;
>>>>>   
>>>>> -	return mtd_device_register(&nor->mtd, data ? data->parts : NULL,
>>>>> -				   data ? data->nr_parts : 0);
>>>>> +	ret = mtd_device_register(&nor->mtd, data ? data->parts : NULL,
>>>>> +				  data ? data->nr_parts : 0);
>>>>> +	if (ret)
>>>>> +		return ret;
>>>>> +
>>>>> +	op = spi_nor_spimem_get_read_op(nor);
>>>>
>>>> Isn't this too specific? I really don't know much about spi-nors, but I
>>>> find odd to have this op being created here, why not moving this into
>>>> the _do_calibration() helper?
>>>
>>> Maybe the naming confused you but this is a function in the SPI NOR
>>> core, not in SPI MEM. SPI NOR supports both SPI MEM based controllers
>>> and "legacy" controllers, so the convention is to add the "spimem"
>>> prefix before SPI MEM specific functions. So I don't get the comment
>>> about it being too specific. It is too specific to what?
>>
>> Mmh right, it's fine then.
>>
>>>
>>> And how can spi_mem_do_calibration() know what op the flash uses to read
>>> data? SPI NOR or SPI NAND would know it, but not SPI MEM. That is why we
>>> pass in that information to spi_mem_do_calibration().
>>
>> But here the op is "spi-nor wide", I would have expected a
>> per-device op. But that is not a big deal, that is something that can
>> also be updated later if needed I guess.
> 
> It is per-device. The op is generated using nor->read_opcode,
> nor->addr_width, nor->read_dummy, etc. So if you have 2 NOR flashes on
> your system with different opcodes, it would work for both.
> 
>>
>> One last question, is there something that mtd_device_register() does
>> that is really needed for the calibration to work? Otherwise I would
>> rather prefer to have that calibration happening before the user gets
>> access to the device.

Which would mean calling it right after :

	ret = spi_nor_create_read_dirmap(nor);
	if (ret)
		return ret;

	ret = spi_nor_create_write_dirmap(nor);
	if (ret)
		return ret;

> The calibration works by reading a known pattern that is already written
> to the flash again and again and seeing what delays work and what don't.
> For that to happen, the controller driver needs to know where the
> pattern is stored. 

Why don't you simply choose some random place, first 16KB for instance,
and check that the data is random enough ? If not, declare calibration
not possible and choose a default safe setting which is anyhow a
requirement for calibration. Retry at reboot as data might have changed.

> This series does that by looking at the MTD
> partitions. For that to happen, we need to create those partitions
> first, which happens after mtd_device_register().

hmm, that might work for some boards. This is not at all the case for
the BMC boards. Vendors can put any kind of flash model and/or layout
and the driver needs to be more generic.

> But I am planning to use device tree to get that information now so this
> should no longer be needed and we can do calibration before registering
> the device with MTD.

Perfect, we can move the calibration hook in spi_nor_create_read_dirmap()
then, or in devm_spi_mem_dirmap_create(), which would make more sense IMHO.

Thanks,

C.
Cédric Le Goater June 27, 2022, 9:43 a.m. UTC | #20
On 6/27/22 11:14, Pratyush Yadav wrote:
> On 18/05/22 10:51AM, Cédric Le Goater wrote:
>> Hello,
>>
>> On 5/18/22 09:56, Pratyush Yadav wrote:
>>> On 18/05/22 09:19AM, Miquel Raynal wrote:
>>>> Hi Pratyush,
>>>>
>>>> p.yadav@ti.com wrote on Wed, 18 May 2022 11:37:05 +0530:
>>>>
>>>>> +Cedric
>>>>>
>>>>> On 17/05/22 04:02PM, Miquel Raynal wrote:
>>>>>> Hi Pratyush,
>>>>>>
>>>>>> p.yadav@ti.com wrote on Fri, 12 Mar 2021 00:42:13 +0530:
>>>>>>> Once the flash is initialized tell the controller it can run
>>>>>>> calibration procedures if needed. This can be useful when calibration is
>>>>>>> needed to run at higher clock speeds.
>>>>>>>
>>>>>>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>>>>>>> ---
>>>>>>>    drivers/mtd/spi-nor/core.c | 12 ++++++++++--
>>>>>>>    1 file changed, 10 insertions(+), 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
>>>>>>> index 88888df009f0..e0cbcaf1be89 100644
>>>>>>> --- a/drivers/mtd/spi-nor/core.c
>>>>>>> +++ b/drivers/mtd/spi-nor/core.c
>>>>>>> @@ -3650,6 +3650,7 @@ static int spi_nor_probe(struct spi_mem *spimem)
>>>>>>>    	 * checking what's really supported using spi_mem_supports_op().
>>>>>>>    	 */
>>>>>>>    	const struct spi_nor_hwcaps hwcaps = { .mask = SNOR_HWCAPS_ALL };
>>>>>>> +	struct spi_mem_op op;
>>>>>>>    	char *flash_name;
>>>>>>>    	int ret;
>>>>>>> @@ -3709,8 +3710,15 @@ static int spi_nor_probe(struct spi_mem *spimem)
>>>>>>>    	if (ret)
>>>>>>>    		return ret;
>>>>>>> -	return mtd_device_register(&nor->mtd, data ? data->parts : NULL,
>>>>>>> -				   data ? data->nr_parts : 0);
>>>>>>> +	ret = mtd_device_register(&nor->mtd, data ? data->parts : NULL,
>>>>>>> +				  data ? data->nr_parts : 0);
>>>>>>> +	if (ret)
>>>>>>> +		return ret;
>>>>>>> +
>>>>>>> +	op = spi_nor_spimem_get_read_op(nor);
>>>>>>
>>>>>> Isn't this too specific? I really don't know much about spi-nors, but I
>>>>>> find odd to have this op being created here, why not moving this into
>>>>>> the _do_calibration() helper?
>>>>>
>>>>> Maybe the naming confused you but this is a function in the SPI NOR
>>>>> core, not in SPI MEM. SPI NOR supports both SPI MEM based controllers
>>>>> and "legacy" controllers, so the convention is to add the "spimem"
>>>>> prefix before SPI MEM specific functions. So I don't get the comment
>>>>> about it being too specific. It is too specific to what?
>>>>
>>>> Mmh right, it's fine then.
>>>>
>>>>>
>>>>> And how can spi_mem_do_calibration() know what op the flash uses to read
>>>>> data? SPI NOR or SPI NAND would know it, but not SPI MEM. That is why we
>>>>> pass in that information to spi_mem_do_calibration().
>>>>
>>>> But here the op is "spi-nor wide", I would have expected a
>>>> per-device op. But that is not a big deal, that is something that can
>>>> also be updated later if needed I guess.
>>>
>>> It is per-device. The op is generated using nor->read_opcode,
>>> nor->addr_width, nor->read_dummy, etc. So if you have 2 NOR flashes on
>>> your system with different opcodes, it would work for both.
>>>
>>>>
>>>> One last question, is there something that mtd_device_register() does
>>>> that is really needed for the calibration to work? Otherwise I would
>>>> rather prefer to have that calibration happening before the user gets
>>>> access to the device.
>>
>> Which would mean calling it right after :
>>
>> 	ret = spi_nor_create_read_dirmap(nor);
>> 	if (ret)
>> 		return ret;
>>
>> 	ret = spi_nor_create_write_dirmap(nor);
>> 	if (ret)
>> 		return ret;
>>
>>> The calibration works by reading a known pattern that is already written
>>> to the flash again and again and seeing what delays work and what don't.
>>> For that to happen, the controller driver needs to know where the
>>> pattern is stored.
>>
>> Why don't you simply choose some random place, first 16KB for instance,
>> and check that the data is random enough ? If not, declare calibration
>> not possible and choose a default safe setting which is anyhow a
>> requirement for calibration. Retry at reboot as data might have changed.
> 
> I did not come up with the pattern myself. But from what I can
> understand, the pattern is not random at all, but is carefully chosen to
> target certain ways a read can fail on the controller. So a random piece
> of data won't work as well as this carefully designed pattern.

True. I don't exactly remember how your proposal worked from the
driver side but I imagine having a specific DT property to locate
that pattern in the setup handler and to use it later on is not
too complex.

>>> This series does that by looking at the MTD
>>> partitions. For that to happen, we need to create those partitions
>>> first, which happens after mtd_device_register().
>>
>> hmm, that might work for some boards. This is not at all the case for
>> the BMC boards. Vendors can put any kind of flash model and/or layout
>> and the driver needs to be more generic.
> 
> Yes, vendors can choose any layout, but one partition on that layout
> would be your calibration pattern. I think you can use a different
> compatible for that partition. 

OK. and that it would become more generic then.

> I have not thought this through yet though.

If a partition is required, that's a dependency on mtdpart.

It could be done from spi_nor_probe() after mtd_device_register() with
some spimem handler using the 'struct mtd_partition' for the {size,offset}
parameters.

>>
>>> But I am planning to use device tree to get that information now so this
>>> should no longer be needed and we can do calibration before registering
>>> the device with MTD.
>>
>> Perfect, we can move the calibration hook in spi_nor_create_read_dirmap()
>> then, or in devm_spi_mem_dirmap_create(), which would make more sense IMHO.
> 
> Sorry, I still don't get why you want to tie dirmap and calibration
> together. Just let them be independent and let flash drivers take care
> of when to call what. SPI MEM should not care.

I know you would prefer a specific handler and that can still be done.

Thanks,

C.