[v13,00/15] Add Multi Circular Queue Support

Hi Martin,

This patch series is an implementation of UFS Multi-Circular Queue.
Please consider this series for next merge window.
This implementation has been verified on a Qualcomm & MediaTek platform.

Thanks,
Asutosh


UFS Multi-Circular Queue (MCQ) has been added in UFSHCI v4.0 to improve storage performance.
The implementation uses the shared tagging mechanism so that tags are shared
among the hardware queues. The number of hardware queues is configurable.
This series doesn't include the ESI implementation for completion handling.

Please take a look and let us know your thoughts.

v12 -> v13:
- Rebased on top of latest Martin's tree and fixed conflicts 
- Fixed merged conflicts due to (3d2f12b Merge patch series "ufs: qcom: Add HS-G4 support") in
  0003-ufs-core-Defer-adding-host-to-scsi-if-mcq-is-support.patch to support the reinitialization
  needed on Qualcomm platforms for HS-G4.
 
v11 -> v12:
- Dropped 0001-ufs-core-Optimize-duplicate-code-to-read-extended-fe.patch.
  Arthur's change series
  (https://lore.kernel.org/r/1670763911-8695-1-git-send-email-Arthur.Simchaev@wdc.com) makes this change unnecessary.
- Pass hwq as a parameter to ufshcd_send_command() in ufshcd_advanced_rpmb_req_handler()
- Added Stanley Chu's Reviewed-by tags

v10 -> v11:
- Added Stanley Chu's Reviewed-by tags

v9 -> v10:
- Fix few checkpatch warnings
- Use div_u64() instead of direct division in ufshcd_mcq_get_tag() to fix a LKP
warning

v8 -> v9:
- Added missing Reviewed-by tags.

v7 -> v8:
- Addressed Eddie's comments

v6 -> v7:
- Added missing Reviewed-by tags.

v5 -> v6:
- Addressed Mani's comments
- Addressed Bart's comments

v4 -> v5:
- Fixed failure to fallback to SDB during initialization
- Fixed failure when rpm-lvl=5 in the ufshcd_host_reset_and_restore() path
- Improved ufshcd_mcq_config_nr_queues() to handle different configurations
- Addressed Bart's comments
- Verified read/write using FIO, clock gating, runtime-pm[lvl=3, lvl=5]

v3 -> v4:
- Added a kernel module parameter to disable MCQ mode
- Added Bart's reviewed-by tag for some patches
- Addressed Bart's comments

v2 -> v3:
- Split ufshcd_config_mcq() into ufshcd_alloc_mcq() and ufshcd_config_mcq()
- Use devm_kzalloc() in ufshcd_mcq_init()
- Free memory and resource allocation on error paths
- Corrected typos in code comments

v1 -> v2:
- Added a non MCQ related change to use a function to extrace ufs extended
feature
- Addressed Mani's comments
- Addressed Bart's comments

v1:
- Split the changes
- Addressed Bart's comments
- Addressed Bean's comments

* RFC versions:
v2 -> v3:
- Split the changes based on functionality
- Addressed queue configuration issues
- Faster SQE tail pointer increments
- Addressed comments from Bart and Manivannan

v1 -> v2:
- Enabled host_tagset
- Added queue num configuration support
- Added one more vops to allow vendor provide the wanted MAC
- Determine nutrs and can_queue by considering both MAC, bqueuedepth and EXT_IID support
- Postponed MCQ initialization and scsi_add_host() to async probe
- Used (EXT_IID, Task Tag) tuple to support up to 4096 tasks (theoretically)


Asutosh Das (15):
  ufs: core: Probe for ext_iid support
  ufs: core: Introduce Multi-circular queue capability
  ufs: core: Defer adding host to scsi if mcq is supported
  ufs: core: mcq: Add support to allocate multiple queues
  ufs: core: mcq: Configure resource regions
  ufs: core: mcq: Calculate queue depth
  ufs: core: mcq: Allocate memory for mcq mode
  ufs: core: mcq: Configure operation and runtime interface
  ufs: core: mcq: Use shared tags for MCQ mode
  ufs: core: Prepare ufshcd_send_command for mcq
  ufs: core: mcq: Find hardware queue to queue request
  ufs: core: Prepare for completion in mcq
  ufs: mcq: Add completion support of a cqe
  ufs: core: mcq: Add completion support in poll
  ufs: core: mcq: Enable Multi Circular Queue

 drivers/ufs/core/Makefile      |   2 +-
 drivers/ufs/core/ufs-mcq.c     | 415 +++++++++++++++++++++++++++++++++++++++++
 drivers/ufs/core/ufshcd-priv.h |  92 ++++++++-
 drivers/ufs/core/ufshcd.c      | 377 +++++++++++++++++++++++++++++++------
 drivers/ufs/host/ufs-qcom.c    | 146 +++++++++++++++
 drivers/ufs/host/ufs-qcom.h    |   5 +
 include/ufs/ufs.h              |   7 +
 include/ufs/ufshcd.h           | 129 +++++++++++++
 include/ufs/ufshci.h           |  64 +++++++
 9 files changed, 1177 insertions(+), 60 deletions(-)
 create mode 100644 drivers/ufs/core/ufs-mcq.c

On Fri, 13 Jan 2023 12:48:37 -0800, Asutosh Das wrote:

> This patch series is an implementation of UFS Multi-Circular Queue.
> Please consider this series for next merge window.
> This implementation has been verified on a Qualcomm & MediaTek platform.
> 
> Thanks,
> Asutosh
> 
> [...]

Applied to 6.3/scsi-queue, thanks!

[01/15] ufs: core: Probe for ext_iid support
        https://git.kernel.org/mkp/scsi/c/6e1d850acff9
[02/15] ufs: core: Introduce Multi-circular queue capability
        https://git.kernel.org/mkp/scsi/c/305a357d3595
[03/15] ufs: core: Defer adding host to scsi if mcq is supported
        https://git.kernel.org/mkp/scsi/c/0cab4023ec7b
[04/15] ufs: core: mcq: Add support to allocate multiple queues
        https://git.kernel.org/mkp/scsi/c/57b1c0ef89ac
[05/15] ufs: core: mcq: Configure resource regions
        https://git.kernel.org/mkp/scsi/c/c263b4ef737e
[06/15] ufs: core: mcq: Calculate queue depth
        https://git.kernel.org/mkp/scsi/c/7224c806876e
[07/15] ufs: core: mcq: Allocate memory for mcq mode
        https://git.kernel.org/mkp/scsi/c/4682abfae2eb
[08/15] ufs: core: mcq: Configure operation and runtime interface
        https://git.kernel.org/mkp/scsi/c/2468da61ea09
[09/15] ufs: core: mcq: Use shared tags for MCQ mode
        https://git.kernel.org/mkp/scsi/c/0d33728fc0e7
[10/15] ufs: core: Prepare ufshcd_send_command for mcq
        https://git.kernel.org/mkp/scsi/c/22a2d563de14
[11/15] ufs: core: mcq: Find hardware queue to queue request
        https://git.kernel.org/mkp/scsi/c/854f84e7feeb
[12/15] ufs: core: Prepare for completion in mcq
        https://git.kernel.org/mkp/scsi/c/c30d8d010b5e
[13/15] ufs: mcq: Add completion support of a cqe
        https://git.kernel.org/mkp/scsi/c/f87b2c41822a
[14/15] ufs: core: mcq: Add completion support in poll
        https://git.kernel.org/mkp/scsi/c/ed975065c31c
[15/15] ufs: core: mcq: Enable Multi Circular Queue
        https://git.kernel.org/mkp/scsi/c/eacb139b77ff